Add full metadata balancing support, v5.
cover Add full metadata balancing support, v5.
This patchset is a fifth round of the metadata balancing support, makes minor revisions and builds on the fourth round of patches submitted (5/26/2010). There are still some definite warts and FIXMEs in the code but this is now a complete patchset for the metadata balance feature.
Changes from the fourth round start around patch 19 and include:
1) Sort mdas so that ignored mdas get committed to disk first. This reduces the time that we may get an inconsistent on-disk state that would require auto-repair.
2) Fixed a few bugs in pv-based metadata balance patchset.
3) Add "phase 2" (vg-based) of metadata balance. Define one function which implements the metadata balancing and call it from the vg_write() path. In most cases, this works just fine, but there are a few checks that need updated for the new 'fid' list of ignored mdas.
4) Update nightly test cases to include all "phase 2" commands such as vgcreate, vgchange, vgsplit, etc.
PATCH 1/35 Change 'filler' to 'flags' in on-disk 'raw_locn' structure.
Future patches will make use of a specific flag in the on-disk 'raw_locn' structure to enable/disable metadata areas, and facilitate metadata balancing.
Note that 'filler' is always set to '0' (see add_mda() - memset), so use of this area as a non-zero flags field is a safe way to provide future code features.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/layout.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
lib/format_text/layout.h +1 -1
@@ -51,7 +51,7 @@ struct raw_locn {uint64_t offset;/* Offset in bytes to start sector */ uint64_t size;/* Bytes */ uint32_t checksum; -uint32_t filler; +uint32_t flags; } __attribute__ ((packed));/* On disk */
Signature · 1 line
1.6.0.6
PATCH 2/35 Add text format specific 'rlocn' ignore flag and access functions.
Adding a flag to the 'rlocn' structure in the mda header of the text format allows us to flip a bit to ignore an area on disk that stores the metadata via the text format specific mda_header. This patch defines the flag and access functions to manage the flag. Other patches will manage the ignore on a format-independent basis, by using a flag in the metadata_area structure.
Signed-off-by Dave Wysochanski <[email protected]>
--- include/.symlinks.in | 1 + lib/format_text/format-text.c | 13 +++++++++++++ lib/format_text/format-text.h | 1 + lib/format_text/layout.h | 10 ++++++++++ 4 files changed, 25 insertions(+), 0 deletions(-)
include/.symlinks.in +1 -0
@@ -26,6 +26,7 @@ @top_srcdir@/lib/format_text/format-text.h @top_srcdir@/lib/format_text/text_export.h @top_srcdir@/lib/format_text/text_import.h + at top_srcdir@/lib/format_text/layout.h @top_srcdir@/lib/label/label.h @top_srcdir@/lib/locking/locking.h @top_srcdir@/lib/log/log.h
lib/format_text/format-text.c +13 -0
@@ -66,6 +66,19 @@ struct text_context {char *desc;/* Description placed inside file */ }; +int rlocn_is_ignored (const struct raw_locn *rlocn) +{ +return (rlocn->flags &RAW_LOCN_IGNORED ?1 :0 ); +} + +void rlocn_set_ignored (struct raw_locn *rlocn,int value) +{ +if (value) + rlocn->flags |=RAW_LOCN_IGNORED ; +else + rlocn->flags &= ~RAW_LOCN_IGNORED ; +} +/* * NOTE: Currently there can be only one vg per text file. */
lib/format_text/format-text.h +1 -0
@@ -17,6 +17,7 @@#define _LVM_FORMAT_TEXT_H#include "lvm-types.h" +#include "layout.h" #include "metadata.h" #define FMT_TEXT_NAME "lvm2"
lib/format_text/layout.h +10 -0
@@ -46,6 +46,13 @@ struct pv_header {struct disk_locn disk_areas_xl[0 ];/* Two lists */ } __attribute__ ((packed)); +/* +* Ignore this raw location. This allows us to +* ignored metadata areas easily, and thus balance +* metadata across VGs with many PVs. +*/ +#define RAW_LOCN_IGNORED 0x00000001 +/* On disk */ struct raw_locn {uint64_t offset;/* Offset in bytes to start sector */
@@ -54,6 +61,9 @@ struct raw_locn {uint32_t flags; } __attribute__ ((packed)); +int rlocn_is_ignored (const struct raw_locn *rlocn); +void rlocn_set_ignored (struct raw_locn *rlocn,int value); +/* On disk */ /* Structure size limited to one sector */ struct mda_header {
Signature · 1 line
1.6.0.6
PATCH 3/35 Add location independent flag and functions to ignore mdas.
First we add a 'flags' field to the location independent metadata_area structure, and a MDA_IGNORE flag. The mda_is_ignored and mda_set_ignored functions are added to manage the flag. Adding the flag and functions gives a library interface to ignore metadata areas independent of the underlying location (disk, file, etc). The location specific read/write functions must then handle the specifics of what this flag means to the location.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/text_label.c | 1 + lib/metadata/metadata.c | 13 +++++++++++++ lib/metadata/metadata.h | 6 ++++++ 3 files changed, 20 insertions(+), 0 deletions(-)
lib/format_text/text_label.c +1 -0
@@ -215,6 +215,7 @@ int add_mda(const struct format_type *fmt, struct dm_pool *mem, struct dm_list * mdal->ops = mda_lists->raw_ops; mdal->metadata_locn = mdac; + mdal->flags =0 ; mdac->area.dev = dev; mdac->area.start = start;
lib/metadata/metadata.c +13 -0
@@ -3855,6 +3855,19 @@ uint32_t pv_pe_alloc_count(const struct physical_volume *pv)return pv_field (pv, pe_alloc_count); } +int mda_is_ignored (struct metadata_area *mda) +{ +return (mda->flags &MDA_IGNORED ); +} + +void mda_set_ignored (struct metadata_area *mda,int value) +{ +if (value) + mda->flags |=MDA_IGNORED ; +else + mda->flags &= ~MDA_IGNORED ; +} +uint32_t pv_mda_count (const struct physical_volume *pv) {struct lvmcache_info *info;
lib/metadata/metadata.h +6 -0
@@ -158,12 +158,18 @@ struct metadata_area_ops { }; +#define MDA_IGNORED 0x00000001 +struct metadata_area {struct dm_list list;struct metadata_area_ops *ops;void *metadata_locn; +uint32_t flags; }; +int mda_is_ignored (struct metadata_area *mda); +void mda_set_ignored (struct metadata_area *mda,int value); +#define seg_pvseg (seg, s) (seg)->areas[(s)].u.pv.pvseg#define seg_dev (seg, s) (seg)->areas[(s)].u.pv.pvseg->pv->dev#define seg_pe (seg, s) (seg)->areas[(s)].u.pv.pvseg->pe
Signature · 1 line
1.6.0.6
PATCH 4/35 Move dev_open / dev_close outside _vg_read_raw_area().
This refactoring moves the device open/close up one level to the caller of _vg_read_raw_area(). Should be no functional change and facilitate future changes related to metadata balancing.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 40 ++++++++++++++++++++++++++++++---------- 1 files changed, 30 insertions(+), 10 deletions(-)
lib/format_text/format-text.c +30 -10
@@ -477,9 +477,6 @@ static struct volume_group *_vg_read_raw_area(struct format_instance *fid,char *desc;uint32_t wrap =0 ; -if (!dev_open (area->dev)) - return_NULL; -if (!(mdah =_raw_read_mda_header (fid->fmt, area))) goto_out;
@@ -514,9 +511,6 @@ static struct volume_group *_vg_read_raw_area(struct format_instance *fid, vg->status |=PRECOMMITTED ; out: -if (!dev_close (area->dev)) - stack; -return vg; }
@@ -525,8 +519,17 @@ static struct volume_group *_vg_read_raw(struct format_instance *fid, struct metadata_area *mda) {struct mda_context *mdac = (struct mda_context *) mda->metadata_locn; +struct volume_group *vg; -return _vg_read_raw_area (fid, vgname, &mdac->area,0 ); +if (!dev_open (mdac->area.dev)) + return_NULL; + + vg =_vg_read_raw_area (fid, vgname, &mdac->area,0 ); + +if (!dev_close (mdac->area.dev)) + stack; + +return vg; } static struct volume_group *_vg_read_precommit_raw(struct format_instance *fid,
@@ -534,8 +537,17 @@ static struct volume_group *_vg_read_precommit_raw(struct format_instance *fid, struct metadata_area *mda) {struct mda_context *mdac = (struct mda_context *) mda->metadata_locn; +struct volume_group *vg; + +if (!dev_open (mdac->area.dev)) + return_NULL; + + vg =_vg_read_raw_area (fid, vgname, &mdac->area,1 ); + +if (!dev_close (mdac->area.dev)) + stack; -return _vg_read_raw_area (fid, vgname, &mdac->area,1 ); +return vg; } static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg,
@@ -1186,9 +1198,17 @@ static int _scan_raw(const struct format_type *fmt)/* FIXME We're reading mdah twice here... */ if ((vgname =vgname_from_mda (fmt, &rl->dev_area, &vgid, &vgstatus,NULL ,NULL ))) { -if ((vg =_vg_read_raw_area (&fid, vgname, - &rl->dev_area,0 ))) +if (!dev_open (rl->dev_area.dev)) { + stack; +continue ; + } + + vg =_vg_read_raw_area (&fid, vgname, &rl->dev_area,0 ); +if (vg)lvmcache_update_vg (vg,0 ); + +if (!dev_close (rl->dev_area.dev)) + stack; } }
Signature · 1 line
1.6.0.6
PATCH 5/35 Move dev_open/dev_close outside vgname_from_mda().
Refactor vgname_from_mda() so caller must open/close the device. Should be no functional change.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 20 +++++++------------- lib/format_text/text_label.c | 6 ++++++ 2 files changed, 13 insertions(+), 13 deletions(-)
lib/format_text/format-text.c +7 -13
@@ -1093,9 +1093,6 @@ const char *vgname_from_mda(const struct format_type *fmt,if (mda_free_sectors) *mda_free_sectors = ((dev_area->size -MDA_HEADER_SIZE ) /2 ) >>SECTOR_SHIFT ; -if (!dev_open (dev_area->dev)) - return_NULL; -if (!(mdah =_raw_read_mda_header (fmt, dev_area))) goto_out;
@@ -1173,9 +1170,6 @@ const char *vgname_from_mda(const struct format_type *fmt, } out: -if (!dev_close (dev_area->dev)) - stack; -return vgname; }
@@ -1196,20 +1190,20 @@ static int _scan_raw(const struct format_type *fmt) dm_list_iterate_items(rl, raw_list) {/* FIXME We're reading mdah twice here... */ +if (!dev_open (rl->dev_area.dev)) { + stack; +continue ; + } +if ((vgname =vgname_from_mda (fmt, &rl->dev_area, &vgid, &vgstatus,NULL ,NULL ))) { -if (!dev_open (rl->dev_area.dev)) { - stack; -continue ; - } - vg =_vg_read_raw_area (&fid, vgname, &rl->dev_area,0 );if (vg)lvmcache_update_vg (vg,0 ); -if (!dev_close (rl->dev_area.dev)) - stack; } +if (!dev_close (rl->dev_area.dev)) + stack; }return 1 ;
lib/format_text/text_label.c +6 -0
@@ -299,6 +299,10 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, dm_list_iterate_items(mda, &info->mdas) { mdac = (struct mda_context *) mda->metadata_locn; +if (!dev_open (mdac->area.dev)) { + stack; +continue ; + } if ((vgname = vgname_from_mda(info->fmt, &mdac->area, &vgid, &vgstatus, &creation_host, &mdac->free_sectors)) &&
@@ -306,6 +310,8 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, (char *) &vgid, vgstatus, creation_host)) return_0; +if (!dev_close (mdac->area.dev)) + stack; } info->status &= ~CACHE_INVALID ;
Signature · 1 line
1.6.0.6
PATCH 6/35 Allow raw_read_mda_header to be called from text_label.c.
We'd like to pass in mda_header to vgname_from_mda(). In order to do this, we need to call raw_read_mda_header() from text_label.c, _text_read(), which gets called from the label_read() path, and peers into the metadata and update vginfo cache. We should check the disable bit here, and if set, not peer into the vg metadata, thus reducing the I/O to disk.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 33 +++++++++++++++++++-------------- lib/format_text/format-text.h | 1 + lib/format_text/layout.h | 4 ++++ lib/format_text/text_label.c | 15 +++++++++++++-- 4 files changed, 37 insertions(+), 16 deletions(-)
lib/format_text/format-text.c +19 -14
@@ -37,9 +37,6 @@#include <dirent.h> #include <ctype.h> -static struct mda_header *_raw_read_mda_header (const struct format_type *fmt, -struct device_area *dev_area); - static struct format_instance *_text_create_text_instance(const struct format_type *fmt, const char *vgname, const char *vgid,
@@ -181,7 +178,7 @@ static int _pv_analyze_mda_raw (const struct format_type * fmt,if (!dev_open (area->dev)) return_0; -if (!(mdah =_raw_read_mda_header (fmt, area))) +if (!(mdah =raw_read_mda_header (fmt, area))) goto_out; rlocn = mdah->raw_locns;
@@ -308,8 +305,8 @@ static void _xlate_mdah(struct mda_header *mdah) } } -static struct mda_header *_raw_read_mda_header (const struct format_type *fmt, -struct device_area *dev_area) +struct mda_header *raw_read_mda_header (const struct format_type *fmt, +struct device_area *dev_area) {struct mda_header *mdah;
@@ -453,7 +450,7 @@ static int _raw_holds_vgname(struct format_instance *fid,if (!dev_open (dev_area->dev)) return_0; -if (!(mdah =_raw_read_mda_header (fid->fmt, dev_area))) +if (!(mdah =raw_read_mda_header (fid->fmt, dev_area))) return_0;if (_find_vg_rlocn (dev_area, mdah, vgname, &noprecommit))
@@ -477,7 +474,7 @@ static struct volume_group *_vg_read_raw_area(struct format_instance *fid,char *desc;uint32_t wrap =0 ; -if (!(mdah =_raw_read_mda_header (fid->fmt, area))) +if (!(mdah =raw_read_mda_header (fid->fmt, area))) goto_out;if (!(rlocn =_find_vg_rlocn (area, mdah, vgname, &precommitted))) {
@@ -577,7 +574,7 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg,if (!dev_open (mdac->area.dev)) return_0; -if (!(mdah =_raw_read_mda_header (fid->fmt, &mdac->area))) +if (!(mdah =raw_read_mda_header (fid->fmt, &mdac->area))) goto_out; rlocn = _find_vg_rlocn(&mdac->area, mdah,
@@ -683,7 +680,7 @@ static int _vg_commit_raw_rlocn(struct format_instance *fid,if (!found)return 1 ; -if (!(mdah =_raw_read_mda_header (fid->fmt, &mdac->area))) +if (!(mdah =raw_read_mda_header (fid->fmt, &mdac->area))) goto_out; if (!(rlocn = _find_vg_rlocn(&mdac->area, mdah,
@@ -794,7 +791,7 @@ static int _vg_remove_raw(struct format_instance *fid, struct volume_group *vg,if (!dev_open (mdac->area.dev)) return_0; -if (!(mdah =_raw_read_mda_header (fid->fmt, &mdac->area))) +if (!(mdah =raw_read_mda_header (fid->fmt, &mdac->area))) goto_out;if (!(rlocn =_find_vg_rlocn (&mdac->area, mdah, vg->name, &noprecommit))) {
@@ -1077,12 +1074,12 @@ static int _scan_file(const struct format_type *fmt) }const char *vgname_from_mda (const struct format_type *fmt, +struct mda_header *mdah,struct device_area *dev_area,struct id *vgid,uint64_t *vgstatus,char **creation_host,uint64_t *mda_free_sectors) {struct raw_locn *rlocn; -struct mda_header *mdah;uint32_t wrap =0 ;const char *vgname =NULL ;unsigned int len =0 ;
@@ -1093,7 +1090,7 @@ const char *vgname_from_mda(const struct format_type *fmt,if (mda_free_sectors) *mda_free_sectors = ((dev_area->size -MDA_HEADER_SIZE ) /2 ) >>SECTOR_SHIFT ; -if (!(mdah =_raw_read_mda_header (fmt, dev_area))) +if (!mdah) goto_out;/* FIXME Cope with returning a list */
@@ -1182,6 +1179,7 @@ static int _scan_raw(const struct format_type *fmt)struct format_instance fid;struct id vgid;uint64_t vgstatus; +struct mda_header *mdah; raw_list = &((struct mda_lists *) fmt->private)->raws;
@@ -1195,13 +1193,20 @@ static int _scan_raw(const struct format_type *fmt)continue ; } -if ((vgname =vgname_from_mda (fmt, &rl->dev_area, &vgid, &vgstatus, +if (!(mdah =raw_read_mda_header (fmt, &rl->dev_area))) { + stack; + goto close_dev; + } + +if ((vgname =vgname_from_mda (fmt, mdah, + &rl->dev_area, &vgid, &vgstatus,NULL ,NULL ))) { vg =_vg_read_raw_area (&fid, vgname, &rl->dev_area,0 );if (vg)lvmcache_update_vg (vg,0 ); } + close_dev:if (!dev_close (rl->dev_area.dev)) stack; }
lib/format_text/format-text.h +1 -0
@@ -61,6 +61,7 @@ int add_mda(const struct format_type *fmt, struct dm_pool *mem, struct dm_list *void del_mdas (struct dm_list *mdas);const char *vgname_from_mda (const struct format_type *fmt, +struct mda_header *mdah,struct device_area *dev_area,struct id *vgid,uint64_t *vgstatus,char **creation_host,uint64_t *mda_free_sectors);
lib/format_text/layout.h +4 -0
@@ -76,6 +76,9 @@ struct mda_header {struct raw_locn raw_locns[0 ];/* NULL-terminated list */ } __attribute__ ((packed)); +struct mda_header *raw_read_mda_header (const struct format_type *fmt, +struct device_area *dev_area); +struct mda_lists {struct dm_list dirs;struct dm_list raws;
@@ -86,6 +89,7 @@ struct mda_lists {struct mda_context {struct device_area area;uint64_t free_sectors; +struct mda_header *mdah;/* copy of the on-disk mdah */ struct raw_locn rlocn;/* Store inbetween write and commit */ };
lib/format_text/text_label.c +13 -2
@@ -262,6 +262,7 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf,const char *vgname;uint64_t vgstatus;char *creation_host; +struct mda_header *mdah; pvhdr = (struct pv_header *) ((void *) buf +xlate32 (lh->offset_xl));
@@ -303,13 +304,23 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, stack;continue ; } -if ((vgname =vgname_from_mda (info->fmt, &mdac->area, +if (!(mdah =raw_read_mda_header (info->fmt, &mdac->area))) { + stack; + goto close_dev; + } + +if ((vgname =vgname_from_mda (info->fmt, mdah, + &mdac->area, &vgid, &vgstatus, &creation_host, &mdac->free_sectors)) && !lvmcache_update_vgname_and_id (info, vgname, (char *) &vgid, vgstatus, - creation_host)) + creation_host)) { +if (!dev_close (mdac->area.dev)) + stack; return_0; + } + close_dev:if (!dev_close (mdac->area.dev)) stack; }
Signature · 1 line
1.6.0.6
PATCH 7/35 Ensure in-memory state matches on-disk state of mda ignore bit.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 3 +++ lib/format_text/text_label.c | 2 ++ 2 files changed, 5 insertions(+), 0 deletions(-)
lib/format_text/format-text.c +3 -0
@@ -720,6 +720,7 @@ static int _vg_commit_raw_rlocn(struct format_instance *fid,"header at %" PRIu64, vg->name, dev_name(mdac->area.dev), mdac->area.start); +rlocn_set_ignored (mdah->raw_locns,mda_is_ignored (mda));if (!_raw_write_mda_header (fid->fmt, mdac->area.dev, mdac->area.start, mdah)) {dm_pool_free (fid->fmt->cmd->mem, mdah);
@@ -802,6 +803,7 @@ static int _vg_remove_raw(struct format_instance *fid, struct volume_group *vg, rlocn->offset =0 ; rlocn->size =0 ; rlocn->checksum =0 ; +rlocn_set_ignored (mdah->raw_locns,mda_is_ignored (mda));if (!_raw_write_mda_header (fid->fmt, mdac->area.dev, mdac->area.start, mdah)) {
@@ -1505,6 +1507,7 @@ static int _text_pv_write(const struct format_type *fmt, struct physical_volume mdac = mda->metadata_locn;memset (&buf,0 ,sizeof (buf)); mdah->size = mdac->area.size; +rlocn_set_ignored (mdah->raw_locns,mda_is_ignored (mda));if (!_raw_write_mda_header (fmt, mdac->area.dev, mdac->area.start, mdah)) {if (!dev_close (pv->dev))
lib/format_text/text_label.c +2 -0
@@ -301,6 +301,7 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, dm_list_iterate_items(mda, &info->mdas) { mdac = (struct mda_context *) mda->metadata_locn;if (!dev_open (mdac->area.dev)) { +mda_set_ignored (mda,1 ); stack;continue ; }
@@ -308,6 +309,7 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, stack; goto close_dev; } +mda_set_ignored (mda,rlocn_is_ignored (mdah->raw_locns)); if ((vgname = vgname_from_mda(info->fmt, mdah, &mdac->area,
Signature · 1 line
1.6.0.6
PATCH 8/35 Add mda_locn_match() internal library function for mapping pv/device to VG mda.
A metadata_area is defined independent of the location. One downside is that there is no obvious mapping from a pv to an mda. For a PV in a VG, we need a way to start with a PV and end up with an MDA, if we are to manage mdas starting with a device/pv. This function provides us a way to go down the list of PVs on a VG, and identify which ones match a particular PV.
I'm not entirely happy with this approach, but it does fit into the existing structures in a reasonable way.
An alternative solution might be to refactor the VG - PV interface such that mdas are a list tied to a PV. However, this seemed a bit tricky since a PV does not come into existence until after the list of mdas is constructed (see _vg_read() - we create a 'fid' and attach mdas to it, then we go through them and attach pvs).
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 15 +++++++++++++++ lib/metadata/metadata.c | 30 ++++++++++++++++++++++++++++++ lib/metadata/metadata.h | 7 +++++++ 3 files changed, 52 insertions(+), 0 deletions(-)
lib/format_text/format-text.c +15 -0
@@ -121,6 +121,20 @@ static int _mda_in_vg_raw(struct format_instance *fid __attribute((unused)),return 0 ; } +static int _mda_locn_match_raw (struct metadata_area *mda1, +struct metadata_area *mda2) +{ +struct mda_context *mda1c = (struct mda_context *) mda1->metadata_locn; +struct mda_context *mda2c = (struct mda_context *) mda2->metadata_locn; + +if ((mda1c->area.dev == mda2c->area.dev) && + (mda1c->area.start == mda2c->area.start) && + (mda1c->area.size == mda2c->area.size)) +return 1 ; + +return 0 ; +} +/* * For circular region between region_start and region_start + region_size, * back up one SECTOR_SIZE from 'region_ptr' and return the value.
@@ -1724,6 +1738,7 @@ static struct metadata_area_ops _metadata_text_raw_ops = { .mda_total_sectors = _mda_total_sectors_raw, .mda_in_vg = _mda_in_vg_raw, .pv_analyze_mda = _pv_analyze_mda_raw, + .mda_locn_match = _mda_locn_match_raw };/* pvmetadatasize in sectors */
lib/metadata/metadata.c +30 -0
@@ -3855,6 +3855,36 @@ uint32_t pv_pe_alloc_count(const struct physical_volume *pv)return pv_field (pv, pe_alloc_count); } +/* +* This function provides a way to answer the question on a format specific +* basis - does the format specfic context of these two metadata areas +* match? +* +* A metatdata_area is defined to be independent of the underlying context. +* This has the benefit that we can use the same abstraction to read disks +* (see _metadata_text_raw_ops) or files (see _metadata_text_file_ops). +* However, one downside is there is no format-independent way to determine +* whether a given metadata_area is attached to a specific device - in fact, +* it may not be attached to a device at all. +* +* Thus, LVM is structured such that an mda is not a member of struct +* physical_volume. The location of the mda depends on whether +* the PV is in a volume group. A PV not in a VG has an mda on the +* 'info->mda' list in lvmcache, while a PV in a VG has an mda on +* the vg->fid->metadata_areas list. For further details, see _vg_read(), +* and the sequence of creating the format_instance with fid->metadata_areas +* list, as well as the construction of the VG, with list of PVs (comes +* after the construction of the fid and list of mdas). +*/ +int mda_locn_match (struct metadata_area *mda1,struct metadata_area *mda2) +{ +if (!mda1->ops->mda_locn_match || !mda2->ops->mda_locn_match || + mda1->ops->mda_locn_match != mda2->ops->mda_locn_match) +return 0 ; + +return mda1->ops->mda_locn_match (mda1, mda2); +} +int mda_is_ignored (struct metadata_area *mda) {return (mda->flags &MDA_IGNORED );
lib/metadata/metadata.h +7 -0
@@ -156,6 +156,12 @@ struct metadata_area_ops {int (*pv_analyze_mda) (const struct format_type * fmt,struct metadata_area *mda); +/* +* Do these two metadata_areas match with respect to their underlying +* location? +*/ +int (*mda_locn_match)(struct metadata_area *mda1, +struct metadata_area *mda2); };#define MDA_IGNORED 0x00000001
@@ -169,6 +175,7 @@ struct metadata_area {int mda_is_ignored (struct metadata_area *mda);void mda_set_ignored (struct metadata_area *mda,int value); +int mda_locn_match (struct metadata_area *mda1,struct metadata_area *mda2);#define seg_pvseg (seg, s) (seg)->areas[(s)].u.pv.pvseg#define seg_dev (seg, s) (seg)->areas[(s)].u.pv.pvseg->pv->dev
Signature · 1 line
1.6.0.6
PATCH 9/35 Add mda location specific mda_copy constructor.
Because of the way mdas are handled internally, where a PV in a VG has mdas on both info->mdas and vg->fid->metadata_areas list, we need a location independent copy constructor for struct metadata_area. Break up the existing format-text specific copy constructor into a format independent piece and a format dependent piece.
This function is necessary to properly implement pv_set_mda_ignored().
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 28 ++++++++++------------------ lib/metadata/metadata.c | 25 +++++++++++++++++++++++++ lib/metadata/metadata.h | 7 +++++++ 3 files changed, 42 insertions(+), 18 deletions(-)
lib/format_text/format-text.c +10 -18
@@ -1614,32 +1614,23 @@ static int _populate_pv_fields(struct lvmcache_info *info, }/* -* Copy constructor for a metadata_area. +* Copy constructor for a metadata_locn. */ -static struct metadata_area *_mda_copy (struct dm_pool *mem, -struct metadata_area *mda) +static void *_mda_copy_raw (struct dm_pool *mem, +struct metadata_area *mda) { -struct metadata_area *mda_new;struct mda_context *mdac, *mdac_new; -if (!(mda_new =dm_pool_alloc (mem,sizeof (*mda_new)))) { -log_error ("metadata_area allocation failed" ); -return NULL ; - } -/* FIXME: Should have a per-format constructor here */ mdac = (struct mda_context *) mda->metadata_locn; +if (!mdac) +return NULL ;if (!(mdac_new =dm_pool_alloc (mem,sizeof (*mdac_new)))) {log_error ("mda_context allocation failed" ); -dm_pool_free (mem, mda_new);return NULL ; } -memcpy (mda_new, mda,sizeof (*mda));memcpy (mdac_new, mdac,sizeof (*mdac)); - mda_new->metadata_locn = mdac_new; - -/* FIXME mda 'list' left invalid here */ -return mda_new; +return mdac_new; }
@@ -1667,7 +1658,7 @@ static int _text_pv_read(const struct format_type *fmt, const char *pv_name,/* Add copy of mdas to supplied list */ dm_list_iterate_items(mda, &info->mdas) { - mda_new =_mda_copy (fmt->cmd->mem, mda); + mda_new =mda_copy (fmt->cmd->mem, mda);if (!mda_new)return 0 ;dm_list_add (mdas, &mda_new->list);
@@ -1734,6 +1725,7 @@ static struct metadata_area_ops _metadata_text_raw_ops = { .vg_precommit = _vg_precommit_raw, .vg_commit = _vg_commit_raw, .vg_revert = _vg_revert_raw, + .mda_copy = _mda_copy_raw, .mda_free_sectors = _mda_free_sectors_raw, .mda_total_sectors = _mda_total_sectors_raw, .mda_in_vg = _mda_in_vg_raw,
@@ -1804,7 +1796,7 @@ static int _text_pv_setup(const struct format_type *fmt,if (found)continue ; - mda_new =_mda_copy (fmt->cmd->mem, mda); + mda_new =mda_copy (fmt->cmd->mem, mda);if (!mda_new) return_0;dm_list_add (mdas, &mda_new->list);
@@ -1972,7 +1964,7 @@ static struct format_instance *_text_create_text_instance(const struct format_ty mdas = &info->mdas; dm_list_iterate_items(mda, mdas) {/* FIXME Check it holds this VG */ - mda_new =_mda_copy (fmt->cmd->mem, mda); + mda_new =mda_copy (fmt->cmd->mem, mda);if (!mda_new) return_NULL;dm_list_add (&fid->metadata_areas, &mda_new->list);
lib/metadata/metadata.c +25 -0
@@ -3856,6 +3856,31 @@ uint32_t pv_pe_alloc_count(const struct physical_volume *pv) }/* +* Copy constructor for a metadata_area. +*/ +struct metadata_area *mda_copy (struct dm_pool *mem, +struct metadata_area *mda) +{ +struct metadata_area *mda_new; + +if (!(mda_new =dm_pool_alloc (mem,sizeof (*mda_new)))) { +log_error ("metadata_area allocation failed" ); +return NULL ; + } +memcpy (mda_new, mda,sizeof (*mda)); +if (mda->ops->mda_copy && mda->metadata_locn) { + mda_new->metadata_locn = mda->ops->mda_copy (mem, mda); +if (!mda_new->metadata_locn) { +dm_pool_free (mem, mda_new); +return NULL ; + } + } + +/* FIXME mda 'list' left invalid here */ + +return mda_new; +} +/* * This function provides a way to answer the question on a format specific * basis - does the format specfic context of these two metadata areas * match?
lib/metadata/metadata.h +7 -0
@@ -136,6 +136,11 @@ struct metadata_area_ops { struct metadata_area * mda);/* +* Per location copy constructor. +*/ +void *(*mda_copy) (struct dm_pool *mem,struct metadata_area *mda); + +/* * Returns number of free sectors in given metadata area. */ uint64_t (*mda_free_sectors) (struct metadata_area *mda);
@@ -172,6 +177,8 @@ struct metadata_area {void *metadata_locn;uint32_t flags; }; +struct metadata_area *mda_copy (struct dm_pool *mem, +struct metadata_area *mda);int mda_is_ignored (struct metadata_area *mda);void mda_set_ignored (struct metadata_area *mda,int value);
Signature · 1 line
1.6.0.6
PATCH 10/35 Use vg_mda_count() in vgdisplay.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/display/display.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
lib/display/display.c +1 -1
@@ -664,7 +664,7 @@ void vgdisplay_full(const struct volume_group *vg)log_print ("Format %s" , vg->fid->fmt->name);if (vg->fid->fmt->features &FMT_MDAS ) {log_print ("Metadata Areas %d" , -dm_list_size (&vg->fid->metadata_areas)); +vg_mda_count (vg));log_print ("Metadata Sequence No %d" , vg->seqno); } access_str = vg->status & (LVM_READ |LVM_WRITE );
Signature · 1 line
1.6.0.6
PATCH 11/35 Add metadata_areas_ignored list and functions to manage ignored mdas.
Add a second mda list, metadata_areas_ignored to fid, and a couple functions, fid_add_mda() and fid_add_mdas() to help manage the lists.
These functions are needed to properly count the ignored mdas and manage the lists attached to the 'fid' and ultimately the 'vg'.
Ensure metadata_areas_ignored is initialized in other formats, even if the list is never used.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format1/format1.c | 1 + lib/format_pool/format_pool.c | 1 + lib/format_text/format-text.c | 2 ++ lib/metadata/metadata-exported.h | 1 + lib/metadata/metadata.c | 23 +++++++++++++++++++++++ lib/metadata/metadata.h | 2 ++ lib/report/report.c | 1 + 7 files changed, 31 insertions(+), 0 deletions(-)
lib/format1/format1.c +1 -0
@@ -464,6 +464,7 @@ static struct format_instance *_format1_create_instance(const struct format_type fid->fmt = fmt;dm_list_init (&fid->metadata_areas); +dm_list_init (&fid->metadata_areas_ignored);/* Define a NULL metadata area */ if (!(mda =dm_pool_alloc (fmt->cmd->mem,sizeof (*mda)))) {
lib/format_pool/format_pool.c +1 -0
@@ -263,6 +263,7 @@ static struct format_instance *_pool_create_instance(const struct format_type *f fid->fmt = fmt;dm_list_init (&fid->metadata_areas); +dm_list_init (&fid->metadata_areas_ignored);/* Define a NULL metadata area */ if (!(mda =dm_pool_zalloc (fmt->cmd->mem,sizeof (*mda)))) {
lib/format_text/format-text.c +2 -0
@@ -1201,6 +1201,7 @@ static int _scan_raw(const struct format_type *fmt) fid.fmt = fmt;dm_list_init (&fid.metadata_areas); +dm_list_init (&fid.metadata_areas_ignored); dm_list_iterate_items(rl, raw_list) {/* FIXME We're reading mdah twice here... */
@@ -1910,6 +1911,7 @@ static struct format_instance *_text_create_text_instance(const struct format_ty fid->fmt = fmt;dm_list_init (&fid->metadata_areas); +dm_list_init (&fid->metadata_areas_ignored);if (!vgname) {if (!(mda =dm_pool_alloc (fmt->cmd->mem,sizeof (*mda))))
lib/metadata/metadata-exported.h +1 -0
@@ -216,6 +216,7 @@ struct physical_volume {struct format_instance {const struct format_type *fmt;struct dm_list metadata_areas;/* e.g. metadata locations */ +struct dm_list metadata_areas_ignored;void *private; };
lib/metadata/metadata.c +23 -0
@@ -3855,6 +3855,29 @@ uint32_t pv_pe_alloc_count(const struct physical_volume *pv)return pv_field (pv, pe_alloc_count); } +void fid_add_mda (struct format_instance *fid,struct metadata_area *mda) +{ +if (mda_is_ignored (mda)) +dm_list_add (&fid->metadata_areas_ignored, + &mda->list); +else +dm_list_add (&fid->metadata_areas, + &mda->list); +} + +int fid_add_mdas (struct format_instance *fid,struct dm_list *mdas) +{ +struct metadata_area *mda, *mda_new; + + dm_list_iterate_items(mda, mdas) { + mda_new =mda_copy (fid->fmt->cmd->mem, mda); +if (!mda_new) + return_0; +fid_add_mda (fid, mda_new); + } +return 1 ; +} +/* * Copy constructor for a metadata_area. */
lib/metadata/metadata.h +2 -0
@@ -183,6 +183,8 @@ struct metadata_area *mda_copy(struct dm_pool *mem,int mda_is_ignored (struct metadata_area *mda);void mda_set_ignored (struct metadata_area *mda,int value);int mda_locn_match (struct metadata_area *mda1,struct metadata_area *mda2); +void fid_add_mda (struct format_instance *fid,struct metadata_area *mda); +int fid_add_mdas (struct format_instance *fid,struct dm_list *mdas);#define seg_pvseg (seg, s) (seg)->areas[(s)].u.pv.pvseg#define seg_dev (seg, s) (seg)->areas[(s)].u.pv.pvseg->pv->dev
lib/report/report.c +1 -0
@@ -1103,6 +1103,7 @@ static int _copypercent_disp(struct dm_report *rh __attribute((unused)),/* necessary for displaying something for PVs not belonging to VG */ static struct format_instance _dummy_fid = { .metadata_areas = { &(_dummy_fid.metadata_areas), &(_dummy_fid.metadata_areas) }, + .metadata_areas_ignored = { &(_dummy_fid.metadata_areas_ignored), &(_dummy_fid.metadata_areas_ignored) }, };static struct volume_group _dummy_vg = {
Signature · 1 line
1.6.0.6
PATCH 12/35 Define new functions and vgs/pvs fields related to mda ignore.
Define a new pvs field, pv_mda_count_ignored, and a new vgs field, vg_mda_count_ignored to match the existing pv_mda_count and vg_mda_count. Also define various supporting functions to implement the counting as well as setting the ignored flag and determining if an mda is ignored. These high level functions call into the lower level location independent mda ignore functions defined by earlier patches.
Note that counting ignored mdas in a vg requires traversing both lists and checking for the ignored bit on the mda. The count of 'ignored' mdas then is defined by having the bit set, not by which list the mda is on. The list does determine whether LVM actually does read/write to the mda, though we must count the bits in order to return accurate numbers for the various counts. Also, pv_mda_set_ignored must search both vg lists for ignored mda. If the state changes and needs to be committed to disk, the ignored mda will be on the non-ignored list.
Note also in pv_mda_set_ignored(), we must properly manage the mda lists. If we change the ignored state of an mda, we must change any mdas on vg->fid->metadata_areas that correspond to this pv. Also, we may need to allocate a copy of the mda, as is done when fid->metadata_areas is populated from _vg_read(), if we are un-ignoring an ignored mda.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata-exported.h | 4 + lib/metadata/metadata.c | 116 +++++++++++++++++++++++++++++++++++++- lib/report/columns.h | 2 + lib/report/report.c | 25 ++++++++ 4 files changed, 146 insertions(+), 1 deletions(-)
lib/metadata/metadata-exported.h +4 -0
@@ -857,6 +857,9 @@ uint64_t pv_pe_start(const struct physical_volume *pv);uint32_t pv_pe_count (const struct physical_volume *pv);uint32_t pv_pe_alloc_count (const struct physical_volume *pv);uint32_t pv_mda_count (const struct physical_volume *pv); +uint32_t pv_mda_count_ignored (const struct physical_volume *pv); +uint32_t pv_mda_is_ignored (const struct physical_volume *pv); +int pv_mda_set_ignored (const struct physical_volume *pv,int value);uint64_t lv_size (const struct logical_volume *lv);
@@ -872,6 +875,7 @@ uint64_t vg_pv_count(const struct volume_group *vg);uint64_t vg_max_pv (const struct volume_group *vg);uint64_t vg_max_lv (const struct volume_group *vg);uint32_t vg_mda_count (const struct volume_group *vg); +uint32_t vg_mda_count_ignored (const struct volume_group *vg);int vg_check_write_mode (struct volume_group *vg);#define vg_is_clustered (vg) (vg_status((vg)) & CLUSTERED)#define vg_is_exported (vg) (vg_status((vg)) & EXPORTED_VG)
lib/metadata/metadata.c +115 -1
128 lines not on this page · raw message
lib/report/columns.h +2 -0
@@ -92,6 +92,7 @@ FIELD(PVS, pv, NUM, "PE", pe_count, 3, uint32, "pv_pe_count", "Total number of P FIELD(PVS, pv, NUM, "Alloc", pe_alloc_count, 5, uint32, "pv_pe_alloc_count", "Total number of allocated Physical Extents.") FIELD(PVS, pv, STR, "PV Tags", tags, 7, tags, "pv_tags", "Tags, if any.") FIELD(PVS, pv, NUM, "#PMda", id, 5, pvmdas, "pv_mda_count", "Number of metadata areas on this device.") +FIELD(PVS, pv, NUM, "#PMdaIgn", id, 8, pvmdas_ignored, "pv_mda_count_ignored", "Number of metadata areas ignored on this device.") FIELD(VGS, vg, STR, "Fmt", cmd, 3, vgfmt, "vg_fmt", "Type of metadata.")FIELD (VGS , vg,STR ,"VG UUID" , id,38 , uuid,"vg_uuid" ,"Unique identifier." )
@@ -111,6 +112,7 @@ FIELD(VGS, vg, NUM, "#SN", cmd, 3, snapcount, "snap_count", "Number of snapshots FIELD(VGS, vg, NUM, "Seq", seqno, 3, uint32, "vg_seqno", "Revision number of internal metadata. Incremented whenever it changes.") FIELD(VGS, vg, STR, "VG Tags", tags, 7, tags, "vg_tags", "Tags, if any.") FIELD(VGS, vg, NUM, "#VMda", cmd, 5, vgmdas, "vg_mda_count", "Number of metadata areas in use by this VG.") +FIELD(VGS, vg, NUM, "#VMdaIgn", cmd, 8, vgmdas_ignored, "vg_mda_count_ignored", "Number of metadata areas ignored in this VG.") FIELD(VGS, vg, NUM, "VMdaFree", cmd, 9, vgmdafree, "vg_mda_free", "Free metadata area space for this VG in current units.")FIELD (VGS , vg,NUM ,"VMdaSize" , cmd,9 , vgmdasize,"vg_mda_size" ,"Size of smallest metadata area for this VG in current units." )
lib/report/report.c +25 -0
@@ -856,6 +856,19 @@ static int _pvmdas_disp(struct dm_report *rh, struct dm_pool *mem,return _uint32_disp (rh, mem, field, &count, private); } +static int _pvmdas_ignored_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +uint32_t count; +const struct physical_volume *pv = + (const struct physical_volume *) data; + + count =pv_mda_count_ignored (pv); + +return _uint32_disp (rh, mem, field, &count, private); +} +static int _vgmdas_disp (struct dm_report *rh,struct dm_pool *mem,struct dm_report_field *field,const void *data,void *private)
@@ -868,6 +881,18 @@ static int _vgmdas_disp(struct dm_report *rh, struct dm_pool *mem,return _uint32_disp (rh, mem, field, &count, private); } +static int _vgmdas_ignored_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +const struct volume_group *vg = (const struct volume_group *) data; +uint32_t count; + + count =vg_mda_count_ignored (vg); + +return _uint32_disp (rh, mem, field, &count, private); +} +static int _pvmdafree_disp (struct dm_report *rh,struct dm_pool *mem,struct dm_report_field *field,const void *data,void *private)
Signature · 1 line
1.6.0.6
PATCH 13/35 Add --metadataignore to pvchange, allowing for ignoring of metadata areas.
This patch just modifies pvchange to call the underlying ignore functions for mdas. Ensure special cases do not reflect changes in metadata (PVs with 0 mdas, setting ignored when already ignored, clearing ignored when not ignored).
Signed-off-by Dave Wysochanski <[email protected]>
--- tools/args.h | 1 + tools/commands.h | 3 ++- tools/pvchange.c | 33 ++++++++++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 2 deletions(-)
tools/args.h +1 -0
@@ -25,6 +25,7 @@ arg(nolocking_ARG, '\0', "nolocking", NULL, 0) arg(pvmetadatacopies_ARG, '\0', "pvmetadatacopies", int_arg, 0) arg(metadatacopies_ARG, '\0', "metadatacopies", int_arg, 0) arg(metadatasize_ARG, '\0', "metadatasize", size_mb_arg, 0) +arg(metadataignore_ARG, '\0', "metadataignore", yes_no_arg, 0) arg(restorefile_ARG, '\0', "restorefile", string_arg, 0) arg(labelsector_ARG, '\0', "labelsector", int_arg, 0)arg (driverloaded_ARG,'\0' ,"driverloaded" , yes_no_arg,0 )
tools/commands.h +2 -1
@@ -455,6 +455,7 @@ xx(pvchange,"\t[-t|--test]\n" "\t[-u|--uuid]\n" "\t[-x|--allocatable y|n]\n" +"\t[--metadataignore y|n]\n" "\t[-v|--verbose]\n" "\t[--addtag Tag]\n" "\t[--deltag Tag]\n"
@@ -462,7 +463,7 @@ xx(pvchange, "\t[PhysicalVolumePath...]\n", all_ARG, allocatable_ARG, allocation_ARG, autobackup_ARG, deltag_ARG, - addtag_ARG, test_ARG, uuid_ARG) + addtag_ARG, metadataignore_ARG, test_ARG, uuid_ARG) xx(pvresize, "Resize physical volume(s)",
tools/pvchange.c +32 -1
@@ -34,6 +34,7 @@ static int _pvchange_single(struct cmd_context *cmd, struct volume_group *vg,int allocatable =0 ;int tagarg =0 ;int r =0 ; +int mda_ignore =0 ;if (arg_count (cmd, addtag_ARG)) tagarg = addtag_ARG;
@@ -43,6 +44,9 @@ static int _pvchange_single(struct cmd_context *cmd, struct volume_group *vg,if (arg_count (cmd, allocatable_ARG)) allocatable = !strcmp (arg_str_value (cmd, allocatable_ARG,"n" ),"y" ); +if (arg_count (cmd, metadataignore_ARG)) + mda_ignore = !strcmp (arg_str_value (cmd, metadataignore_ARG,"n" ), +"y" );else if (tagarg && !(tag =arg_str_value (cmd, tagarg,NULL ))) {log_error ("Failed to get tag" );return 0 ;
@@ -117,6 +121,32 @@ static int _pvchange_single(struct cmd_context *cmd, struct volume_group *vg, goto out; } } + }else if (arg_count (cmd, metadataignore_ARG)) { +if (mda_ignore &&pv_mda_is_ignored (pv)) { +log_error ("Physical volume \"%s\" metadata already " +"ignored" , pv_name); + goto out; + } +if (!mda_ignore && !pv_mda_is_ignored (pv)) { +log_error ("Physical volume \"%s\" metadata already " +"not ignored" , pv_name); + goto out; + } +if (!pv_mda_count (pv)) { +log_error ("Physical volume \"%s\" has no metadata " +"areas " , pv_name); + goto out; + } +if (mda_ignore) { +log_verbose ("Setting physical volume \"%s\" " +"metadata ignored" , pv_name); + }else { +log_verbose ("Setting physical volume \"%s\" " +"metadata not ignored" , pv_name); + } +if (!pv_mda_set_ignored (pv, mda_ignore)) { + goto out; + } }else {/* --uuid: Change PV ID randomly */ if (!id_create (&pv->id)) {
@@ -187,7 +217,8 @@ int pvchange(struct cmd_context *cmd, int argc, char **argv)struct str_list *sll;if (arg_count (cmd, allocatable_ARG) +arg_count (cmd, addtag_ARG) + -arg_count (cmd, deltag_ARG) +arg_count (cmd, uuid_ARG) !=1 ) { +arg_count (cmd, deltag_ARG) +arg_count (cmd, uuid_ARG) + +arg_count (cmd, metadataignore_ARG) !=1 ) {log_error ("Please give exactly one option of -x, -uuid, " "--addtag or --deltag" );return EINVALID_CMD_LINE ;
Signature · 1 line
1.6.0.6
PATCH 14/35 Update pvchange, pvs and vgs man pages for metadata ignore.
Explain --metadataignore argument to pvchange, add new fields to pvs / vgs.
Signed-off-by Dave Wysochanski <[email protected]>
--- man/pvchange.8.in | 10 +++++++++- man/pvs.8.in | 3 ++- man/vgs.8.in | 2 +- 3 files changed, 12 insertions(+), 3 deletions(-)
man/pvchange.8.in +9 -1
@@ -6,6 +6,7 @@ pvchange \- change attributes of a physical volume [\-\-addtag Tag] [\-A|\-\-autobackup y|n] [\-d|\-\-debug] [\-\-deltag Tag] +[\-\-metadataignore y|n] [\-h|\-?|\-\-help] [\-t|\-\-test] [\-v|\-\-verbose] [\-a|\-\-all] [\-x|\-\-allocatable y|n]
@@ -25,10 +26,17 @@ Generate new random UUID for specified physical volumes. .TP .I \-x, \-\-allocatable y|n Enable or disable allocation of physical extents on this physical volume. +.TP +.I \-\-metadataignore y|n +Ingore or un-ignore metadata areas on this physical volume. If metadata +areas on a physical volume are ignored, LVM will treat the physical volume +as though it has been created with \fB\-\-pvmetadatacopies\fP of 0. LVM +operations that would normally result in reading or writing of metadata +will skip over any ignored metadata areas on physical volumes. .SH Example "pvchange -x n /dev/sdk1" disallows the allocation of physical extents on this physical volume (possibly because of disk errors, or because it will be removed after freeing it. .SH SEE ALSO -.BR lvm (8), +.BR lvm (8), .BR pvcreate (8)
man/pvs.8.in +2 -1
@@ -54,7 +54,8 @@ Use \fb-o help\fP to view the full list of columns available. .IP Column names include: pv_fmt, pv_uuid, dev_size, pv_name, pv_mda_free, pv_mda_size, pe_start, pv_size, pv_free, pv_used, pv_attr, pv_pe_count, -pv_pe_alloc_count, pv_tags, pv_mda_count, pvseg_start, and pvseg_size. +pv_pe_alloc_count, pv_tags, pv_mda_count, pv_mda_count_ignored, +pvseg_start, and pvseg_size. .IP With --segments, any "pvseg_" prefixes are optional; otherwise any "pv_" prefixes are optional. Columns mentioned in \fBvgs (8)\fP can also
man/vgs.8.in +1 -1
@@ -49,7 +49,7 @@ Use \fb-o help\fP to view the full list of columns available. Column names include: vg_fmt, vg_uuid, vg_name, vg_attr, vg_size, vg_free, vg_sysid, vg_extent_size, vg_extent_count, vg_free_count, max_lv, max_pv, pv_count, lv_count, snap_count, vg_seqno, vg_tags, vg_mda_count, vg_mda_free, -and vg_mda_size. +and vg_mda_size, vg_mda_count_ignored. .IP Any "vg_" prefixes are optional. Columns mentioned in either \fBpvs (8)\fP or \fBlvs (8)\fP can also be chosen, but columns cannot be taken from both
Signature · 1 line
1.6.0.6
PATCH 15/35 Implement ignore of mda if bit set by skipping r/w of metadata.
We implement ignore of an mda at label_read time by checking for the ignore bit, and then skipping the reading of the vgname and other information in the metadata. This will have an effect similar to a PV found with no mdas. Thus, it will look like an orphan in the cache until we scan the rest of the system and find a PV with metadata, and the mda will not be on the vg->fid->metadata_areas list so no read/writes will be done to the metadata area.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/text_label.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
lib/format_text/text_label.c +6 -0
@@ -311,6 +311,12 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, }mda_set_ignored (mda,rlocn_is_ignored (mdah->raw_locns)); +if (mda_is_ignored (mda)) { +if (!dev_close (mdac->area.dev)) + stack; +continue ; + } + if ((vgname = vgname_from_mda(info->fmt, mdah, &mdac->area, &vgid, &vgstatus, &creation_host,
Signature · 1 line
1.6.0.6
PATCH 16/35 Add mdas_empty_or_ignored() helper function.
Add a helper function to consolidate checking for an empty mdas list or ignored mdas. Ignored mdas should behave almost identically to an empty mda list - the metadata areas should not be read or written to. This function will make it easier to implement metadata balancing and easier to track pvs with an empty mda list or ignored mdas.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata.c | 13 +++++++++++++ lib/metadata/metadata.h | 1 + 2 files changed, 14 insertions(+), 0 deletions(-)
lib/metadata/metadata.c +13 -0
@@ -4045,6 +4045,19 @@ int pv_mda_set_ignored(const struct physical_volume *pv, int value)return 1 ; } +int mdas_empty_or_ignored (struct dm_list *mdas) +{ +struct metadata_area *mda; + +if (!dm_list_size (mdas)) +return 1 ; + dm_list_iterate_items(mda, mdas) { +if (mda_is_ignored (mda)) +return 1 ; + } +return 0 ; +} +uint32_t vg_seqno (const struct volume_group *vg) {return vg->seqno;
lib/metadata/metadata.h +1 -0
@@ -185,6 +185,7 @@ void mda_set_ignored(struct metadata_area *mda, int value);int mda_locn_match (struct metadata_area *mda1,struct metadata_area *mda2);void fid_add_mda (struct format_instance *fid,struct metadata_area *mda);int fid_add_mdas (struct format_instance *fid,struct dm_list *mdas); +int mdas_empty_or_ignored (struct dm_list *mdas);#define seg_pvseg (seg, s) (seg)->areas[(s)].u.pv.pvseg#define seg_dev (seg, s) (seg)->areas[(s)].u.pv.pvseg->pv->dev
Signature · 1 line
1.6.0.6
PATCH 17/35 Use mdas_empty_or_ignored() in place of checks for empty mda list.
With the addition of ignored mdas, we replace all checks for an empty mda list with a new function to look for either an empty mda list or ignored mdas.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/cache/lvmcache.c | 3 ++- lib/metadata/metadata.c | 31 ++++++++++++++++++------------- 2 files changed, 20 insertions(+), 14 deletions(-)
lib/cache/lvmcache.c +2 -1
@@ -1197,7 +1197,8 @@ int lvmcache_update_vgname_and_id(struct lvmcache_info *info, }/* If PV without mdas is already in a real VG, don't make it orphan */ -if (is_orphan_vg (vgname) && info->vginfo && !dm_list_size (&info->mdas) && +if (is_orphan_vg (vgname) && info->vginfo && +mdas_empty_or_ignored (&info->mdas) && !is_orphan_vg (info->vginfo->vgname) &&memlock ())return 1 ;
lib/metadata/metadata.c +18 -13
@@ -1305,7 +1305,7 @@ static int pvcreate_check(struct cmd_context *cmd, const char *name,* this means checking every VG by scanning every PV on the * system. */ -if (pv &&is_orphan (pv) && !dm_list_size (&mdas)) { +if (pv &&is_orphan (pv) &&mdas_empty_or_ignored (&mdas)) {if (!scan_vgs_for_pvs (cmd)) return_0; pv =pv_read (cmd, name,NULL ,NULL ,0 ,0 );
@@ -1798,7 +1798,7 @@ static struct physical_volume *_find_pv_by_name(struct cmd_context *cmd,return NULL ; } -if (is_orphan_vg (pv->vg_name) && !dm_list_size (&mdas)) { +if (is_orphan_vg (pv->vg_name) &&mdas_empty_or_ignored (&mdas)) {/* If a PV has no MDAs - need to search all VGs for it */ if (!scan_vgs_for_pvs (cmd)) return_NULL;
@@ -2722,8 +2722,8 @@ static struct volume_group *_vg_read(struct cmd_context *cmd,/* Ensure every PV in the VG was in the cache */ if (correct_vg) {/* -* If the VG has PVs without mdas, they may still be -* orphans in the cache: update the cache state here. +* If the VG has PVs without mdas, or ignored mdas, they may +* still be orphans in the cache: update the cache state here. */ if (!inconsistent &&dm_list_size (&correct_vg->pvs) >dm_list_size (pvids)) {
@@ -2738,11 +2738,12 @@ static struct volume_group *_vg_read(struct cmd_context *cmd,/* * PV not marked as belonging to this VG in cache. -* Check it's an orphan without metadata area. +* Check it's an orphan without metadata area +* not ignored. */ if (!(info =info_from_pvid (pvl->pv->dev->pvid,1 )) || !info->vginfo || !is_orphan_vg (info->vginfo->vgname) || -dm_list_size (&info->mdas)) { + !mdas_empty_or_ignored (&info->mdas)) { inconsistent_pvs =1 ;break ; }
@@ -3128,20 +3129,24 @@ const char *find_vgname_from_pvid(struct cmd_context *cmd, return_NULL; }/* -* If an orphan PV has no MDAs it may appear to be an -* orphan until the metadata is read off another PV in -* the same VG. Detecting this means checking every VG -* by scanning every PV on the system. +* If an orphan PV has no MDAs, or it has MDAs but the +* MDA is ignored, it may appear to be an orphan until +* the metadata is read off another PV in the same VG. +* Detecting this means checking every VG by scanning +* every PV on the system. */ -if (!dm_list_size (&info->mdas)) { +if (mdas_empty_or_ignored (&info->mdas)) {if (!scan_vgs_for_pvs (cmd)) {log_error ("Rescan for PVs without " "metadata areas failed." );return NULL ; } +/* +* Ask lvmcache again - we may have a non-orphan +* name now +*/ + vgname =lvmcache_vgname_from_pvid (cmd, pvid); } -/* Ask lvmcache again - we may have a non-orphan name now */ - vgname =lvmcache_vgname_from_pvid (cmd, pvid); }return vgname; }
Signature · 1 line
1.6.0.6
PATCH 18/35 Update _vg_read and _text_create_text_instance to use fid_add_mda[s].
When we are constructing the vg, we may need to adjust the list of metadata_areas if there are ignored mdas. At label read time, we do not read the metadata of ignored mdas, and as a result, they do not get placed on vg->fid->metadata_areas inside _text_create_text_instance since lvmcache does not have these areas attached to vginfo->infos. However, when we're checking the pvids inside _vg_read, after having read another metadata area from another PV, we do have the opportunity to update the metadata_area and metadata_areas_ignored lists based on the read metadata_area. We need accurate mda lists for the reporting functions that count the ignored mdas, as well as general correctness of mda balancing.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/format-text.c | 20 +++++++------------- lib/metadata/metadata.c | 6 +++++- 2 files changed, 12 insertions(+), 14 deletions(-)
lib/format_text/format-text.c +7 -13
@@ -1886,11 +1886,11 @@ static struct format_instance *_text_create_text_instance(const struct format_ty {struct format_instance *fid;struct text_fid_context *fidtc; -struct metadata_area *mda, *mda_new; +struct metadata_area *mda;struct mda_context *mdac;struct dir_list *dl;struct raw_list *rl; -struct dm_list *dir_list, *raw_list, *mdas; +struct dm_list *dir_list, *raw_list;char path[PATH_MAX ];struct lvmcache_vginfo *vginfo;struct lvmcache_info *info;
@@ -1918,7 +1918,7 @@ static struct format_instance *_text_create_text_instance(const struct format_ty return_NULL; mda->ops = &_metadata_text_file_backup_ops; mda->metadata_locn = context; -dm_list_add (&fid->metadata_areas, &mda->list); +fid_add_mda (fid, mda); }else { dir_list = &((struct mda_lists *) fmt->private)->dirs;
@@ -1935,7 +1935,7 @@ static struct format_instance *_text_create_text_instance(const struct format_ty return_NULL; mda->ops = &_metadata_text_file_ops; mda->metadata_locn = context; -dm_list_add (&fid->metadata_areas, &mda->list); +fid_add_mda (fid, mda); } raw_list = &((struct mda_lists *) fmt->private)->raws;
@@ -1955,7 +1955,7 @@ static struct format_instance *_text_create_text_instance(const struct format_tymemcpy (&mdac->area, &rl->dev_area,sizeof (mdac->area)); mda->ops = &_metadata_text_raw_ops;/* FIXME MISTAKE? mda->metadata_locn = context; */ -dm_list_add (&fid->metadata_areas, &mda->list); +fid_add_mda (fid, mda); }/* Scan PVs in VG for any further MDAs */
@@ -1963,14 +1963,8 @@ static struct format_instance *_text_create_text_instance(const struct format_tyif (!(vginfo =vginfo_from_vgname (vgname, vgid))) goto_out; dm_list_iterate_items(info, &vginfo->infos) { - mdas = &info->mdas; - dm_list_iterate_items(mda, mdas) { -/* FIXME Check it holds this VG */ - mda_new =mda_copy (fmt->cmd->mem, mda); -if (!mda_new) - return_NULL; -dm_list_add (&fid->metadata_areas, &mda_new->list); - } +if (!fid_add_mdas (fid, &info->mdas)) + return_NULL; }/* FIXME Check raw metadata area count - rescan if required */ }
lib/metadata/metadata.c +5 -1
@@ -2723,7 +2723,8 @@ static struct volume_group *_vg_read(struct cmd_context *cmd,if (correct_vg) {/* * If the VG has PVs without mdas, or ignored mdas, they may -* still be orphans in the cache: update the cache state here. +* still be orphans in the cache: update the cache state here, +* and update the metadata lists in the vg. */ if (!inconsistent &&dm_list_size (&correct_vg->pvs) >dm_list_size (pvids)) {
@@ -2747,6 +2748,9 @@ static struct volume_group *_vg_read(struct cmd_context *cmd, inconsistent_pvs =1 ;break ; } +if (dm_list_size (&info->mdas) && + !fid_add_mdas (fid, &info->mdas)) + return_NULL; }/* If the check passed, let's update VG and recalculate pvids */
Signature · 1 line
1.6.0.6
PATCH 19/35 Refactor vg_commit() to add _vg_commit_mdas().
Factor out calling mda->ops->vg_commit() for each mda. No functional change.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata.c | 26 +++++++++++++++++--------- 1 files changed, 17 insertions(+), 9 deletions(-)
lib/metadata/metadata.c +17 -9
@@ -2424,18 +2424,11 @@ int vg_write(struct volume_group *vg)return 1 ; } -/* Commit pending changes */ -int vg_commit (struct volume_group *vg) +static int _vg_commit_mdas (struct volume_group *vg) {struct metadata_area *mda; -int cache_updated =0 ;int failed =0 ; - -if (!vgname_is_locked (vg->name)) { -log_error (INTERNAL_ERROR "Attempt to write new VG metadata " -"without locking %s" , vg->name); -return cache_updated; - } +int cache_updated =0 ;/* Commit to each copy of the metadata area */ dm_list_iterate_items(mda, &vg->fid->metadata_areas) {
@@ -2451,6 +2444,21 @@ int vg_commit(struct volume_group *vg) cache_updated =1 ; } } +return cache_updated; +} + +/* Commit pending changes */ +int vg_commit (struct volume_group *vg) +{ +int cache_updated =0 ; + +if (!vgname_is_locked (vg->name)) { +log_error (INTERNAL_ERROR "Attempt to write new VG metadata " +"without locking %s" , vg->name); +return cache_updated; + } + + cache_updated =_vg_commit_mdas (vg);if (cache_updated) {/* Instruct remote nodes to upgrade cached metadata. */
Signature · 1 line
1.6.0.6
PATCH 20/35 Before committing each mda, arrange mdas so ignored mdas get committed first.
Arrange mdas so mdas that are to be ignored come first. This is an optimization that ensures consistency on disk for the longest period of time. This was noted by agk in review of the v4 patchset of pvchange-based mda balance.
Note the following example for an explanation of the background:
Assume the initial state on disk is as follows:
PV0 (v1, non-ignored)
PV1 (v1, non-ignored)
PV2 (v1, non-ignored)
PV3 (v1, non-ignored)
If we did not sort the list, we would have a commit sequence something like this:
PV0 (v2, non-ignored)
PV1 (v2, ignored)
PV2 (v2, ignored)
PV3 (v2, non-ignored)
After the commit of PV0's mdas, we'd have an on-disk state like this:
PV0 (v2, non-ignored)
PV1 (v1, non-ignored)
PV2 (v1, non-ignored)
PV3 (v1, non-ignored)
This is an inconsistent state of the disk. If the machine fails, the next time it was brought back up, the auto-correct mechanism in vg_read would update the metadata on PV1-PV3. However, if possible we try to avoid inconsistent on-disk states. Clearly, because we did not sort, we have a greater chance of on-disk inconsistency - from the time the commit of PV0 is complete until the time PV3 is complete.
We could improve the amount of time the on-disk state is consistent by simply sorting the commit order as follows:
PV1 (v2, ignored)
PV2 (v2, ignored)
PV0 (v2, non-ignored)
PV3 (v2, non-ignored)
Thus, after the first PV is committed (in this case PV1), on-disk we would have:
PV0 (v1, non-ignored)
PV1 (v2, ignored)
PV2 (v1, non-ignored)
PV3 (v1, non-ignored)
This is clearly a consistent state. PV1 will be read but the mda will be ignored. All other PVs contain v1 metadata, and no auto-correct will be required. In fact, if we commit all PVs with ignored mdas first, we'll only have an inconsistent state when we start writing non-ignored PVs, and thus the chances we'll get an inconsistent state on disk is much less with the sorted method.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata.c | 13 ++++++++++++- 1 files changed, 12 insertions(+), 1 deletions(-)
lib/metadata/metadata.c +12 -1
@@ -2426,10 +2426,21 @@ int vg_write(struct volume_group *vg)static int _vg_commit_mdas (struct volume_group *vg) { -struct metadata_area *mda; +struct metadata_area *mda, *tmda; +struct dm_list ignored;int failed =0 ;int cache_updated =0 ; +/* Rearrange the metadata_areas list so ignored mdas come first */ +dm_list_init (&ignored); + dm_list_iterate_items_safe(mda, tmda, &vg->fid->metadata_areas) { +if (mda_is_ignored (mda)) +dm_list_move (&ignored, &mda->list); + } + dm_list_iterate_items_safe(mda, tmda, &ignored) { +dm_list_move (&vg->fid->metadata_areas, &mda->list); + } +/* Commit to each copy of the metadata area */ dm_list_iterate_items(mda, &vg->fid->metadata_areas) { failed =0 ;
Signature · 1 line
1.6.0.6
PATCH 21/35 Add tests for phase 1 of metadata balance - manage per-PV ignore bit.
Signed-off-by Dave Wysochanski <[email protected]>
--- test/t-metadata-balance.sh | 41 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 41 insertions(+), 0 deletions(-) create mode 100755 test/t-metadata-balance.sh
test/t-metadata-balance.sh new file +41 -0
@@ -0,0 +1,41 @@ +# Copyright (C) 2008 Red Hat, Inc. All rights reserved. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions +# of the GNU General Public License v.2. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +. ./test-utils.sh + +aux prepare_devs 5 + +# Make sure we can ignore / un-ignore mdas on a per-PV basis +for pv_in_vg in 1 0; do +for mdacp in 1 2; do + pvcreate --metadatacopies $mdacp $dev1 $dev2 + pvcreate --metadatacopies 0 $dev3 + if [ $pv_in_vg = 1 ]; then + vgcreate -c n "$vg" $dev1 $dev2 $dev3 + fi + pvchange --metadataignore y $dev1 + check_pv_field_ $dev1 pv_mda_count $mdacp + check_pv_field_ $dev1 pv_mda_count_ignored $mdacp + check_pv_field_ $dev2 pv_mda_count $mdacp + check_pv_field_ $dev2 pv_mda_count_ignored 0 + if [ $pv_in_vg = 1 ]; then + check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) + check_vg_field_ $vg vg_mda_count_ignored $mdacp + fi + pvchange --metadataignore n $dev1 + check_pv_field_ $dev1 pv_mda_count $mdacp + check_pv_field_ $dev1 pv_mda_count_ignored 0 + if [ $pv_in_vg = 1 ]; then + check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) + check_vg_field_ $vg vg_mda_count_ignored 0 + vgremove -f $vg + fi +done +done
Signature · 1 line
1.6.0.6
PATCH 22/35 Define vgmetadatacopies in vgchange man page.
This patch adds a vgmetadatacopies parameter for metadata balancing. This parameter provides a simple way for users to create a policy for placing metadata on PVs automatically by LVM. The behavior is implemented inside LVM by managing the 'ignore' mda bits. We chose the name 'vgmetadatacopies' as this is a natural extension to the existing parameter 'pvmetadatacopies' / 'metadatacopies' in pvcreate.
This is a first step at VG parameter based metadata balancing. Most users will probably want to state that they want a certain number of PVs to contai metadata, and they may be less concerned about a specific number of metadata copies in the volume group. However, for default values (pvmetadatacopies is 1 by default), the number of metadatacopies in the volume group, and the number of PVs with metadata are the same. In the future we could add vgmetadatacopiespvs to define more specifically the number of pvs in the VG that contain metadata, but for now we start with this parameter.
Another possible future extension would be to define a specific pv tag to mark the set of PVs that should be used for metadata balancing. This tag based approach could be used in conjunction with 'vgmetadatacopies'.
Signed-off-by Dave Wysochanski <[email protected]>
--- man/vgchange.8.in | 27 ++++++++++++++++++++------- 1 files changed, 20 insertions(+), 7 deletions(-)
man/vgchange.8.in +20 -7
@@ -25,6 +25,8 @@ vgchange \- change attributes of a volume group .IR MaxLogicalVolumes ] .RB [ -p | \-\-maxphysicalvolumes .IR MaxPhysicalVolumes ] +.RB [ \-\-[vg]metadatacopies ] +.IR copies ] .RB [ \-P | \-\-partial] .RB [ \-s | \-\-physicalextentsize .IR PhysicalExtentSize [ \fBbBsSkKmMgGtTpPeE\fR ]]
@@ -127,13 +129,24 @@ volume group. Changes the maximum number of physical volumes that can belong to this volume group. For volume groups with metadata in lvm1 format, the limit is 255. -If the metadata uses lvm2 format, the value 0 -removes this restriction: there is then no limit. -If you have a large number of physical volumes in -a volume group with metadata in lvm2 format, -for tool performance reasons, you should consider -some use of \fB--pvmetadatacopies 0\fP -as described in \fBpvcreate(8)\fP. +If the metadata uses lvm2 format, the value 0 removes this restriction: +there is then no limit. If you have a large number of physical volumes in +a volume group with metadata in lvm2 format, for tool performance reasons, +you should consider some use of \fB--pvmetadatacopies 0\fP as described in +\fBpvcreate(8)\fP, and/or use \fB--vgmetadatacopies\fP. +.TP +.BR \-\-vgmetadatacopies " " \fI#copies\fR +Sets the targetted number of metadata copies in the volume group. +If set to a non-zero value, LVM will attempt to use \fB#copies\fP to +determine the set of physical volumes on which to read and write metadata. +LVM manages the set automatically by setting or clearing the 'metadataignore' +flag on the physical volumes (see \fBpvchange\fP), and in this way balances +metadata across PVs in the volume group. Only physical volumes created +with a non-zero value of \fBmetadatacopies\fP are used in the set of physical +volumes. This option is useful for volume groups containing large numbers +of physical volumes containing metadata as it may be used to minimize metadata +read and write overhead. +Setting \fB--vgmetadatacopies\fP to 0 disables LVM metadata balancing. .TP .BR \-s ", " \-\-physicalextentsize " " \fIPhysicalExtentSize\fR[\fBbBsSkKmMgGtTpPeE\fR] Changes the physical extent size on physical volumes of this volume group.
Signature · 1 line
1.6.0.6
PATCH 23/35 Add mda_copies to VG structures and initialization.
Add a field to struct volume_group to later implement metadata balancing:
- mda_copies: target # of non-ignored mdas in the VG; default 0 (do not control pv 'ignore mdas' bit.
This patch just adds the parameter to the structures with the default values but does not modify any commands. Should be no functional change.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/config/defaults.h | 1 + lib/metadata/metadata-exported.h | 2 ++ lib/metadata/metadata.c | 1 + tools/toollib.c | 2 ++ 4 files changed, 6 insertions(+), 0 deletions(-)
lib/config/defaults.h +1 -0
@@ -61,6 +61,7 @@#define DEFAULT_STRIPESIZE 64/* KB */ #define DEFAULT_PVMETADATASIZE 255#define DEFAULT_PVMETADATACOPIES 1 +#define DEFAULT_VGMETADATACOPIES 0#define DEFAULT_LABELSECTOR UINT64_C(1)#define DEFAULT_READ_AHEAD "auto"#define DEFAULT_UDEV_RULES 1
lib/metadata/metadata-exported.h +2 -0
@@ -281,6 +281,7 @@ struct volume_group {* 0 for success else appropriate FAILURE_* bits set. */ uint32_t read_status; +uint32_t mda_copies;/* target number of mdas for this VG */ };/* There will be one area for each stripe */
@@ -891,6 +892,7 @@ struct vgcreate_params {size_t max_lv;alloc_policy_t alloc;int clustered;/* FIXME: put this into a 'status' variable instead? */ +uint32_t metadata_copies; }; int vgcreate_params_validate(struct cmd_context *cmd,
lib/metadata/metadata.c +1 -0
@@ -910,6 +910,7 @@ struct volume_group *vg_create(struct cmd_context *cmd, const char *vg_name) vg->max_pv =DEFAULT_MAX_PV ; vg->alloc =DEFAULT_ALLOC_POLICY ; + vg->mda_copies =DEFAULT_VGMETADATACOPIES ; vg->pv_count =0 ;dm_list_init (&vg->pvs);
tools/toollib.c +2 -0
@@ -1187,6 +1187,7 @@ void vgcreate_params_set_defaults(struct vgcreate_params *vp_def, vp_def->max_lv = vg->max_lv; vp_def->alloc = vg->alloc; vp_def->clustered =vg_is_clustered (vg); + vp_def->metadata_copies = vg->mda_copies; }else { vp_def->vg_name =NULL ; vp_def->extent_size =DEFAULT_EXTENT_SIZE *2 ;
@@ -1194,6 +1195,7 @@ void vgcreate_params_set_defaults(struct vgcreate_params *vp_def, vp_def->max_lv =DEFAULT_MAX_LV ; vp_def->alloc =DEFAULT_ALLOC_POLICY ; vp_def->clustered =DEFAULT_CLUSTERED ; + vp_def->metadata_copies =DEFAULT_VGMETADATACOPIES ; } }
Signature · 1 line
1.6.0.6
PATCH 24/35 Add vg get/set methods for VG metadata copies.
This patch adds the get and partially implemented set function. The 'set' function should probably ignore or un-ignore metadata areas based on new values.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata-exported.h | 2 ++ lib/metadata/metadata.c | 16 ++++++++++++++++ 2 files changed, 18 insertions(+), 0 deletions(-)
lib/metadata/metadata-exported.h +2 -0
@@ -877,6 +877,8 @@ uint64_t vg_max_pv(const struct volume_group *vg);uint64_t vg_max_lv (const struct volume_group *vg);uint32_t vg_mda_count (const struct volume_group *vg);uint32_t vg_mda_count_ignored (const struct volume_group *vg); +uint32_t vg_mda_copies (const struct volume_group *vg); +int vg_set_mda_copies (struct volume_group *vg,uint32_t value);int vg_check_write_mode (struct volume_group *vg);#define vg_is_clustered (vg) (vg_status((vg)) & CLUSTERED)#define vg_is_exported (vg) (vg_status((vg)) & EXPORTED_VG)
lib/metadata/metadata.c +16 -0
@@ -989,6 +989,22 @@ static int _recalc_extents(uint32_t *extents, const char *desc1,return 1 ; } +uint32_t vg_mda_copies (const struct volume_group *vg) +{ +return vg->mda_copies; +} + +int vg_set_mda_copies (struct volume_group *vg,uint32_t value) +{ +/* FIXME: add checks, etc, and set the value */ +/* +* FIXME: Before we set a larger value, we may need to +* enable some mdas on PVS +*/ + vg->mda_copies = value; +return 1 ; +} +int vg_set_extent_size (struct volume_group *vg,uint32_t new_size) {uint32_t old_size = vg->extent_size;
Signature · 1 line
1.6.0.6
PATCH 25/35 Make vg->mda_copies persistent in on disk vg metadata.
This patch adds the ability to read/write the vg->mda_copies values from/to the vg metadata.
If we read the VG metadata and this field does not exist, we set mda_copies to the default value of 0. Later in the code, we use this special '0' value to indicate a disable of metadata balancing. This should preserve existing LVM behavior and ensure metadata balancing can be turned off should the need arise.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/format_text/export.c | 1 + lib/format_text/import_vsn1.c | 5 +++++ 2 files changed, 6 insertions(+), 0 deletions(-)
lib/format_text/export.c +1 -0
@@ -397,6 +397,7 @@ static int _print_vg(struct formatter *f, struct volume_group *vg)outf (f,"allocation_policy = \"%s\"" ,get_alloc_string (vg->alloc)); } +outf (f,"metadata_copies = %u" , vg->mda_copies);return 1 ; }
lib/format_text/import_vsn1.c +5 -0
@@ -23,6 +23,7 @@#include "pv_alloc.h" #include "segtype.h" #include "text_import.h" +#include "defaults.h" typedef int (*section_fn) (struct format_instance * fid, struct dm_pool * mem, struct volume_group * vg, struct config_node * pvn,
@@ -745,6 +746,10 @@ static struct volume_group *_read_vg(struct format_instance *fid, return_0; } +if (!_read_uint32 (vgn,"metadata_copies" , &vg->mda_copies)) { + vg->mda_copies =DEFAULT_VGMETADATACOPIES ; + } +/* * The pv hash memorises the pv section names -> pv * structures.
Signature · 1 line
1.6.0.6
PATCH 26/35 Add vg_mda_copies display field to 'vgs' command.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/report/columns.h | 1 + lib/report/report.c | 12 ++++++++++++ 2 files changed, 13 insertions(+), 0 deletions(-)
lib/report/columns.h +1 -0
@@ -115,6 +115,7 @@ FIELD(VGS, vg, NUM, "#VMda", cmd, 5, vgmdas, "vg_mda_count", "Number of metadata FIELD(VGS, vg, NUM, "#VMdaIgn", cmd, 8, vgmdas_ignored, "vg_mda_count_ignored", "Number of metadata areas ignored in this VG.") FIELD(VGS, vg, NUM, "VMdaFree", cmd, 9, vgmdafree, "vg_mda_free", "Free metadata area space for this VG in current units.") FIELD(VGS, vg, NUM, "VMdaSize", cmd, 9, vgmdasize, "vg_mda_size", "Size of smallest metadata area for this VG in current units.") +FIELD(VGS, vg, NUM, "#VMdaCps", cmd, 8, vgmda_copies, "vg_mda_copies", "Target number of non-ignored metadata areas in the VG.") FIELD(SEGS, seg, STR, "Type", list, 4, segtype, "segtype", "Type of LV segment.")FIELD (SEGS , seg,NUM ,"#Str" , area_count,4 , uint32,"stripes" ,"Number of stripes or mirror legs." )
lib/report/report.c +12 -0
@@ -893,6 +893,18 @@ static int _vgmdas_ignored_disp(struct dm_report *rh, struct dm_pool *mem,return _uint32_disp (rh, mem, field, &count, private); } +static int _vgmda_copies_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +const struct volume_group *vg = (const struct volume_group *) data; +uint32_t count; + + count =vg_mda_copies (vg); + +return _uint32_disp (rh, mem, field, &count, private); +} +static int _pvmdafree_disp (struct dm_report *rh,struct dm_pool *mem,struct dm_report_field *field,const void *data,void *private)
Signature · 1 line
1.6.0.6
PATCH 27/35 Update vgchange tool to accept --vgmetadatacopies.
Update logic in vgchange to handle --vgmetadatacopies, allow --metadatacopies as a synonym to --vgmetadatacopies, and add these parameters to args.h and commands.h
Signed-off-by Dave Wysochanski <[email protected]>
--- tools/args.h | 1 + tools/commands.h | 4 +++- tools/vgchange.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 51 insertions(+), 3 deletions(-)
tools/args.h +1 -0
@@ -23,6 +23,7 @@ arg(physicalvolumesize_ARG, '\0', "setphysicalvolumesize", size_mb_arg, 0) arg(ignorelockingfailure_ARG, '\0', "ignorelockingfailure", NULL, 0) arg(nolocking_ARG, '\0', "nolocking", NULL, 0) arg(pvmetadatacopies_ARG, '\0', "pvmetadatacopies", int_arg, 0) +arg(vgmetadatacopies_ARG, '\0', "vgmetadatacopies", int_arg, 0) arg(metadatacopies_ARG, '\0', "metadatacopies", int_arg, 0) arg(metadatasize_ARG, '\0', "metadatasize", size_mb_arg, 0)arg (metadataignore_ARG,'\0' ,"metadataignore" , yes_no_arg,0 )
tools/commands.h +3 -1
@@ -710,6 +710,7 @@ xx(vgchange,"\t[--ignorelockingfailure]\n" "\t[--ignoremonitoring]\n" "\t[--monitor {y|n}]\n" +"\t[--[vg]metadatacopies #copies] " "\n" "\t[--poll {y|n}]\n" "\t[--noudevsync]\n" "\t[--refresh]\n"
@@ -731,7 +732,8 @@ xx(vgchange, addtag_ARG, alloc_ARG, allocation_ARG, autobackup_ARG, available_ARG, clustered_ARG, deltag_ARG, ignorelockingfailure_ARG, ignoremonitoring_ARG, logicalvolume_ARG, maxphysicalvolumes_ARG, monitor_ARG, noudevsync_ARG, - partial_ARG, physicalextentsize_ARG, poll_ARG, refresh_ARG, resizeable_ARG, + metadatacopies_ARG, vgmetadatacopies_ARG, partial_ARG, + physicalextentsize_ARG, poll_ARG, refresh_ARG, resizeable_ARG, resizable_ARG, sysinit_ARG, test_ARG, uuid_ARG) xx(vgck,
tools/vgchange.c +47 -2
69 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 28/35 Implement _vg_metadata_balance and call from vg_write() path.
Implement metadata balancing by comparing the value of the newly added vg_mda_copies field (--vgmetadatacopies parameter) with the current count of of non-ignored mdas and ignoring or unignoring mdas as necessary to get to the target count. Also, as a safety check before returning, ensure we have at least one mda enabled.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata.c | 88 +++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 88 insertions(+), 0 deletions(-)
lib/metadata/metadata.c +88 -0
@@ -58,6 +58,8 @@ static struct pv_list *_find_pv_in_vg_by_uuid(const struct volume_group *vg,static uint32_t _vg_bad_status_bits (const struct volume_group *vg,uint64_t status); +static int _vg_metadata_balance (struct volume_group *vg); +const char _really_init[] ="Really INITIALIZE physical volume \"%s\" of volume group \"%s\" [y/n]? " ;
@@ -989,6 +991,91 @@ static int _recalc_extents(uint32_t *extents, const char *desc1,return 1 ; } +static int _vg_ignore_mdas (struct volume_group *vg,uint32_t count) +{ +struct metadata_area *mda; + +if (!count) +return 1 ; +/* FIXME: flip bits on random mdas */ + dm_list_iterate_items(mda, &vg->fid->metadata_areas) { +if (!mda_is_ignored (mda)) { +mda_set_ignored (mda,1 ); + count--; + } +if (!count) +return 1 ; + } +log_error ("Unable to find %" PRIu32" metadata areas to ignore " +"on volume group %s" , count, vg->name); +return 0 ; +} + +static int _vg_unignore_mdas (struct volume_group *vg,uint32_t count) +{ +struct metadata_area *mda, *tmda; + +if (!count) +return 1 ; +/* FIXME: flip bits on random mdas */ + dm_list_iterate_items_safe(mda, tmda, &vg->fid->metadata_areas_ignored) { +if (mda_is_ignored (mda)) { +mda_set_ignored (mda,0 ); +dm_list_move (&vg->fid->metadata_areas, &mda->list); + count--; + } +if (!count) +return 1 ; + } + dm_list_iterate_items(mda, &vg->fid->metadata_areas) { +if (mda_is_ignored (mda)) { +mda_set_ignored (mda,0 ); + count--; + } +if (!count) +return 1 ; + } +log_error ("Unable to find %" PRIu32" metadata areas to un-ignore " +"on volume group %s" , count, vg->name); +return 0 ; +} + +static int _vg_metadata_balance (struct volume_group *vg) +{ +uint32_t mda_copies, count; +int ret =1 ; + + mda_copies =vg_mda_count (vg) -vg_mda_count_ignored (vg); +if (!vg->mda_copies) + goto skip_balance; + +if (mda_copies > vg->mda_copies) { + ret =_vg_ignore_mdas (vg, mda_copies - vg->mda_copies); + }else if (mda_copies < vg->mda_copies) { +/* not an error to have vg_mda_count larger than total mdas */ +if (vg->mda_copies >=vg_mda_count (vg)) + count =vg_mda_count_ignored (vg); +else + count = vg->mda_copies - mda_copies; + ret =_vg_unignore_mdas (vg, count); + } +if (!ret) +return ret; + +skip_balance: +/* +* Ensure at least one enabled mda. +* FIXME: check size of fid->metadata_areas; reason is because of how +* pv_setup works in the case of a pv with 2 mdas, one ignored and +* another not ignored; function needs refactoring to be correct. +*/ +if (!dm_list_size (&vg->fid->metadata_areas) || +vg_mda_count (vg) ==vg_mda_count_ignored (vg)) { + ret =_vg_unignore_mdas (vg,1 ); + } +return ret; +} +uint32_t vg_mda_copies (const struct volume_group *vg) {return vg->mda_copies;
@@ -2378,6 +2465,7 @@ int vg_write(struct volume_group *vg)return 0 ; } +_vg_metadata_balance (vg);if (dm_list_empty (&vg->fid->metadata_areas)) {log_error ("Aborting vg_write: No metadata areas to write to!" );
Signature · 1 line
1.6.0.6
PATCH 29/35 Set vg_mda_copies when pvchange --metadataignore is given.
When a user explicitly sets a new mda ignore value for a PV, we should update vg_mda_copies accordingly. When the VG is written out, the user would not want the new ignore state to get lost as a result of the metadata balancing policy.
Signed-off-by Dave Wysochanski <[email protected]>
--- tools/pvchange.c | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-)
tools/pvchange.c +12 -0
@@ -147,6 +147,18 @@ static int _pvchange_single(struct cmd_context *cmd, struct volume_group *vg,if (!pv_mda_set_ignored (pv, mda_ignore)) { goto out; } +/* +* Update vg_mda_copies based on the mdas in this PV. +* This is most likely what the usre would expect - if they +* specify a specific PV to be ignored/un-ignored, they will +* most likely not want LVM to turn around and change the +* ignore / un-ignore value when it writes the VG to disk. +* This does not guarantee this PV's ignore bits will be +* preserved in future operations. +*/ +if (!is_orphan (pv) &&vg_mda_copies (vg)) { +vg_set_mda_copies (vg,vg_mda_copies (vg)); + } }else {/* --uuid: Change PV ID randomly */ if (!id_create (&pv->id)) {
Signature · 1 line
1.6.0.6
PATCH 30/35 Add --vgmetadatacopies to vgcreate man page, command, and lvm.conf.
Allow parsing of --vgmetadatacopies for vgcreate. Accept --metadatacopies as a synonym for --vgmetadatacopies. Other patches will add balancing algorithm before the metadata is written in vg_write().
Signed-off-by Dave Wysochanski <[email protected]>
--- man/lvm.conf.5.in | 23 +++++++++++++++++++++++ man/vgcreate.8.in | 15 +++++++++++++++ tools/commands.h | 5 +++-- tools/toollib.c | 12 ++++++++++++ tools/vgcreate.c | 8 ++------ 5 files changed, 55 insertions(+), 8 deletions(-)
man/lvm.conf.5.in +23 -0
@@ -406,6 +406,29 @@ The metadata areas are treated as circular buffers, so unused space becomes filled with an archive of the most recent previous versions of the metadata. .IP +\fBvgmetadatacopies\fP \(em When creating a volume group using the +LVM2 metadata format, this is the default number of copies of metadata +desired across all the physical volumes in the volume group. If set to +a non-zero value, LVM will attempt to balance metadata areas across +the physical volumes in the volume group to achieve the number +of metadata copies specified. LVM implements the balancing through +automatically setting and clearing metadata ignore +(See \fBpvchange --metadataignore\fP) on physical volumes created with +a non-zero value of \fBpvmetadatacopies\fP. Balancing may be triggered +by an LVM command that adds or removes physical volumes (\fBvgextend\fP +or \fBvgreduce\fP, \fBvgsplit\fP, \fBvgmerge\fP), when physical +volumes go missing or reappear, or when a new value is explicity set +(See \fBvgchange --vgmetadatacopies\fP). +Setting \fBvgmetadatacopies\fP to 0 disables metadata balancing. It +is not an error to set a value larger than the sum of all metadata areas +on all physical volumes. The value can be overridden on the command line +with \fB--vgmetadatacopies\fP. This option may be useful for volume groups +containing large numbers of physical volumes. It offers a simple way +to specify a reasonable number of metadata copies (for example 5) without +having to know ahead of time which physical volumes should have metadata. +Having a smaller number of metadata areas offers better performance +than the default of reading and writing metadata on all physical volumes. +.IP \fBdirs\fP \(em List of directories holding live copies of LVM2 metadata as text files. These directories must not be on logical volumes. It is possible to use LVM2 with a couple of directories
man/vgcreate.8.in +15 -0
@@ -16,6 +16,8 @@ vgcreate \- create a volume group .RB [ -M | \-\-metadatatype type] .RB [ -p | \-\-maxphysicalvolumes .IR MaxPhysicalVolumes ] +.RB [ \-\-[vg]metadatacopies ] +.IR copies ] .RB [ \-s | \-\-physicalextentsize .IR PhysicalExtentSize [ \fBbBsSkKmMgGtTpPeE\fR ]] .RB [ \-t | \-\-test ]
@@ -74,6 +76,19 @@ for tool performance reasons, you should consider some use of \fB--pvmetadatacopies 0\fP as described in \fBpvcreate(8)\fP. .TP +.BR \-\-vgmetadatacopies " " \fI#copies\fR +Sets the targetted number of metadata copies in the volume group. +If set to a non-zero value, LVM will attempt to use \fB#copies\fP to +determine the set of physical volumes on which to read and write metadata. +LVM manages the set automatically by setting or clearing the 'metadataignore' +flag on the physical volumes (see \fBpvchange\fP), and in this way balances +metadata across PVs in the volume group. Only physical volumes created +with a non-zero value of \fBmetadatacopies\fP are used in the set of physical +volumes. This option is useful for volume groups containing large numbers +of physical volumes containing metadata as it may be used to minimize metadata +read and write overhead. +The default value is 0, which disables LVM metadata balancing. +.TP .BR \-s ", " \-\-physicalextentsize " " \fIPhysicalExtentSize\fR[\fBbBsSkKmMgGtTpPeE\fR] Sets the physical extent size on physical volumes of this volume group. A size suffix (k for kilobytes up to t for terabytes) is optional, megabytes
tools/commands.h +3 -2
@@ -776,6 +776,7 @@ xx(vgcreate,"\t[-h|--help]" "\n" "\t[-l|--maxlogicalvolumes MaxLogicalVolumes]" "\n" "\t[-M|--metadatatype 1|2] " "\n" +"\t[--[vg]metadatacopies #copies] " "\n" "\t[-p|--maxphysicalvolumes MaxPhysicalVolumes] " "\n" "\t[-s|--physicalextentsize PhysicalExtentSize[bBsSkKmMgGtTpPeE]] " "\n" "\t[-t|--test] " "\n"
@@ -787,8 +788,8 @@ xx(vgcreate, addtag_ARG, alloc_ARG, autobackup_ARG, clustered_ARG, maxlogicalvolumes_ARG, maxphysicalvolumes_ARG, metadatatype_ARG, physicalextentsize_ARG, test_ARG, force_ARG, yes_ARG, zero_ARG, labelsector_ARG, metadatasize_ARG, - pvmetadatacopies_ARG, metadatacopies_ARG, dataalignment_ARG, - dataalignmentoffset_ARG) + pvmetadatacopies_ARG, metadatacopies_ARG, vgmetadatacopies_ARG, + dataalignment_ARG, dataalignmentoffset_ARG) xx(vgdisplay, "Display volume group information",
tools/toollib.c +12 -0
@@ -1243,6 +1243,18 @@ int vgcreate_params_set_from_args(struct cmd_context *cmd,return 1 ; } +if (arg_count (cmd, metadatacopies_ARG)) { + vp_new->metadata_copies =arg_int_value (cmd, metadatacopies_ARG, +DEFAULT_VGMETADATACOPIES ); + }else if (arg_count (cmd, vgmetadatacopies_ARG)) { + vp_new->metadata_copies =arg_int_value (cmd, vgmetadatacopies_ARG, +DEFAULT_VGMETADATACOPIES ); + }else { + vp_new->metadata_copies =find_config_tree_int (cmd, +"metadata/vgmetadatacopies" , +DEFAULT_VGMETADATACOPIES ); + } +return 0 ; }
tools/vgcreate.c +2 -6
@@ -35,11 +35,6 @@ int vgcreate(struct cmd_context *cmd, int argc, char **argv) argc--; argv++; -if (arg_count (cmd, metadatacopies_ARG)) { -log_error ("Invalid option --metadatacopies, " -"use --pvmetadatacopies instead." ); -return EINVALID_CMD_LINE ; - }pvcreate_params_set_defaults (&pp);if (!pvcreate_params_validate (cmd, argc, argv, &pp)) {return EINVALID_CMD_LINE ;
@@ -68,7 +63,8 @@ int vgcreate(struct cmd_context *cmd, int argc, char **argv) !vg_set_max_lv (vg, vp_new.max_lv) || !vg_set_max_pv (vg, vp_new.max_pv) || !vg_set_alloc_policy (vg, vp_new.alloc) || - !vg_set_clustered(vg, vp_new.clustered)) + !vg_set_clustered (vg, vp_new.clustered) || + !vg_set_mda_copies(vg, vp_new.metadata_copies)) goto bad_orphan;if (!lock_vol (cmd,VG_ORPHANS ,LCK_VG_WRITE )) {
Signature · 1 line
1.6.0.6
PATCH 31/35 Ensure fid mda lists are populated correctly during vgextend.
The vgextend path calls add_pv_to_vg(). Inside add_pv_to_vg(), we must ensure we pass the correct mdas list into pv_setup(), as copies of mdas are placed on the vg->fid list. If we don't place the mdas on the correct vg->fid list, the various counts may be incorrect and the metadata balance algorithm will not work when called from vg_write() path.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata.c | 18 +++++++++++++++++- 1 files changed, 17 insertions(+), 1 deletions(-)
lib/metadata/metadata.c +17 -1
@@ -174,6 +174,7 @@ int add_pv_to_vg(struct volume_group *vg, const char *pv_name,struct format_instance *fid = vg->fid;struct dm_pool *mem = vg->vgmem; char uuid[64] __attribute((aligned(8))); +struct dm_list *mdas;log_verbose ("Adding physical volume '%s' to volume group '%s'" , pv_name, vg->name);
@@ -217,9 +218,24 @@ int add_pv_to_vg(struct volume_group *vg, const char *pv_name,*/ pv->pe_alloc_count =0 ; +/* +* FIXME: this does not work entirely correctly in the case where a PV +* has 2 mdas and only one is ignored; ideally all non-ignored mdas +* should be placed on metadata_areas list and ignored on the +* metadata_areas_ignored list; however this requires another +* fairly complex refactoring to remove the 'mdas' parameter from both +* pv_setup and pv_write. For now, we only put ignored mdas on the +* metadata_areas_ignored list if all mdas in the PV are ignored; +* otherwise, we use the non-ignored list. +*/ +if (pv_mda_is_ignored (pv) ==pv_mda_count (pv)) + mdas = &fid->metadata_areas_ignored; +else + mdas = &fid->metadata_areas; +if (!fid->fmt->ops->pv_setup (fid->fmt,UINT64_C (0 ),0 , vg->extent_size,0 ,0 ,0UL ,UINT64_C (0 ), - &fid->metadata_areas, pv, vg)) { + mdas, pv, vg)) {log_error ("Format-specific setup of physical volume '%s' " "failed." , pv_name);return 0 ;
Signature · 1 line
1.6.0.6
PATCH 32/35 Update check in vg_split_mdas to account for ignored mdas list.
The check in vg_split_mdas will trigger an error if the 'from' vg list is empty. However, this might be ok in some instances now that we have ignored mdas. Relax this check so an error is triggered only in the case where there's truly no more mdas in the 'from' vg.
One example of where this makes a difference is with vgreduce. If we try to vgreduce a PV with un-ignored mdas, this should trigger the balancing function to un-ignore mdas on another PV in the VG. However, we don't get to vg_write() before we fail because this list size check fails, and we see an error message indicating:
"Cannot remove final metadata area ..."
Another example is with vgsplit into a new VG, where the PVs being moved contain all ignored mdas. We must move the mdas on fid->metadata_areas_ignored from 'vg_from' to 'vg_to'.
Signed-off-by Dave Wysochanski <[email protected]>
--- lib/metadata/metadata.c | 40 ++++++++++++++++++++++++++++------------ 1 files changed, 28 insertions(+), 12 deletions(-)
lib/metadata/metadata.c +28 -12
@@ -1364,20 +1364,12 @@ int vg_set_clustered(struct volume_group *vg, int clustered)return 1 ; } -/* -* Separate metadata areas after splitting a VG. -* Also accepts orphan VG as destination (for vgreduce). -*/ -int vg_split_mdas (struct cmd_context *cmd __attribute((unused)), -struct volume_group *vg_from,struct volume_group *vg_to) +static int _move_mdas (struct volume_group *vg_from,struct volume_group *vg_to, +struct dm_list *mdas_from,struct dm_list *mdas_to) {struct metadata_area *mda, *mda2; -struct dm_list *mdas_from, *mdas_to;int common_mda =0 ; - mdas_from = &vg_from->fid->metadata_areas; - mdas_to = &vg_to->fid->metadata_areas; - dm_list_iterate_items_safe(mda, mda2, mdas_from) {if (!mda->ops->mda_in_vg) { common_mda =1 ;
@@ -1391,9 +1383,33 @@ int vg_split_mdas(struct cmd_context *cmd __attribute((unused)),dm_list_move (mdas_to, &mda->list); } } +return common_mda; +} + +/* +* Separate metadata areas after splitting a VG. +* Also accepts orphan VG as destination (for vgreduce). +*/ +int vg_split_mdas (struct cmd_context *cmd __attribute((unused)), +struct volume_group *vg_from,struct volume_group *vg_to) +{ +struct dm_list *mdas_from, *mdas_to; +struct dm_list *mdas_from_ignored, *mdas_to_ignored; +int common_mda =0 ; + + mdas_from = &vg_from->fid->metadata_areas; + mdas_from_ignored = &vg_from->fid->metadata_areas_ignored; + mdas_to = &vg_to->fid->metadata_areas; + mdas_to_ignored = &vg_to->fid->metadata_areas_ignored; + + common_mda =_move_mdas (vg_from, vg_to, + mdas_from, mdas_to); + common_mda =_move_mdas (vg_from, vg_to, + mdas_from_ignored, mdas_to_ignored); -if (dm_list_empty (mdas_from) || - (!is_orphan_vg (vg_to->name) &&dm_list_empty (mdas_to))) +if ((dm_list_empty (mdas_from) &&dm_list_empty (mdas_from_ignored)) || + ((!is_orphan_vg (vg_to->name) &&dm_list_empty (mdas_to) && +dm_list_empty (mdas_to_ignored))))return common_mda;return 1 ;
Signature · 1 line
1.6.0.6
PATCH 33/35 Update _vgmerge_single() to move fid->metadata_areas_ignored.
When vgmerge is called we move the mdas from the source to the destination. With metadata balancing we now have another mda list, fid->metadata_areas_ignored, so move the mdas on this list as well.
This patch should not matter as the code is written today. However we include it for completeness in the case that _vgmerge_single() is refactored and/or moved into a library function.
Signed-off-by Dave Wysochanski <[email protected]>
--- tools/vgmerge.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
tools/vgmerge.c +6 -0
@@ -126,6 +126,12 @@ static int _vgmerge_single(struct cmd_context *cmd, const char *vg_name_to,dm_list_move (&vg_to->fid->metadata_areas, mdah); } +while (!dm_list_empty (&vg_from->fid->metadata_areas_ignored)) { +struct dm_list *mdah = vg_from->fid->metadata_areas_ignored.n; + +dm_list_move (&vg_to->fid->metadata_areas_ignored, mdah); + } + vg_to->extent_count += vg_from->extent_count; vg_to->free_count += vg_from->free_count;
Signature · 1 line
1.6.0.6
PATCH 34/35 Add --vgmetadatacopies to vgsplit man page and command.
Signed-off-by Dave Wysochanski <[email protected]>
--- tools/commands.h | 3 ++- tools/vgsplit.c | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-)
tools/commands.h +2 -1
@@ -1007,6 +1007,7 @@ xx(vgsplit,"\t[-h|--help] " "\n" "\t[-l|--maxlogicalvolumes MaxLogicalVolumes]" "\n" "\t[-M|--metadatatype 1|2] " "\n" +"\t[--[vg]metadatacopies #copies] " "\n" "\t[-n|--name LogicalVolumeName]\n" "\t[-p|--maxphysicalvolumes MaxPhysicalVolumes] " "\n" "\t[-t|--test] " "\n"
@@ -1017,7 +1018,7 @@ xx(vgsplit, alloc_ARG, autobackup_ARG, clustered_ARG, maxlogicalvolumes_ARG, maxphysicalvolumes_ARG, - metadatatype_ARG, name_ARG, test_ARG) + metadatatype_ARG, vgmetadatacopies_ARG, name_ARG, test_ARG) xx(version, "Display software and driver version information",
tools/vgsplit.c +4 -2
@@ -272,7 +272,8 @@ static int new_vg_option_specified(struct cmd_context *cmd)return (arg_count (cmd, clustered_ARG) ||arg_count (cmd, alloc_ARG) ||arg_count (cmd, maxphysicalvolumes_ARG) || -arg_count (cmd, maxlogicalvolumes_ARG)); +arg_count (cmd, maxlogicalvolumes_ARG) || +arg_count (cmd, vgmetadatacopies_ARG)); }int vgsplit (struct cmd_context *cmd,int argc,char **argv)
@@ -382,7 +383,8 @@ int vgsplit(struct cmd_context *cmd, int argc, char **argv) !vg_set_max_lv (vg_to, vp_new.max_lv) || !vg_set_max_pv (vg_to, vp_new.max_pv) || !vg_set_alloc_policy (vg_to, vp_new.alloc) || - !vg_set_clustered(vg_to, vp_new.clustered)) + !vg_set_clustered (vg_to, vp_new.clustered) || + !vg_set_mda_copies(vg_to, vp_new.metadata_copies)) goto_bad; }
Signature · 1 line
1.6.0.6
PATCH 35/35 Update tests to handle phase 2 (vg based) metadata balance.
Test vgcreate/vgchange --vgmetadatacopies, vgextend, vgreduce, vgsplit, vgmerge.
Signed-off-by Dave Wysochanski <[email protected]>
--- test/t-metadata-balance.sh | 153 +++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 151 insertions(+), 2 deletions(-)
test/t-metadata-balance.sh +151 -2
@@ -10,9 +10,9 @@ . ./test-utils.sh -aux prepare_devs 5 +aux prepare_devs 6 -# Make sure we can ignore / un-ignore mdas on a per-PV basis +echo Make sure we can ignore / un-ignore mdas on a per-PV basis for pv_in_vg in 1 0; do for mdacp in 1 2; do pvcreate --metadatacopies $mdacp $dev1 $dev2
@@ -28,6 +28,7 @@ for mdacp in 1 2; do if [ $pv_in_vg = 1 ]; then check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) check_vg_field_ $vg vg_mda_count_ignored $mdacp + check_vg_field_ $vg vg_mda_copies 0 fi pvchange --metadataignore n $dev1 check_pv_field_ $dev1 pv_mda_count $mdacp
@@ -35,7 +36,155 @@ for mdacp in 1 2; do if [ $pv_in_vg = 1 ]; then check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) check_vg_field_ $vg vg_mda_count_ignored 0 + check_vg_field_ $vg vg_mda_copies 0 vgremove -f $vg fi done done + +# Check if a PV has non-ignored mdas, and if so, ignore +pvignore_ () { + pv_ignored=$(get_pv_field "$1" pv_mda_count_ignored) + if [ $pv_ignored = 0 ]; then + pvchange --metadataignore y $1 + fi +} + +pvunignore_ () { + pv_ignored=$(get_pv_field "$1" pv_mda_count_ignored) + if [ $pv_ignored != 0 ]; then + pvchange --metadataignore n $1 + fi +} + +echo Test of vgmetadatacopies with vgcreate and vgchange +for mdacp in 1 2; do + pvcreate --metadatacopies $mdacp $dev1 $dev2 $dev4 $dev5 + check_pv_field_ $dev1 pv_mda_count_ignored 0 + check_pv_field_ $dev2 pv_mda_count_ignored 0 + check_pv_field_ $dev4 pv_mda_count_ignored 0 + check_pv_field_ $dev5 pv_mda_count_ignored 0 + pvcreate --metadatacopies 0 $dev3 + vgcreate -c n "$vg" $dev1 $dev2 $dev3 + check_vg_field_ $vg vg_mda_copies 0 + echo ensure both --vgmetadatacopies and --metadatacopies accepted + vgchange --metadatacopies $(($mdacp * 1)) $vg + echo --vgmetadatacopies is persistent on disk + echo --vgmetadatacopies affects underlying pv mda ignore + check_vg_field_ $vg vg_mda_copies $(($mdacp * 1)) + check_vg_field_ $vg vg_mda_count_ignored $(($mdacp * 1)) + vgchange --vgmetadatacopies $(($mdacp * 2)) $vg + check_vg_field_ $vg vg_mda_copies $(($mdacp * 2)) + check_vg_field_ $vg vg_mda_count_ignored 0 + echo allow setting metadatacopies larger than number of PVs + vgchange --vgmetadatacopies $(($mdacp * 5)) $vg + check_vg_field_ $vg vg_mda_copies $(($mdacp * 5)) + check_vg_field_ $vg vg_mda_count_ignored 0 + echo setting to 0 disables automatic balancing + vgchange --vgmetadatacopies 0 $vg + check_vg_field_ $vg vg_mda_copies 0 + vgremove -f $vg + echo vgcreate succeeds even when creating a VG w/all ignored mdas + pvchange --metadataignore y $dev1 $dev2 + check_pv_field_ $dev1 pv_mda_count $mdacp + check_pv_field_ $dev2 pv_mda_count_ignored $mdacp + vgcreate -c n "$vg" $dev1 $dev2 + check_vg_field_ $vg vg_mda_copies 0 + vgremove -f $vg + echo vgcreate succeeds with a specific number of metadata copies + vgcreate -c n --vgmetadatacopies $(($mdacp * 2)) "$vg" $dev1 $dev2 + check_vg_field_ $vg vg_mda_copies $(($mdacp * 2)) + vgremove -f $vg + vgcreate -c n --vgmetadatacopies $(($mdacp * 1)) "$vg" $dev1 $dev2 + check_vg_field_ $vg vg_mda_copies $(($mdacp * 1)) + vgremove -f $vg + echo vgcreate succeeds with a larger value than total metadatacopies + vgcreate -c n --vgmetadatacopies $(($mdacp * 5)) "$vg" $dev1 $dev2 + check_vg_field_ $vg vg_mda_copies $(($mdacp * 5)) + vgremove -f $vg + echo vgcreate succeeds with --vgmetadatacopies 0 + vgcreate -c n --vgmetadatacopies 0 "$vg" $dev1 $dev2 + check_vg_field_ $vg vg_mda_copies 0 + vgremove -f $vg + pvunignore_ $dev1 + pvunignore_ $dev2 + pvunignore_ $dev4 + pvunignore_ $dev5 + echo vgcreate succeds with small value of --metadatacopies, ignores mdas + vgcreate -c n --vgmetadatacopies 1 "$vg" $dev1 $dev2 $dev4 $dev5 + check_vg_field_ $vg vg_mda_copies 1 + check_vg_field_ $vg vg_mda_count $(($mdacp * 4)) + check_vg_field_ $vg vg_mda_count_ignored $(($mdacp * 4 - 1)) + echo Setting a larger value should trigger non-ignore of mdas + vgchange --metadatacopies 3 $vg + check_vg_field_ $vg vg_mda_copies 3 + check_vg_field_ $vg vg_mda_count_ignored $(($mdacp * 4 - 3)) + echo Setting a smaller value should trigger ignore of mdas + vgremove -f $vg +done + +echo Test vgextend / vgreduce with vgmetadatacopies +for mdacp in 1 2; do + pvcreate --metadatacopies $mdacp $dev1 $dev2 $dev4 $dev5 + pvcreate --metadatacopies 0 $dev3 + echo Set a large value of vgmetadatacopies + vgcreate -c n --vgmetadatacopies $(($mdacp * 5)) "$vg" $dev1 $dev2 $dev3 + check_vg_field_ $vg vg_mda_copies $(($mdacp * 5)) + echo Ignore mdas on devices to be used for vgextend + echo Large value of vgetadatacopies should automatically un-ignore mdas + pvchange --metadataignore y $dev4 $dev5 + check_pv_field_ $dev4 pv_mda_count_ignored $mdacp + vgextend $vg $dev4 $dev5 + check_pv_field_ $dev4 pv_mda_count_ignored 0 + check_pv_field_ $dev5 pv_mda_count_ignored 0 + vgremove -f $vg + echo Set a small value of vgmetadatacopies + vgcreate -c n --vgmetadatacopies $(($mdacp * 1)) "$vg" $dev1 $dev2 $dev3 + check_vg_field_ $vg vg_mda_copies $(($mdacp * 1)) + echo Ignore mdas on devices to be used for vgextend + echo Small value of vgetadatacopies should leave mdas as ignored + pvchange --metadataignore y $dev4 $dev5 + check_pv_field_ $dev4 pv_mda_count_ignored $mdacp + vgextend $vg $dev4 $dev5 + check_pv_field_ $dev4 pv_mda_count_ignored $mdacp + check_pv_field_ $dev5 pv_mda_count_ignored $mdacp + echo vgreduce of ignored pv w/mda should not trigger any change to ignore bits + vgreduce $vg $dev4 + check_pv_field_ $dev4 pv_mda_count_ignored $mdacp + check_pv_field_ $dev5 pv_mda_count_ignored $mdacp + echo vgreduce of un-ignored pv w/mda should trigger un-ignore on an mda + vgreduce $vg $dev1 $dev2 $dev3 + check_pv_field_ $dev5 pv_mda_count_ignored 0 + check_vg_field_ $vg vg_mda_copies $(($mdacp * 1)) + pvunignore_ $dev1 + pvunignore_ $dev2 + echo setting vgmetadatacopies to 0 should allow vgextend to add w/out balancing + vgchange --vgmetadatacopies 0 $vg + vgextend $vg $dev1 $dev2 + check_vg_field_ $vg vg_mda_copies 0 + check_vg_field_ $vg vg_mda_count $(($mdacp * 3)) + check_vg_field_ $vg vg_mda_count_ignored 0 + check_pv_field_ $dev1 pv_mda_count_ignored 0 + check_pv_field_ $dev2 pv_mda_count_ignored 0 + vgremove -f $vg +done + +echo Test special situations, vgsplit, vgmerge, etc +for mdacp in 1 2; do + pvcreate --metadatacopies $mdacp $dev1 $dev2 $dev3 $dev4 $dev5 + vgcreate -c n --vgmetadatacopies 2 $vg1 $dev1 $dev2 $dev3 + vgcreate -c n --vgmetadatacopies $(($mdacp * 1)) $vg2 $dev4 $dev5 + echo vgsplit/vgmerge preserves value of metadata copies + check_vg_field_ $vg1 vg_mda_copies 2 + check_vg_field_ $vg2 vg_mda_copies $(($mdacp * 1)) + vgsplit $vg1 $vg2 $dev1 + check_vg_field_ $vg2 vg_mda_copies $(($mdacp * 1)) + vgmerge $vg1 $vg2 + check_vg_field_ $vg1 vg_mda_copies 2 + check_vg_field_ $vg1 vg_mda_count $(($mdacp * 5)) + echo vgsplit into new vg sets proper value of vgmetadatacopies + vgsplit --vgmetadatacopies $(($mdacp * 2)) $vg1 $vg2 $dev1 $dev2 + check_vg_field_ $vg2 vg_mda_copies $(($mdacp * 2)) + vgremove -f $vg1 + vgremove -f $vg2 +done
Signature · 1 line
1.6.0.6
PATCH 25/35 Make vg->mda_copies persistent in on disk vg metadata.
In reply to Dave WysochanskiDne 22.6.2010 05:05, Dave Wysochanski napsal(a):
Quoted 20 lines
This patch adds the ability to read/write the
vg->mda_copiesvalues from/to the vg metadata.If we read the VG metadata and this field does not exist, we set
mda_copiesto the default value of 0. Later in the code, we use this special '0' value to indicate a disable of metadata balancing. This should preserve existing LVM behavior and ensure metadata balancing can be turned off should the need arise.Signed-off-by Dave Wysochanski<[email protected]>--- lib/format_text/export.c | 1 + lib/format_text/import_vsn1.c | 5 +++++ 2 files changed, 6 insertions(+), 0 deletions(-)lib/format_text/export.c +0
-0
lib/format_text/export.c static int _print_vg(struct formatter *f, struct volume_group *vg) · line 402
Quoted 6 lines
lib/format_text/export.c +1
-0@@ -397,6 +397,7 @@ static int _print_vg(struct formatter *f, struct volume_group *vg)outf (f,"allocation_policy = \"%s\"" ,get_alloc_string (vg->alloc)); } +outf (f,"metadata_copies = %u" , vg->mda_copies);return 1 ;
Hmm - so we add a new element to the metadata format with the same version number? If the user will update metadata with older tools which do not know about this parameter - will he get at least a warning that he will miss some information ?
Zdenek
PATCH 25/35 Make vg->mda_copies persistent in on disk vg metadata.
Quoted 26 lines · Zdenek Kabelac
Dne 22.6.2010 05:05, Dave Wysochanski napsal(a):
Quoted 25 lines
This patch adds the ability to read/write the
vg->mda_copiesvalues from/to the vg metadata.If we read the VG metadata and this field does not exist, we set
mda_copiesto the default value of 0. Later in the code, we use this special '0' value to indicate a disable of metadata balancing. This should preserve existing LVM behavior and ensure metadata balancing can be turned off should the need arise.Signed-off-by Dave Wysochanski<[email protected]>--- lib/format_text/export.c | 1 + lib/format_text/import_vsn1.c | 5 +++++ 2 files changed, 6 insertions(+), 0 deletions(-)lib/format_text/export.c +1
-0@@ -397,6 +397,7 @@ static int _print_vg(struct formatter *f, struct volume_group *vg)outf (f,"allocation_policy = \"%s\"" ,get_alloc_string (vg->alloc)); } +outf (f,"metadata_copies = %u" , vg->mda_copies);
Quoted 8 lines
lib/format_text/export.c static int _print_vg(struct formatter *f, struct volume_group *vg) · line 402
Quoted 1 line
lib/format_text/export.c +0
-0return 1 ;Hmm - so we add a new element to the metadata format with the same version number? If the user will update metadata with older tools which do not know about this parameter - will he get at least a warning that he will miss some information ?
LVM ignores unknown fields in the vg metadata, and if older tools are used after metadata is written with new tools and this value, yes, you'll lose this field (it will go back to default value of "disabled metadata balance").
If we update FORMAT_VERSION_VALUE, older tools will just refuse to parse the metadata completely and the tools are useless. I don't think this is what users would want. Perhaps there should be a warning of sorts when unknown fields are encountered, though I've not looked into such a patch.
It's not clear what process (if any) we are using when updating on-disk metadata and fields. Another case is my first few patches, where I take a previously unused area of the LVM header area and use it for flags. I have not tested whether older tools would wipe the value of the "ignored" flag or whether it would be preserved.
Was someone working on patches or rules for updating on-disk fields/metadata? It does not seem like we test these types of scenarios but I guess so far it has "just worked".
PATCH 25/35 Make vg->mda_copies persistent in on disk vg metadata.
Indeed, that version number would only be changed if chose to make a major change that broke compatibility.
Generally, if people access LVM metadata with older tools than the ones used to write it they can expect any new features in the metadata to get dropped - but the code should still work OK. Sometimes there'll be warning messages; other times not.
We do try to phase-in new features, so code goes in that recognises the new feature several releases before the feature can be enabled by default.
So for now, this new feature will require positive action by the sysadmin to enable. (E.g. editing lvm.conf or using new cmdline options.)
Alasdair