Replicator 100412

PATCH New Zdenek Kabelac 5 participants 41 messages Apr 12, 2010 – Apr 26, 2010 Library code

Share & tools

Zdenek Kabelac

author

2010-04-12 15:21 UTC #0 lore raw

cover Replicator 100412

Rebased set of replicator patches.
Log creation code is still kept separately as many changes are still on going in this code.

Updated parts:

* previous set had wrong month number.
* converted Makefiles.
* renamed some functions (removed lvm_ prefix).
* renamed vg_name_list -> vgs_list.
* moved code from validation to segment checking.

* Added dm_list_splice
* Changed API for process_each_lv_in_vg()
* lock_lv_vol() detects missing VGs before calling lock_vol().

* Using common typedef between dm and lvm part for operation mode.
* Minor code reoganization based on review
* Extended validation code.
* Added manpages updates to patches to document API
still this is ongoing task to make all the options working.
* API around vg_name_list which is now autonomous entity.
* Processed VG contains list of opened VGs and flag for missing VGs.

Zdenek Kabelac (22):
Add dm_list_splice() for list join.
API change for args of process_each_lv_in_vg()
Replicator: add libdm support
Replicator: suspend/deactivate replicator
Replicator: add lvm support
Replicator: check replicator segment
Replicator: extend _lv_each_dependency() rep. deps
Replicator: extend volume_group with list of VGs
Replicator: add sorted vgs_list
Replicator: add find_replicator_vgs
Replicator: add read and release VGs for rsites
Replicator: use vgs_list for _process_one_vg()
Replicator: use vgs_list for process_each_lv_in_vg()
Replicator: lock_vol() finds missing VGs
Replicator: activate checks for missing vgs
Replicator: update _create_partial_dtree()
Replicator: vg with missing_vgs does not print
Replicator: add replicator to dtree
Replicator: do not remove of replicators' LVs
Replicator: activate change for vgchange
Replicator: add new options for replicator
Replicator: man pages for lvcreate, lvchange

 configure.in                     |   17 +
 lib/Makefile.in                  |   10 +
 lib/activate/activate.c          |   22 +-
 lib/activate/dev_manager.c       |  135 +++++++-
 lib/commands/toolcontext.c       |    5 +
 lib/config/defaults.h            |    1 +
 lib/format_text/flags.c          |    2 +
 lib/locking/locking.h            |    4 +-
 lib/metadata/lv_manip.c          |   10 +
 lib/metadata/merge.c             |   17 +
 lib/metadata/metadata-exported.h |  105 +++++-
 lib/metadata/metadata.c          |    4 +
 lib/metadata/replicator_manip.c  |  718 ++++++++++++++++++++++++++++++++++
 lib/metadata/segtype.h           |    8 +
 lib/replicator/.exported_symbols |    1 +
 lib/replicator/Makefile.in       |   25 ++
 lib/replicator/replicator.c      |  794 ++++++++++++++++++++++++++++++++++++++
 libdm/.exported_symbols          |    4 +
 libdm/datastruct/list.c          |   12 +
 libdm/libdevmapper.h             |   38 ++
 libdm/libdm-deptree.c            |  359 +++++++++++++++++-
 man/lvchange.8.in                |   57 +++
 man/lvcreate.8.in                |   55 +++
 tools/args.h                     |   13 +
 tools/commands.h                 |    7 +
 tools/lvcreate.c                 |   55 +++
 tools/lvmcmdline.c               |    9 +
 tools/toollib.c                  |  115 ++++--
 tools/toollib.h                  |    1 +
 tools/tools.h                    |    1 +
 tools/vgchange.c                 |    9 +-
 tools/vgdisplay.c                |    2 +-
 32 files changed, 2567 insertions(+), 48 deletions(-)
 create mode 100644 lib/metadata/replicator_manip.c
 create mode 100644 lib/replicator/.exported_symbols
 create mode 100644 lib/replicator/Makefile.in
 create mode 100644 lib/replicator/replicator.c

Zdenek Kabelac

author

2010-04-12 15:21 UTC #1 lore raw

PATCH 1/22 Add dm_list_splice() for list join.

Introduce dm_list_splice to join two lists.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 libdm/.exported_symbols |    1 +
 libdm/datastruct/list.c |   12 ++++++++++++
 libdm/libdevmapper.h    |    5 +++++
 3 files changed, 18 insertions(+), 0 deletions(-)

libdm/.exported_symbols +1 -0

index fd9d0b9..ea65c8b 100644
@@ -153,6 +153,7 @@ dm_list_add
 dm_list_add_h
 dm_list_del
 dm_list_move
+dm_list_splice
 dm_list_empty
 dm_list_start
 dm_list_end

libdm/datastruct/list.c +12 -0

index 3d9ca8b..a7f9324 100644
@@ -144,3 +144,15 @@ unsigned int dm_list_size(const struct dm_list *head)
 
 	return s;
 }
+
+void dm_list_splice(struct dm_list *list, struct dm_list *head)
+{
+	struct dm_list *first = list->n;
+	struct dm_list *last = list->p;
+	struct dm_list *at = head->n;
+
+	first->p = head;
+	head->n = first;
+	last->n = at;
+	at->p = last;
+}

libdm/libdevmapper.h +5 -0

index 1ae930f..aa8ad5c 100644
@@ -715,6 +715,11 @@ void dm_list_del(struct dm_list *elem);
 void dm_list_move(struct dm_list *head, struct dm_list *elem);
 
 /*
+ * Join 'list' to 'head'.
+ */
+void dm_list_splice(struct dm_list *list, struct dm_list *head);
+
+/*
  * Is the list empty?
  */
 int dm_list_empty(const struct dm_list *head);

Signature · 1 line

1.7.0.1

Petr Rockai

2010-04-14 11:38 UTC #2 lore raw

PATCH 1/22 Add dm_list_splice() for list join.

Quoted 23 lines · Zdenek Kabelac

libdm/datastruct/list.c +12 -0

@@ -144,3 +144,15 @@ unsigned int dm_list_size(const struct dm_list *head)
 
 	return s;
 }
+
+void dm_list_splice(struct dm_list *list, struct dm_list *head)
+{
+	struct dm_list *first = list->n;
+	struct dm_list *last = list->p;
+	struct dm_list *at = head->n;
+
+	first->p = head;
+	head->n = first;
+	last->n = at;
+	at->p = last;
+}

libdm/libdevmapper.h +0 -0

@@ -715,6 +715,11 @@ void dm_list_del(struct dm_list *elem);
 void dm_list_move(struct dm_list *head, struct dm_list *elem);

libdm/libdevmapper.h void dm_list_del(struct dm_list *elem); · line 725

Quoted 10 lines

libdm/libdevmapper.h +5 -0

 
 /*
+ * Join 'list' to 'head'.
+ */
+void dm_list_splice(struct dm_list *list, struct dm_list *head);
+
+/*
  * Is the list empty?
  */
 int dm_list_empty(const struct dm_list *head);

Ack. Please check in.

Yours,
Petr.

Alasdair G Kergon

2010-04-26 13:15 UTC #3 lore raw

PATCH 1/22 Add dm_list_splice() for list join.

In reply to Zdenek Kabelac

Quoted 1 line · Zdenek Kabelac

Introduce dm_list_splice to join two lists.

OK, but do please retain the style and conventions of the existing functions.

Quoted 1 line

+void dm_list_splice(struct dm_list *list, struct dm_list *head)

Based on existing conventions, the code appears to implement:
dm_list_splice_h(struct dm_list *head, struct dm_list *list)

Try the following (untested) instead:

/*
* Join two lists together.
* This moves all the elements of the list 'head1' to the end of the list
* 'head', leaving 'head1' empty.
*/
void dm_list_splice(struct dm_list *head, struct dm_list *head1)

{
	assert(head->n);
	assert(head1->n);

	head1->p->n = head;
	head1->n->p = head->p;

	head->p->n = head1->n;
	head->p = head1->p;

	dm_list_init(head1);
}

Of if you genuinely require the _h version (i.e. the elements are ordered and you can't choose to retain 'head1' for the combined list instead of 'head.)

/*
* Join two lists together.
* This moves all the elements of the list 'head1' to the front of the list
* 'head', leaving 'head1' empty.
*/
void dm_list_splice_h(struct dm_list *head, struct dm_list *head1)

{
	assert(head->n);
	assert(head1->n);

	head1->n->p = head;
	head1->p->n = head->n;

	head->n->p = head1->p;
	head->n = head1->n;

	dm_list_init(head1);
}

Alasdair

Alasdair G Kergon

2010-04-26 13:23 UTC #4 lore raw

PATCH 1/22 Add dm_list_splice() for list join.

Need to check for empty lists too.

Alasdair

Zdenek Kabelac

author

2010-04-12 15:21 UTC #5 lore raw

PATCH 2/22 API change for args of process_each_lv_in_vg()

In reply to Zdenek Kabelac

Patch adds failed_lvs to the list of parameters for process_each_lv_in_vg(). If the list is not NULL it will be filled with lv_names of failing LVs during function execution.

Application could later reiterate only on failed LVs.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 tools/toollib.c   |   14 +++++++++++++-
 tools/toollib.h   |    1 +
 tools/vgdisplay.c |    2 +-
 3 files changed, 15 insertions(+), 2 deletions(-)

tools/toollib.c +13 -1

index 52ea4a6..c31315c 100644
@@ -86,6 +86,7 @@ int process_each_lv_in_vg(struct cmd_context *cmd,
 			  struct volume_group *vg,
 			  const struct dm_list *arg_lvnames,
 			  const struct dm_list *tags,
+			  struct dm_list *failed_lvs,
 			  void *handle,
 			  process_single_lv_fn_t process_single)
 {

@@ -99,6 +100,9 @@ int process_each_lv_in_vg(struct cmd_context *cmd,
 
 	struct lv_list *lvl;
 
+	if (failed_lvs)
+		dm_list_init(failed_lvs);
+
 	if (!vg_check_status(vg, EXPORTED_VG))
 		return ECMD_FAILED;

@@ -148,6 +152,14 @@ int process_each_lv_in_vg(struct cmd_context *cmd,
 			continue;
 
 		ret = process_single(cmd, lvl->lv, handle);
+		if (ret != ECMD_PROCESSED && failed_lvs) {
+			if (!str_list_add(cmd->mem, failed_lvs,
+					  dm_pool_strdup(cmd->mem,
+							 lvl->lv->name))) {
+				log_error("Allocation failed for str_list.");
+				return ECMD_FAILED;
+			}
+		}
 		if (ret > ret_max)
 			ret_max = ret;
 		if (sigint_caught())

@@ -320,7 +332,7 @@ int process_each_lv(struct cmd_context *cmd, int argc, char **argv,
 		}
 
 		ret = process_each_lv_in_vg(cmd, vg, &lvnames, tags_arg,
-					    handle, process_single);
+					    NULL, handle, process_single);
 		unlock_and_release_vg(cmd, vg, vgname);
 		if (ret > ret_max)
 			ret_max = ret;

tools/toollib.h +1 -0

index 0ea7eba..8f22cc4 100644
@@ -79,6 +79,7 @@ int process_each_lv_in_vg(struct cmd_context *cmd,
 			  struct volume_group *vg,
 			  const struct dm_list *arg_lvnames,
 			  const struct dm_list *tags,
+			  struct dm_list *failed_lvs,
 			  void *handle,
 			  process_single_lv_fn_t process_single);

tools/vgdisplay.c +1 -1

index 29bc7f1..954e456 100644
@@ -37,7 +37,7 @@ static int vgdisplay_single(struct cmd_context *cmd, const char *vg_name,
 	if (arg_count(cmd, verbose_ARG)) {
 		vgdisplay_extents(vg);
 
-		process_each_lv_in_vg(cmd, vg, NULL, NULL, NULL,
+		process_each_lv_in_vg(cmd, vg, NULL, NULL, NULL, NULL,
 				      (process_single_lv_fn_t)lvdisplay_full);
 
 		log_print("--- Physical volumes ---");

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 13:34 UTC #6 lore raw

PATCH 2/22 API change for args of process_each_lv_in_vg()

Quoted 4 lines · Zdenek Kabelac

Patch adds failed_lvs to the list of parameters for process_each_lv_in_vg(). If the list is not NULL it will be filled with lv_names of failing LVs during function execution.
Application could later reiterate only on failed LVs.

OK.

Alasdair

Mike Snitzer

2010-04-26 13:46 UTC #7 lore raw

PATCH 2/22 API change for args of process_each_lv_in_vg()

In reply to Zdenek Kabelac

Quoted 5 lines · Zdenek Kabelac

Patch adds failed_lvs to the list of parameters for process_each_lv_in_vg(). If the list is not NULL it will be filled with lv_names of failing LVs during function execution.
Application could later reiterate only on failed LVs.

Alasdair's recent comment highlighted this change. Minor nit, maybe rename the 'failed_lvs' to 'failed_lvnames'?

Mike

Zdenek Kabelac

author

2010-04-12 15:21 UTC #8 lore raw

PATCH 3/22 Replicator: add libdm support

In reply to Zdenek Kabelac

Introducing dm_tree_node_add_replicator_target() and dm_tree_node_add_replicator_dev_target(). Define typedef dm_replicator_mode_t

Signed-off-by Zdenek Kabelac <[email protected]>

---
 libdm/.exported_symbols |    2 +
 libdm/libdevmapper.h    |   31 +++++
 libdm/libdm-deptree.c   |  297 ++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 325 insertions(+), 5 deletions(-)

libdm/.exported_symbols +2 -0

index ea65c8b..d9ef125 100644
@@ -80,6 +80,8 @@ dm_tree_node_add_crypt_target
 dm_tree_node_add_mirror_target
 dm_tree_node_add_mirror_target_log
 dm_tree_node_add_target_area
+dm_tree_node_add_replicator_target
+dm_tree_node_add_replicator_dev_target
 dm_tree_node_set_read_ahead
 dm_tree_skip_lockfs
 dm_tree_use_no_flush_suspend

libdm/libdevmapper.h +31 -0

index aa8ad5c..3ab5ea2 100644
@@ -439,6 +439,37 @@ int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node,
 					  const char *log_uuid,
 					  unsigned area_count,
 					  uint32_t flags);
+
+/* Replicator operation mode */
+typedef enum {
+	DM_REPLICATOR_SYNC,			/* Synchronous replication */
+	DM_REPLICATOR_ASYNC_WARN,		/* Warn if async replicator is slow */
+	DM_REPLICATOR_ASYNC_STALL,		/* Stall replicator if not fast enough */
+	DM_REPLICATOR_ASYNC_DROP,		/* Drop sites out of sync */
+	DM_REPLICATOR_ASYNC_FAIL,		/* Fail replicator if slow */
+	NUM_DM_REPLICATOR_MODES
+} dm_replicator_mode_t;
+
+int dm_tree_node_add_replicator_target(struct dm_tree_node *node,
+				       uint64_t size,
+				       const char *rlog_uuid,
+				       const char *rlog_type,
+				       unsigned rsite_index,
+				       dm_replicator_mode_t mode,
+				       uint32_t async_timeout,
+				       uint64_t fall_behind_data,
+				       uint32_t fall_behind_ios);
+
+int dm_tree_node_add_replicator_dev_target(struct dm_tree_node *node,
+					   uint64_t size,
+					   const char *replicator_uuid,	/* Replicator control device */
+					   uint64_t rdevice_index,
+					   const char *rdev_uuid,	/* Rimage device name/uuid */
+					   unsigned rsite_index,
+					   const char *slog_uuid,
+					   uint32_t slog_flags,		/* Mirror log flags */
+					   uint32_t slog_region_size);
+
 int dm_tree_node_add_target_area(struct dm_tree_node *node,
 				    const char *dev_name,
 				    const char *dlid,

libdm/libdm-deptree.c +292 -5

380 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 13:46 UTC #9 lore raw

PATCH 3/22 Replicator: add libdm support

libdm/libdevmapper.h int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node, · line 441

Quoted 7 lines · Zdenek Kabelac

index aa8ad5c..3ab5ea2 100644

libdm/libdevmapper.h +0 -0

@@ -439,6 +439,37 @@ int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node,
 					  const char *log_uuid,
 					  unsigned area_count,
 					  uint32_t flags);

Add comment lines that clearly separate out this section of the header file, indicating that the functionality and API are not yet frozen.

libdm/libdevmapper.h int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node,

Quoted 6 lines

libdm/libdevmapper.h +5 -0

+
+/* Replicator operation mode */
+typedef enum {
 
+/* Replicator-log has a list of sites */
+/* CHECKME: maybe move to seg_area too? */

Stick with 'FIXME' for anything like that.

libdm/libdevmapper.h int dm_tree_preload_children(struct dm_tree_node *dnode, · line 1812

Quoted 7 lines

libdm/libdevmapper.h +3 -0

@@ -1688,6 +1807,9 @@ int dm_tree_preload_children(struct dm_tree_node *dnode,
 			}
 		}
 
+		if (child->activation_priority != 0)
+			continue;
+

That change could affect non-replicator code: please separate it out and explain the reasoning.

libdm/libdevmapper.h int dm_tree_preload_children(struct dm_tree_node *dnode, · line 1813

Quoted 1 line

libdm/libdevmapper.h +1 -0
+	if (rsite_index == 0) {

Stick with !rsite_index. (Or if '0' has a special meaning use #define.)

Alasdair

Zdenek Kabelac

author

2010-04-26 14:12 UTC #10 lore raw

PATCH 3/22 Replicator: add libdm support

Dne 26.4.2010 15:46, Alasdair G Kergon napsal(a):

Quoted 25 lines · Zdenek Kabelac

libdm/libdevmapper.h int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node, · line 441
Quoted 7 lines
index aa8ad5c..3ab5ea2 100644
libdm/libdevmapper.h +0 -0
@@ -439,6 +439,37 @@ int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node,
 					  const char *log_uuid,
 					  unsigned area_count,
 					  uint32_t flags);
Add comment lines that clearly separate out this section of the header file, indicating that the functionality and API are not yet frozen.
libdm/libdevmapper.h int dm_tree_node_add_mirror_target_log(struct dm_tree_node *node,
Quoted 6 lines
libdm/libdevmapper.h +5 -0
+
+/* Replicator operation mode */
+typedef enum {
 
+/* Replicator-log has a list of sites */
+/* CHECKME: maybe move to seg_area too? */
Stick with 'FIXME' for anything like that.
Quoted 4 lines
libdm/libdevmapper.h +0 -0
@@ -1688,6 +1807,9 @@ int dm_tree_preload_children(struct dm_tree_node *dnode,
 			}
 		}
 

Quoted 6 lines

libdm/libdevmapper.h int dm_tree_preload_children(struct dm_tree_node *dnode, · line 1812
Quoted 3 lines
libdm/libdevmapper.h +3 -0
+		if (child->activation_priority != 0)
+			continue;
+
That change could affect non-replicator code: please separate it out and explain the reasoning.

True - this code seems to be no longer needed - as it's been hack to avoid some strange tree loading in some past versions of replicator.

Zdenek

Zdenek Kabelac

author

2010-04-12 15:21 UTC #11 lore raw

PATCH 4/22 Replicator: suspend/deactivate replicator

In reply to Zdenek Kabelac

Introducing dm_tree_set_suspend_priority() for suspending replicator control target before deactivation of replicator-dev target.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 libdm/.exported_symbols |    1 +
 libdm/libdevmapper.h    |    2 +
 libdm/libdm-deptree.c   |   62 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 65 insertions(+), 0 deletions(-)

libdm/.exported_symbols +1 -0

index d9ef125..b977383 100644
@@ -82,6 +82,7 @@ dm_tree_node_add_mirror_target_log
 dm_tree_node_add_target_area
 dm_tree_node_add_replicator_target
 dm_tree_node_add_replicator_dev_target
+dm_tree_set_suspend_priority
 dm_tree_node_set_read_ahead
 dm_tree_skip_lockfs
 dm_tree_use_no_flush_suspend

libdm/libdevmapper.h +2 -0

index 3ab5ea2..52f594c 100644
@@ -470,6 +470,8 @@ int dm_tree_node_add_replicator_dev_target(struct dm_tree_node *node,
 					   uint32_t slog_flags,		/* Mirror log flags */
 					   uint32_t slog_region_size);
 
+int dm_tree_set_suspend_priority(struct dm_tree *dtree, const char *uuid, int priority);
+
 int dm_tree_node_add_target_area(struct dm_tree_node *node,
 				    const char *dev_name,
 				    const char *dlid,

libdm/libdm-deptree.c +62 -0

86 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 13:50 UTC #12 lore raw

PATCH 4/22 Replicator: suspend/deactivate replicator

Quoted 2 lines · Zdenek Kabelac

Introducing dm_tree_set_suspend_priority() for suspending replicator control target before deactivation of replicator-dev target.

Existing code paths are affected by this.

Again, please provide a more-detailed explanation of the new logic and why it will not break any existing code.

Alasdair

Zdenek Kabelac

author

2010-04-12 15:21 UTC #13 lore raw

PATCH 5/22 Replicator: add lvm support

In reply to Zdenek Kabelac

Adding configure.in support for Replicators. Adding basic lib lvm support for Replicators. Adding flags REPLICATOR and REPLICATOR_LOG. Adding segments SEG_REPLICATOR and SEG_REPLICATOR_DEV. Adding basic methods for handling replicator metadata.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 configure.in                     |   17 +
 lib/Makefile.in                  |   10 +
 lib/commands/toolcontext.c       |    5 +
 lib/format_text/flags.c          |    2 +
 lib/metadata/lv_manip.c          |    1 +
 lib/metadata/merge.c             |   14 +
 lib/metadata/metadata-exported.h |   74 ++++-
 lib/metadata/replicator_manip.c  |  340 ++++++++++++++++
 lib/metadata/segtype.h           |    8 +
 lib/replicator/.exported_symbols |    1 +
 lib/replicator/Makefile.in       |   25 ++
 lib/replicator/replicator.c      |  794 ++++++++++++++++++++++++++++++++++++++
 12 files changed, 1290 insertions(+), 1 deletions(-)
 create mode 100644 lib/metadata/replicator_manip.c
 create mode 100644 lib/replicator/.exported_symbols
 create mode 100644 lib/replicator/Makefile.in
 create mode 100644 lib/replicator/replicator.c

configure.in +17 -0

index ed9e631..c6b6a31 100644
@@ -303,6 +303,21 @@ if test x$MIRRORS = xinternal; then
 fi
 
 ################################################################################
+dnl -- asynchronous volume replicator inclusion type
+AC_MSG_CHECKING(whether to include replicators)
+AC_ARG_WITH(replicators, AC_HELP_STRING([--with-replicators=TYPE],
+	[Replicator support: internal/shared/none [TYPE=none] ]),
+	[REPLICATORS=$withval], [REPLICATORS="none"])
+AC_MSG_RESULT($REPLICATORS)
+
+case "$REPLICATORS" in
+  none|shared) ;;
+  internal) AC_DEFINE([REPLICATOR_INTERNAL], 1,
+		[Define to 1 to include built-in support for replicators.]) ;;
+  *) AC_MSG_ERROR([--with-replicators parameter invalid ($REPLICATORS)]) ;;
+esac
+
+################################################################################
 dnl -- Disable readline
 AC_MSG_CHECKING(whether to enable readline)
 AC_ARG_ENABLE([readline],

@@ -1104,6 +1119,7 @@ AC_SUBST(LVM_PATCHLEVEL)
 AC_SUBST(LVM_RELEASE)
 AC_SUBST(LVM_RELEASE_DATE)
 AC_SUBST(MIRRORS)
+AC_SUBST(REPLICATORS)
 AC_SUBST(MSGFMT)
 AC_SUBST(PKGCONFIG)
 AC_SUBST(POOL)

@@ -1156,6 +1172,7 @@ lib/format1/Makefile
 lib/format_pool/Makefile
 lib/locking/Makefile
 lib/mirror/Makefile
+lib/replicator/Makefile
 lib/misc/lvm-version.h
 lib/snapshot/Makefile
 libdm/Makefile

lib/Makefile.in +10 -0

index 3498141..ae81040 100644
@@ -32,6 +32,10 @@ ifeq ("@MIRRORS@", "shared")
   SUBDIRS += mirror
 endif
 
+ifeq ("@REPLICATORS@", "shared")
+  SUBDIRS += replicator
+endif
+
 SOURCES =\
 	activate/activate.c \
 	cache/lvmcache.c \

@@ -74,6 +78,7 @@ SOURCES =\
 	metadata/mirror.c \
 	metadata/pv_manip.c \
 	metadata/pv_map.c \
+	metadata/replicator_manip.c \
 	metadata/segtype.c \
 	metadata/snapshot_manip.c \
 	misc/crc.c \

@@ -129,6 +134,10 @@ ifeq ("@MIRRORS@", "internal")
   SOURCES += mirror/mirrored.c
 endif
 
+ifeq ("@REPLICATORS@", "internal")
+  SOURCES += replicator/replicator.c
+endif
+
 ifeq ("@DEVMAPPER@", "yes")
   SOURCES +=\
 	activate/dev_manager.c \

@@ -155,6 +164,7 @@ ifeq ($(MAKECMDGOALS),distclean)
 	format_pool \
 	snapshot \
 	mirror \
+	replicator \
 	locking
 endif

lib/commands/toolcontext.c +5 -0

index 818e452..388c559 100644
@@ -917,6 +917,11 @@ static int _init_segtypes(struct cmd_context *cmd)
 	dm_list_add(&cmd->segtypes, &segtype->list);
 #endif
 
+#ifdef REPLICATOR_INTERNAL
+	if (!init_replicator_segtype(&seglib))
+		return 0;
+#endif
+
 #ifdef HAVE_LIBDL
 	/* Load any formats in shared libs unless static */
 	if (!is_static() &&

lib/format_text/flags.c +2 -0

index a41dc1a..994adf2 100644
@@ -67,6 +67,8 @@ static const struct flag _lv_flags[] = {
 	{PARTIAL_LV, NULL, 0},
 	{POSTORDER_FLAG, NULL, 0},
 	{VIRTUAL_ORIGIN, NULL, 0},
+	{REPLICATOR, NULL, 0},
+	{REPLICATOR_LOG, NULL, 0},
 	{0, NULL, 0}
 };

lib/metadata/lv_manip.c +1 -0

index f8bed4b..43ca423 100644
@@ -2008,6 +2008,7 @@ struct logical_volume *alloc_lv(struct dm_pool *mem)
 	dm_list_init(&lv->segments);
 	dm_list_init(&lv->tags);
 	dm_list_init(&lv->segs_using_this_lv);
+	dm_list_init(&lv->rsites);
 
 	return lv;
 }

lib/metadata/merge.c +14 -0

index df90743..66ff57b 100644
@@ -72,6 +72,8 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 	uint32_t area_multiplier, s;
 	struct seg_list *sl;
 	int error_count = 0;
+	struct replicator_site *rsite;
+	struct replicator_device *rdev;
 
 	dm_list_iterate_items(seg, &lv->segments) {
 		seg_count++;

@@ -213,6 +215,18 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 			if (lv == seg_lv(seg, s))
 				seg_found++;
 		}
+		if (seg_is_replicator_dev(seg)) {
+			dm_list_iterate_items(rsite, &seg->replicator->rsites) {
+				dm_list_iterate_items(rdev, &rsite->rdevices) {
+					if (lv == rdev->lv || lv == rdev->slog)
+						seg_found++;
+				}
+			}
+			if (lv == seg->replicator)
+				seg_found++;
+		}
+		if (seg_is_replicator(seg) && lv == seg->rlog_lv)
+				seg_found++;
 		if (seg->log_lv == lv)
 			seg_found++;
 		if (!seg_found) {

lib/metadata/metadata-exported.h +73 -1

110 lines not on this page · raw message

lib/metadata/replicator_manip.c new file +340 -0

340 lines not on this page · raw message

lib/metadata/segtype.h +8 -0

index 7b31c1d..d352928 100644
@@ -35,9 +35,13 @@ struct dev_manager;
 #define SEG_VIRTUAL		0x00000020U
 #define SEG_CANNOT_BE_ZEROED	0x00000040U
 #define SEG_MONITORED		0x00000080U
+#define SEG_REPLICATOR		0x00000100U
+#define SEG_REPLICATOR_DEV	0x00000200U
 #define SEG_UNKNOWN		0x80000000U
 
 #define seg_is_mirrored(seg)	((seg)->segtype->flags & SEG_AREAS_MIRRORED ? 1 : 0)
+#define seg_is_replicator(seg)	((seg)->segtype->flags & SEG_REPLICATOR ? 1 : 0)
+#define seg_is_replicator_dev(seg) ((seg)->segtype->flags & SEG_REPLICATOR_DEV ? 1 : 0)
 #define seg_is_striped(seg)	((seg)->segtype->flags & SEG_AREAS_STRIPED ? 1 : 0)
 #define seg_is_snapshot(seg)	((seg)->segtype->flags & SEG_SNAPSHOT ? 1 : 0)
 #define seg_is_virtual(seg)	((seg)->segtype->flags & SEG_VIRTUAL ? 1 : 0)

@@ -110,6 +114,10 @@ struct segment_type *init_error_segtype(struct cmd_context *cmd);
 struct segment_type *init_free_segtype(struct cmd_context *cmd);
 struct segment_type *init_unknown_segtype(struct cmd_context *cmd, const char *name);
 
+#ifdef REPLICATOR_INTERNAL
+int init_replicator_segtype(struct segtype_library *seglib);
+#endif
+
 #ifdef SNAPSHOT_INTERNAL
 struct segment_type *init_snapshot_segtype(struct cmd_context *cmd);
 #endif

lib/replicator/.exported_symbols new file +1 -0

new file mode 100644
index 0000000..1c92c6a
@@ -0,0 +1 @@
+init_segtype

lib/replicator/Makefile.in new file +25 -0

new file mode 100644
index 0000000..ac42730
@@ -0,0 +1,25 @@
+#
+# Copyright (C) 2009-2010 Red Hat, Inc. All rights reserved.
+#
+# This file is part of LVM2.
+#
+# This copyrighted material is made available to anyone wishing to use,
+# modify, copy, or redistribute it subject to the terms and conditions
+# of the GNU General Public License v.2.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+srcdir = @srcdir@
+top_srcdir = @top_srcdir@
+top_builddir = @top_builddir@
+
+SOURCES = replicator.c
+
+LIB_SHARED = liblvm2replicator.$(LIB_SUFFIX)
+LIB_VERSION = $(LIB_VERSION_LVM)
+
+include $(top_builddir)/make.tmpl
+
+install: install_lib_shared_plugin

lib/replicator/replicator.c new file +794 -0

794 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 14:10 UTC #14 lore raw

PATCH 5/22 Replicator: add lvm support

Quoted 14 lines · Zdenek Kabelac

Adding configure.in support for Replicators. Adding basic lib lvm support for Replicators. Adding flags REPLICATOR and REPLICATOR_LOG. Adding segments SEG_REPLICATOR and SEG_REPLICATOR_DEV. Adding basic methods for handling replicator metadata.
+++ b/lib/metadata/replicator_manip.c
lib/metadata/replicator_manip.c new file +6 -0
+#include <inttypes.h>
+#include <sys/types.h>
+#include "libdevmapper.h"
+#include "lvm-types.h"
+
+#include "activate.h"

lib/metadata/replicator_manip.c line 30

Quoted 10 lines

lib/metadata/replicator_manip.c new file +10 -0

+#include "archiver.h"
+#include "defaults.h"
+#include "label.h"
+#include "lib.h"
+#include "locking.h"
+#include "lv_alloc.h"
+#include "lvm-string.h"
+#include "metadata.h"
+#include "segtype.h"
+#include "toolcontext.h"

Please clean up those includes: First one is required to be "lib.h" and then you'll find several of the others listed are already included!

Alasdair

Zdenek Kabelac

author

2010-04-12 15:21 UTC #15 lore raw

PATCH 6/22 Replicator: check replicator segment

In reply to Zdenek Kabelac

Check for possible problems within replicator structures. Used also by vg_validate.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/merge.c             |    3 +
 lib/metadata/metadata-exported.h |    1 +
 lib/metadata/replicator_manip.c  |  152 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 156 insertions(+), 0 deletions(-)

lib/metadata/merge.c +3 -0

index 66ff57b..9a34daf 100644
@@ -137,6 +137,9 @@ int check_lv_segments(struct logical_volume *lv, int complete_vg)
 			}
 		}
 
+		if (seg_is_replicator(seg) && !replicator_check_segment(seg))
+			inc_error_count;
+
 		for (s = 0; s < seg->area_count; s++) {
 			if (seg_type(seg, s) == AREA_UNASSIGNED) {
 				log_error("LV %s: segment %u has unassigned "

lib/metadata/metadata-exported.h +1 -0

index 7b6c7fa..6d40365 100644
@@ -767,6 +767,7 @@ struct logical_volume *replicator_dev_remove_slog(struct replicator_device *rdev
 int replicator_dev_add_rimage(struct replicator_device *rdev, struct logical_volume *lv);
 struct logical_volume *replicator_dev_remove_rimage(struct replicator_device *rdev);
 
+int replicator_check_segment(const struct lv_segment *replicator_seg);
 int lv_is_active_replicator_dev(const struct logical_volume *lv);
 int lv_is_replicator(const struct logical_volume *lv);
 int lv_is_replicator_dev(const struct logical_volume *lv);

lib/metadata/replicator_manip.c +152 -0

158 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #16 lore raw

PATCH 7/22 Replicator: extend _lv_each_dependency() rep. deps

In reply to Zdenek Kabelac

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/metadata.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

lib/metadata/metadata.c +4 -0

index a4f1948..dcbaeff 100644
@@ -1976,6 +1976,8 @@ static int _lv_each_dependency(struct logical_volume *lv,
 	struct lv_segment *lvseg;
 
 	struct logical_volume *deps[] = {
+		(lv->rdevice && lv != lv->rdevice->lv) ? lv->rdevice->lv : 0,
+		(lv->rdevice && lv != lv->rdevice->slog) ? lv->rdevice->slog : 0,
 		lv->snapshot ? lv->snapshot->origin : 0,
 		lv->snapshot ? lv->snapshot->cow : 0 };
 	for (i = 0; i < sizeof(deps) / sizeof(*deps); ++i) {

@@ -1986,6 +1988,8 @@ static int _lv_each_dependency(struct logical_volume *lv,
 	dm_list_iterate_items(lvseg, &lv->segments) {
 		if (lvseg->log_lv && !fn(lvseg->log_lv, data))
 			return_0;
+		if (lvseg->rlog_lv && !fn(lvseg->rlog_lv, data))
+			return_0;
 		for (s = 0; s < lvseg->area_count; ++s) {
 			if (seg_type(lvseg, s) == AREA_LV && !fn(seg_lv(lvseg,s), data))
 				return_0;

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #17 lore raw

PATCH 8/22 Replicator: extend volume_group with list of VGs

In reply to Zdenek Kabelac

Add linked list of opened VGs. List keeps the information about needed and locked and opened VGs for replicator target. Also add missing_vgs status information for quick check and also for possible continuos process_each_lv() where we need to detect whether failure has been caused by missing VG or some other reason.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/metadata-exported.h |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

lib/metadata/metadata-exported.h +4 -0

index 6d40365..9c73dba 100644
@@ -250,6 +250,10 @@ struct volume_group {
 
 	struct dm_list tags;
 
+        /* List of wanted/locked and opened VGs */
+	struct dm_list vgs;
+	uint32_t missing_vgs;
+
 	/*
 	 * FIXME: Move the next fields into a different struct?
 	 */

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 14:26 UTC #18 lore raw

PATCH 8/22 Replicator: extend volume_group with list of VGs

Quoted 12 lines · Zdenek Kabelac

Add linked list of opened VGs. List keeps the information about needed and locked and opened VGs for replicator target. Also add missing_vgs status information for quick check and also for possible continuos process_each_lv() where we need to detect whether failure has been caused by missing VG or some other reason.
lib/metadata/metadata-exported.h +0 -0
index 6d40365..9c73dba 100644

lib/metadata/metadata-exported.h struct volume_group { · line 252

Quoted 3 lines

lib/metadata/metadata-exported.h +3 -0

@@ -250,6 +250,10 @@ struct volume_group {
+        /* List of wanted/locked and opened VGs */
+	struct dm_list vgs;
+	uint32_t missing_vgs;

Can we think up more-specific names?

vg->vgs isn't very informative.

Alasdair

Zdenek Kabelac

author

2010-04-26 14:39 UTC #19 lore raw

PATCH 8/22 Replicator: extend volume_group with list of VGs

Dne 26.4.2010 16:26, Alasdair G Kergon napsal(a):

Quoted 19 lines · Zdenek Kabelac

Quoted 12 lines
Add linked list of opened VGs. List keeps the information about needed and locked and opened VGs for replicator target. Also add missing_vgs status information for quick check and also for possible continuos process_each_lv() where we need to detect whether failure has been caused by missing VG or some other reason.
lib/metadata/metadata-exported.h +0 -0
index 6d40365..9c73dba 100644
lib/metadata/metadata-exported.h struct volume_group { · line 252
Quoted 3 lines
lib/metadata/metadata-exported.h +3 -0
@@ -250,6 +250,10 @@ struct volume_group {
+        /* List of wanted/locked and opened VGs */
+	struct dm_list vgs;
+	uint32_t missing_vgs;
Can we think up more-specific names?

Quoted 2 lines

vg->vgs isn't very informative.

In fact I've had there some different names - like remote_vgs, needed_vgs, deps_vgs....
but I could not decide - and as we now already have pvs, lvs, vgs seems to
be natural extension of naming concept.

vgs - is list of vgnames and flags for openning, including the vg itself - thus remote_vgs doesn't fit. Needed - seems plain ugly.

So I'm open for new names here - note there are some manipulation functions which should also use this name.

Zdenek

Alasdair G Kergon

2010-04-26 14:27 UTC #20 lore raw

PATCH 8/22 Replicator: extend volume_group with list of VGs

In reply to Zdenek Kabelac

Also initialise the new fields and add to vg_validate as part of this patch, to ensure internal data consistency.

Alasdair

Zdenek Kabelac

author

2010-04-26 14:35 UTC #21 lore raw

PATCH 8/22 Replicator: extend volume_group with list of VGs

Dne 26.4.2010 16:27, Alasdair G Kergon napsal(a):

Quoted 2 lines

Also initialise the new fields and add to vg_validate as part of this patch, to ensure internal data consistency.

My plan here is to introduce one common routine for creation of volume_group structure - not yet finished - so it should be handled there.

For now there is 'lazy' initialization of fields before it's usage - of course, requires to not touch these fields on unexpected occasions. This should be fixed once the common routine will be there.

Zdenek

Zdenek Kabelac

author

2010-04-12 15:21 UTC #22 lore raw

PATCH 9/22 Replicator: add sorted vgs_list

In reply to Zdenek Kabelac

Introduce struct vgs_list to store information about needed volume group name, vgid, flags and the pointer to opened VG.

Keep VGs list in alphabetical order for locking order.

Introduce functions:
vgs_list_add() add new vgs_list entry. vgs_list_lookup() search for vgs_list for vg_name. vgs_list_read() open list of VGs. vgs_list_release() close VGs in reversed order.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/metadata-exported.h |   20 ++++++
 lib/metadata/replicator_manip.c  |  138 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 158 insertions(+), 0 deletions(-)

lib/metadata/metadata-exported.h +20 -0

index 9c73dba..0f00d7a 100644
@@ -288,6 +288,15 @@ struct lv_segment_area {
 
 struct segment_type;
 
+/* List with vg_name, vgid and flags */
+struct vgs_list {
+	struct dm_list list;
+	const char *vg_name;
+	const char *vgid;
+	uint32_t flags;
+	struct volume_group *vg;
+};
+
 /* ++ Replicator datatypes */
 typedef enum {
 	REPLICATOR_STATE_PASSIVE,

@@ -780,6 +789,17 @@ int lv_is_rlog(const struct logical_volume *lv);
 int lv_is_slog(const struct logical_volume *lv);
 struct logical_volume *first_replicator_dev(const struct logical_volume *lv);
 /* --  metadata/replicator_manip.c */
+struct vgs_list *vgs_list_add(struct dm_pool *mem,
+			      struct dm_list *vg_nlist,
+			      const char *vg_name, const char *vgid,
+			      uint32_t flags);
+struct vgs_list *vgs_list_lookup(struct dm_list *vg_nlist,
+				 const char *vg_name);
+
+int vgs_list_read(struct cmd_context *cmd, struct dm_list *vgs_list,
+		  struct vgs_list *vnl_vg);
+void vgs_list_release(struct dm_list *vgs_list,
+		      struct vgs_list *vnl_vg);
 
 struct logical_volume *find_pvmove_lv(struct volume_group *vg,
 				      struct device *dev, uint32_t lv_type);

lib/metadata/replicator_manip.c +138 -0

141 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 14:34 UTC #23 lore raw

PATCH 9/22 Replicator: add sorted vgs_list

Quoted 2 lines · Zdenek Kabelac

Introduce struct vgs_list to store information about needed volume group name, vgid, flags and the pointer to opened VG.

Why does this need a new structure?
How does this interact with internal caches?
How does this interact with 'struct volume_group' itself? (The fields are already part of it so why is a new struct needed?)

Alasdair

Zdenek Kabelac

author

2010-04-26 14:56 UTC #24 lore raw

PATCH 9/22 Replicator: add sorted vgs_list

Dne 26.4.2010 16:34, Alasdair G Kergon napsal(a):

Quoted 5 lines · Zdenek Kabelac

Quoted 2 lines
Introduce struct vgs_list to store information about needed volume group name, vgid, flags and the pointer to opened VG.
Why does this need a new structure?

We don't have any similar one - I would be aware of.

Using new structure here seems to simplify modification of routines to be able to process some loops multiple times - i.e. code changes are minimal to extend it for work with remote VGs.

Quoted 1 line

How does this interact with internal caches?

There is no interaction - VGs are opened in alphabetical order via standard lvm mechanism - there are no peeks into internal caches.

Technically we could find names in caches - but we should also keep locks for all VGs during activation so noone changes them.

Quoted 2 lines

How does this interact with 'struct volume_group' itself? (The fields are already part of it so why is a new struct needed?)

We keep list of needed VGs outside in command - once we open 'master' VG - list is moved (spliced) to internal VG list - then it's filled with potentially missing 'other VGs' - later - if there was found problem in processing caused by missing VG - list is taken out of VG - and in alphabetical order opened again. And this loop iterates until either all missing VGs are satisfied or some VG couldn't be opened -> failing resource. This is a bit tricky - but we do not need to put things to global structure like cmd_context in this case.

Zdenek

Dave Wysochanski

2010-04-26 18:01 UTC #25 lore raw

PATCH 9/22 Replicator: add sorted vgs_list

Quoted 9 lines · Zdenek Kabelac

Dne 26.4.2010 16:34, Alasdair G Kergon napsal(a):
Quoted 5 lines · Zdenek Kabelac
Quoted 2 lines
Introduce struct vgs_list to store information about needed volume group name, vgid, flags and the pointer to opened VG.
Why does this need a new structure?
We don't have any similar one - I would be aware of.

It will likely also be very useful for certain library operations that involve multiple VGs (vgsplit, vgmerge) - I started looking at refactoring the vgsplit/vgmerge code to use this new structure (or something very close to it).

Zdenek Kabelac

author

2010-04-12 15:21 UTC #26 lore raw

PATCH 10/22 Replicator: add find_replicator_vgs

In reply to Zdenek Kabelac

Adding function find_replicator_vgs() to find all needed VGs for replicator.

This function is later called before taking lock_vol().

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/metadata-exported.h |    2 ++
 lib/metadata/replicator_manip.c  |   32 ++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 0 deletions(-)

lib/metadata/metadata-exported.h +2 -0

index 0f00d7a..1ff8986 100644
@@ -801,6 +801,8 @@ int vgs_list_read(struct cmd_context *cmd, struct dm_list *vgs_list,
 void vgs_list_release(struct dm_list *vgs_list,
 		      struct vgs_list *vnl_vg);
 
+int find_replicator_vgs(struct logical_volume *lv);
+
 struct logical_volume *find_pvmove_lv(struct volume_group *vg,
 				      struct device *dev, uint32_t lv_type);
 struct logical_volume *find_pvmove_lv_from_pvname(struct cmd_context *cmd,

lib/metadata/replicator_manip.c +32 -0

index 8b5d8fe..208efea 100644
@@ -628,3 +628,35 @@ void vgs_list_release(struct dm_list *list, struct vgs_list *vgs_vg)
 		vgsl->vg = NULL;
 	}
 }
+
+/**
+ * Find all needed remote VGs for processing given LV.
+ * Missing VGs are added to VG's vgs list.
+ */
+int find_replicator_vgs(struct logical_volume *lv)
+{
+	struct replicator_site *rsite;
+	int ret = 1;
+
+	if (!lv_is_replicator_dev(lv))
+		return 1;
+
+	dm_list_iterate_items(rsite, &first_seg(lv)->replicator->rsites) {
+		if (!rsite->vg_name ||
+		    vgs_list_lookup(&lv->vg->vgs, rsite->vg_name))
+			continue;
+		ret = 0;
+		/* Using cmd memory pool for list allocation */
+		if (!vgs_list_add(lv->vg->cmd->mem, &lv->vg->vgs,
+				      rsite->vg_name, NULL, 0)) {
+			lv->vg->missing_vgs = 0; /* do not retry */
+			stack;
+			break;
+		}
+
+		log_debug("VG: %s added as missing.", rsite->vg_name);
+		lv->vg->missing_vgs++;
+	}
+
+	return ret;
+}

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #27 lore raw

PATCH 11/22 Replicator: add read and release VGs for rsites

In reply to Zdenek Kabelac

Add function to read and release remote VGs for replicator sites in activation context.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/metadata-exported.h |    3 ++
 lib/metadata/replicator_manip.c  |   56 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 0 deletions(-)

lib/metadata/metadata-exported.h +3 -0

index 1ff8986..d0c86d9 100644
@@ -803,6 +803,9 @@ void vgs_list_release(struct dm_list *vgs_list,
 
 int find_replicator_vgs(struct logical_volume *lv);
 
+int lv_read_replicator_vgs(struct logical_volume *lv);
+void lv_release_replicator_vgs(struct logical_volume *lv);
+
 struct logical_volume *find_pvmove_lv(struct volume_group *vg,
 				      struct device *dev, uint32_t lv_type);
 struct logical_volume *find_pvmove_lv_from_pvname(struct cmd_context *cmd,

lib/metadata/replicator_manip.c +56 -0

59 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #28 lore raw

PATCH 12/22 Replicator: use vgs_list for _process_one_vg()

In reply to Zdenek Kabelac

Patch modifes behavior of _process_one_vg().

In the first pass the plain vg_read() for the given VG is used. Replicator collects sorted list of additional VGs during lock_vol(). If any VG needed by replicator is not yet opened then next iteration loop is taken with collected VGs.

Flag vg->missing_vgs detects missing VGs.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 tools/toollib.c |   56 ++++++++++++++++++++++++++++++++----------------------
 1 files changed, 33 insertions(+), 23 deletions(-)

tools/toollib.c +33 ~~-23~~

index c31315c..2c5ba63 100644
@@ -436,37 +436,47 @@ static int _process_one_vg(struct cmd_context *cmd, const char *vg_name,
 						  struct volume_group * vg,
 						  void *handle))
 {
-	struct volume_group *vg;
+	struct dm_list vgs_list;
+	struct vgs_list *vgs_vg;
 	int ret = 0;
 
 	log_verbose("Finding volume group \"%s\"", vg_name);
 
-	vg = vg_read(cmd, vg_name, vgid, flags);
-	/* Allow FAILED_INCONSISTENT through only for vgcfgrestore */
-	if (vg_read_error(vg) &&
-	    !((vg_read_error(vg) == FAILED_INCONSISTENT) &&
-	      (flags & READ_ALLOW_INCONSISTENT))) {
-		ret_max = ECMD_FAILED;
-		goto_out;
-	}
+	dm_list_init(&vgs_list);
+	vgs_vg = vgs_list_add(cmd->mem, &vgs_list, vg_name, vgid, flags);
+	if (!vgs_vg)
+		return_0;
+
+	while (!sigint_caught()) {
+		if (!vgs_list_read(cmd, &vgs_list, vgs_vg))
+			/* Allow FAILED_INCONSISTENT through only for vgcfgrestore */
+			if (vg_read_error(vgs_vg->vg) &&
+			    (!((flags & READ_ALLOW_INCONSISTENT) &&
+			       (vg_read_error(vgs_vg->vg) == FAILED_INCONSISTENT)))) {
+				ret = ECMD_FAILED;
+				break;
+			}
+
+		if (!dm_list_empty(tags) &&
+		    /* Only process if a tag matches or it's on arg_vgnames */
+		    !str_list_match_item(arg_vgnames, vg_name) &&
+		    !str_list_match_list(tags, &vgs_vg->vg->tags))
+			break;
 
-	if (!dm_list_empty(tags)) {
-		/* Only process if a tag matches or it's on arg_vgnames */
-		if (!str_list_match_item(arg_vgnames, vg_name) &&
-		    !str_list_match_list(tags, &vg->tags))
-			goto out;
+		ret = process_single(cmd, vg_name, vgs_vg->vg, handle);
+
+		if (vg_read_error(vgs_vg->vg)) /* FAILED_INCONSISTENT */
+			break;
+
+		if (!vgs_vg->vg->missing_vgs)
+			break;
+
+		vgs_list_release(&vgs_list, vgs_vg);
 	}
 
-	if ((ret = process_single(cmd, vg_name, vg,
-				  handle)) > ret_max)
-		ret_max = ret;
+	vgs_list_release(&vgs_list, vgs_vg);
 
-out:
-	if (vg_read_error(vg))
-		vg_release(vg);
-	else
-		unlock_and_release_vg(cmd, vg, vg_name);
-	return ret_max;
+	return (ret > ret_max) ? ret : ret_max;
 }
 
 int process_each_vg(struct cmd_context *cmd, int argc, char **argv,

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #29 lore raw

PATCH 13/22 Replicator: use vgs_list for process_each_lv_in_vg()

In reply to Zdenek Kabelac

As for _process_one_vg() we need similar retry loop for process_each_lv_in_vg(). This patch retries to process failed LVs with reopened VGs.

Patch does not add any extra repeated invocations if there is not any missing VG during LV processing.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 tools/toollib.c |   47 +++++++++++++++++++++++++++++++++++++----------
 1 files changed, 37 insertions(+), 10 deletions(-)

tools/toollib.c +37 ~~-10~~

index 2c5ba63..598710b 100644
@@ -151,6 +151,7 @@ int process_each_lv_in_vg(struct cmd_context *cmd,
 		if (!process_lv)
 			continue;
 
+		lvl->lv->vg->missing_vgs = 0;
 		ret = process_single(cmd, lvl->lv, handle);
 		if (ret != ECMD_PROCESSED && failed_lvs) {
 			if (!str_list_add(cmd->mem, failed_lvs,

@@ -159,6 +160,8 @@ int process_each_lv_in_vg(struct cmd_context *cmd,
 				log_error("Allocation failed for str_list.");
 				return ECMD_FAILED;
 			}
+			if (lvl->lv->vg->missing_vgs)
+				ret = ECMD_PROCESSED;
 		}
 		if (ret > ret_max)
 			ret_max = ret;

@@ -188,7 +191,9 @@ int process_each_lv(struct cmd_context *cmd, int argc, char **argv,
 	struct dm_list *tags_arg;
 	struct dm_list *vgnames;	/* VGs to process */
 	struct str_list *sll, *strl;
-	struct volume_group *vg;
+	struct vgs_list *vgs_vg;
+	struct dm_list vgs_list;
+	struct dm_list failed_lvs;
 	struct dm_list tags, lvnames;
 	struct dm_list arg_lvnames;	/* Cmdline vgname or vgname/lvname */
 	char *vglv;

@@ -198,6 +203,7 @@ int process_each_lv(struct cmd_context *cmd, int argc, char **argv,
 
 	dm_list_init(&tags);
 	dm_list_init(&arg_lvnames);
+	dm_list_init(&failed_lvs);
 
 	if (argc) {
 		struct dm_list arg_vgnames;

@@ -293,13 +299,17 @@ int process_each_lv(struct cmd_context *cmd, int argc, char **argv,
 		}
 	}
 
-	vg = NULL;
 	dm_list_iterate_items(strl, vgnames) {
 		vgname = strl->str;
-		vg = vg_read(cmd, vgname, NULL, flags);
+		dm_list_init(&vgs_list);
+		if (!(vgs_vg = vgs_list_add(cmd->mem, &vgs_list,
+					    vgname, NULL, flags))) {
+			stack;
+			return ECMD_FAILED;
+		}
 
-		if (vg_read_error(vg)) {
-			vg_release(vg);
+		if (!vgs_list_read(cmd, &vgs_list, vgs_vg)) {
+			vgs_list_release(&vgs_list, vgs_vg);
 			if (ret_max < ECMD_FAILED) {
 				log_error("Skipping volume group %s", vgname);
 				ret_max = ECMD_FAILED;

@@ -325,19 +335,36 @@ int process_each_lv(struct cmd_context *cmd, int argc, char **argv,
 						  dm_pool_strdup(cmd->mem,
 							      lv_name + 1))) {
 					log_error("strlist allocation failed");
-					unlock_and_release_vg(cmd, vg, vgname);
+					vgs_list_release(&vgs_list,vgs_vg);
 					return ECMD_FAILED;
 				}
 			}
 		}
 
-		ret = process_each_lv_in_vg(cmd, vg, &lvnames, tags_arg,
-					    NULL, handle, process_single);
-		unlock_and_release_vg(cmd, vg, vgname);
+		while (!sigint_caught()) {
+			ret = process_each_lv_in_vg(cmd, vgs_vg->vg, &lvnames,
+						    tags_arg, &failed_lvs,
+						    handle, process_single);
+			if (ret != ECMD_PROCESSED ||
+			    dm_list_empty(&failed_lvs))
+				break;
+
+			/* Try again with failed LVs in this VG */
+			dm_list_init(&lvnames);
+			dm_list_splice(&failed_lvs, &lvnames);
+
+			vgs_list_release(&vgs_list, vgs_vg);
+			if (!vgs_list_read(cmd, &vgs_list, vgs_vg)) {
+				ret = ECMD_FAILED; /* break */
+				break;
+			}
+		}
 		if (ret > ret_max)
 			ret_max = ret;
+
+		vgs_list_release(&vgs_list, vgs_vg);
 		if (sigint_caught())
-			break;
+			return ECMD_FAILED;
 	}
 
 	return ret_max;

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #30 lore raw

PATCH 14/22 Replicator: lock_vol() finds missing VGs

In reply to Zdenek Kabelac

Find and check for all needed VGs before calling lock_vol().

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/locking/locking.h |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

lib/locking/locking.h +3 -1

index e7f25b9..8f6f52b 100644
@@ -129,7 +129,9 @@ int check_lvm1_vg_inactive(struct cmd_context *cmd, const char *vgname);
 	(vg_is_clustered((lv)->vg) ? LCK_CLUSTER_VG : 0)
 
 #define lock_lv_vol(cmd, lv, flags)	\
-	lock_vol(cmd, (lv)->lvid.s, flags | LCK_LV_CLUSTERED(lv))
+	(find_replicator_vgs((lv)) ? \
+		lock_vol(cmd, (lv)->lvid.s, flags | LCK_LV_CLUSTERED(lv)) : \
+		0)
 
 #define unlock_vg(cmd, vol)	lock_vol(cmd, vol, LCK_VG_UNLOCK)
 #define unlock_and_release_vg(cmd, vg, vol) \

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #31 lore raw

PATCH 15/22 Replicator: activate checks for missing vgs

In reply to Zdenek Kabelac

Activation needs to have all remote VGs locked to pass for replicator.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/activate/activate.c |   22 ++++++++++++++++++----
 1 files changed, 18 insertions(+), 4 deletions(-)

lib/activate/activate.c +18 -4

index 1844cea..4aabc38 100644
@@ -865,6 +865,9 @@ static int _lv_suspend(struct cmd_context *cmd, const char *lvid_s,
 		goto out;
 	}
 
+	if (!lv_read_replicator_vgs(lv))
+		goto_out;
+
 	lv_calculate_readahead(lv, NULL);
 
 	/* If VG was precommitted, preload devices for the LV */

@@ -894,9 +897,10 @@ static int _lv_suspend(struct cmd_context *cmd, const char *lvid_s,
 out:
 	if (lv_pre)
 		vg_release(lv_pre->vg);
-	if (lv)
+	if (lv) {
+		lv_release_replicator_vgs(lv);
 		vg_release(lv->vg);
-
+	}
 	return r;
 }

@@ -1025,6 +1029,9 @@ int lv_deactivate(struct cmd_context *cmd, const char *lvid_s)
 			goto_out;
 	}
 
+	if (!lv_read_replicator_vgs(lv))
+		goto_out;
+
 	lv_calculate_readahead(lv, NULL);
 
 	if (!monitor_dev_for_events(cmd, lv, 0))

@@ -1038,8 +1045,10 @@ int lv_deactivate(struct cmd_context *cmd, const char *lvid_s)
 	if (!lv_info(cmd, lv, &info, 1, 0) || info.exists)
 		r = 0;
 out:
-	if (lv)
+	if (lv) {
+		lv_release_replicator_vgs(lv);
 		vg_release(lv->vg);
+	}
 
 	return r;
 }

@@ -1118,6 +1127,9 @@ static int _lv_activate(struct cmd_context *cmd, const char *lvid_s,
 		goto out;
 	}
 
+	if (!lv_read_replicator_vgs(lv))
+		goto_out;
+
 	lv_calculate_readahead(lv, NULL);
 
 	if (exclusive)

@@ -1133,8 +1145,10 @@ static int _lv_activate(struct cmd_context *cmd, const char *lvid_s,
 		stack;
 
 out:
-	if (lv)
+	if (lv) {
+		lv_release_replicator_vgs(lv);
 		vg_release(lv->vg);
+	}
 
 	return r;
 }

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 14:39 UTC #32 lore raw

PATCH 15/22 Replicator: activate checks for missing vgs

lib/activate/activate.c static int _lv_suspend(struct cmd_context *cmd, const char *lvid_s, · line 902

Quoted 6 lines · Zdenek Kabelac

Activation needs to have all remote VGs locked to pass for replicator.
lib/activate/activate.c +2 -1
-	if (lv)
+	if (lv) {
+		lv_release_replicator_vgs(lv);
 		vg_release(lv->vg);

Perhaps vg_release() should be sufficient here on its own?
- If other VGs are linked of it, they must necessarily also be released without the caller needing to specify that explicitly?

Alasdair

Zdenek Kabelac

author

2010-04-12 15:21 UTC #33 lore raw

PATCH 16/22 Replicator: update _create_partial_dtree()

In reply to Zdenek Kabelac

Adding function _create_partial_replicator() to create partial tree for Replicator target.

Using dm_tree_set_suspend_priority() for Replicator.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/activate/dev_manager.c |   75 ++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 75 insertions(+), 0 deletions(-)

lib/activate/dev_manager.c +75 -0

index dcd5000..c904858 100644
@@ -867,6 +867,75 @@ static int _add_lv_to_dtree(struct dev_manager *dm, struct dm_tree *dtree, struc
 	return 1;
 }
 
+/*
+ * Add devices from replicator
+ * use _add_dev_to_dtree() directly instead of _add_lv_to_dtree()
+ * no need to extend its check with  _slog, _rlog, _rimage
+ */
+static int _create_partial_replicator(struct dev_manager *dm,
+				      struct dm_tree *dtree,
+				      struct lv_segment *seg)
+{
+	struct replicator_device *rdev;
+	struct replicator_site *rsite;
+	const char *replicator_dlid;
+
+	if (!lv_is_active_replicator_dev(seg->lv)) {
+		if (!_add_dev_to_dtree(dm, dtree, seg->lv->rdevice->lv,
+				       NULL))
+			return_0;
+		return 1;
+	}
+
+	/* Add all related devs for replicator */
+	dm_list_iterate_items(rsite, &seg->replicator->rsites) {
+		dm_list_iterate_items(rdev, &rsite->rdevices) {
+			if (rsite->state == REPLICATOR_STATE_ACTIVE) {
+				/* Add _rimage LV */
+				if (rdev->lv &&
+				    !_add_dev_to_dtree(dm, dtree,
+						       rdev->lv, NULL))
+					return_0;
+
+				/* Add replicator-dev LV */
+				if (seg->lv != rdev->replicator_dev->lv &&
+				    !_add_dev_to_dtree(dm, dtree,
+						       rdev->replicator_dev->lv,
+						       NULL))
+					return_0;
+			}
+
+			if (!rdev->rsite->vg_name)
+				continue;
+
+			if (rdev->lv &&
+			    !_add_dev_to_dtree(dm, dtree, rdev->lv, NULL))
+				return_0;
+
+			if (rdev->slog &&
+			    !_add_dev_to_dtree(dm, dtree, rdev->slog, NULL))
+				return_0;
+
+		}
+	}
+
+	/* Add rlog and replicator devices */
+	if (first_seg(seg->replicator)->rlog_lv &&
+	    !_add_dev_to_dtree(dm, dtree,
+			       first_seg(seg->replicator)->rlog_lv, NULL))
+		return_0;
+	if (!_add_dev_to_dtree(dm, dtree, seg->replicator, NULL))
+		return_0;
+
+	/* Set suspend priority for replicator device */
+	if (!(replicator_dlid = build_dlid(dm, seg->replicator->lvid.s, NULL)))
+		return_0;
+	if (!dm_tree_set_suspend_priority(dtree, replicator_dlid, 1))
+		return_0;
+
+	return 1;
+}
+
 static struct dm_tree *_create_partial_dtree(struct dev_manager *dm, struct logical_volume *lv)
 {
 	struct dm_tree *dtree;

@@ -887,6 +956,12 @@ static struct dm_tree *_create_partial_dtree(struct dev_manager *dm, struct logi
 		if (!_add_lv_to_dtree(dm, dtree, dm_list_struct_base(snh, struct lv_segment, origin_list)->cow))
 			goto_bad;
 
+	/* Add any replicator-dev LVs of this LV */
+	dm_list_iterate_items(seg, &lv->segments)
+		if (seg_is_replicator_dev(seg) &&
+		    !_create_partial_replicator(dm, dtree, seg))
+			goto_bad;
+
 	/* Add any LVs used by segments in this LV */
 	dm_list_iterate_items(seg, &lv->segments)
 		for (s = 0; s < seg->area_count; s++)

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #34 lore raw

PATCH 17/22 Replicator: vg with missing_vgs does not print

In reply to Zdenek Kabelac

If missed VGs do not print message and wait for retry.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 tools/vgchange.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

tools/vgchange.c +3 -2

index 81c95d1..68fac86 100644
@@ -224,8 +224,9 @@ static int _vgchange_available(struct cmd_context *cmd, struct volume_group *vg)
 
 	ret = _activate_lvs_in_vg(cmd, vg, available);
 
-	log_print("%d logical volume(s) in volume group \"%s\" now active",
-		  lvs_in_vg_activated(vg), vg->name);
+	if (!vg->missing_vgs)
+		log_print("%d logical volume(s) in volume group \"%s\" now active",
+			  lvs_in_vg_activated(vg), vg->name);
 	return ret;
 }

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #35 lore raw

PATCH 18/22 Replicator: add replicator to dtree

In reply to Zdenek Kabelac

Adding all replicator related LVs to dtree.
Start of one replicator_dev initiate start of all other related.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/activate/dev_manager.c |   60 +++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 59 insertions(+), 1 deletions(-)

lib/activate/dev_manager.c +59 -1

index c904858..cf39c58 100644
@@ -1173,6 +1173,61 @@ static int _add_target_to_dtree(struct dev_manager *dm,
 static int _add_new_lv_to_dtree(struct dev_manager *dm, struct dm_tree *dtree,
 				  struct logical_volume *lv, const char *layer);
 
+/* Add all replicators' LVs from active segment */
+static int _add_replicator_dev_target_to_dtree(struct dev_manager *dm,
+					       struct dm_tree *dtree,
+					       struct lv_segment *seg)
+{
+	struct replicator_device *rdev;
+	struct replicator_site *rsite;
+
+	/* Only active replicator */
+	if (!lv_is_active_replicator_dev(seg->lv)) {
+		if (!_add_new_lv_to_dtree(dm, dtree, seg->lv->rdevice->lv, NULL))
+			return_0;
+		return 1;
+	}
+
+	/* Add rlog and replicator nodes */
+	if (!seg->replicator ||
+            !first_seg(seg->replicator)->rlog_lv ||
+	    !_add_new_lv_to_dtree(dm, dtree,
+				  first_seg(seg->replicator)->rlog_lv, NULL) ||
+	    !_add_new_lv_to_dtree(dm, dtree, seg->replicator, NULL))
+	    return_0;
+
+	/* Activation of one replicator_dev node activates all other nodes */
+	dm_list_iterate_items(rsite, &seg->replicator->rsites) {
+		dm_list_iterate_items(rdev, &rsite->rdevices) {
+			if (rdev->lv &&
+			    !_add_new_lv_to_dtree(dm, dtree, rdev->lv, NULL))
+				return_0;
+
+			if (rdev->slog &&
+			    !_add_new_lv_to_dtree(dm, dtree,
+						  rdev->slog, NULL))
+				return_0;
+		}
+	}
+	/* Add remaining replicator-dev nodes in the second loop
+	 * to avoid multiple retries for inserting all elements */
+	dm_list_iterate_items(rsite, &seg->replicator->rsites) {
+		if (rsite->state != REPLICATOR_STATE_ACTIVE)
+			continue;
+		dm_list_iterate_items(rdev, &rsite->rdevices) {
+			if (rdev->replicator_dev->lv == seg->lv)
+				continue;
+			if (!rdev->replicator_dev->lv ||
+			    !_add_new_lv_to_dtree(dm, dtree,
+						  rdev->replicator_dev->lv,
+						  NULL))
+				return_0;
+		}
+	}
+
+	return 1;
+}
+
 static int _add_segment_to_dtree(struct dev_manager *dm,
 				   struct dm_tree *dtree,
 				   struct dm_tree_node *dnode,

@@ -1203,9 +1258,12 @@ static int _add_segment_to_dtree(struct dev_manager *dm,
 	    !_add_new_lv_to_dtree(dm, dtree, seg->log_lv, NULL))
 		return_0;
 
+	if (seg_is_replicator_dev(seg)) {
+		if (!_add_replicator_dev_target_to_dtree(dm, dtree, seg))
+			return_0;
 	/* If this is a snapshot origin, add real LV */
 	/* If this is a snapshot origin + merging snapshot, add cow + real LV */
-	if (lv_is_origin(seg->lv) && !layer) {
+	} else if (lv_is_origin(seg->lv) && !layer) {
 		if (vg_is_clustered(seg->lv->vg)) {
 			log_error("Clustered snapshots are not yet supported");
 			return 0;

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #36 lore raw

PATCH 19/22 Replicator: do not remove of replicators' LVs

In reply to Zdenek Kabelac

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/metadata/lv_manip.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

lib/metadata/lv_manip.c +9 -0

index 43ca423..55040cc 100644
@@ -2230,6 +2230,15 @@ int lv_remove_single(struct cmd_context *cmd, struct logical_volume *lv,
 		return 0;
 	}
 
+	if (lv_is_replicator_dev(lv) ||
+	    lv_is_rimage(lv) ||
+	    lv_is_rlog(lv) ||
+	    lv_is_slog(lv)) {
+		log_error("Can't remove logical volume %s used by a "
+			  "replicator.", lv->name);
+		return 0;
+	}
+
 	if (lv->status & LOCKED) {
 		log_error("Can't remove locked LV %s", lv->name);
 		return 0;

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #37 lore raw

PATCH 20/22 Replicator: activate change for vgchange

In reply to Zdenek Kabelac

Activate only the first replicator-dev LV, that activates all other related LVs from Replicator.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 tools/vgchange.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

tools/vgchange.c +4 -0

index 68fac86..139a1a5 100644
@@ -104,6 +104,10 @@ static int _activate_lvs_in_vg(struct cmd_context *cmd,
 		if ((lv->status & MIRROR_IMAGE) || (lv->status & MIRROR_LOG))
 			continue;
 
+		/* Only request activation of the first replicator-dev LV */
+		if (lv_is_replicator_dev(lv) && (lv != first_replicator_dev(lv)))
+			continue;
+
 		/* Can't deactivate a pvmove LV */
 		/* FIXME There needs to be a controlled way of doing this */
 		if (((activate == CHANGE_AN) || (activate == CHANGE_ALN)) &&

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #38 lore raw

PATCH 21/22 Replicator: add new options for replicator

In reply to Zdenek Kabelac

New options: addsite, delsite, replicator,
replicatorlog, replicatorlogtype.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 lib/config/defaults.h            |    1 +
 lib/metadata/metadata-exported.h |    1 +
 tools/args.h                     |   13 +++++++++
 tools/commands.h                 |    7 +++++
 tools/lvcreate.c                 |   55 ++++++++++++++++++++++++++++++++++++++
 tools/lvmcmdline.c               |    9 ++++++
 tools/tools.h                    |    1 +
 7 files changed, 87 insertions(+), 0 deletions(-)

lib/config/defaults.h +1 -0

index 22a085d..4ef94ff 100644
@@ -108,6 +108,7 @@
 
 #define DEFAULT_STRIPE_FILLER "error"
 #define DEFAULT_MIRROR_REGION_SIZE 512	/* KB */
+#define DEFAULT_REPLICATOR_REGION_SIZE 512	/* KB */
 #define DEFAULT_INTERVAL 15
 
 #ifdef READLINE_SUPPORT

lib/metadata/metadata-exported.h +1 -0

index d0c86d9..fee76d1 100644
@@ -595,6 +595,7 @@ uint64_t extents_from_size(struct cmd_context *cmd, uint64_t size,
 struct lvcreate_params {
 	/* flags */
 	int snapshot; /* snap */
+	int replicator; /* replicator */
 	int zero; /* all */
 	int major; /* all */
 	int minor; /* all */

tools/args.h +13 -0

index 981f45f..4623ff6 100644
@@ -66,6 +66,19 @@ arg(virtualoriginsize_ARG, '\0', "virtualoriginsize", size_mb_arg, 0)
 arg(virtualsize_ARG, '\0', "virtualsize", size_mb_arg, 0)
 arg(noudevsync_ARG, '\0', "noudevsync", NULL, 0)
 arg(poll_ARG, '\0', "poll", yes_no_arg, 0)
+arg(replicator_ARG, '\0', "replicator", NULL, 0)
+arg(replicatordev_ARG, '\0', "replicatordev", NULL, 0)
+arg(replicatorlog_ARG, '\0', "replicatorlog", NULL, 0)
+arg(replicatorlogtype_ARG, '\0', "replicatorlogtype", NULL, 0)
+arg(addsite_ARG, '\0', "addsite", site_arg, 0)
+arg(delsite_ARG, '\0', "delsite", site_arg, 0)
+arg(fallbehinddata_ARG, '\0', "fallbehinddata", size_mb_arg, 0)
+arg(fallbehindios_ARG, '\0', "fallbehindios", int_arg, 0)
+arg(fallbehindtimeout_ARG, '\0', "fallbehindtimeout", int_arg, 0)
+arg(sitemode_ARG, '\0', "sitemode", string_arg, 0)
+arg(activesite_ARG, '\0', "activesite", string_arg, 0)
+arg(passivesite_ARG, '\0', "passivesite", string_arg, 0)
+arg(remotevolumegroup_ARG, '\0', "remotevolumegroup", string_arg, 0)
 
 /* Allow some variations */
 arg(resizable_ARG, '\0', "resizable", yes_no_arg, 0)

tools/commands.h +7 -0

index 9b611de..6b43d94 100644
@@ -62,10 +62,12 @@ xx(lvchange,
    "lvchange\n"
    "\t[-A|--autobackup y|n]\n"
    "\t[-a|--available [e|l]y|n]\n"
+   "\t[--addsite Site]\n"
    "\t[--addtag Tag]\n"
    "\t[--alloc AllocationPolicy]\n"
    "\t[-C|--contiguous y|n]\n"
    "\t[-d|--debug]\n"
+   "\t[--delsite Site]\n"
    "\t[--deltag Tag]\n"
    "\t[-f|--force]\n"
    "\t[-h|--help]\n"

@@ -90,6 +92,7 @@ xx(lvchange,
    ignorelockingfailure_ARG, ignoremonitoring_ARG, major_ARG, minor_ARG,
    monitor_ARG, noudevsync_ARG, partial_ARG, permission_ARG, persistent_ARG,
    poll_ARG, readahead_ARG, resync_ARG, refresh_ARG, addtag_ARG, deltag_ARG,
+   addsite_ARG, delsite_ARG, 
    test_ARG, yes_ARG)
 
 xx(lvconvert,

@@ -162,6 +165,9 @@ xx(lvcreate,
    "\t[-p|--permission {r|rw}]\n"
    "\t[-r|--readahead ReadAheadSectors|auto|none]\n"
    "\t[-R|--regionsize MirrorLogRegionSize]\n"
+   "\t[--replicator]\n"
+   "\t[--replicatorlog LogicalVolumeName]\n"
+   "\t[--replicatorlogtype ringbuffer]\n"
    "\t[-t|--test]\n"
    "\t[--type VolumeType]\n"
    "\t[-v|--verbose]\n"

@@ -200,6 +206,7 @@ xx(lvcreate,
    mirrorlog_ARG, mirrors_ARG, monitor_ARG, name_ARG, nosync_ARG, noudevsync_ARG,
    permission_ARG, persistent_ARG, readahead_ARG, regionsize_ARG, size_ARG,
    snapshot_ARG, stripes_ARG, stripesize_ARG, test_ARG, type_ARG,
+   replicator_ARG, replicatorlog_ARG, replicatorlogtype_ARG,
    virtualoriginsize_ARG, virtualsize_ARG, zero_ARG)
 
 xx(lvdisplay,

tools/lvcreate.c +55 -0

85 lines not on this page · raw message

tools/lvmcmdline.c +9 -0

index d9ea0c7..367e8ee 100644
@@ -488,6 +488,15 @@ int readahead_arg(struct cmd_context *cmd __attribute((unused)), struct arg *a)
 	return 1;
 }
 
+int site_arg(struct cmd_context *cmd __attribute((unused)), struct arg *a)
+{
+	char *pos = a->value;
+	if (!validate_name(pos))
+		return 0;
+
+	return 1;
+}
+
 static void __alloc(int size)
 {
 	if (!(_cmdline.commands = dm_realloc(_cmdline.commands, sizeof(*_cmdline.commands) * size))) {

tools/tools.h +1 -0

index 75575e7..8dbabbc 100644
@@ -152,6 +152,7 @@ int units_arg(struct cmd_context *cmd, struct arg *a);
 int segtype_arg(struct cmd_context *cmd, struct arg *a);
 int alloc_arg(struct cmd_context *cmd, struct arg *a);
 int readahead_arg(struct cmd_context *cmd, struct arg *a);
+int site_arg(struct cmd_context *cmd, struct arg *a);
 
 /* we use the enums to access the switches */
 unsigned arg_count(const struct cmd_context *cmd, int a);

Signature · 1 line

1.7.0.1

Zdenek Kabelac

author

2010-04-12 15:21 UTC #39 lore raw

PATCH 22/22 Replicator: man pages for lvcreate, lvchange

In reply to Zdenek Kabelac

Update man pages to cover basic usage of lvcreate and lvchange utility to handle basic replicator funtionality.

Current desriptions shows how to create replicator, add site, add replicated device.

Signed-off-by Zdenek Kabelac <[email protected]>

---
 man/lvchange.8.in |   57 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 man/lvcreate.8.in |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 112 insertions(+), 0 deletions(-)

man/lvchange.8.in +57 -0

index c3cb3b2..280e6d6 100644
@@ -3,10 +3,18 @@
 lvchange \- change attributes of a logical volume
 .SH SYNOPSIS
 .B lvchange
+[\-\-addsite Site]
 [\-\-addtag Tag]
 [\-A|\-\-autobackup y|n] [\-a|\-\-available y|n|ey|en|ly|ln]
 [\-\-alloc AllocationPolicy]
 [\-C|\-\-contiguous y|n] [\-d|\-\-debug] [\-\-deltag Tag]
+[\-\-delsite Site]
+[[\-\-fallbehinddata Size[bBsSkKmMgGtTpPeE]] |
+[\-\-fallbehindios  IOS] |
+[\-\-fallbehindtimeout sec]]
+[\-\-sitemode [sync|warn|stall|drop|fail]
+[\-\-activesite Site]
+[\-\-passivesite Site]
 [\-\-resync]
 [\-h|\-?|\-\-help]
 [\-\-ignorelockingfailure]

@@ -19,6 +27,7 @@ lvchange \- change attributes of a logical volume
 [\-p|\-\-permission r|rw] [\-r/\-\-readahead ReadAheadSectors|auto|none]
 [\-\-refresh]
 [\-t|\-\-test]
+[\-\-remotevolumegroup VolumeGroupName]
 [\-v|\-\-verbose] LogicalVolumePath [LogicalVolumePath...]
 .SH DESCRIPTION
 lvchange allows you to change the attributes of a logical volume

@@ -102,10 +111,58 @@ If the logical volume is active, reload its metadata.
 This is not necessary in normal operation, but may be useful
 if something has gone wrong or if you're doing clustering 
 manually without a clustered lock manager.
+.TP
+.I \-\-addsite
+Add new site to given replicator LV.
+.TP
+.I \-\-delsite
+Remove given site from replicator LV.
+.TP
+.I \-\-fallbehinddata
+How many data could get replicator behind. Cannot be mixed
+with --fallbehindios or --fallbehindtimeout.
+.TP
+.I \-\-fallbehindios
+How many io operation could be queued in replicator backing store device.
+Cannot be mixed with --fallbehinddata or --fallbehindtimeout.
+.TP
+.I \-\-fallbehindtimeout
+How many seconds could be replicator behind actual data. Cannot be mixed
+with --fallbehinddata or --fallbehindios.
+.TP
+.I \-\-sitemode
+Select replication mode. Default is Warn.
+Sync is synchronous mode. Warn,Stall,Drop,Fail are asynchronous modes.
+Warn issues warning when remote site is switched to sync log.
+Stall stalls replicator until remote site is back in range.
+Drop drops whole remote site, if it get too behind.
+Fail whole replicator fails is remote sites gets too behind.
+Local site works usually in stall mode. There is no synclog for local
+replicated volumes.
+.TP
+.I \-\-activesite
+Marks site as active. Note: Only one site from replicator could be
+marked active with actual implementation and this site must be
+local site.
+.TP
+.I \-\-passivesite
+Marks site as passive which is default.
+.TP
+.I \-\-remotevolumegroup
+Selects remote volume group for remote site that contains logical volumes
+with same names for replication as the volume group for site. Local site
+for replicator is assumed if remotevolumegroup is not specified.
+.TP
 .SH Examples
 "lvchange -pr vg00/lvol1" changes the permission on 
 volume lvol1 in volume group vg00 to be read-only.
 
+"lvchange --addsite NewYork vg00/replicator0"
+adds new local site NewYork in replicator vg00/replicator0.
+
+"lvchange --activesite NewYork vg00/replicator0"
+marks site NewYork in replicator vg00/replicator0 as active.
+
 .SH SEE ALSO
 .BR lvm (8), 
 .BR lvcreate (8),

man/lvcreate.8.in +55 -0

83 lines not on this page · raw message

Signature · 1 line

1.7.0.1

Alasdair G Kergon

2010-04-26 14:49 UTC #40 lore raw

PATCH 22/22 Replicator: man pages for lvcreate, lvchange

man/lvchange.8.in If the logical volume is active, reload its metadata. · line 149

Quoted 10 lines · Zdenek Kabelac

+[\-\-activesite Site]
+[\-\-passivesite Site]

+.I \-\-activesite

man/lvchange.8.in +6 -0

+Marks site as active. Note: Only one site from replicator could be
+marked active with actual implementation and this site must be
+local site.
+.TP
+.I \-\-passivesite
+Marks site as passive which is default.

Inconsistency. Does it or does it not take 'Site' as an argument? Can both be set together as the synopsis implies, or not?

What combinations of those args are meant to be permitted, or not? --activesite site1 --passivesite site1 ?

(If it's either/or, use '|' or split into 2 args.)

Alasdair