Skip to contentSkip to conversation

dm raid: fix raid set extensions not being synchronized

PATCH New Heinz Mauelshagen 1 participant 5 messages Oct 1, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover dm raid: fix raid set extensions not being synchronized

This patch series fixes flaws causing raid set extensions not to be synchronized in case a grown MD bitmap has to allocate additional pages.

When a raid set's grown, additional pages in the MD write-intent bitmap may have to be allocated as a result. Because so far no md_bitmap_resize() occurs in that process, additional new regions (aka MD bitmap chunks) aren't set dirty during creation, thus they fail to be synchronized. Solve by using md_bitmap_resize() in raid set size extensions to ensure dirtying of additional, new regions.

Also address another flaw causing new regions not to be synchronized after a raid set size reduction followed by an extension and streamline recovery setup and progress status functions.

Passes all lvm2 test suite RAID tests.

Heinz Mauelshagen (4):
dm raid: change rs_set_dev_and_array_sectors API and callers dm raid: to ensure resynchronization, perform raid set grow in preresume dm raid: simplify rs_setup_recovery call chain
dm raid: streamline rs_get_progress() and its raid_status() caller side

 .../admin-guide/device-mapper/dm-raid.rst     |   2 +
 drivers/md/dm-raid.c                          | 168 ++++++++++--------
 2 files changed, 91 insertions(+), 79 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/4 dm raid: change rs_set_dev_and_array_sectors API and callers

Add a size argument to rs_set_dev_and_array_sectors as prerequisite to fixing grown device resynchronization not occuring when new MD bitmap pages have to be allocated as a result of the extension in a follwup patch.

Also avoid code duplication by using rs_set_rdev_sectors in the aforementioned function.

---
 drivers/md/dm-raid.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)
drivers/md/dm-raid.c +5 -9
index b0aa595e4375..89f805e851cf 100644
@@ -1615,13 +1615,12 @@ static int _check_data_dev_sectors(struct raid_set *rs)
 }
 
 /* Calculate the sectors per device and per array used for @rs */
-static int rs_set_dev_and_array_sectors(struct raid_set *rs, bool use_mddev)
+static int rs_set_dev_and_array_sectors(struct raid_set *rs, sector_t sectors, bool use_mddev)
 {
 	int delta_disks;
 	unsigned int data_stripes;
+	sector_t array_sectors = sectors, dev_sectors = sectors;
 	struct mddev *mddev = &rs->md;
-	struct md_rdev *rdev;
-	sector_t array_sectors = rs->ti->len, dev_sectors = rs->ti->len;
 
 	if (use_mddev) {
 		delta_disks = mddev->delta_disks;
@@ -1656,12 +1655,9 @@ static int rs_set_dev_and_array_sectors(struct raid_set *rs, bool use_mddev)
 		/* Striped layouts */
 		array_sectors = (data_stripes + delta_disks) * dev_sectors;
 
-	rdev_for_each(rdev, mddev)
-		if (!test_bit(Journal, &rdev->flags))
-			rdev->sectors = dev_sectors;
-
 	mddev->array_sectors = array_sectors;
 	mddev->dev_sectors = dev_sectors;
+	rs_set_rdev_sectors(rs);
 
 	return _check_data_dev_sectors(rs);
 bad:
@@ -2911,7 +2907,7 @@ static int rs_setup_reshape(struct raid_set *rs)
 
 	/* Remove disk(s) */
 	} else if (rs->delta_disks < 0) {
-		r = rs_set_dev_and_array_sectors(rs, true);
+		r = rs_set_dev_and_array_sectors(rs, rs->ti->len, true);
 		mddev->reshape_backwards = 1; /* removing disk(s) -> backward reshape */
 
 	/* Change layout and/or chunk size */
@@ -3067,7 +3063,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 	 *
 	 * Any existing superblock will overwrite the array and device sizes
 	 */
-	r = rs_set_dev_and_array_sectors(rs, false);
+	r = rs_set_dev_and_array_sectors(rs, rs->ti->len, false);
 	if (r)
 		goto bad;
 
Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/4 dm raid: to ensure resynchronization, perform raid set grow in preresume

In reply to Heinz Mauelshagen

This fixes a flaw causing raid set extensions not to be synchronized in case the MD bitmap resize required additional pages to be allocated.

Also share resize code in the raid constructor between new size changes and those occuring during recovery.

Bump the target version to define the change and document it in Documentation/admin-guide/device-mapper/dm-raid.rst.

---
 .../admin-guide/device-mapper/dm-raid.rst     |  2 +
 drivers/md/dm-raid.c                          | 81 ++++++++++++++-----
 2 files changed, 62 insertions(+), 21 deletions(-)
Documentation/admin-guide/device-mapper/dm-raid.rst +2 -0
index 2fe255b130fb..f6344675e395 100644
@@ -417,3 +417,5 @@ Version History
 	deadlock/potential data corruption.  Update superblock when
 	specific devices are requested via rebuild.  Fix RAID leg
 	rebuild errors.
+ 1.15.0 Fix size extensions not being synchronized in case of new MD bitmap
+        pages allocated;  also fix those not occuring after previous reductions
drivers/md/dm-raid.c +60 -21

158 lines not on this page · raw message

Signature · 1 line
2.21.0
#3 lore raw

PATCH 3/4 dm raid: simplify rs_setup_recovery call chain

In reply to Heinz Mauelshagen

rs_setup_recovery() sets the starting recovery offset.

Drop superfluous rs_setup_recovery() and replace with __rs_setup_recovery().

---
 drivers/md/dm-raid.c | 27 ++++++---------------------
 1 file changed, 6 insertions(+), 21 deletions(-)
drivers/md/dm-raid.c +6 -21
index b600b335ce88..9aa830c53844 100644
@@ -1670,7 +1670,7 @@ static int rs_set_dev_and_array_sectors(struct raid_set *rs, sector_t sectors, b
 }
 
 /* Setup recovery on @rs */
-static void __rs_setup_recovery(struct raid_set *rs, sector_t dev_sectors)
+static void rs_setup_recovery(struct raid_set *rs, sector_t dev_sectors)
 {
 	/* raid0 does not recover */
 	if (rs_is_raid0(rs))
@@ -1691,22 +1691,6 @@ static void __rs_setup_recovery(struct raid_set *rs, sector_t dev_sectors)
 				     ? MaxSector : dev_sectors;
 }
 
-/* Setup recovery on @rs based on raid type, device size and 'nosync' flag */
-static void rs_setup_recovery(struct raid_set *rs, sector_t dev_sectors)
-{
-	if (!dev_sectors)
-		/* New raid set or 'sync' flag provided */
-		__rs_setup_recovery(rs, 0);
-	else if (dev_sectors == MaxSector)
-		/* Prevent recovery */
-		__rs_setup_recovery(rs, MaxSector);
-	else if (__rdev_sectors(rs) < dev_sectors)
-		/* Grown raid set */
-		__rs_setup_recovery(rs, __rdev_sectors(rs));
-	else
-		__rs_setup_recovery(rs, MaxSector);
-}
-
 static void do_table_event(struct work_struct *ws)
 {
 	struct raid_set *rs = container_of(ws, struct raid_set, md.event_work);
@@ -2474,7 +2458,7 @@ static int super_validate(struct raid_set *rs, struct md_rdev *rdev)
 		return -EINVAL;
 	}
 
-	/* Enable bitmap creation for RAID levels != 0 */
+	/* Enable bitmap creation on @rs unless no metadevs or raid0 or journaled raid4/5/6 set. */
 	mddev->bitmap_info.offset = (rt_is_raid0(rs->raid_type) || rs->journal_dev.dev) ? 0 : to_sector(4096);
 	mddev->bitmap_info.default_offset = mddev->bitmap_info.offset;
 
@@ -3173,7 +3157,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 		rs_setup_recovery(rs, MaxSector);
 		rs_set_new(rs);
 	} else if (rs_reshape_requested(rs)) {
-		/* Only on size extensions, not on reshapes. */
+		/* Only request grow on raid set size extensions, not on reshapes. */
 		clear_bit(RT_FLAG_RS_GROW, &rs->runtime_flags);
 
 		/*
@@ -3211,10 +3195,11 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 		if (test_bit(__CTR_FLAG_REBUILD, &rs->ctr_flags)) {
 			clear_bit(RT_FLAG_RS_GROW, &rs->runtime_flags);
 			set_bit(RT_FLAG_UPDATE_SBS, &rs->runtime_flags);
+			rs_setup_recovery(rs, MaxSector);
 		} else if (test_bit(RT_FLAG_RS_GROW, &rs->runtime_flags)) {
 			/*
-			 * Set raid set to current size, i.e. non-grown size
-			 * as of superblocks to grow to new size in preresume.
+			 * Set raid set to current size, i.e. size as of
+			 * superblocks to grow to larger size in preresume.
 			 */
 			r = rs_set_dev_and_array_sectors(rs, sb_array_sectors, false);
 			if (r)
Signature · 1 line
2.21.0
#4 lore raw

PATCH 4/4 dm raid: streamline rs_get_progress() and its raid_status() caller side

In reply to Heinz Mauelshagen

Pass already deciphered state into rs_get_progress, simplify recovery offset definition and combine two st_resync, st_reshape conditionals into one as is already the case with st_check and st_repair.

---
 drivers/md/dm-raid.c | 52 ++++++++++++++++++--------------------------
 1 file changed, 21 insertions(+), 31 deletions(-)
drivers/md/dm-raid.c +21 -31
index 9aa830c53844..153654ae84ae 100644
@@ -3413,10 +3413,9 @@ static const char *__raid_dev_status(struct raid_set *rs, struct md_rdev *rdev)
 
 /* Helper to return resync/reshape progress for @rs and runtime flags for raid set in sync / resynching */
 static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery,
-				sector_t resync_max_sectors)
+				enum sync_state state, sector_t resync_max_sectors)
 {
 	sector_t r;
-	enum sync_state state;
 	struct mddev *mddev = &rs->md;
 
 	clear_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags);
@@ -3427,12 +3426,7 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery,
 		set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags);
 
 	} else {
-		state = decipher_sync_action(mddev, recovery);
-
-		if (state == st_idle && !test_bit(MD_RECOVERY_INTR, &recovery))
-			r = mddev->recovery_cp;
-		else
-			r = mddev->curr_resync_completed;
+		r = max(mddev->recovery_cp, mddev->curr_resync_completed);
 
 		if (state == st_idle && r >= resync_max_sectors) {
 			/*
@@ -3446,18 +3440,14 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery,
 			/*
 			 * In case we are recovering, the array is not in sync
 			 * and health chars should show the recovering legs.
+			 *
+			 * Already retrieved recovery offset from curr_resync_completed above.
 			 */
 			;
-		else if (state == st_resync)
-			/*
-			 * If "resync" is occurring, the raid set
-			 * is or may be out of sync hence the health
-			 * characters shall be 'a'.
-			 */
-			set_bit(RT_FLAG_RS_RESYNCING, &rs->runtime_flags);
-		else if (state == st_reshape)
+
+		else if (state == st_resync || state == st_reshape)
 			/*
-			 * If "reshape" is occurring, the raid set
+			 * If "resync/reshape" is occurring, the raid set
 			 * is or may be out of sync hence the health
 			 * characters shall be 'a'.
 			 */
@@ -3471,22 +3461,22 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery,
 			 */
 			set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags);
 
-		else {
-			struct md_rdev *rdev;
-
+		else if (test_bit(MD_RECOVERY_NEEDED, &recovery))
 			/*
 			 * We are idle and recovery is needed, prevent 'A' chars race
 			 * caused by components still set to in-sync by constructor.
 			 */
-			if (test_bit(MD_RECOVERY_NEEDED, &recovery))
-				set_bit(RT_FLAG_RS_RESYNCING, &rs->runtime_flags);
+			set_bit(RT_FLAG_RS_RESYNCING, &rs->runtime_flags);
 
+		else {
 			/*
-			 * The raid set may be doing an initial sync, or it may
-			 * be rebuilding individual components.	 If all the
-			 * devices are In_sync, then it is the raid set that is
-			 * being initialized.
+			 * We are idle and the raid set may be doing an initial
+			 * sync, or it may be rebuilding individual components.
+			 * If all the devices are In_sync, then it is the raid set
+			 * that is being initialized.
 			 */
+			struct md_rdev *rdev;
+
 			set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags);
 			rdev_for_each(rdev, mddev)
 				if (!test_bit(Journal, &rdev->flags) &&
@@ -3519,7 +3509,7 @@ static void raid_status(struct dm_target *ti, status_type_t type,
 	unsigned int rebuild_disks;
 	unsigned int write_mostly_params = 0;
 	sector_t progress, resync_max_sectors, resync_mismatches;
-	const char *sync_action;
+	enum sync_state state;
 	struct raid_type *rt;
 
 	switch (type) {
@@ -3533,14 +3523,14 @@ static void raid_status(struct dm_target *ti, status_type_t type,
 
 		/* Access most recent mddev properties for status output */
 		smp_rmb();
-		recovery = rs->md.recovery;
 		/* Get sensible max sectors even if raid set not yet started */
 		resync_max_sectors = test_bit(RT_FLAG_RS_PRERESUMED, &rs->runtime_flags) ?
 				      mddev->resync_max_sectors : mddev->dev_sectors;
-		progress = rs_get_progress(rs, recovery, resync_max_sectors);
+		recovery = rs->md.recovery;
+		state = decipher_sync_action(mddev, recovery);
+		progress = rs_get_progress(rs, recovery, state, resync_max_sectors);
 		resync_mismatches = (mddev->last_sync_action && !strcasecmp(mddev->last_sync_action, "check")) ?
 				    atomic64_read(&mddev->resync_mismatches) : 0;
-		sync_action = sync_str(decipher_sync_action(&rs->md, recovery));
 
 		/* HM FIXME: do we want another state char for raid0? It shows 'D'/'A'/'-' now */
 		for (i = 0; i < rs->raid_disks; i++)
@@ -3568,7 +3558,7 @@ static void raid_status(struct dm_target *ti, status_type_t type,
 		 *   See Documentation/admin-guide/device-mapper/dm-raid.rst for
 		 *   information on each of these states.
 		 */
-		DMEMIT(" %s", sync_action);
+		DMEMIT(" %s", sync_str(state));
 
 		/*
 		 * v1.5.0+:
Signature · 1 line
2.21.0