Skip to contentSkip to conversation

mirroring: [patch 1 of 8] device failure tolerance

PATCH Jonathan E Brassow 3 participants 6 messages Jun 29, 2005 – Jun 30, 2005 Device mapper

lore
Share & tools
lore
#0 lore raw

This patch defines a couple more states that logs can return, and checks for those states in the mirror code. The states are useful for logs that have cluster support.

brassow

drivers/md/dm-log.c +1 -1
diff -urN linux-2.6.12-orig/drivers/md/dm-log.c linux-2.6.12-001/drivers/md/dm-log.c
@@ -517,13 +517,13 @@
 static int core_is_clean(struct dirty_log *log, region_t region)
 {
 	struct log_c *lc = (struct log_c *) log->context;
-	return log_test_bit(lc->clean_bits, region);
+	return (log_test_bit(lc->clean_bits, region)) ? LOG_CLEAN : LOG_DIRTY;
 }
 
 static int core_in_sync(struct dirty_log *log, region_t region, int 

block)
{
struct log_c *lc = (struct log_c *) log->context;
- return log_test_bit(lc->sync_bits, region);
+ return (log_test_bit(lc->sync_bits, region)) ? LOG_CLEAN : LOG_NOSYNC;
}

static int core_flush(struct dirty_log *log)

drivers/md/dm-log.h +6 -0
diff -urN linux-2.6.12-orig/drivers/md/dm-log.h linux-2.6.12-001/drivers/md/dm-log.h
@@ -9,6 +9,12 @@
 
 #include "dm.h"
 
+#define LOG_DIRTY              0
+#define LOG_CLEAN              1
+#define LOG_NOSYNC             2
+#define LOG_RECOVERING         3
+#define LOG_REMOTE_RECOVERING  4
+
 typedef sector_t region_t;
 
 struct dirty_log_type;
drivers/md/dm-raid1.c +21 -7
diff -urN linux-2.6.12-orig/drivers/md/dm-raid1.c linux-2.6.12-001/drivers/md/dm-raid1.c
@@ -91,7 +91,8 @@
 	RH_CLEAN,
 	RH_DIRTY,
 	RH_NOSYNC,
-	RH_RECOVERING
+	RH_RECOVERING,
+	RH_REMOTE_RECOVERING
 };
 
 struct region {
@@ -234,7 +235,7 @@
 
 	read_unlock(&rh->hash_lock);
 	nreg = mempool_alloc(rh->region_pool, GFP_NOIO);
-	nreg->state = rh->log->type->in_sync(rh->log, region, 1) ?
+	nreg->state = (rh->log->type->in_sync(rh->log, region, 1) == LOG_CLEAN) ?
 		RH_CLEAN : RH_NOSYNC;
 	nreg->rh = rh;
 	nreg->key = region;
@@ -292,13 +293,26 @@
 	 * The region wasn't in the hash, so we fall back to the
 	 * dirty log.
 	 */
-	r = rh->log->type->in_sync(rh->log, region, may_block);
-
+	switch (rh->log->type->in_sync(rh->log, region, may_block)) {
+	case LOG_CLEAN:
+		r = RH_CLEAN;
+		break;
+	case LOG_DIRTY:
+		r = RH_DIRTY;
+		break;
+	case LOG_REMOTE_RECOVERING:
+		r = RH_REMOTE_RECOVERING;
+		break;
+	default:
+		r = RH_NOSYNC;
+		break;
+	}
+	
 	/*
 	 * Any error from the dirty log (eg. -EWOULDBLOCK) gets
 	 * taken as a RH_NOSYNC
 	 */
-	return r == 1 ? RH_CLEAN : RH_NOSYNC;
+	return r;
 }
 
 static inline int rh_in_sync(struct region_hash *rh,
@@ -702,7 +716,7 @@
 		/*
 		 * We can only read balance if the region is in sync.
 		 */
-		if (rh_in_sync(&ms->rh, region, 0))
+		if (rh_in_sync(&ms->rh, region, 0) == RH_CLEAN)
 			m = choose_mirror(ms, bio->bi_sector);
 		else
 			m = ms->mirror + DEFAULT_MIRROR;
@@ -1117,7 +1131,7 @@
 	if (r < 0 && r != -EWOULDBLOCK)
 		return r;
 
-	if (r == -EWOULDBLOCK)	/* FIXME: ugly */
+	if (r != RH_CLEAN)
 		r = 0;
 
 	/*
#1 lore raw
Quoted 3 lines · Jonathan E Brassow

This patch defines a couple more states that logs can return, and checks for those states in the mirror code. The states are useful for logs that have cluster support.

OK. Let me just ask one question, which I'll preface with the following:

I totally love the idea of md / DM convergence. I'm not yet quite clear how this is to be achieved, but having two separate frameworks which at least considerable overlap doesn't make sense - this isn't like two filesystems, this is like two VFS layers and obviously silly. So, I appreciate movement into that direction.

However, what's going on now is that features are being added to the DM raid code, instead of figuring out a path of convergence.

RAID code takes, that's the experience we've got with Linux so far, quite some time to stabilize. Duplicating it definetely doesn't seem smart, if I may be so blunt. The RAID code in md is, nowadays, very stable, and even already has patches for faster resync etc (merged in -mm).

md is, in certain ways, more mature, which is exemplified by md's ability to actually provide sane logging. DM has new cool features (like the online remapping and stuff).

Why is the approach of reinventing a wheel with a slightly different number of edges taken?

Is it believed that DM will supersede all aspects of md one day? Ain't that slightly silly? ;-)

Sincerely,
Lars Marowsky-Brée <[email protected]>

Signature · 4 lines
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business	 -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"
#2 lore raw
In reply to Jonathan E Brassow
Quoted 5 lines · Jonathan E Brassow

This patch defines a couple more states that logs can return, and checks for those states in the mirror code. The states are useful for logs that have cluster support.

brassow

Have I been asleep? This is the first I heard of DM having cluster mirror support. I assume the goal is something along the lines of what DRBD provides.

Can someone give a very high-level overview of current status, plans, etc. related to DM Cluster Mirror support. And if this related to the DRBD project?

Thanks
Greg

Signature · 3 lines
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
#3 lore raw
Quoted 7 lines · Greg Freemyer
Quoted 5 lines · Jonathan E Brassow

This patch defines a couple more states that logs can return, and checks for those states in the mirror code. The states are useful for logs that have cluster support.

brassow

Quoted 7 lines

Have I been asleep? This is the first I heard of DM having cluster mirror support. I assume the goal is something along the lines of what DRBD provides.

Can someone give a very high-level overview of current status, plans, etc. related to DM Cluster Mirror support. And if this related to the DRBD project?

Please note that the "mirroring [patch x of 8]..." set for device fault tolerance has been replaced with the "mirroring [patch x of 6]..." set.

The cluster mirroring that I'm talking about is active/active and capable of handling more than 2 nodes. It would be used in conjunction with a clustered file system - like GFS. The application sitting on the cluster mirror should be cluster-aware. You wouldn't be able to just put ext3 on there and expect it to work in a cluster capacity.

Although there are some minor things that need to happen in the mirror proper code (as can be seen from the smallness of the cluster patch), the heavy lifting is done by a cluster-aware log. That is were you are keeping track of clean/dirty/recovering state.

An out-dated (but still useful for background) dock can be found at http://www.brassow.com/mirroring/index.html

brassow

#4 lore raw
Quoted 32 lines · Jonathan E Brassow
Quoted 7 lines · Greg Freemyer
Quoted 5 lines · Jonathan E Brassow

This patch defines a couple more states that logs can return, and checks for those states in the mirror code. The states are useful for logs that have cluster support.

brassow

Quoted 7 lines

Have I been asleep? This is the first I heard of DM having cluster mirror support. I assume the goal is something along the lines of what DRBD provides.

Can someone give a very high-level overview of current status, plans, etc. related to DM Cluster Mirror support. And if this related to the DRBD project?

Please note that the "mirroring [patch x of 8]..." set for device fault tolerance has been replaced with the "mirroring [patch x of 6]..." set.

The cluster mirroring that I'm talking about is active/active and capable of handling more than 2 nodes. It would be used in conjunction with a clustered file system - like GFS. The application sitting on the cluster mirror should be cluster-aware. You wouldn't be able to just put ext3 on there and expect it to work in a cluster capacity.

Although there are some minor things that need to happen in the mirror proper code (as can be seen from the smallness of the cluster patch), the heavy lifting is done by a cluster-aware log. That is were you are keeping track of clean/dirty/recovering state.

Quoted 5 lines

An out-dated (but still useful for background) dock can be found at
http://www.brassow.com/mirroring/index.html

brassow

Sounds like you have more ambitious plans than DRBD. (They have
active-passive now and IIRC they plan the next major release to have active-active, but I think only on 2 nodes.)

Is the GFS team planning on supporting this infrastructure. I hope so.

Greg

Signature · 3 lines
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
#5 lore raw
Quoted 36 lines · Greg Freemyer
Quoted 35 lines · Jonathan E Brassow
Quoted 7 lines · Greg Freemyer
Quoted 5 lines · Jonathan E Brassow

This patch defines a couple more states that logs can return, and checks for those states in the mirror code. The states are useful for logs that have cluster support.

brassow

Quoted 7 lines

Have I been asleep? This is the first I heard of DM having cluster mirror support. I assume the goal is something along the lines of what DRBD provides.

Can someone give a very high-level overview of current status, plans, etc. related to DM Cluster Mirror support. And if this related to the DRBD project?

Please note that the "mirroring [patch x of 8]..." set for device fault tolerance has been replaced with the "mirroring [patch x of 6]..." set.

The cluster mirroring that I'm talking about is active/active and capable of handling more than 2 nodes. It would be used in conjunction with a clustered file system - like GFS. The application sitting on the cluster mirror should be cluster-aware. You wouldn't be able to just put ext3 on there and expect it to work in a cluster capacity.

Although there are some minor things that need to happen in the mirror proper code (as can be seen from the smallness of the cluster patch), the heavy lifting is done by a cluster-aware log. That is were you are keeping track of clean/dirty/recovering state.

An out-dated (but still useful for background) dock can be found at
http://www.brassow.com/mirroring/index.html

Quoted 9 lines
Quoted 2 lines

brassow

Sounds like you have more ambitious plans than DRBD. (They have
active-passive now and IIRC they plan the next major release to have active-active, but I think only on 2 nodes.)

Is the GFS team planning on supporting this infrastructure. I hope so.

Yes. In fact, the cluster log being developed leverages the Red hat cluster manager (cman) - the same one that GFS uses.

CMAN is moving to user space (I think in the FC5 timeframe?), so the cluster log will have to adapt to that. I won't be submitting the cluster log code until that takes place. It is available in the cluster repository though (sources.redhat.com/cluster), but i will have to clean it up again to take advantage of the hooks I added to the logging code.

brassow