Write barriers on MD RAID1

Support for write barriers on MD RAID1 devices appears to be broken on 2.6.29 and later kernels. Mounting an ext3 or ext4 filesystem over an md raid1 block device gives the kernel message:

JBD: barrier-based sync failed on md0 - disabling barriers

Is this intentional? It looks like a regression to me, since write barriers (ostensibly) worked for MD RAID1 on earlier kernels (well, all of the 2.6.27 series at least...)

Doing a git bisect suggested that this is the culprit:

commit cec0707e40ae25794b5a2de7b7f03c51961f80d9

Author: Jens Axboe <[email protected]>
Date:   Tue Jan 13 15:28:32 2009 +0100

block: silently error an unsupported barrier bio

Rolling this back from 2.6.29.4 made the JBD error message go away for me. I tried to do some disk benchmarking to determine if write barriers were really still working, but the results were not very conclusive.

Details of test setup:
- 2.6.29.4 kernel (config from slackware-current)
- ext3 filesystem (mounted with barrier=1), also tried ext4
- MD RAID1 block device over two SATA disks
- Intel motherboard AHCI controller (82801)

I got similar behaviour on a virtualised setup using qemu, with the RAID over two emulated IDE disks.

Please contact me if I can provide any further information.

-Ken Milmore.

Quoted 10 lines · Ken Milmore

Support for write barriers on MD RAID1 devices appears to be broken on 2.6.29 and later kernels. Mounting an ext3 or ext4 filesystem over an md raid1 block device gives the kernel message:

JBD: barrier-based sync failed on md0 - disabling barriers

Is this intentional? It looks like a regression to me, since write barriers (ostensibly) worked for MD RAID1 on earlier kernels (well, all of the 2.6.27 series at least...)

Quoted 7 lines

Doing a git bisect suggested that this is the culprit:

commit cec0707e40ae25794b5a2de7b7f03c51961f80d9

Author: Jens Axboe <[email protected]>
Date:   Tue Jan 13 15:28:32 2009 +0100

block: silently error an unsupported barrier bio

Yes, that patch is broken.
the new code it introduces should probably go in the top of __make_request rather than just before the call to ->make_request_fn. make_request_fn functions other than __make_request generally don't use ->next_ordered at all.

Jens? Would you agree?

NeilBrown

Quoted 17 lines

Rolling this back from 2.6.29.4 made the JBD error message go away for me. I tried to do some disk benchmarking to determine if write barriers were really still working, but the results were not very conclusive.

Details of test setup:
- 2.6.29.4 kernel (config from slackware-current)
- ext3 filesystem (mounted with barrier=1), also tried ext4
- MD RAID1 block device over two SATA disks
- Intel motherboard AHCI controller (82801)

I got similar behaviour on a virtualised setup using qemu, with the RAID over two emulated IDE disks.

Please contact me if I can provide any further information.

-Ken Milmore.

I thought I had better bump my previous post as this regression is still present in 2.6.29.5.

To recap, commit cec0707e40ae25794b5a2de7b7f03c51961f80d9 has broken write barriers on md raid1 block devices in 2.6.29 and later kernels. Reversing this commit appears to fix the problem.

Please let me know if I'm harassing the wrong maintainers here!

From addd8b129835a63d6df9a38eae20caaa15de5988 Mon Sep 17 00:00:00 2001

Since this test was added, barriers have not worked on md, and (I think) dm and similar devices that don't use __make_request and so don't bother to set next_ordered.

Quoted 3 lines · Neil Brown

Since this test was added, barriers have not worked on md, and (I think) dm and similar devices that don't use __make_request and so don't bother to set next_ordered.

In dm, we added this:

+ blk_queue_ordered(md->queue, QUEUE_ORDERED_DRAIN, NULL);

Quoted 2 lines · On Sunday June 21, [email protected] wrote:

I thought I had better bump my previous post as this regression is still present in 2.6.29.5.

Thanks Ken.

Quoted 1 line

Quoted 5 lines

To recap, commit cec0707e40ae25794b5a2de7b7f03c51961f80d9 has broken write barriers on md raid1 block devices in 2.6.29 and later kernels. Reversing this commit appears to fix the problem.

Please let me know if I'm harassing the wrong maintainers here!

Jens,
have you had a chance to look at this?

I think the following patch is appropriate and should go in to -stable.

Thanks,
NeilBrown

From addd8b129835a63d6df9a38eae20caaa15de5988 Mon Sep 17 00:00:00 2001
From: NeilBrown <[email protected]>
Date: Wed, 24 Jun 2009 13:39:15 +1000
Subject: [PATCH] Restore barrier support for md and probably other virtual devices.

The next_ordered flag is only meaningful for devices that use __make_request. So move the test against next_ordered out of generic code and in to __make_request

Since this test was added, barriers have not worked on md, and (I think) dm and similar devices that don't use __make_request and so don't bother to set next_ordered.

Cc [email protected]

Cc Ken Milmore <[email protected]>

Signed-off-by NeilBrown <[email protected]>

---
 block/blk-core.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

block/blk-core.c +5 -5

index b06cf5c..fc221aa 100644
@@ -1172,6 +1172,11 @@ static int __make_request(struct request_queue *q, struct bio *bio)
 	const int unplug = bio_unplug(bio);
 	int rw_flags;
 
+	if (bio_barrier(bio) && bio_has_data(bio) &&
+	    (q->next_ordered == QUEUE_ORDERED_NONE)) {
+		bio_endio(bio, -EOPNOTSUPP)
+			return 0;
+	}
 	/*
 	 * low level driver can indicate that it wants pages above a
 	 * certain limit bounced to low memory (ie for highmem, or even

@@ -1472,11 +1477,6 @@ static inline void __generic_make_request(struct bio *bio)
 			err = -EOPNOTSUPP;
 			goto end_io;
 		}
-		if (bio_barrier(bio) && bio_has_data(bio) &&
-		    (q->next_ordered == QUEUE_ORDERED_NONE)) {
-			err = -EOPNOTSUPP;
-			goto end_io;
-		}
 
 		ret = q->make_request_fn(q, bio);
 	} while (ret);

Signature · 1 line

1.6.3.1

Quoted 11 lines · Neil Brown

Quoted 2 lines · On Sunday June 21, [email protected] wrote:

I thought I had better bump my previous post as this regression is still present in 2.6.29.5.

Thanks Ken.

Quoted 5 lines

To recap, commit cec0707e40ae25794b5a2de7b7f03c51961f80d9 has broken write barriers on md raid1 block devices in 2.6.29 and later kernels. Reversing this commit appears to fix the problem.

Quoted 4 lines

Quoted 1 line

Please let me know if I'm harassing the wrong maintainers here!

Jens,
have you had a chance to look at this?

Yeah, I think it's the right way to go. I'll queue it up for .31 and we should put in in -stable as well.

Quoted 60 lines

I think the following patch is appropriate and should go in to -stable.

Thanks,
NeilBrown

From addd8b129835a63d6df9a38eae20caaa15de5988 Mon Sep 17 00:00:00 2001
From: NeilBrown <[email protected]>
Date: Wed, 24 Jun 2009 13:39:15 +1000
Subject: [PATCH] Restore barrier support for md and probably other virtual devices.

The next_ordered flag is only meaningful for devices that use __make_request. So move the test against next_ordered out of generic code and in to __make_request

Since this test was added, barriers have not worked on md, and (I think) dm and similar devices that don't use __make_request and so don't bother to set next_ordered.

Cc [email protected]

Cc Ken Milmore <[email protected]>

Signed-off-by NeilBrown <[email protected]>

---
 block/blk-core.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

block/blk-core.c +5 -5

index b06cf5c..fc221aa 100644
@@ -1172,6 +1172,11 @@ static int __make_request(struct request_queue *q, struct bio *bio)
 	const int unplug = bio_unplug(bio);
 	int rw_flags;
 
+	if (bio_barrier(bio) && bio_has_data(bio) &&
+	    (q->next_ordered == QUEUE_ORDERED_NONE)) {
+		bio_endio(bio, -EOPNOTSUPP)
+			return 0;
+	}
 	/*
 	 * low level driver can indicate that it wants pages above a
 	 * certain limit bounced to low memory (ie for highmem, or even

@@ -1472,11 +1477,6 @@ static inline void __generic_make_request(struct bio *bio)
 			err = -EOPNOTSUPP;
 			goto end_io;
 		}
-		if (bio_barrier(bio) && bio_has_data(bio) &&
-		    (q->next_ordered == QUEUE_ORDERED_NONE)) {
-			err = -EOPNOTSUPP;
-			goto end_io;
-		}
 
 		ret = q->make_request_fn(q, bio);
 	} while (ret);

Signature · 1 line

1.6.3.1

Signature · 1 line

Jens Axboe