Skip to contentSkip to conversation

hybrid raid1 with trim support

DISCUSSION Markus 2 participants 7 messages Mar 12, 2013 – Apr 28, 2013 MD RAID

lore
Share & tools
lore

Markus

author
#0 lore raw

Hello!

I created a hybrid raid1 with one ssd and one hdd. Used writemostly and writebehind and put ext4 with discard enabled on it. This setup worked quite well for the last months (last kernel was 3.7.6). But now as I booted 3.8.2 the hdd was dropped from the raid with:

Quoted 2 lines

md/raid1:md2: Disk failure on sdb1, disabling device. md/raid1:md2: Operation continuing on 1 devices.

Re-adding this drive it will try to resync but the hdd will be dropped short time after. Now I remounted the device without the discard flag and the resync and usage works as before. After remounting it again with discard enabled the hdd is dropped again. So I think this is the culprit as the hdd does obviously not support TRIM...

As it worked fine before, I think this is a regression? Or is this an intended change?

Thanks,
Markus

author
#1 lore raw

Hi!

Still the same with 3.8.4 ... anybody?

Best regards,
Markus

Markus schrieb am 12.03.2013:

Quoted 15 lines

Hello!

I created a hybrid raid1 with one ssd and one hdd. Used writemostly and writebehind and put ext4 with discard enabled on it. This setup worked quite well for the last months (last kernel was 3.7.6). But now as I booted 3.8.2 the hdd was dropped from the raid with:

Quoted 2 lines

md/raid1:md2: Disk failure on sdb1, disabling device. md/raid1:md2: Operation continuing on 1 devices.

Re-adding this drive it will try to resync but the hdd will be dropped short time after. Now I remounted the device without the discard flag and the resync and usage works as before. After remounting it again with discard enabled the hdd is dropped again. So I think this is the culprit as the hdd does obviously not support TRIM...

As it worked fine before, I think this is a regression? Or is this an intended change?

Thanks,
Markus

author
#2 lore raw

Re: hybrid raid1 with trim support [REGRESSION]

Hi!

Now I had the time to bisect, started with 3.7 as good and 3.8 as bad. 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 is the bad commit. [1]
block: add plug for blkdev_issue_discard

While 3.8.10 was still bad, the same kernel with the reverted patch applied is fine.

I found another report. [2]

Thanks,
Markus

[1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9
[2] http://www.spinics.net/lists/raid/msg42758.html

Markus schrieb am 24.03.2013:

Quoted 23 lines

Hi!

Still the same with 3.8.4 ... anybody?

Best regards,
Markus

Markus schrieb am 12.03.2013:

Quoted 15 lines

Hello!

I created a hybrid raid1 with one ssd and one hdd. Used writemostly and writebehind and put ext4 with discard enabled on it. This setup worked quite well for the last months (last kernel was 3.7.6). But now as I booted 3.8.2 the hdd was dropped from the raid with:

Quoted 2 lines

md/raid1:md2: Disk failure on sdb1, disabling device. md/raid1:md2: Operation continuing on 1 devices.

Re-adding this drive it will try to resync but the hdd will be dropped short time after. Now I remounted the device without the discard flag and the resync and usage works as before. After remounting it again with discard enabled the hdd is dropped again. So I think this is the culprit as the hdd does obviously not support TRIM...

As it worked fine before, I think this is a regression? Or is this an intended change?

Thanks,
Markus

#3 lore raw

Re: hybrid raid1 with trim support [REGRESSION]

Quoted 7 lines · Markus

Hi!

Now I had the time to bisect, started with 3.7 as good and 3.8 as bad. 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 is the bad commit. [1]
block: add plug for blkdev_issue_discard

While 3.8.10 was still bad, the same kernel with the reverted patch applied is fine.

Thanks for the reporting. Does below patch work for you?

Thanks,
Shaohua

---
 drivers/md/raid1.c |    4 ++++
 1 file changed, 4 insertions(+)
drivers/md/raid1.c +4 -0
Index: linux/drivers/md/raid1.c
===================================================================
@@ -981,6 +981,10 @@ static void raid1_unplug(struct blk_plug
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
 		generic_make_request(bio);
 		bio = next;
 	}
#4 lore raw

Re: hybrid raid1 with trim support [REGRESSION]

Quoted 9 lines · Shaohua Li
Quoted 7 lines · Markus

Hi!

Now I had the time to bisect, started with 3.7 as good and 3.8 as bad. 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 is the bad commit. [1]
block: add plug for blkdev_issue_discard

While 3.8.10 was still bad, the same kernel with the reverted patch applied is fine.

Thanks for the reporting. Does below patch work for you?

Oops, there is a typo there, should be this one:

---
 drivers/md/raid1.c  |    7 ++++++-
 drivers/md/raid10.c |    7 ++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)
drivers/md/raid1.c +6 -1
Index: linux/drivers/md/raid1.c
===================================================================
@@ -981,7 +981,12 @@ static void raid1_unplug(struct blk_plug
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
-		generic_make_request(bio);
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
+		else
+			generic_make_request(bio);
 		bio = next;
 	}
 	kfree(plug);
drivers/md/raid10.c +6 -1
Index: linux/drivers/md/raid10.c
===================================================================
@@ -1133,7 +1133,12 @@ static void raid10_unplug(struct blk_plu
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
-		generic_make_request(bio);
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
+		else
+			generic_make_request(bio);
 		bio = next;
 	}
 	kfree(plug);
author
#5 lore raw

Re: hybrid raid1 with trim support [REGRESSION]

Hi!

Thanks for your work. The patch seems to work for me on a vanilla 3.8.10, at least the hdds are no longer dropped from the raid.
The code now ignores some request? What was the reason the disks fell off the raid? The discards are still passed to the ssd?

Thanks,
Markus

Shaohua Li schrieb am 28.04.2013:

Quoted 9 lines · Shaohua Li
Quoted 8 lines · Markus
Quoted 7 lines

Hi!

Now I had the time to bisect, started with 3.7 as good and 3.8 as bad. 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 is the bad commit. [1]
block: add plug for blkdev_issue_discard

While 3.8.10 was still bad, the same kernel with the reverted patch

applied is fine.

Quoted 44 lines
Quoted 1 line

Thanks for the reporting. Does below patch work for you?

Oops, there is a typo there, should be this one:

---
 drivers/md/raid1.c  |    7 ++++++-
 drivers/md/raid10.c |    7 ++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)

Index: linux/drivers/md/raid1.c
===================================================================

drivers/md/raid1.c +6 -1
@@ -981,7 +981,12 @@ static void raid1_unplug(struct blk_plug
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
-		generic_make_request(bio);
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
+		else
+			generic_make_request(bio);
 		bio = next;
 	}
 	kfree(plug);

Index: linux/drivers/md/raid10.c
===================================================================

drivers/md/raid10.c +6 -1
@@ -1133,7 +1133,12 @@ static void raid10_unplug(struct blk_plu
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
-		generic_make_request(bio);
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
+		else
+			generic_make_request(bio);
 		bio = next;
 	}
 	kfree(plug);
#6 lore raw

Re: hybrid raid1 with trim support [REGRESSION]

Quoted 6 lines · Markus

Hi!

Thanks for your work. The patch seems to work for me on a vanilla 3.8.10, at least the hdds are no longer dropped from the raid.
The code now ignores some request? What was the reason the disks fell off the raid? The discards are still passed to the ssd?

Thanks for testing, I'll send to Neil soon.

Yes, the discard will still be passed to SSD, we just ignore the request for harddisk.

Thanks,
Shaohua

Quoted 59 lines

Thanks,
Markus

Shaohua Li schrieb am 28.04.2013:

Quoted 9 lines · Shaohua Li
Quoted 8 lines · Markus
Quoted 7 lines

Hi!

Now I had the time to bisect, started with 3.7 as good and 3.8 as bad. 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 is the bad commit. [1]
block: add plug for blkdev_issue_discard

While 3.8.10 was still bad, the same kernel with the reverted patch

applied is fine.

Quoted 44 lines
Quoted 1 line

Thanks for the reporting. Does below patch work for you?

Oops, there is a typo there, should be this one:

---
 drivers/md/raid1.c  |    7 ++++++-
 drivers/md/raid10.c |    7 ++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)

Index: linux/drivers/md/raid1.c
===================================================================

drivers/md/raid1.c +6 -1
@@ -981,7 +981,12 @@ static void raid1_unplug(struct blk_plug
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
-		generic_make_request(bio);
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
+		else
+			generic_make_request(bio);
 		bio = next;
 	}
 	kfree(plug);

Index: linux/drivers/md/raid10.c
===================================================================

drivers/md/raid10.c +6 -1
@@ -1133,7 +1133,12 @@ static void raid10_unplug(struct blk_plu
 	while (bio) { /* submit pending writes */
 		struct bio *next = bio->bi_next;
 		bio->bi_next = NULL;
-		generic_make_request(bio);
+		if (unlikely((bio->bi_rw & REQ_DISCARD) &&
+		    !blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
+			/* Just ignore it */
+			bio_endio(bio, 0);
+		else
+			generic_make_request(bio);
 		bio = next;
 	}
 	kfree(plug);