Skip to contentSkip to conversation

Fix data corruption with dm-crypt over RAID5

PATCH New Christophe Saout 3 participants 9 messages Dec 2, 2006 – Feb 23, 2007 7 messages not in the archive Device mapper

lore
Share & tools
lore
#7 lore raw

PATCH 1/1 Fix data corruption with dm-crypt over RAID5

In reply to a message not in the archive

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

drivers/md/dm-crypt.c +4 -2
@@ -717,13 +717,15 @@
 	if (bio->bi_size)
 		return 1;
 
+	if (!bio_flagged(bio, BIO_UPTODATE) && !error)
+		error = -EIO;
+                        
 	bio_put(bio);
 
 	/*
 	 * successful reads are decrypted by the worker thread
 	 */
-	if ((bio_data_dir(bio) == READ)
-	    && bio_flagged(bio, BIO_UPTODATE)) {
+	if (bio_data_dir(io->bio) == READ && !error) {
 		kcryptd_queue_io(io);
 		return 0;
 	}
#8 lore raw

PATCH 1/1 Fix data corruption with dm-crypt over RAID5

In reply to a message not in the archive

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

drivers/md/dm-crypt.c +4 -2
@@ -717,13 +717,15 @@
 	if (bio->bi_size)
 		return 1;
 
+	if (!bio_flagged(bio, BIO_UPTODATE) && !error)
+		error = -EIO;
+                        
 	bio_put(bio);
 
 	/*
 	 * successful reads are decrypted by the worker thread
 	 */
-	if ((bio_data_dir(bio) == READ)
-	    && bio_flagged(bio, BIO_UPTODATE)) {
+	if (bio_data_dir(io->bio) == READ && !error) {
 		kcryptd_queue_io(io);
 		return 0;
 	}
#9 lore raw

[Note: please Cc: [email protected] on -stable patches]

Quoted 4 lines · * Christophe Saout ([email protected]) wrote:

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

I take it this is fixed a different way in 2.6.19? Mind clarifying the difference?

drivers/md/dm-crypt.c line 722

Quoted 12 lines
drivers/md/dm-crypt.c +3 -0
@@ -717,13 +717,15 @@
 	if (bio->bi_size)
 		return 1;
 
+	if (!bio_flagged(bio, BIO_UPTODATE) && !error)
+		error = -EIO;
+                        

Minor nit: introduces trailing whitespaces, cleaned it up locally.

thanks,
-chris
--

This is a note to let you know that we have just queued up the patch titled

Subject: dm crypt: Fix data corruption with dm-crypt over RAID5

to the 2.6.18-stable tree. Its filename is

dm-crypt-fix-data-corruption-with-dm-crypt-over-raid5.patch

A git repo of this tree can be found at
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

Quoted 1 line

From linux-kernel-owner+chrisw=40sous-sol.org-S1162719AbWLBC2Z@vger.kernel.org Fri Dec 1 18:36:19 2006

Date: Sat, 02 Dec 2006 03:27:56 +0100
From: Christophe Saout <[email protected]>
To: [email protected]
Cc: [email protected], Andrey <[email protected]>, Andrew Morton <[email protected]>, [email protected], Neil Brown <[email protected]>, Jens Axboe <[email protected]>, Chris Wright <[email protected]>
Subject: dm crypt: Fix data corruption with dm-crypt over RAID5

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

---
 drivers/md/dm-crypt.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/md/dm-crypt.c +4 -2
@@ -717,13 +717,15 @@ static int crypt_endio(struct bio *bio, 
 	if (bio->bi_size)
 		return 1;
 
+	if (!bio_flagged(bio, BIO_UPTODATE) && !error)
+		error = -EIO;
+
 	bio_put(bio);
 
 	/*
 	 * successful reads are decrypted by the worker thread
 	 */
-	if ((bio_data_dir(bio) == READ)
-	    && bio_flagged(bio, BIO_UPTODATE)) {
+	if (bio_data_dir(io->bio) == READ && !error) {
 		kcryptd_queue_io(io);
 		return 0;
 	}
#10 lore raw

Am Freitag, den 01.12.2006, 19:49 -0800 schrieb Chris Wright:

Quoted 8 lines · * Christophe Saout ([email protected]) wrote:
Quoted 4 lines

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

I take it this is fixed a different way in 2.6.19? Mind clarifying the difference?

It's more or less fixed the same way in 2.6.19. The function was reordered by Milan Broz to accommodate the changed write path (commit 8b004457168995f2ae2a35327f885183a9e74141):

Quoted 7 lines

[PATCH] dm crypt: restructure for workqueue change

Restructure part of the dm-crypt code in preparation for workqueue changes.

Use 'base_bio' or 'clone' variable names consistently throughout. No functional changes are included in this patch.

"No functional changes" actually included the correct fix, accidental or not.

Quoted 1 line

Minor nit: introduces trailing whitespaces, cleaned it up locally.

Ouch, sorry.

#11 lore raw
In reply to Christophe Saout

On Sat, 2006-12-02 at 03:27 +0100, Christophe Saout wrote:

Hi Christophe:

I'm wondering about trying out your patch with dm-crypt on 2.6.12. The code in drivers/md/dm-crypt.c`crypt_endio() appears to be the same.

Is there a reason that this isn't necessary or would be a bad idea. Looks like the existing code isn't checking the BIO_UPTODATE flag before doing the bio_put(). Looks the the second part of not calling kcryptd_queue_io() and forwarding the processing to the cryptd is effectively the same. The 1st change will set error if BIO_UPTODATE isn't set and that will cause the 2nd change to skip calling kcryptd_queue_io().

I'm not sure about the change in the arg to bio_data_dir() changing from bio to io->bio. Perhaps they are equivalent; care to comment on that.

Unless I hear otherwise I'll try it out Tomorrow.

-piet

Quoted 34 lines

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

drivers/md/dm-crypt.c +4 -3
@@ -717,13 +717,15 @@
 	if (bio->bi_size)
 		return 1;
 
+	if (!bio_flagged(bio, BIO_UPTODATE) && !error)
+		error = -EIO;
+                        
 	bio_put(bio);
 
 	/*
 	 * successful reads are decrypted by the worker thread
 	 */
-	if ((bio_data_dir(bio) == READ)
-	    && bio_flagged(bio, BIO_UPTODATE)) {
+	if (bio_data_dir(io->bio) == READ && !error) {
 		kcryptd_queue_io(io);
 		return 0;
 	}
 
 
--

dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

Signature · 4 lines
Piet Delaney                                    Phone: (408) 200-5256
Blue Lane Technologies                          Fax:   (408) 200-5299
10450 Bubb Rd.
Cupertino, Ca. 95014                            Email: piet-QZGUWFDCMdVWk0Htik3J/[email protected]
#12 lore raw

Am Mittwoch, den 17.01.2007, 00:44 -0800 schrieb Piet Delaney:

Quoted 2 lines

I'm wondering about trying out your patch with dm-crypt on 2.6.12. The code in drivers/md/dm-crypt.c`crypt_endio() appears to be the same.

Yes, it should apply as well.

Quoted 7 lines

Is there a reason that this isn't necessary or would be a bad idea. Looks like the existing code isn't checking the BIO_UPTODATE flag before doing the bio_put(). Looks the the second part of not calling kcryptd_queue_io() and forwarding the processing to the cryptd is effectively the same. The 1st change will set error if BIO_UPTODATE isn't set and that will cause the 2nd change to skip calling kcryptd_queue_io().

Correct.

Quoted 3 lines

I'm not sure about the change in the arg to bio_data_dir() changing from bio to io->bio. Perhaps they are equivalent; care to comment on that.

It's just to fix the use-after-free case, apart from that all bio's involved have the same data direction, of course.

PS: Sorry because of the Wiki defacement, when I find some time I will look into it. Just reverting won't help this time as the bots are currently changing the pages at least once per hour.

#13 lore raw
Quoted 5 lines · Christophe Saout
Quoted 2 lines

I'm wondering about trying out your patch with dm-crypt on 2.6.12. The code in drivers/md/dm-crypt.c`crypt_endio() appears to be the

same.

Yes, it should apply as well.

Quoted 4 lines · Christophe Saout

Am Mittwoch, den 17.01.2007, 00:44 -0800 schrieb Piet Delaney:

Quoted 2 lines

I'm wondering about trying out your patch with dm-crypt on 2.6.12. The code in drivers/md/dm-crypt.c`crypt_endio() appears to be the

same.

Quoted 2 lines

Yes, it should apply as well.

I'll assume that also means that applying the patch is reasonable.

In our case we aren't using raid5; how ever we are using raid to mirror two root partitions. Your explanation to Andrew indicates the problem should only occur when using raid5. So applying the patch should be only have the effect of making it now possible to use raid5 with dm-crypt. Since we are currently running raid with root encryption it feels like it's better to include your patch.

-piet

Signature · 4 lines
Piet Delaney                                    Phone: (408) 200-5256
Blue Lane Technologies                          Fax:   (408) 200-5299
10450 Bubb Rd.
Cupertino, Ca. 95014                            Email: [email protected]
#14 lore raw
In reply to Christophe Saout
Quoted 6 lines · Christophe Saout

Am Mittwoch, den 17.01.2007, 00:44 -0800 schrieb Piet Delaney:

Quoted 2 lines

I'm wondering about trying out your patch with dm-crypt on 2.6.12. The code in drivers/md/dm-crypt.c`crypt_endio() appears to be the same.

Yes, it should apply as well.

I'll assume that also means that applying the patch is reasonable.

In our case we aren't using raid5; how ever we are using raid to mirror two root partitions. Your explanation to Andrew indicates the problem should only occur when using raid5. So applying the patch should be only have the effect of making it now possible to use raid5 with dm-crypt. Since we are currently running raid with root encryption it feels like it's better to include your patch.

-piet

#15 lore raw
In reply to Christophe Saout
Quoted 17 lines · Christophe Saout

Fix corruption issue with dm-crypt on top of software raid5. Cancelled readahead bio's that report no error, just have BIO_UPTODATE cleared were reported as successful reads to the higher layers (and leaving random content in the buffer cache). Already fixed in 2.6.19.

--- linux-2.6.18.orig/drivers/md/dm-crypt.c 2006-09-20 05:42:06.000000000
+0200
+++ linux-2.6.18/drivers/md/dm-crypt.c 2006-12-02 03:03:36.000000000 +0100

drivers/md/dm-crypt.c +2 -0
@@ -717,13 +717,15 @@
 if (bio->bi_size)
 return 1;
 
+     if (!bio_flagged(bio, BIO_UPTODATE) && !error)
+             error = -EIO;

drivers/md/dm-crypt.c line 729

Quoted 10 lines
drivers/md/dm-crypt.c +2 -2
+
 bio_put(bio);
 
 /*
 * successful reads are decrypted by the worker thread
 */
-     if ((bio_data_dir(bio) == READ)
-         && bio_flagged(bio, BIO_UPTODATE)) {
+     if (bio_data_dir(io->bio) == READ && !error) {
 kcryptd_queue_io(io);

Why doesn't kcryptd_queue_io() check the return value from queue_work()?

    476 static void kcryptd_queue_io(struct crypt_io *io)
    477 {
    478         INIT_WORK(&io->work, kcryptd_do_work, io);
    479         queue_work(_kcryptd_workqueue, &io->work);
    480 }

-piet

Quoted 2 lines
drivers/md/dm-crypt.c +0 -0
 return 0;
 }