Skip to contentSkip to conversation

Worrisome IDE PIO transfers...

DISCUSSION Jeff Garzik 7 participants 17 messages Feb 28, 2004 – Mar 1, 2004

lore
Share & tools
lore
#0 lore raw

Looking at the function that is used to transfer data when in PIO mode...

void taskfile_output_data (ide_drive_t *drive, void *buffer, u32 wcount)

{
        if (drive->bswap) {
                ata_bswap_data(buffer, wcount);
                HWIF(drive)->ata_output_data(drive, buffer, wcount);
                ata_bswap_data(buffer, wcount);
        } else {
                HWIF(drive)->ata_output_data(drive, buffer, wcount);
        }
}

Swapping the data in-place is very, very wrong... you don't want to be touching the data that userspace might have mmap'd ... Additionally, byteswapping back and forth for each PIO sector chews unnecessary CPU.

Seems to me the architecture's OUTS[WL] hook (or a new, similar hook) that swaps as it writes would be _much_ preferred, and eliminate this possible data corruption issue.

Jeff

#1 lore raw

[ Geert added to cc: ]

Quoted 13 lines · Jeff Garzik

Looking at the function that is used to transfer data when in PIO mode...

void taskfile_output_data (ide_drive_t *drive, void *buffer, u32 wcount)

{
         if (drive->bswap) {
                 ata_bswap_data(buffer, wcount);
                 HWIF(drive)->ata_output_data(drive, buffer, wcount);
                 ata_bswap_data(buffer, wcount);
         } else {
                 HWIF(drive)->ata_output_data(drive, buffer, wcount);
         }
}
Quoted 3 lines

Swapping the data in-place is very, very wrong... you don't want to be
touching the data that userspace might have mmap'd ... Additionally, byteswapping back and forth for each PIO sector chews unnecessary CPU.

This is used for accessing "normal" disks on beasts with byte-swapped IDE bus (Atari/Q40/TiVo) and "byteswapped" disks on normal machines.

[ Hm. actually I don't see how it can be used for accessing "normal" disks,
as data is byteswapped by IDE bus and then swapped back by IDE driver. ]

Manfred noticed the same issue some time ago:
http://www.ussg.iu.edu/hypermail/linux/kernel/0201.0/0768.html
but discussion ended without final conclusion.

I like Alan's idea to use loopback instead of "bswap".

Quoted 3 lines

Seems to me the architecture's OUTS[WL] hook (or a new, similar hook) that swaps as it writes would be _much_ preferred, and eliminate this possible data corruption issue.

I think something similar has been already done
(grep for insw_swapw/outsw_swapw in ide-iops.c and asm-m68k/ide.h).

Bartlomiej

#2 lore raw
Quoted 22 lines · Bartlomiej Zolnierkiewicz

[ Geert added to cc: ]

Quoted 13 lines · Jeff Garzik

Looking at the function that is used to transfer data when in PIO mode...

void taskfile_output_data (ide_drive_t *drive, void *buffer, u32 wcount)

{
        if (drive->bswap) {
                ata_bswap_data(buffer, wcount);
                HWIF(drive)->ata_output_data(drive, buffer, wcount);
                ata_bswap_data(buffer, wcount);
        } else {
                HWIF(drive)->ata_output_data(drive, buffer, wcount);
        }
}
Quoted 3 lines

Swapping the data in-place is very, very wrong... you don't want to be
touching the data that userspace might have mmap'd ... Additionally, byteswapping back and forth for each PIO sector chews unnecessary CPU.

Quoted 5 lines

This is used for accessing "normal" disks on beasts with byte-swapped IDE bus (Atari/Q40/TiVo) and "byteswapped" disks on normal machines.

[ Hm. actually I don't see how it can be used for accessing "normal" disks,
as data is byteswapped by IDE bus and then swapped back by IDE driver. ]

Yeah, just byteswapped disks are affected.

Quoted 5 lines

Manfred noticed the same issue some time ago:
http://www.ussg.iu.edu/hypermail/linux/kernel/0201.0/0768.html
but discussion ended without final conclusion.

I like Alan's idea to use loopback instead of "bswap".

Neat but no more zerocopy that way. I much prefer a swap-as-you-go...

Quoted 6 lines
Quoted 3 lines

Seems to me the architecture's OUTS[WL] hook (or a new, similar hook) that swaps as it writes would be _much_ preferred, and eliminate this possible data corruption issue.

I think something similar has been already done
(grep for insw_swapw/outsw_swapw in ide-iops.c and asm-m68k/ide.h).

Yeah, but this would need to be per-device... I agree all the other pieces are already present.

Jeff

#3 lore raw
Quoted 3 lines · Jeff Garzik
Quoted 1 line

I like Alan's idea to use loopback instead of "bswap".

Neat but no more zerocopy that way. I much prefer a swap-as-you-go...

Okay, better solution:

- on Atari/Q40:
if drive->bswap use insw/outsw instead of swapping variants

- on others:
use device mapper as suggested by Matt Mackall
(no extra copying and you can use DMA to read disk from TiVo!)

Bartlomiej

#4 lore raw
Quoted 9 lines · Bartlomiej Zolnierkiewicz
Quoted 3 lines · Jeff Garzik
Quoted 1 line

I like Alan's idea to use loopback instead of "bswap".

Neat but no more zerocopy that way. I much prefer a swap-as-you-go...

Okay, better solution:

- on Atari/Q40:
if drive->bswap use insw/outsw instead of swapping variants

Yep, that sounds the most logical. Richard?

Quoted 3 lines

- on others:
use device mapper as suggested by Matt Mackall
(no extra copying and you can use DMA to read disk from TiVo!)

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#5 lore raw
Quoted 4 lines · Geert Uytterhoeven
Quoted 3 lines · Bartlomiej Zolnierkiewicz
Quoted 2 lines · Jeff Garzik
Quoted 1 line

I like Alan's idea to use loopback instead of "bswap".

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Neat but no more zerocopy that way. I much prefer a swap-as-you-go...

Okay, better solution:

- on Atari/Q40:
if drive->bswap use insw/outsw instead of swapping variants

Yep, that sounds the most logical. Richard?

looks good.

However it appears to fix only part of the problem - we need some logic to ensure only disk data is swapped.
Bswapping WIN_DOWNLOAD_MICROCODE data would not be very clever I guess.

Richard

#6 lore raw
Quoted 13 lines · Richard Zidlicky
Quoted 12 lines · Geert Uytterhoeven
Quoted 10 lines · Bartlomiej Zolnierkiewicz
Quoted 4 lines · Jeff Garzik
Quoted 1 line

I like Alan's idea to use loopback instead of "bswap".

Neat but no more zerocopy that way. I much prefer a swap-as-you-go...

Okay, better solution:

- on Atari/Q40:
if drive->bswap use insw/outsw instead of swapping variants

Quoted 8 lines
Quoted 1 line

Yep, that sounds the most logical. Richard?

looks good.

However it appears to fix only part of the problem - we need some logic to ensure only disk data is swapped.
Bswapping WIN_DOWNLOAD_MICROCODE data would not be very clever I guess.

Actually drive->bswap should die as I overlooked the fact that we are _not_ swapping disk data (byteswapped data is used for FS) on Atari/Q40.

Therefore the real solution is to use device-mapper instead of drive->bswap and on Atari/Q40 use standard insw/outs only if blk_fs_request(drive->rq), for everything else insw_swapw/outsw_swapw should be used.

Does it make any sense? :)

Bartlomiej

#7 lore raw
Quoted 18 lines · Bartlomiej Zolnierkiewicz
Quoted 17 lines · Richard Zidlicky
Quoted 7 lines · Geert Uytterhoeven
Quoted 6 lines · Bartlomiej Zolnierkiewicz
Quoted 4 lines · Jeff Garzik
Quoted 1 line

I like Alan's idea to use loopback instead of "bswap".

Neat but no more zerocopy that way. I much prefer a swap-as-you-go...

Quoted 6 lines
Quoted 4 lines

Okay, better solution:

- on Atari/Q40:
if drive->bswap use insw/outsw instead of swapping variants

Yep, that sounds the most logical. Richard?

looks good.

Quoted 7 lines
Quoted 4 lines

However it appears to fix only part of the problem - we need some logic to ensure only disk data is swapped.
Bswapping WIN_DOWNLOAD_MICROCODE data would not be very clever I guess.

Actually drive->bswap should die as I overlooked the fact that we are _not_ swapping disk data (byteswapped data is used for FS) on Atari/Q40.

correct, on those machines the IDE bus is wired "reversed" and we take the data without any correction, except for IDENTIFY and atapi requests. That means that quite a few ioctls (SMART etc) are most likely broken right now.

Quoted 5 lines

Therefore the real solution is to use device-mapper instead of drive->bswap and on Atari/Q40 use standard insw/outs only if blk_fs_request(drive->rq), for everything else insw_swapw/outsw_swapw should be used.

Does it make any sense? :)

that would be the perfect solution. Note that atapi transfers are already correct the way they are - nothing to fix here.

Richard

#8 lore raw
In reply to Bartlomiej Zolnierkiewicz
Quoted 1 line · Bartlomiej Zolnierkiewicz

I like Alan's idea to use loopback instead of "bswap".

Or, more likely, device mapper.

Signature · 1 line
Matt Mackall : http://www.selenic.com : Linux development and consulting
#9 lore raw
Quoted 6 lines · Matt Mackall
Quoted 1 line · Bartlomiej Zolnierkiewicz

I like Alan's idea to use loopback instead of "bswap".

Or, more likely, device mapper.

Somehow I doubt anybody cares enough to write a whole driver just for this unlikely case.

For now let's at least record the knowledge... (patch attached)

Jeff

===== drivers/ide/ide-taskfile.c 1.28 vs edited =====

drivers/ide/ide-taskfile.c +7 -1
@@ -29,6 +29,7 @@
 
 #include <linux/config.h>
 #include <linux/module.h>
+#include <linux/compiler.h>
 #include <linux/types.h>
 #include <linux/string.h>
 #include <linux/kernel.h>
@@ -81,7 +82,12 @@
 
 void taskfile_output_data (ide_drive_t *drive, void *buffer, u32 wcount)
 {
-	if (drive->bswap) {
+	if (unlikely(drive->bswap)) {
+		/* FIXME: Besides the inefficiency each sector
+		 * twice, this can lead to data corruption on
+		 * SMP.  Fortunately drives that need this swapping
+		 * are quite uncommon.
+		 */
 		ata_bswap_data(buffer, wcount);
 		HWIF(drive)->ata_output_data(drive, buffer, wcount);
 		ata_bswap_data(buffer, wcount);
#10 lore raw
Quoted 8 lines · Jeff Garzik
Quoted 4 lines · Matt Mackall
Quoted 1 line · Bartlomiej Zolnierkiewicz

I like Alan's idea to use loopback instead of "bswap".

Or, more likely, device mapper.

Somehow I doubt anybody cares enough to write a whole driver just for this unlikely case.

Any TiVo hacker reading this? :)

Quoted 1 line

For now let's at least record the knowledge... (patch attached)

Ok, thanks.

Bartlomiej

#11 lore raw
Quoted 12 lines · Bartlomiej Zolnierkiewicz
Quoted 8 lines · Jeff Garzik
Quoted 4 lines · Matt Mackall
Quoted 1 line · Bartlomiej Zolnierkiewicz

I like Alan's idea to use loopback instead of "bswap".

Or, more likely, device mapper.

Somehow I doubt anybody cares enough to write a whole driver just for this unlikely case.

Any TiVo hacker reading this? :)

ahem. now for the whole list instead of just Bart, (sorry Bart) in case any of you gurus have insights for me (even if it's just to tell me I'm biting off more than I can chew.)

I'm definitely reading, but not necessarily comprehending. I upgraded a 60GB TiVo to 2x80GB in about 2002 using 2.4 and some userland tools. in fact i was just contemplating doing another upgrade since the 80s in it now are on their way out i think, so i'll even have some spare TiVo drives to fool around with during the development of this thing. I'm probably not qualified to begin an undertaking like this, I'm just a student, but....I've always wanted to be a kernel hacker. ;-) but i'm not sure I understand what would be involved. And, does this mean I'll have to use 2.4 to do my next upcoming drive replacement? (replacing the drives in the tivo requires a host linux box on which to do it)

If someone would be kind enough to give me a little more info about the dm driver for tivo drives, i'd at least entertain the idea...

-Aubin

#12 lore raw
In reply to Jeff Garzik
Quoted 8 lines · Jeff Garzik
Quoted 4 lines · Matt Mackall
Quoted 1 line · Bartlomiej Zolnierkiewicz

I like Alan's idea to use loopback instead of "bswap".

Or, more likely, device mapper.

Somehow I doubt anybody cares enough to write a whole driver just for this unlikely case.

http://www.kernel.org/pub/linux/kernel/people/bart/dm-byteswap-2.6.4-rc1.patch

Guess what's this? :)

[ Not that I care so much but I always wanted to learn more about
device-mapper and crypto API. ]

It is simply a stripped down dm-crypt.c, so all credits go to Christophe. I have tested it quickly with loop device and it seems to work.

Bartlomiej

#13 lore raw

Am Mo, den 01.03.2004 schrieb Bartlomiej Zolnierkiewicz um 01:47:

Quoted 3 lines

http://www.kernel.org/pub/linux/kernel/people/bart/dm-byteswap-2.6.4-rc1.patch

Guess what's this? :)

The thieves... they've stolen my precioussss. ;)

Quoted 2 lines

It is simply a stripped down dm-crypt.c, so all credits go to Christophe. I have tested it quickly with loop device and it seems to work.

Yes, it's not that complicated. Looks good.
BTW: You don't need the km_types voodoo as the conversion routine is never called from a softirq context and you are allowed (but should try to avoid it) to sleep. You could add a conditional reschedule after kunmapping the buffers to keep the latency low on non-preempt kernels.

BTW: I've got some cleanups and a small fix in Andrew's latest tree (using a #define for the log prefix and I bvec array thingy).

#14 lore raw
Quoted 8 lines · Christophe Saout

Am Mo, den 01.03.2004 schrieb Bartlomiej Zolnierkiewicz um 01:47:

Quoted 4 lines

http://www.kernel.org/pub/linux/kernel/people/bart/dm-byteswap-2.6.4-rc1.
patch

Guess what's this? :)

The thieves... they've stolen my precioussss. ;)

Quoted 8 lines
Quoted 2 lines

It is simply a stripped down dm-crypt.c, so all credits go to Christophe. I have tested it quickly with loop device and it seems to work.

Yes, it's not that complicated. Looks good.
BTW: You don't need the km_types voodoo as the conversion routine is never called from a softirq context and you are allowed (but should try to avoid it) to sleep. You could add a conditional reschedule after kunmapping the buffers to keep the latency low on non-preempt kernels.

Yes, you are of course right. I will remove "km_types voodoo".

Quoted 2 lines

BTW: I've got some cleanups and a small fix in Andrew's latest tree (using a #define for the log prefix and I bvec array thingy).

Yep, I've seen them already.

Thanks,
Bartlomiej

#15 lore raw
In reply to Bartlomiej Zolnierkiewicz
Quoted 20 lines · Bartlomiej Zolnierkiewicz

[ Geert added to cc: ]

Quoted 13 lines · Jeff Garzik

Looking at the function that is used to transfer data when in PIO mode...

void taskfile_output_data (ide_drive_t *drive, void *buffer, u32 wcount)

{
         if (drive->bswap) {
                 ata_bswap_data(buffer, wcount);
                 HWIF(drive)->ata_output_data(drive, buffer, wcount);
                 ata_bswap_data(buffer, wcount);
         } else {
                 HWIF(drive)->ata_output_data(drive, buffer, wcount);
         }
}
Quoted 3 lines

Swapping the data in-place is very, very wrong... you don't want to be
touching the data that userspace might have mmap'd ... Additionally, byteswapping back and forth for each PIO sector chews unnecessary CPU.

Quoted 5 lines

This is used for accessing "normal" disks on beasts with byte-swapped IDE bus (Atari/Q40/TiVo) and "byteswapped" disks on normal machines.

[ Hm. actually I don't see how it can be used for accessing "normal" disks,
as data is byteswapped by IDE bus and then swapped back by IDE driver. ]

Why not? The only difference between normal' and byteswapped' disks is that the bytes in a 16-bit word are swapped (sic :-), so you can convert in both directions by swapping the bytes. Normal disks have been used on Atari before, so it should (still) work.

BTW, the generic tree misses this patch, which was deemed inappropriate before, but is needed to make sure the drive identification block is correct on those machines:

drivers/ide/ide-iops.c +17 -0
@@ -360,6 +360,23 @@
 	int i;
 	u16 *stringcast;
 
+#ifdef __mc68000__
+	if (!MACH_IS_AMIGA && !MACH_IS_MAC && !MACH_IS_Q40 && !MACH_IS_ATARI)
+		return;
+
+#ifdef M68K_IDE_SWAPW
+	if (M68K_IDE_SWAPW) {	/* fix bus byteorder first */
+		u_char *p = (u_char *)id;
+		u_char t;
+		for (i = 0; i < 512; i += 2) {
+			t = p[i];
+			p[i] = p[i+1];
+			p[i+1] = t;
+		}
+	}
+#endif
+#endif /* __mc68000__ */
+
 	id->config         = __le16_to_cpu(id->config);
 	id->cyls           = __le16_to_cpu(id->cyls);
 	id->reserved2      = __le16_to_cpu(id->reserved2);

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#16 lore raw
Quoted 26 lines · Geert Uytterhoeven
Quoted 25 lines · Bartlomiej Zolnierkiewicz

[ Geert added to cc: ]

Quoted 14 lines · Jeff Garzik

Looking at the function that is used to transfer data when in PIO mode...

void taskfile_output_data (ide_drive_t *drive, void *buffer, u32

wcount) {
         if (drive->bswap) {
                 ata_bswap_data(buffer, wcount);
                 HWIF(drive)->ata_output_data(drive, buffer, wcount);
                 ata_bswap_data(buffer, wcount);
         } else {
                 HWIF(drive)->ata_output_data(drive, buffer, wcount);
         }
}
Quoted 4 lines

Swapping the data in-place is very, very wrong... you don't want to
be touching the data that userspace might have mmap'd ... Additionally, byteswapping back and forth for each PIO sector chews unnecessary CPU.

This is used for accessing "normal" disks on beasts with byte-swapped IDE bus (Atari/Q40/TiVo) and "byteswapped" disks on normal machines.

Quoted 8 lines
Quoted 3 lines

[ Hm. actually I don't see how it can be used for accessing "normal" disks, as data is byteswapped by IDE bus and then swapped back by IDE driver. ]

Why not? The only difference between normal' and byteswapped' disks is that the bytes in a 16-bit word are swapped (sic :-), so you can convert in both directions by swapping the bytes. Normal disks have been used on Atari before, so it should (still) work.

Oh yes, IDE driver fixes byteorder on Atari/Q40 only when "bswap" is used.

Quoted 3 lines

BTW, the generic tree misses this patch, which was deemed inappropriate before, but is needed to make sure the drive identification block is correct on those machines:

Thanks. I can't see a smarter way to fix this now. :/

Bartlomiej