libata extension

Good Day

Say i want to implement extended set of ATA commands available to userspace for building diagnostic tools.
I need 0x40 -- read verify and 0x32 -- write long with error handling,

for example. I was trying ide driver through ioctl's, but seems it lack of functionality and full of gotchas. Furthermore it oopses sometimes.

Is it possible to use libata for such purpose or i need to write separate IDE driver ?

Good Day

Say i want to implement extended set of ATA commands available to userspace for building diagnostic tools.
I need 0x40 -- read verify and 0x32 -- write long with error handling, for example. I was trying ide driver through ioctl's, but seems it lack of functionality and full of gotchas. Furthermore it oopses sometimes.

Use the SCSI SG_IO ioctl() with opcode=ATA_16, which gives you access to the ATA Passthrough mechanism. This will work for most ATA commands.

I already use it in hdparm and in some other utilities for scanning/repairing drives.

A notable exeception are the READ/WRITE LONG opcodes, which require an extra kernel patch from me, awaiting merge into libata some year.

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

Quoted 3 lines · On Mon, 12 Mar 2007 15:08:19 -0400 Mark Lord <[email protected]> wrote:

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

Quoted 3 lines

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

Do you still get hangs with ata_piix if you use R/W LONG and these changes ?

Quoted 10 lines · Alan Cox

Quoted 6 lines · On Mon, 12 Mar 2007 15:08:19 -0400 Mark Lord <[email protected]> wrote:

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

Quoted 2 lines

Do you still get hangs with ata_piix if you use R/W LONG and these changes ?

Err.. I *never* got hangs with ata_piix and these patches. Either that, or the Alzheimer's is getting worse. ;)

Umm... someone reported originally getting problems with R/W LONG and PIIX but not AHCI and we discussed whether we might need to turn the prefetch/postwrite off for it. Would have been about October last year.

If it works ok then I'll just cross it off the list

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

This patch adds support to libata for READ/WRITE LONG commands issued via SG_IO/ATA_16.

This patch was generated against a 2.6.21-rc3-git7 base:

Quoted 7 lines · Mark Lord

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

Quoted 8 lines

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

This patch adds support to libata for READ/WRITE LONG commands issued via SG_IO/ATA_16.

This patch was generated against a 2.6.21-rc3-git7 base:

I think it would be better if this comes in two patches. One to add qc->sect_size and convert all users of ATA_SECT_SIZE to qc->sect_size and the other one to implement READ/WRITE LONG. Another question is whether this needs to be included into mainline. This is definitely useful but it is mostly for debugging/testing.

Hmmm... But we're gonna need qc->sect_size anyway for devices with larger sector sizes and overhead for supporting READ/WRITE LONG is nearly nill, so I'm voting for inclusion.

Thanks.

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

This patch adds support to libata for READ/WRITE LONG commands issued via SG_IO/ATA_16.

This patch was generated against a 2.6.21-rc3-git7 base:

drivers/ata/libata-core.c |   20 ++++++++++----------
drivers/ata/libata-scsi.c |   12 ++++++++++++
include/linux/ata.h       |    6 ++++++
include/linux/libata.h    |    2 ++
4 files changed, 30 insertions(+), 10 deletions(-)

Signed-off-by Mark Lord <[email protected]>

Quoted 2 lines

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

Do we always know the worst case here as this breaks my pending patch to use bounce buffers for PIO so we transfer with IRQ enabled and I need to know the correct new worst case size.

Ugh. The drives default to 520 bytes (always), but can specify a different, preferred, length in the IDENTIFY data. A SETFEATURES command is required to change the setting from the default of 520.

Quoted 6 lines · Alan Cox

Quoted 2 lines

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

Do we always know the worst case here as this breaks my pending patch to use bounce buffers for PIO so we transfer with IRQ enabled and I need to know the correct new worst case size.

Ugh. The drives default to 520 bytes (always),

but can specify a different, preferred, length in the IDENTIFY data. A SETFEATURES command is required to change the setting from the default of 520.

In practice, I only ever use 520, because (1) it always works, and (2) I'm only trying to corrupt drives, not write correct stuff. ;)

So we could modify the patch to limit it to a specific maximum, or always require exactly 520 bytes without any hardship.

???

Quoted 1 line · Mark Lord

Ugh. The drives default to 520 bytes (always),

The old IDE drives used to have 4 and 7 byte ECC (seen in ancient
Phoenix BIOS).

9.9.8 Word 22: ECC bytes available on read/write long commands

If the contents of this field are set to a value other than 4, the only way to use this information is via the Set Features commands.

Section 2.1.7: ... The default length of the vendor specific bytpes associated with the READ LONG and WRITE LONG commands is four bytes, but may be changed by use of the SET FEATURES command.

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

This patch adds support to libata for READ/WRITE LONG commands issued via SG_IO/ATA_16.

Signed-off-by Mark Lord <[email protected]>

---
--- linux/include/linux/libata.h.orig 2007-03-07 09:20:04.000000000
-0500
+++ linux/include/linux/libata.h 2007-03-12 13:16:29.000000000 -0400

(quoted diff) +1 -0

@@ -1171,6 +1172,7 @@
    qc->n_elem = 0;
    qc->err_mask = 0;
    qc->pad_len = 0;
+    qc->sect_size = ATA_SECT_SIZE;
 
    ata_tf_init(qc->dev, &qc->tf);
 

include/linux/ata.h 2007-03-12 13:15:45.000000000 -0400 renamed from include/linux/ata.h.orig 2007-03-07 09:21:25.000000000 -0500 +6 -1

@@ -163,6 +163,12 @@
    /* READ_LOG_EXT pages */
    ATA_LOG_SATA_NCQ    = 0x10,
 
+    /* READ/WRITE LONG (obsolete) */
+    ATA_CMD_READ_LONG    = 0x22,
+    ATA_CMD_READ_LONG_ONCE    = 0x23,
+    ATA_CMD_WRITE_LONG    = 0x32,
+    ATA_CMD_WRITE_LONG_ONCE    = 0x33,
+
    /* SETFEATURES stuff */
    SETFEATURES_XFER    = 0x03,
    XFER_UDMA_7        = 0x47,
--- linux/drivers/ata/libata-scsi.c.orig    2007-03-07 

09:21:13.000000000 -0500
+++ linux/drivers/ata/libata-scsi.c 2007-03-12 13:14:46.000000000 -0400

include/linux/ata.h 2007-03-12 13:15:45.000000000 -0400 renamed from include/linux/ata.h.orig 2007-03-07 09:21:25.000000000 -0500 +12 -1

@@ -2678,6 +2678,18 @@
        tf->device = qc->dev->devno ?
            tf->device | ATA_DEV1 : tf->device & ~ATA_DEV1;
 
+    /* READ/WRITE LONG use a non-standard sect_size */
+    qc->sect_size = ATA_SECT_SIZE;
+    switch (tf->command) {
+    case ATA_CMD_READ_LONG:
+    case ATA_CMD_READ_LONG_ONCE:
+    case ATA_CMD_WRITE_LONG:
+    case ATA_CMD_WRITE_LONG_ONCE:
+        if (tf->protocol != ATA_PROT_PIO || tf->nsect != 1)
+            goto invalid_fld;
+        qc->sect_size = scmd->request_bufflen;
+    }
+
    /*
     * Filter SET_FEATURES - XFER MODE command -- otherwise,
     * SET_FEATURES - XFER MODE must be preceded/succeeded
--- linux/drivers/ata/libata-core.c.orig    2007-03-12 

11:22:43.000000000 -0400
+++ linux/drivers/ata/libata-core.c 2007-03-12 13:08:28.000000000 -0400

include/linux/ata.h 2007-03-12 13:15:45.000000000 -0400 renamed from include/linux/ata.h.orig 2007-03-07 09:21:25.000000000 -0500 +5 -5

@@ -4032,10 +4032,10 @@
 
 
 /**
- *    ata_pio_sector - Transfer ATA_SECT_SIZE (512 bytes) of data.
+ *    ata_pio_sector - Transfer a sector of data.
 *    @qc: Command on going
 *
- *    Transfer ATA_SECT_SIZE of data from/to the ATA device.
+ *    Transfer qc->sect_size bytes of data from/to the ATA device.
 *
 *    LOCKING:
 *    Inherited from caller.

@@ -4050,7 +4050,7 @@
    unsigned int offset;
    unsigned char *buf;
 
-    if (qc->curbytes == qc->nbytes - ATA_SECT_SIZE)
+    if (qc->curbytes == qc->nbytes - qc->sect_size)
        ap->hsm_task_state = HSM_ST_LAST;
 
    page = sg[qc->cursg].page;

@@ -4070,17 +4070,17 @@
        buf = kmap_atomic(page, KM_IRQ0);
 
        /* do the actual data transfer */
-        ap->ops->data_xfer(qc->dev, buf + offset, ATA_SECT_SIZE, 
 do_write);
+        ap->ops->data_xfer(qc->dev, buf + offset, qc->sect_size, 
 do_write);
 
 
        kunmap_atomic(buf, KM_IRQ0);
        local_irq_restore(flags);
    } else {
        buf = page_address(page);
-        ap->ops->data_xfer(qc->dev, buf + offset, ATA_SECT_SIZE, 
 do_write);
+        ap->ops->data_xfer(qc->dev, buf + offset, qc->sect_size, 

Which requires from the drivers to be able to turn off IDE prefetch (and
maybe posting too). I don't see that in this patch (or are you expecting them to just "snoop' the commands and do it automagically?).

Quoted 2 lines

Which requires from the drivers to be able to turn off IDE prefetch (and
maybe posting too). I don't see that in this patch (or are you expecting them to just "snoop' the commands and do it automagically?).

We need to see which drivers have that problem and if its only the odd one then we can handle it in qc_issue. We've already got controllers that wrap qc_issue to peek at commands and reject some of them (eg the 8212 in hardware raid mode)

Alan

Quoted 2 lines

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

Which requires from the drivers to be able to turn off IDE prefetch
(and maybe posting too). I don't see that in this patch (or are you expecting them to just "snoop' the commands and do it automagically?).

Again, ata_data_xfer() doesn't seem capable of performing ECC read/writes
-- the ECC bytes must be transferred in 8-bit mode, AFAIR. ata_data_xfer() can oinly do that for optionally trailing odd byte.

Quoted 5 lines

Quoted 2 lines

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

Which requires from the drivers to be able to turn off IDE prefetch
(and maybe posting too). I don't see that in this patch (or are you expecting them to just "snoop' the commands and do it automagically?).

Most controllers seem to be behave well enough
and just not muck up. And if it fails on any "too clever" controllers, then no big deal -- we just switch to a controller that works instead.

This is not an essential command for normal operation. What it is for, is injecting errors so that we can experience firsthand the (excellent or crappy) error handling in the kernel and in applications. And then work out and test better strategies.

So 100% working everywhere, for an obsolete opcode, isn't really essential to me. But it should work most places, and it does.

Quoted 5 lines

Again, ata_data_xfer() doesn't seem capable of performing ECC
read/writes
-- the ECC bytes must be transferred in 8-bit mode, AFAIR. ata_data_xfer()
can oinly do that for optionally trailing odd byte.

I have no idea what that was all about. Care to explain again?

RWLONG transfer the ECC info 8-bits at a time, using 16-bit words to do so, no different from normal. ???

Here it is again with the whitespace not so messed up.

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

This patch adds support to libata for READ/WRITE LONG commands issued via SG_IO/ATA_16.

This patch was generated against a 2.6.21-rc3-git7 base:

Hello.

Mark Lord wrote:

Quoted 7 lines

Quoted 5 lines

Again, ata_data_xfer() doesn't seem capable of performing ECC
read/writes
-- the ECC bytes must be transferred in 8-bit mode, AFAIR. ata_data_xfer()
can oinly do that for optionally trailing odd byte.

I have no idea what that was all about. Care to explain again?

Care to read the standards? :-/

Quoted 2 lines

RWLONG transfer the ECC info 8-bits at a time, using 16-bit words to do so, no different from normal. ???

From ATA-1:

"The transfer of the vendor specific bytes shall be one byte at a
time over bits DD0-7 only (8-bits wide)."

Yes, I said that already. 8-bits at a time, but using 16-bit transfers. Kinda like it says here, in ATA-3 section 7.16:

The transfer of the vendor specifc bytes shall be 16 bit transfers with the vendor specific byte in bits 7 through 0. Bits 15 through 8 shall be ignored by the host.

Hello.

Mark Lord wrote:

Quoted 5 lines

Quoted 3 lines

Again, ata_data_xfer() doesn't seem capable of performing ECC read/writes
-- the ECC bytes must be transferred in 8-bit mode, AFAIR. ata_data_xfer() can oinly do that for optionally trailing odd byte.

I have no idea what that was all about. Care to explain again?

Care to read the standards? :-/

Quoted 2 lines

RWLONG transfer the ECC info 8-bits at a time, using 16-bit words to do so, no different from normal. ???

From ATA-1:

"The transfer of the vendor specific bytes shall be one byte at a time over bits DD0-7 only (8-bits wide)."

The transfer of the vendor specifc bytes shall be 16 bit transfers with the vendor specific byte in bits 7 through 0. Bits 15 through 8 shall be ignored by the host.

Quoted 12 lines

Quoted 7 lines

Quoted 5 lines

Again, ata_data_xfer() doesn't seem capable of performing ECC
read/writes
-- the ECC bytes must be transferred in 8-bit mode, AFAIR. ata_data_xfer()
can oinly do that for optionally trailing odd byte.

I have no idea what that was all about. Care to explain again?

Care to read the standards? :-/

Quoted 2 lines

RWLONG transfer the ECC info 8-bits at a time, using 16-bit words to do so, no different from normal. ???

Quoted 4 lines

From ATA-1:

"The transfer of the vendor specific bytes shall be one byte at a
time over bits DD0-7 only (8-bits wide)."

Yes, I said that already. 8-bits at a time, but using 16-bit transfers. Kinda like it says here, in ATA-3 section 7.16:

Quoted 3 lines

The transfer of the vendor specifc bytes shall be 16 bit transfers with the vendor specific byte in bits 7 through 0. Bits 15 through 8 shall be ignored by the host.

But if we really want to be 100% compliant, we could consider dropping to PIO0 for the command.

Not worth it, though, as in practice this is
not necessary, and it would mess up libata far more than is worthwhile for this effort.

Cheers

..
Well, those ATA specs have always been quite messy: for example,
polling protocol had an unnoticed race for years (device was allowed to clear BSY before asserting INTRQ, so there was no guarantee that the host's reading of the status reg. will actually clear drive's interrupt)...

NOTE 31 - Some ATA-1 devices are not capable of delivering the 8 bit ECC immediately after the word sector data. BIOS and driver developers should use PIO mode 0 for 8 bit ECC accesses.

But if we really want to be 100% compliant, we could consider dropping to PIO0 for the command. Not worth it, though, as in practice this is not necessary, and it would mess up libata far more than is worthwhile for this effort.

Cheers

I think it only requires the *host* to drop to PIO0 timings. In which case it should be achievable w/o libata modification -- if the driver has to "snoop" command and turn off prefetch, why not switch to PIO0 temporarily?

I think it only requires the *host* to drop to PIO0 timings. In which
case it should be achievable w/o libata modification -- if the driver has to "snoop" command and turn off prefetch, why not switch to PIO0 temporarily?

Quoted 2 lines

I think it only requires the *host* to drop to PIO0 timings. In which
case it should be achievable w/o libata modification -- if the driver has to "snoop" command and turn off prefetch, why not switch to PIO0 temporarily?

This isn't a big issue. Eventually we have to support sending speed change commands and once we do that the caller will be able to switch to PIO0 and back again.

Quoted 6 lines · Alan Cox

Quoted 2 lines

I think it only requires the *host* to drop to PIO0 timings. In
which case it should be achievable w/o libata modification -- if the driver has to "snoop" command and turn off prefetch, why not switch to PIO0 temporarily?

This isn't a big issue. Eventually we have to support sending speed change commands and once we do that the caller will be able to switch to PIO0 and back again.

We should be pretty close to that. One of the bigger blockages is needing to stop (freeze) all ports, before performing an out-of-band SET FEATURES and related tuning.

Quoted 8 lines · Jeff Garzik

Quoted 6 lines · Alan Cox

Quoted 2 lines

I think it only requires the *host* to drop to PIO0 timings. In
which case it should be achievable w/o libata modification -- if the driver has to "snoop" command and turn off prefetch, why not switch to PIO0 temporarily?

This isn't a big issue. Eventually we have to support sending speed change commands and once we do that the caller will be able to switch to PIO0 and back again.

Quoted 1 line

We should be pretty close to that. One of the bigger blockages is needing to stop (freeze) all ports, before performing an out-of-band SET FEATURES and related tuning.

For this particular feature, READ/WRITE LONG,
none of this is actually necessary in practice on common controllers.

I've tested ATA1 and newer PATA drives, and various SATA drives with these commands without bothering to drop to PIO0, and none of them had issues.

Quoted 10 lines · Jeff Garzik

Quoted 8 lines · Alan Cox

Quoted 4 lines

I think it only requires the *host* to drop to PIO0 timings. In
which case it should be achievable w/o libata modification -- if the driver has to "snoop" command and turn off prefetch, why not switch to PIO0 temporarily?

This isn't a big issue. Eventually we have to support sending speed change commands and once we do that the caller will be able to switch to PIO0 and back again.

Quoted 3 lines

We should be pretty close to that. One of the bigger blockages is needing to stop (freeze) all ports, before performing an out-of-band SET FEATURES and related tuning.

For this particular feature, READ/WRITE LONG,
none of this is actually necessary in practice on common controllers.

I've tested ATA1 and newer PATA drives, and various SATA drives with these commands without bothering to drop to PIO0, and none of them had issues.

On Friday 16 March 2007, Mark Lord wrote:

Quoted 3 lines

I've tested ATA1 and newer PATA drives, and various SATA drives with these commands without bothering to drop to PIO0, and none of them had issues.

This is what really matters wrt libata READ/WRITE LONG support.

These commands are not supposed to be used by normal users anyway so patch while not being "perfect" is "good enough" to be merged IMO.

Quoted 1 line · Bartlomiej Zolnierkiewicz

On Friday 16 March 2007, Mark Lord wrote:

..

Quoted 4 lines

Quoted 3 lines

I've tested ATA1 and newer PATA drives, and various SATA drives with these commands without bothering to drop to PIO0, and none of them had issues.

Quoted 4 lines

This is what really matters wrt libata READ/WRITE LONG support.

These commands are not supposed to be used by normal users anyway so patch while not being "perfect" is "good enough" to be merged IMO.

Heh.. I have a similar patch for IDE as well,

but not up-to-date for the latest kernels.

I've tested ATA1 and newer PATA drives, and various SATA drives with these commands without bothering to drop to PIO0, and none of them had issues.

Cheers

Quoted 5 lines · Sergei Shtylyov

Quoted 2 lines

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

Which requires from the drivers to be able to turn off IDE prefetch
(and maybe posting too). I don't see that in this patch (or are you expecting them to just "snoop' the commands and do it automagically?).

Most controllers seem to be behave well enough
and just not muck up. And if it fails on any "too clever" controllers, then no big deal -- we just switch to a controller that works instead.

This is not an essential command for normal operation. What it is for, is injecting errors so that we can experience firsthand the (excellent or crappy) error handling in the kernel and in applications. And then work out and test better strategies.

So 100% working everywhere, for an obsolete opcode, isn't really essential to me. But it should work most places, and it does.

Quoted 5 lines

Again, ata_data_xfer() doesn't seem capable of performing ECC
read/writes
-- the ECC bytes must be transferred in 8-bit mode, AFAIR. ata_data_xfer()
can oinly do that for optionally trailing odd byte.

I have no idea what that was all about. Care to explain again? RWLONG transfer the ECC info 8-bits at a time, using 16-bit words to do so, no different from normal. ???

Here it is again with the whitespace not so messed up.

Thanks.

The READ/WRITE LONG commands are theoretically obsolete, but the majority of drives in existance still implement them.

The WRITE_LONG and WRITE_LONG_ONCE commands are of particular interest for fault injection testing -- eg. creating "media errors" at specific locations on a disk.

The fussy bit is that these commands require a non-standard sector size, usually 520 bytes instead of 512.

This patch adds support to libata for READ/WRITE LONG commands issued via SG_IO/ATA_16.

This patch was generated against a 2.6.21-rc3-git7 base:

drivers/ata/libata-core.c |   20 ++++++++++----------
drivers/ata/libata-scsi.c |   12 ++++++++++++
include/linux/ata.h       |    6 ++++++
include/linux/libata.h    |    2 ++
4 files changed, 30 insertions(+), 10 deletions(-)

Signed-off-by Mark Lord <[email protected]>

Why is the access to Control register needed?

Quoted 2 lines

In the perfect case i would like to be able to execute vendor command set (reverse engineered).

Sounds interesting. :-)

Could you give some more details on what are you going to implement?

Quoted 1 line

Why is the access to Control register needed?

To execute soft reset for example.

Quoted 3 lines

Quoted 2 lines

In the perfect case i would like to be able to execute vendor command set (reverse engineered).

Quoted 3 lines

Sounds interesting. :-)

Could you give some more details on what are you going to implement?

Reading/writing service area, uploading, downloading modules, working with flash etc.