a new SCSI driver

Hi all.

My little project is now ready to face the masses. Put simply, this driver allows one to present ATA (and soon ATAPI) devices through the Linux SCSI layer.

Since we already have an ATA (a.k.a. IDE) driver, I suppose the main question on a lot of people's minds is "why?" So I present some quick notes, in bullet form, that can form the basis of discussion.

Review is requested by all interested: driver authors, SCSI mavens, ATA mavens, whatever.

I request that people try to avoid this posting devolving into an ata-versus-scsi flamewar. Maybe I can defuse that before the fact, by
saying: "when in doubt, use drivers/ide."

So let's begin... comments and questions welcome!

Why SCSI?
---------

* Many of the advantages are derived the existence of the scsi mid-layer. It does a lot of work on our behalf, allowing me to focus on the ATA command protocols (PIO-in, PIO-out, DMA, etc.) almost exclusively.

* The SCSI mid-layer has shrunk and benefitted a lot from Axboe's block layer work. It's much more lightweight in 2.5.x.

* Serial ATA is looming quickly on the horizon. Both device and host controller SATA implementations really lend themselves to behaviors that have existed in SCSI for a while. SATA even defines use of SCSI Enclosure Services.

* The Linux SCSI layer handles hotplugging, and is more modular. It already has refcounted devices and sysfs and such. Creating a new block device driver from scratch means handling all those little details.

* SCSI has been doing basic error recovery and queue control for a while now. Upcoming SATA2 will benefit greatly from this, as well ATA TCQ if I ever get around to implementing the latter.

* ATAPI is SCSI-like.

* Note that PATA in my driver is only an afterthought. The main area of focus, now and in the future, is SATA. It has strict PATA device and host controller restrictions: must do UDMA, LBA, be at least ATA-3, etc. See "when in doubt..." statement above :)

Build notes
-----------

* Patch below requires you move drivers/scsi/{scsi,hosts,scsi_obsolete}.h to include/linux/scsi_{defs,hosts,obsolete}.h in order to build. These changes are present in the FTP site patches and BK repos, but are not included below to save space. They are obvious changes that can even be recreated by hand.

* You should disable CONFIG_IDE. Both drivers should request_region properly, but if you're not careful the IDE driver will grab the hardware before my driver does.

Testing status
--------------

* SATA stressed with iozone, bonnie, dbench, and some proprietary stress tools, on multiple boxes.

* PATA stressed similarly, on one box.

Near-future directions
----------------------

* ATAPI (see below)

* libata.c DMA and taskfile handling is still host-controller specific. It's the most widely used host controller standard, sure. But that mainly applies to PATA devices.

Future host controllers, with a tiny additional bit of abstracting-out, will simply ignore these functions (provided as defaults for most host controllers) and use their own.

* Better error handling (see below), and more command queueing work

ATA notes
---------

* Supports max UDMA/33 for PATA right now. Temporary limitation because I'm too slack to worry about cable detection.

* No packet command -- yet. And thus, no ATAPI cdroms/burners/etc. Coming soon!

* Does polling PIO in a kthread. Watch katad chew up CPU.

ATA hardware notes
------------------

* Intentionally concentrated on modern hardware: mainly SATA, with a little bit of PATA thrown in when convenient.

* Only supports Intel PATA and SATA right now

* Code is structured such that DMA engine and command queueing engine can be replaced by different hardware. Next hardware driver will demonstrate this.

SCSI protocol notes
-------------------

* claims compliance with latest scsi3 standards (sam3, spc3, sbc2). bug reports in spec compliance are welcome.

Resources
---------

* BitKeeper for 2.4.x and 2.5.x:

	bk://kernel.bkbits.net/jgarzik/atascsi-2.4
	bk://kernel.bkbits.net/jgarzik/atascsi-2.5

* Patchkits for 2.4.x and 2.5.x:

ftp://ftp.kernel.org/pub/linux/kernel/people/jgarzik/patchkits/2.4/2.4.21-rc3-atascsi1.patch.bz2 ftp://ftp.kernel.org/pub/linux/kernel/people/jgarzik/patchkits/2.5/2.5.69-bk17-atascsi1.patch.bz2

Open issues (currently being addressed)
---------------------------------------

* ATAPI. FWIW, yes, I do know that ATAPI is "SCSI-like" and not 100% conformant SCSI.

* Error handling. VERY primitive right now. Handling of errors is often "we'll stop talking to the device forever". Hey, it's better than data corruption. :)

Kudos (in alpha order for fairness :))
--------------------------------------

* Jens Axboe, for tons of block layer knowledge and general reality checks.

* James Bottomley, for patiently answering a lot of my SCSI questions.

* Alan Cox, for key inspirational ideas.

* Andre Hedrick, for tons of ATA knowledge. I've soaked up a ton of time talking ATA with him, for which I'll be forever grateful.

* Lots of people on IRC and linux-scsi for helpful answers and suggestions.

* myself, for progressing quite a bit on one of my Big Projects To Do Before I Die.

drivers/scsi/Kconfig    |   27 
drivers/scsi/Makefile   |    1 
drivers/scsi/ata_piix.c |  312 ++++++
drivers/scsi/libata.c   | 2239 ++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/ata.h     |  483 ++++++++++
5 files changed, 3062 insertions(+)

diff -Nru a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
@@ -400,6 +400,33 @@
 	  say M here and read <file:Documentation/modules.txt>.  The module
 	  will be called megaraid.
 
+config SCSI_ATA
+	bool "ATA support (EXPERIMENTAL)"
+	depends on SCSI && EXPERIMENTAL
+	help
+	  This driver family supports ATA host controllers and devices
+	  (a.k.a. IDE drives).
+
+	  If unsure, say N.
+
+config SCSI_ATA_PATA
+	bool "Parallel ATA support (EXPERIMENTAL)"
+	depends on SCSI_ATA && EXPERIMENTAL
+	help
+	  This option enables support for parallel ATA host controllers.
+
+	  If unsure, say N.
+
+config SCSI_ATA_PIIX
+	tristate "Intel PIIX/ICH PATA/SATA support"
+	depends on SCSI_ATA && PCI
+	help
+	  This option enables support for ICH5 Serial ATA.
+	  If PATA support was enabled previously, this enables
+	  support for select Intel PIIX/ICH PATA host controllers.
+
+	  If unsure, say N.
+
 config SCSI_BUSLOGIC
 	tristate "BusLogic SCSI support"
 	depends on SCSI

diff -Nru a/drivers/scsi/Makefile b/drivers/scsi/Makefile
@@ -113,6 +113,7 @@
 obj-$(CONFIG_SCSI_CPQFCTS)	+= cpqfc.o
 obj-$(CONFIG_SCSI_LASI700)	+= lasi700.o 53c700.o
 obj-$(CONFIG_SCSI_NSP32)	+= nsp32.o
+obj-$(CONFIG_SCSI_ATA_PIIX)	+= ata_piix.o libata.o
 
 obj-$(CONFIG_ARM)		+= arm/
 

Part of the diff is not shown here. View the raw message.

<snip/>

That last statement is interesting. SATA-1 has a maximum length of 1 metre and no external connector/cable definitions (I think). Serial attached SCSI (SAS) uses SATA-1 connector/cable definitions for internal connections and defines its own external connector/cable definitions (up to 6 metres). The SCSI Enclosure Services (SES) command set "is defined for managing various non-SCSI elements contained within the enclosure" (according to its draft standard abstract). So it seems that the 1 metre of SATA may be going to a external enclosure (e.g. RAID or JBOD). Recent SATA versus SAS comparisons have noted that even though SATA is a point to point protocol, fan out devices have been demonstrated.

BTW The SATA people at t13.org signed an agreement with the SAS folks at t10.org in February this year so hopefully there will be a convergence in several areas in the future.

This driver is very relevant to a SAS HBA which allows both SAS disks and SATA disks to be controlled **. The SAS standard envisages that such SATA disks will be controlled by a "normal" ATA application layer in the host machine.

The MMC standards that define the command set for CD/DVDs across ATAPI (as well as other transports, including 1394 and USB) are under the SCSI "umbrella" at t10.org . The MMC drafts have transport specific appendices that outline some of the compromises. Is the ATAPI transport used for anything other than carrying SCSI command sets?

<snip>

I loaded your patch and built it against lk 2.5.69-bk17. It compiled clean (without the patch to move the scsi headers into the ~linux/include/scsi directory), built and loaded.

However there were no signs of life from my Maxtor D740X-6L (P)ATA-5 disk. The PATA selection in the .config file doesn't have a module option. Actually there seems no specific code for PATA in your patch (apart from libata.c) and no mention of CONFIG_SCSI_ATA_PATA in drivers/scsi/Makefile .

Perhaps my IDE interface chip isn't supported:

00:04.1 IDE interface: VIA Technologies, Inc. VT82C586/B/686A/B PIPC
Bus Master IDE (rev 06)

Look forward to that.

<snip>

Thinking about how existing applications such as hdparm and smartmontools will cope with an ATA device (e.g. a disk) with a device node like "/dev/sdb". Those apps would make the assumption from the device name that such a device should be sent a SCSI command set. So perhaps we need a "transport" indicator in the sysfs directory for that device (dare I mention another ioctl).

** When a SATA disk is connected to a SAS HBA there needs to be an expander in between them that does the physical and link layer translation.

Doug Gilbert

Hi,

As of now, smartmontools uses /etc/smartd.conf in which we can specify a device type is ata or scsi
if the device name is not clear enough. (-d scsi or -d ata). I am not sure why this feature is there. Maybe devfs name thing.

Upon cursor examination,
I am not entirely sure whether we can
*always* put a meaningful "transport indicator" in the sysfs directory when there will be multiple
combination of transport layer(s) over the long term/haul. For this particular situation of SATA and SCSI, yes, though. (As the technology trend goes, I won't be surprised to find a home PC that hooks SCSI device via a few different transport layers such as serial, another different serial, say, USB, and other transport layer, say, firewire with some glue gadgets in between.
Whether such beast will be supported
under linux, I am not sure. But we do support USB storage device as a SCSI device, so there may be some demand. I am sure there will be some cheap interface boxes that probably work under some other OSs when everything works perfectly. I am not recommending it, but some people will be hooked to such setup.)

int main(void){int j=2003;/*(c)2003 cishikawa. */
char t[] ="<CI> @abcdefghijklmnopqrstuvwxyz.,\n\"";
char *i ="g>qtCIuqivb,gCwe\[email protected]\"tqkvv is>dnamz";
while(*i)((j+=strchr(t,*i++)-(int)t),(j%=sizeof t-1),
(putchar(t[j])));return 0;}/* under GPL */

A small correction to the code -
While translating the GET_CAPACITY to ATA, you are returning the number of sectors the ATA drive but the GET_CAPACITY requests the last addressable sector. So you should return (n_sectors-1).

static void ata_scsiop_read_cap(struct ata_scsi_args *args, u8 *reqbuf,
unsigned int buflen)

{
	u64 n_sectors = args->dev->n_sectors;
	u32 tmp;
+++ n_sectors --;
	VPRINTK("ENTER\n");

<snip>

Jeff,
Your patch applies to lk 2.5.70 without problems.

I found that my dual Celeron (abit mobo) has this
Intel IDE controller:
00:07.1 IDE interface: Intel Corp. 82371AB/EB/MB PIIX4 IDE (rev 01)

so I tried a 2.5.70 kernel with this patch. Two disks connected:
- SCSI disk to a Tekram 390u3w controller (sym53c8xx_2 driver). Root partition on this disk
- a Maxtor ATA disk (D740X-6L) to the Intel IDE controller.

Didn't get too far. After finding the SCSI controller and SCSI disk at boot up, this came out just before it locked up:
ata1:PATA max UDMA/33 cmd 0x1F0 ctl 0x3F6 bmdma 0xF000 irq 14

So I disabled the IDE controllers in the BIOS and then this came out:
....
PCI: Enabling device 00:07.1 (0000->0001)
ata1:PATA max UDMA/33 cmd 0x1F0 ctl 0x3F6 bmdma 0xF000 irq 14
ATA: abnormal status 0xFF on port 0x1F7
ATA: abnormal status 0xFF on port 0x1F7

and it locked up again.

Doug Gilbert