New QStor SATA/RAID Driver for 2.6.9-rc2

My first attempt at posting this seems to have gone AWOL, so here it comes again. Also being posted to linux-scsi.

Here is the first public release of the 2.6.xx driver source code for the Pacific Digital Corporation QStor SATA/RAID chip.

This 4-channel chip has hardware-assisted RAID0/RAID1/RAID10, host-queuing, per-request TCQ/NCQ support, support for hot insertion and removal of drives, etc.. The 64-bit/66Mhz chip shows throughput in excess of 200MByte/sec on my ancient P3-1GHz test system, and can do much better when installed in a PCI-X slot.

The driver (attached) supports most of the chip features, including host, native and legacy tagged queuing, but does not yet include boot-from-raid support (coming soon).

Both hdparm and smartmontools are fully supported by this driver.

This patch is against linux-2.6.9-rc2.

Please email me any errors or corrections you may deem necessary for kernel inclusion.

Mark Lord
(hdparm keeper & the original "Linux IDE Guy")

How much of the RAID is actually hardware-assisted?

Linus vetoed future SCSI->ATA translators. He only allowed libata because I promised to remove the translation and make it a native block driver in the future, which I have been working towards.

Jeff

This is why the Promise SX8 driver is a block driver, for example...

Jeff

Oh, this is incredibly good to hear. It definitely clarifies the path towards implementing new features such as power management...

Is this written up in a FAQ somewhere?

Nathan

Not AFAIK.

The basic rule is "don't translate the native protocol into something else." and it has been in place for quite a while.

I was allowed a one-time exception for libata, which translates the native ATA into SCSI. The SCSI portions are separated into libata-scsi.c so that they may be easily excised or made optional (likely).

Jeff

Mark,
I'm curious how smartmontools is supported. Does the driver support SCSI LOG SENSE commands on the physical units or does it take the 3ware/Marvell route?
There is no sign of QStor specific code in smartmontools's CVS.

As Jeff is aware, SCSI->ATA translation (SAT) is in the process of being standardized at t10.org . The model being used outlines a SAT layer going into any of three places:
- in the host OS above a SATA/ATA HBA driver (i.e. what libata does and Linus frowns upon)
- in the host OS above a SAS HBA driver which, amongst other protocols, has the SAS Tunnelling Protocol (STP) which conveys ATA/ATAPI7 commands through SAS infrastructure
- somewhere in the service delivery subsystem, specifically in SAS expanders which have phys connected to SATA disks (Linus vetoes have no influence here)

Consider this pathological situation:
Start with one SATA II disk and connect it to a port selector (a SATA II device) which effectively makes the SATA disk dual ported. Connect one of those ports to a SATA HBA that lives in the Linux ATA subsystem (where no SAT is allowed). Connect the other port to a SAS expander which includes a SAT layer and then connect that expander to a SAS HBA in the Linux SCSI subsytem. That should confuse the !@#$ out of any application client trying to get a handle on what is happening.

I assume that OS-based ATA->SCSI translation is also out of the question.

Doug Gilbert

In order to keep compatibility with existing libata users after libata becomes a block driver, the ATA->SCSI translation will live on as a completely separate, independent, optional kernel module.

If we are really lucky we can use it as a library for some of the other drivers that do some amount of ATA->SCSI translation, or ide-scsi...

Jeff

Please fix all the style issues in here, aka

- linebreaks after 80 characters
- don't typedef structs unless they're used opaqueuely
- please avoid C++-style comments even if C99 allows them
- don't use "scsi.h", only headers from <scsi/*.h>, that also means getting rid of the Scsi_Foo typedefs
- please don't include headers in other headers unless nessecary

I also don't want another SCSI->ATA translator in drivers/scsi, please find some way to share codce with an existing one.

I'll do a real review once the driver is readable.

This driver simply implements the HDIO_DRIVE_CMD/HDIO_DRIVE_TASK ioctl call (from Linux IDE). By doing so, it automatically gains compatibility with smartmontools (-d ata), and with hdparm.

Currently, the implementation of those is native to the controller. But eventually, I'd like to see it implement Curtis's ATA Passthrough spec (to which I've been contributing).

Port selectors aside --> a nightmare for any sysadmin or driver writer --> Linux has a huge unresolved issue for block devices.

It is currently next-to-impossible for a vendor to support a new drive controller card under Linux unless it uses either SCSI or libata. Doing one as a new block driver is not an option, because of the installation issues for the end-user during the first couple of years of deployment --> until the distros begin to include the required /dev/ entries everywhere.

To solve this requires a nice generic /dev/ interface for disks, common to all drivers which talk to such devices.

Currently on Linux, that interface is called "SCSI". I think it might not be unreasonable to gradually evolve the SCSI host interface to include, say, a non-translating queuecommand() method, and associated pals.

This would get rid of much (not all) of the double-translation objection, and provide a smooth path for supporting Firewire, USB, parallel-port, SATA, ATA, and of course SCSI, disks all under one naming subsystem.

We practically have that already today.
The SCSI mid-layer is a nice generic block device glue system. We just need perhaps to make it less SCSI-specific.

Cheers

Mark Lord
(hdparm keeper & the original "Linux IDE Guy")

This is what we have been doing for quite some time. If you look, a lot of the helper functions have actually been moved _out_ of SCSI and into the block layer. Most of it is already there. The SCSI mid layer is getting smaller, not bigger.

The remaining piece (well the biggest one at least) is a nice /dev/disk abstraction. That's currently the biggest obstacle to nice generic block drivers, not the infrastructure.

Jens Axboe

I'll just take this opportunity to plug /dev/drive over /dev/disk. Not only does it avoid the disc/disk dialect problem, it is also more accurate in the case of removable media.

"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

I'll take this opportunity to recommend that posters avoid following this sub-thread, for fear of distracting from the real technical issues....

Jeff

[...]

You seem to have independent reached the same conclusion I did :)

To be specific, SCSI provides LLD infrastructure that block does not:
1) infrastructure for queueing, retrying, and timing out requests
2) an error handling thread.
3) a standard method of addressing attached devices
4) a standard method of submitting raw commands from userspace

It is my goal to shift this infrastructure from SCSI to block over time.
There is a fair amount of queueing infrastructure now in 2.6 (part of
#1), and Jens already has test code for #4.

I had hoped to start working on this in 2.7, but with the new dev model 2.7 is postponed indefinitely.... so I guess I'll start working on it sooner rather than later :)

Jeff