SATA "target mode" (or "Channel-to-Channel" comm mode)

Guys,

Within the next couple of weeks, I would like to submit patches for 2.6.30 for a simple form of what Marvell likes to call "target mode", or C2C (channel-to-channel communications).

This is for sata_mv.

The question is, how to expose an interface to actually access it?

Quick background on Marvell C2C:

1. C2C is only for Gen2 and Gen2e chipsets.

2. Requires a special SATA cross-over (simple twist) cable between two SATA ports. Ports can be on the same host adaptor or on separate adaptors and/or machines.

3. Each sata_mv port can be either a (0) normal SATA host, or (1) special SATA C2C initiator, or (2) a SATA target device.

4. A Gen2e mode (2) target can connect/communicate with either a mode (0) host or a mode (1) initiator. I'm not yet sure whether an older Gen2 target can connect with a mode (0) host.

5. Mode (1) initiator appears to relax requirements such as waiting for a device BUSY bit to clear etc., and is intended for simple channel-to-channel communications.

6. A boot/module parameter seems to be the best way to enable this feature, as otherwise libata wastes a lot of time and effort probing for non-existant drives and slowing down the boot process.

7. Initially, all that we want is a way to use two SATA ports (on the same or different machines) as a simple byte-stream communications channel, between a mode (1) inititiator and a mode (2) target. This is used in real-life as a high-speed local comm channel between halves of split server machines.

8. Transparently emulating a SATA drive is possible on Gen2e chips at least, and perhaps also on Gen2 chips. This is not being worked on at this time.

9. Using two ports in tandem, one mode (0) host and one mode (2) target, one can construct a quite capable SATA capture/analyzer device which could be inserted in between any other SATA host and device. Quite useful, and something I intend to work on later this year.

So, starting with simple stuff, I want to expose an interface for point 7 above. The thought is to use netlink for this, on both ends.

An alternative might be to tie it into the SCSI Target Framework (tgt). But that is more for full target device emulation than for simple comms. And SATA is not SCSI, so it could really restrict/prevent us from doing a full SATA emulation (eg. point 9) in the end.

Time is short, so I'd like to spend it on something that Jeff would actually accept. Thus this email.

Quoted 57 lines · Mark Lord

Guys,

Within the next couple of weeks, I would like to submit patches for 2.6.30 for a simple form of what Marvell likes to call "target mode", or C2C (channel-to-channel communications).

This is for sata_mv.

The question is, how to expose an interface to actually access it?

Quick background on Marvell C2C:

1. C2C is only for Gen2 and Gen2e chipsets.

2. Requires a special SATA cross-over (simple twist) cable between two SATA ports. Ports can be on the same host adaptor or on separate adaptors and/or machines.

3. Each sata_mv port can be either a (0) normal SATA host, or (1) special SATA C2C initiator, or (2) a SATA target device.

4. A Gen2e mode (2) target can connect/communicate with either a mode (0) host or a mode (1) initiator. I'm not yet sure whether an older Gen2 target can connect with a mode (0) host.

5. Mode (1) initiator appears to relax requirements such as waiting for a device BUSY bit to clear etc., and is intended for simple channel-to-channel communications.

6. A boot/module parameter seems to be the best way to enable this feature, as otherwise libata wastes a lot of time and effort probing for non-existant drives and slowing down the boot process.

7. Initially, all that we want is a way to use two SATA ports (on the same or different machines) as a simple byte-stream communications channel, between a mode (1) inititiator and a mode (2) target. This is used in real-life as a high-speed local comm channel between halves of split server machines.

8. Transparently emulating a SATA drive is possible on Gen2e chips at least, and perhaps also on Gen2 chips. This is not being worked on at this time.

9. Using two ports in tandem, one mode (0) host and one mode (2) target, one can construct a quite capable SATA capture/analyzer device which could be inserted in between any other SATA host and device. Quite useful, and something I intend to work on later this year.

So, starting with simple stuff, I want to expose an interface for point 7 above. The thought is to use netlink for this, on both ends.

An alternative might be to tie it into the SCSI Target Framework (tgt). But that is more for full target device emulation than for simple comms. And SATA is not SCSI, so it could really restrict/prevent us from doing a full SATA emulation (eg. point 9) in the end.

Quoted 2 lines

Time is short, so I'd like to spend it on something that Jeff would actually accept. Thus this email.

It depends on the task.

The miscdev (i.e. chrdev) interface found in drivers/scsi/scsi_tgt_if.c of repo [1] seems pretty generic, simple, small and applicable to portions of the problem presented here... The basic task in scsi_tgt_if's case is just shoveling packets to/from userspace.

SATA packet capture: highly useful, but implies _copying_ the packets before passing them on to regular channels. So, does this imply packets will be copied kernel->userspace->kernel ? kernel->kernel? The interface will be vastly different in each case.

Also, in general, it sounds like we need a general way to put a port into a specific mode (initiator, target, vendor special) during runtime.
Then create a module parameter that allows boot-time initialization of
this port mode selector.

Quoted 63 lines · Jeff Garzik

Quoted 52 lines · Mark Lord

Guys,

Within the next couple of weeks, I would like to submit patches for 2.6.30 for a simple form of what Marvell likes to call "target mode", or C2C (channel-to-channel communications).

This is for sata_mv.

The question is, how to expose an interface to actually access it?

Quick background on Marvell C2C:

1. C2C is only for Gen2 and Gen2e chipsets.

2. Requires a special SATA cross-over (simple twist) cable between two SATA ports. Ports can be on the same host adaptor or on separate adaptors and/or machines.

3. Each sata_mv port can be either a (0) normal SATA host, or (1) special SATA C2C initiator, or (2) a SATA target device.

4. A Gen2e mode (2) target can connect/communicate with either a mode (0) host or a mode (1) initiator. I'm not yet sure whether an older Gen2 target can connect with a mode (0) host.

5. Mode (1) initiator appears to relax requirements such as waiting for a device BUSY bit to clear etc., and is intended for simple channel-to-channel communications.

6. A boot/module parameter seems to be the best way to enable this feature, as otherwise libata wastes a lot of time and effort probing for non-existant drives and slowing down the boot process.

7. Initially, all that we want is a way to use two SATA ports (on the same or different machines) as a simple byte-stream communications channel, between a mode (1) inititiator and a mode (2) target. This is used in real-life as a high-speed local comm channel between halves of split server machines.

8. Transparently emulating a SATA drive is possible on Gen2e chips at least, and perhaps also on Gen2 chips. This is not being worked on at this time.

9. Using two ports in tandem, one mode (0) host and one mode (2) target, one can construct a quite capable SATA capture/analyzer device which could be inserted in between any other SATA host and device. Quite useful, and something I intend to work on later this year.

So, starting with simple stuff, I want to expose an interface for point 7 above. The thought is to use netlink for this, on both ends.

Quoted 7 lines

An alternative might be to tie it into the SCSI Target Framework (tgt). But that is more for full target device emulation than for simple comms. And SATA is not SCSI, so it could really restrict/prevent us from doing a full SATA emulation (eg. point 9) in the end.

Time is short, so I'd like to spend it on something that Jeff would actually accept. Thus this email.

It depends on the task.

Quoted 1 line

The miscdev (i.e. chrdev) interface found in drivers/scsi/scsi_tgt_if.c of repo [1] seems pretty generic, simple, small and applicable to portions of the problem presented here... The basic task in scsi_tgt_if's case is just shoveling packets to/from userspace.

..

Except it's rather SCSI specific, and the userspace frontend even more so. The code expects a SCSI command block, LUN, TAG, and other fields that a SATA FIS won't have. Seems clumsy, particularly when we (in theory)
are trying to decouple libata from SCSI. But if that's the way,
then I can clumsily wrap each FIS in a fake ATA_16 header or something.

Quoted 1 line

SATA packet capture: highly useful, but implies _copying_ the packets before passing them on to regular channels. So, does this imply packets will be copied kernel->userspace->kernel ? kernel->kernel? The interface will be vastly different in each case.

..

Absolutely, which is why I'm leaving that for (much) later,
and looking for a simple (userspace) comms method to begin with,
using the special (non-SATA compliant) "initiator" on one end,
and a "target" on the other end. This is different from the SATA interceptor/emulator configurations.

Quoted 1 line · Mark Lord

Jeff Garzik wrote:

Quoted 8 lines

Quoted 1 line

The miscdev (i.e. chrdev) interface found in drivers/scsi/scsi_tgt_if.c of repo [1] seems pretty generic, simple, small and applicable to portions of the problem presented here... The basic task in scsi_tgt_if's case is just shoveling packets to/from userspace.

..

Except it's rather SCSI specific, and the userspace frontend even more so. The code expects a SCSI command block, LUN, TAG, and other fields that a SATA FIS won't have. Seems clumsy, particularly when we (in theory)
are trying to decouple libata from SCSI. But if that's the way,
then I can clumsily wrap each FIS in a fake ATA_16 header or something.

The kernel bits are not terribly SCSI-specific. I'd ignore the userland bits.

We just have to get the interface right. Even if we "cp scsi_tgt_if.c ata_tgt_if.c" and then go from there, we should make an effort to create similar packet push/pull interfaces.

Creating something wildly different would be counterproductive.

Quoted 10 lines · Jeff Garzik

Quoted 2 lines · Mark Lord

Time is short, so I'd like to spend it on something that Jeff would actually accept. Thus this email.

It depends on the task.

The miscdev (i.e. chrdev) interface found in drivers/scsi/scsi_tgt_if.c of repo [1] seems pretty generic, simple, small and applicable to portions of the problem presented here... The basic task in scsi_tgt_if's case is just shoveling packets to/from userspace.

..

Except it's rather SCSI specific, and the userspace frontend even more so. The code expects a SCSI command block, LUN, TAG, and other fields that a SATA FIS won't have. Seems clumsy, particularly when we (in theory)
are trying to decouple libata from SCSI. But if that's the way,
then I can clumsily wrap each FIS in a fake ATA_16 header or something.

Yes, but SG_IO began life as being entirely SCSI specific, but it's now the block packet infrastructure. I don't think anyone has an objection to the SCSI target infrastructure becoming block target infrastructure ... the fields can be made optional or more generic to further this. Probably it should speak a generic packet with a protocol identifier and let the user space sort out the details.

Quoted 11 lines · Mark Lord

Quoted 10 lines · Jeff Garzik

Quoted 2 lines · Mark Lord

Time is short, so I'd like to spend it on something that Jeff would actually accept. Thus this email.

It depends on the task.

The miscdev (i.e. chrdev) interface found in drivers/scsi/scsi_tgt_if.c of repo [1] seems pretty generic, simple, small and applicable to portions of the problem presented here... The basic task in scsi_tgt_if's case is just shoveling packets to/from userspace.

Quoted 7 lines

..

Except it's rather SCSI specific, and the userspace frontend even more so. The code expects a SCSI command block, LUN, TAG, and other fields that a SATA FIS won't have. Seems clumsy, particularly when we (in theory)
are trying to decouple libata from SCSI. But if that's the way,
then I can clumsily wrap each FIS in a fake ATA_16 header or something.

Yes, but SG_IO began life as being entirely SCSI specific, but it's now the block packet infrastructure. I don't think anyone has an objection to the SCSI target infrastructure becoming block target infrastructure ... the fields can be made optional or more generic to further this. Probably it should speak a generic packet with a protocol identifier and let the user space sort out the details.