staging: Add firewire-serial driver

Please consider this serial driver for review for submission to staging. The firewire-serial driver implements TTY over IEEE 1394. In its default configuration, it creates 4 TTY devices and one loopback device per firewire card (respectively, named fwtty<n>~fwtty<n+3> and fwloop<n>).

Currently, the TTY devices auto-connect to every cabled peer (the TODO list includes plans for providing a sysfs interface to control virtual cabling with whitelist/blacklist support per GUID).

Efforts are still ongoing for a companion console driver, with plans to eventually add early_printk & kgdb support (via additional drivers).

Some issues did arise with both the TTY and Firewire subsystems which are noted in the TODO file. Please review these workarounds.

Peter Hurley (1):
staging: fwserial: Add TTY-over-Firewire serial driver

Quoted 12 lines · Peter Hurley

Please consider this serial driver for review for submission to staging. The firewire-serial driver implements TTY over IEEE 1394. In its default configuration, it creates 4 TTY devices and one loopback device per firewire card (respectively, named fwtty<n>~fwtty<n+3> and fwloop<n>).

Currently, the TTY devices auto-connect to every cabled peer (the TODO list includes plans for providing a sysfs interface to control virtual cabling with whitelist/blacklist support per GUID).

Efforts are still ongoing for a companion console driver, with plans to eventually add early_printk & kgdb support (via additional drivers).

Quoted 5 lines

Some issues did arise with both the TTY and Firewire subsystems which are noted in the TODO file. Please review these workarounds.

Peter Hurley (1):
staging: fwserial: Add TTY-over-Firewire serial driver

I'd like to get an Ack from Stefan here, before I'll add this to the staging tree.

Quoted 13 lines · Greg Kroah-Hartman

Quoted 12 lines · Peter Hurley

Please consider this serial driver for review for submission to staging. The firewire-serial driver implements TTY over IEEE 1394. In its default configuration, it creates 4 TTY devices and one loopback device per firewire card (respectively, named fwtty<n>~fwtty<n+3> and fwloop<n>).

Currently, the TTY devices auto-connect to every cabled peer (the TODO list includes plans for providing a sysfs interface to control virtual cabling with whitelist/blacklist support per GUID).

Efforts are still ongoing for a companion console driver, with plans to eventually add early_printk & kgdb support (via additional drivers).

Quoted 8 lines

Quoted 5 lines

Some issues did arise with both the TTY and Firewire subsystems which are noted in the TODO file. Please review these workarounds.

Peter Hurley (1):
staging: fwserial: Add TTY-over-Firewire serial driver

I'd like to get an Ack from Stefan here, before I'll add this to the staging tree.

Of course.

Jiri's newest series ("TTY buffer in tty_port and other stuff") does break this driver, so I'll need to resubmit to address those changes. Two of the workarounds mentioned in the TODO are related to the tty_buffer interface; specifically,

1. The ldisc drops the contents of tty_buffer on hangup (rather than waiting for completion). Maybe for other devices this isn't so noticeable because the ldisc can mostly keep up with the device, but on firewire the ldisc lags well behind. Right now, this driver works around this by holding off the hangup until the ldisc empties the tty_buffer. The driver determines how much data is still left in the tty_buffer by walking the flip buffers.

2. Because this driver can fill the entire tty_buffer (64K +) before the ldisc even runs once, this driver has to self-throttle when feeding the tty_buffer. So as not to drop data, the driver has to know when a watermark is reached in the tty_buffer, to ensure that in-flight + already-accepted data can safely be moved to what remains avail in the tty_buffer. (The tty_buffer maxs at 64K). Currently, the avail level in the tty_buffer is computed by this driver (in the same manner as tty_buffer_alloc()).

1. The ldisc drops the contents of tty_buffer on hangup (rather than waiting for completion). Maybe for other devices this isn't so noticeable because the ldisc can mostly keep up with the device, but on firewire the ldisc lags well behind. Right now, this driver works around this by holding off the hangup until the ldisc empties the tty_buffer. The driver determines how much data is still left in the tty_buffer by walking the flip buffers.

2. Because this driver can fill the entire tty_buffer (64K +) before the ldisc even runs once, this driver has to self-throttle when feeding the tty_buffer.

Quoted 7 lines

1. The ldisc drops the contents of tty_buffer on hangup (rather than waiting for completion). Maybe for other devices this isn't so noticeable because the ldisc can mostly keep up with the device, but on firewire the ldisc lags well behind. Right now, this driver works around this by holding off the hangup until the ldisc empties the tty_buffer. The driver determines how much data is still left in the tty_buffer by walking the flip buffers.

The driver should not be trying to look at this, so it's good that it broke now. If you need it to process the data on then hangup then the core code wants fixing to allow this to occur.

Quoted 3 lines

2. Because this driver can fill the entire tty_buffer (64K +) before the ldisc even runs once, this driver has to self-throttle when feeding the tty_buffer.

That wants investigating properly. We do multiple megabits quite happily via the tty layer with 3G modems over USB. What is your data rate ?

most controllers allow spillover into
the portion of bus cycle assigned for sync tx (which is another 4Kbytes per 125us).

(Technical note: the actual total max for combined async and sync tx is 6144 bytes per 125us clock)

Quoted 12 lines · Peter Hurley

Please consider this serial driver for review for submission to staging. The firewire-serial driver implements TTY over IEEE 1394. In its default configuration, it creates 4 TTY devices and one loopback device per firewire card (respectively, named fwtty<n>~fwtty<n+3> and fwloop<n>).

Currently, the TTY devices auto-connect to every cabled peer (the TODO list includes plans for providing a sysfs interface to control virtual cabling with whitelist/blacklist support per GUID).

Efforts are still ongoing for a companion console driver, with plans to eventually add early_printk & kgdb support (via additional drivers).

Quoted 5 lines

Some issues did arise with both the TTY and Firewire subsystems which are noted in the TODO file. Please review these workarounds.

Peter Hurley (1):
staging: fwserial: Add TTY-over-Firewire serial driver

I'd like to get an Ack from Stefan here, before I'll add this to the staging tree.

Quoted 4 lines

 drivers/staging/fwserial/dma_fifo.c |  310 ++++
 drivers/staging/fwserial/dma_fifo.h |  130 ++
 drivers/staging/fwserial/fwserial.c | 2885 +++++++++++++++++++++++++++++++++++
 drivers/staging/fwserial/fwserial.h |  355 +++++

Quoted 4 lines

Quoted 4 lines

 drivers/staging/fwserial/dma_fifo.c |  310 ++++
 drivers/staging/fwserial/dma_fifo.h |  130 ++
 drivers/staging/fwserial/fwserial.c | 2885 +++++++++++++++++++++++++++++++++++
 drivers/staging/fwserial/fwserial.h |  355 +++++

drivers/firewire/net.c is 1721 lines, sbp_target is 2868 lines. Why is fwserial bigger?

This patch provides the kernel driver for high-speed TTY communication over the IEEE 1394 bus.

Signed-off-by Peter Hurley <[email protected]>

---
v2

---
 drivers/staging/Kconfig             |    2 +
 drivers/staging/Makefile            |    1 +
 drivers/staging/fwserial/Kconfig    |    9 +
 drivers/staging/fwserial/Makefile   |    2 +
 drivers/staging/fwserial/TODO       |   37 +
 drivers/staging/fwserial/dma_fifo.c |  310 ++++
 drivers/staging/fwserial/dma_fifo.h |  130 ++
 drivers/staging/fwserial/fwserial.c | 2946 +++++++++++++++++++++++++++++++++++
 drivers/staging/fwserial/fwserial.h |  387 +++++
 9 files changed, 3824 insertions(+)
 create mode 100644 drivers/staging/fwserial/Kconfig
 create mode 100644 drivers/staging/fwserial/Makefile
 create mode 100644 drivers/staging/fwserial/TODO
 create mode 100644 drivers/staging/fwserial/dma_fifo.c
 create mode 100644 drivers/staging/fwserial/dma_fifo.h
 create mode 100644 drivers/staging/fwserial/fwserial.c
 create mode 100644 drivers/staging/fwserial/fwserial.h

drivers/staging/fwserial/TODO new file +4 -0

@@ -0,0 +1,37 @@
+TODOs
+-----
+1. Implement retries for RCODE_BUSY, RCODE_NO_ACK and RCODE_SEND_ERROR
+   - I/O is handled asynchronously which presents some issues when error

Quoted 4 lines · On Nov 02 Peter Hurley wrote:

This patch provides the kernel driver for high-speed TTY communication over the IEEE 1394 bus.

Signed-off-by Peter Hurley <[email protected]>

Hi Greg,

you asked for an Ack. Here it is: I am OK with this getting added to the staging tree.

Quoted 4 lines · Stefan Richter

Quoted 3 lines · On Nov 02 Peter Hurley wrote:

This patch provides the kernel driver for high-speed TTY communication over the IEEE 1394 bus.

Quoted 6 lines

Quoted 1 line

Signed-off-by Peter Hurley <[email protected]>

Hi Greg,

you asked for an Ack. Here it is: I am OK with this getting added to the staging tree.

Thanks, I'll queue it up later this week when I return from my travels.

Quoted 12 lines · Greg Kroah-Hartman

Quoted 10 lines · Stefan Richter

Quoted 4 lines · On Nov 02 Peter Hurley wrote:

This patch provides the kernel driver for high-speed TTY communication over the IEEE 1394 bus.

Signed-off-by Peter Hurley <[email protected]>

Hi Greg,

you asked for an Ack. Here it is: I am OK with this getting added to the staging tree.

Quoted 1 line

Thanks, I'll queue it up later this week when I return from my travels.

Thanks, Greg.

Except for your and Alan's earlier concerns, which I think I addressed, I didn't get any other feedback on the TTY side of things. Are you personally reviewing this submission?

Quoted 4 lines · On Nov 02 Peter Hurley wrote:

This patch provides the kernel driver for high-speed TTY communication over the IEEE 1394 bus.

Signed-off-by Peter Hurley <[email protected]>

Hi Greg,

you asked for an Ack. Here it is: I am OK with this getting added to the staging tree.

(Sorry for the delay, I was ill and at the same time buried in work at the day job.) Note, I only quickly scrolled through v1 and v2. I will do an actual review when somebody asks to move this from staging into a proper mainline place.

Peter,
below are some high-level comments to your TODOs.

drivers/staging/fwserial/Kconfig line 9

Quoted 12 lines

drivers/staging/fwserial/Kconfig new file +9 -0

@@ -0,0 +1,9 @@
+config FIREWIRE_SERIAL
+       tristate "TTY over Firewire"
+       depends on FIREWIRE
+       help
+          This enables TTY over IEEE 1394, providing high-speed serial
+	  connectivity to cabled peers.
+
+	  To compile this driver as a module, say M here:  the module will
+	  be called firewire-serial.

This should mention that this is a Linux-only affair at this time: It uses an ad-hoc defined protocol which no other platform is known to support for the time being.

Quoted 7 lines

drivers/staging/fwserial/TODO new file +4 -0

@@ -0,0 +1,37 @@
+TODOs
+-----
+1. Implement retries for RCODE_BUSY, RCODE_NO_ACK and RCODE_SEND_ERROR
+   - I/O is handled asynchronously which presents some issues when error

drivers/staging/fwserial/TODO line 14

Quoted 10 lines

drivers/staging/fwserial/TODO new file +10 -0

+     conditions occur.
+2. Implement _robust_ console on top of this. The existing prototype console
+   driver is not ready for the big leagues yet.
+3. Expose means of controlling attach/detach of peers via sysfs. Include
+   GUID-to-port matching/whitelist/blacklist.
+
+-- Issues with firewire stack --
+1. This driver uses the same unregistered vendor id that the firewire core does
+     (0xd00d1e). Perhaps this could be exposed as a define in
+     firewire-constants.h?

I guess there is no better OUI for this purpose.

The constant could be added to linux/firewire.h, but IMO rather not to uapi/linux/firewire-constants.h. We don't want this definition to leak out to userland; userland can hardwire its own hacks itself. :-)

This trivial move to a header should be deferred until the driver moves out of staging.

drivers/staging/fwserial/TODO line 17

Quoted 3 lines

drivers/staging/fwserial/TODO new file +3 -0

+2. MAX_ASYNC_PAYLOAD needs to be publicly exposed by core/ohci
+   - otherwise how will this driver know the max size of address window to
+     open for one packet write?

Hmm, don't firewire-sbp2 and firewire-net deal with this very problem already on their own? Firewire-sbp2 tells the target what maximum payload the local node is ready to accept, and firewire-net figures out whether it needs to fragment the datagrams in unicast TX depending on the remote node's capabilities.

drivers/staging/fwserial/TODO line 19

Quoted 2 lines

drivers/staging/fwserial/TODO new file +2 -0

+3. Maybe device_max_receive() and link_speed_to_max_payload() should be
+     taken up by the firewire core?

Sounds like an extension of item 2, and is easier resolved after the driver moves out of staging.

drivers/staging/fwserial/TODO line 21

Quoted 2 lines

drivers/staging/fwserial/TODO new file +2 -0

+4. To avoid dropping rx data while still limiting the maximum buffering,
+     the size of the AR context must be known. How to expose this to drivers?

I don't see a requirement to know the local or remote node's size of AR DMA buffer. Rather, keep the traffic throttled such that too frequent ack-busy are avoided. As far as I have learned from firewire-net, just limiting the number or in-flight transactions should work reasonably. Of course, hardware ack-busy retries which the OHCI performs on its own won't be noticed by software but take away from overall bus bandwidth. IOW the optimum queue depth is by a certain margin less than the depth at which software starts to see ack-busy transaction terminations.

Furthermore: In order to avoid dropping data, you don't need to know the buffer size at all. You need to implement proper software retries. I.e. either those which you noted yourself at the top of this TODO file, or you let the userland on top of the TTY handle it if this is possible. (I don't know how the TTY subsystem and TTY based applications work.)

drivers/staging/fwserial/TODO line 24

Quoted 3 lines

drivers/staging/fwserial/TODO new file +3 -0

+5. Explore if bigger AR context will reduce RCODE_BUSY responses
+   (or auto-grow to certain max size -- but this would require major surgery
+    as the current AR is contiguously mapped)

For the particular application "firewire-net", the present AR-Req DMA buffer size has apparently been determined as suitable. Of course if you find a different optimum, that could certainly be changed.

I haven't looked closely, but I suppose your bandwidth requirement concerns AR-Request DMA, not so much AR-Response DMA, right?

Quoted 5 lines · Stefan Richter

Quoted 3 lines · On Nov 02 Peter Hurley wrote:

+2. MAX_ASYNC_PAYLOAD needs to be publicly exposed by core/ohci

drivers/staging/fwserial/TODO new file +2 -0

+   - otherwise how will this driver know the max size of address window to
+     open for one packet write?

Quoted 5 lines

Hmm, don't firewire-sbp2 and firewire-net deal with this very problem already on their own? Firewire-sbp2 tells the target what maximum payload the local node is ready to accept, and firewire-net figures out whether it needs to fragment the datagrams in unicast TX depending on the remote node's capabilities.

I wasn't as clear as I should have been here. This is only about how big to make the address handler window.

Just like firewire-net, this driver
communicates with remotes at S100 speeds (and packet sizes) to query the largest packet size that the remote supports.

Currently, firewire-net sets an arbitrary address handler length of 4096. This works because the largest AR packet size the current firewire-ohci driver handles is 4096 (value of MAX_ASYNC_PAYLOAD) + header/trailer. Note that firewire-ohci does not limit card->max_receive to this value.

So if the ohci driver changes to handle 8K+ AR packets and the hardware supports it, these address handler windows will be too small.

Quoted 5 lines

Quoted 2 lines

+3. Maybe device_max_receive() and link_speed_to_max_payload() should be
+ taken up by the firewire core?

Sounds like an extension of item 2, and is easier resolved after the driver moves out of staging.

Ok.

Quoted 6 lines

Quoted 2 lines

+4. To avoid dropping rx data while still limiting the maximum buffering,
+ the size of the AR context must be known. How to expose this to drivers?

I don't see a requirement to know the local or remote node's size of AR DMA buffer. Rather, keep the traffic throttled such that too frequent ack-busy are avoided.

I do need to implement retries but that will be of limited benefit to this particular problem.

Data is written from the remote as a posted write into the AR buffer.

Besides the inefficiency of re-sending data that has already been received, retrying may not be possible. Consider when a console driver (work-in-progress) writes to the remote using the same interface. It could be one of the last things the remote does. And the most crucial information is that last write.

Quoted 12 lines · On Nov 13 Peter Hurley wrote:

Quoted 10 lines · Stefan Richter

drivers/staging/fwserial/TODO line 17

Quoted 3 lines · On Nov 02 Peter Hurley wrote:

+2. MAX_ASYNC_PAYLOAD needs to be publicly exposed by core/ohci

drivers/staging/fwserial/TODO new file +2 -0

+   - otherwise how will this driver know the max size of address window to
+     open for one packet write?

Hmm, don't firewire-sbp2 and firewire-net deal with this very problem already on their own? Firewire-sbp2 tells the target what maximum payload the local node is ready to accept, and firewire-net figures out whether it needs to fragment the datagrams in unicast TX depending on the remote node's capabilities.

Quoted 2 lines

I wasn't as clear as I should have been here. This is only about how big to make the address handler window.

Isn't fw_card.max_receive what you were looking for?

But since the
current firewire-core API requires you to allocate one handler address and size for all cards --- even for cards which will be hot-added later while your driver is already running ---, you should surely just allocate 4096 bytes; see below.

Quoted 3 lines

Just like firewire-net, this driver
communicates with remotes at S100 speeds (and packet sizes) to query the largest packet size that the remote supports.

(RFC 2734 additionally keeps things sane for itself by requiring all 2734 compliant nodes to support at least max packet = 512 bytes.)

Could it be easier to take this information from fw_device.max_speed and the max_rec in fw_device.config_rom? RFC 2734 was designed to work without a local node management layer which is capable to gather such information, but we have this information on Linux (at the time when core_device.c finished its node discovery).

Quoted 8 lines

Currently, firewire-net sets an arbitrary address handler length of 4096. This works because the largest AR packet size the current firewire-ohci driver handles is 4096 (value of MAX_ASYNC_PAYLOAD) + header/trailer. Note that firewire-ohci does not limit card->max_receive to this value.

So if the ohci driver changes to handle 8K+ AR packets and the hardware supports it, these address handler windows will be too small.

While the IEEE 1394:2008 link layer specification (section 6) provides for asynchronous packet payloads of up to 16384 bytes (table 6-4), the IEEE 1394 beta mode port specification (section 13) only allows up to 4096 bytes (table 16-18). And alpha mode is of course limited to 2048 bytes.

So, asynchronous packet payloads greater than 4096 bytes are out of scope of the current revision of IEEE 1394.

Quoted 6 lines

Quoted 6 lines

Quoted 2 lines

+4. To avoid dropping rx data while still limiting the maximum buffering,
+ the size of the AR context must be known. How to expose this to drivers?

I don't see a requirement to know the local or remote node's size of AR DMA buffer. Rather, keep the traffic throttled such that too frequent ack-busy are avoided.

[...]

Quoted 4 lines

I do need to implement retries but that will be of limited benefit to this particular problem.

Data is written from the remote as a posted write into the AR buffer.

[...]

Oh, I forgot about posted writes. However, you only lose data with posted writes if the OHCI somehow fails to access the host bus. Merely getting to the end of the DMA program before all of the PCI posting FIFO could be emptied is not a reason to drop data. See OHCI-1394 section 3.3 which speaks about the possibility to drop selfID/ IR/ AR-resp data if out of buffer, but not AR-req data.

Well, maybe there are bad OHCI implementations which drop posted AR-req data...? I don't know.

firewire-net uses posted writes too. But with IP-over-1394, delivery guarantee is handled further above if needed, not by the 1394 encapsulation. firewire-sbp2 also uses posted writes, in two ways: With physical DMA to SCSI READ command data buffers, and for the SBP status FIFO. But physical DMA does not have a backing DMA program, and data loss during a status FIFO write would likely be covered by a SCSI transaction retry.

[...]

Quoted 5 lines

Besides the inefficiency of re-sending data that has already been received, retrying may not be possible. Consider when a console driver (work-in-progress) writes to the remote using the same interface. It could be one of the last things the remote does. And the most crucial information is that last write.

As mentioned, I have no idea how TTY works or is supposed to work. But even if the sending application is allowed to quit before its last outbound datagram was delivered, fwserial could still be made to deliver this last datagram at its own pace, with as many software retries as prudent.

Quoted 14 lines

Quoted 8 lines

Currently, firewire-net sets an arbitrary address handler length of 4096. This works because the largest AR packet size the current firewire-ohci driver handles is 4096 (value of MAX_ASYNC_PAYLOAD) + header/trailer. Note that firewire-ohci does not limit card->max_receive to this value.

So if the ohci driver changes to handle 8K+ AR packets and the hardware supports it, these address handler windows will be too small.

While the IEEE 1394:2008 link layer specification (section 6) provides for asynchronous packet payloads of up to 16384 bytes (table 6-4), the IEEE 1394 beta mode port specification (section 13) only allows up to 4096 bytes (table 16-18). And alpha mode is of course limited to 2048 bytes.

Quoted 2 lines

So, asynchronous packet payloads greater than 4096 bytes are out of scope of the current revision of IEEE 1394.

You should look at this 1394ta.org video
http://www.youtube.com/watch?v=xVXNvXHNQTY of DAP Technologies S1600 OHCI controllers running S1600 cameras using beta cables.

Quoted 1 line

Quoted 1 line

Quoted 14 lines · On Nov 27 Peter Hurley wrote:

Quoted 14 lines

Quoted 8 lines

Currently, firewire-net sets an arbitrary address handler length of 4096. This works because the largest AR packet size the current firewire-ohci driver handles is 4096 (value of MAX_ASYNC_PAYLOAD) + header/trailer. Note that firewire-ohci does not limit card->max_receive to this value.

So if the ohci driver changes to handle 8K+ AR packets and the hardware supports it, these address handler windows will be too small.

While the IEEE 1394:2008 link layer specification (section 6) provides for asynchronous packet payloads of up to 16384 bytes (table 6-4), the IEEE 1394 beta mode port specification (section 13) only allows up to 4096 bytes (table 16-18). And alpha mode is of course limited to 2048 bytes.

Quoted 6 lines

Quoted 2 lines

So, asynchronous packet payloads greater than 4096 bytes are out of scope of the current revision of IEEE 1394.

You should look at this 1394ta.org video
http://www.youtube.com/watch?v=xVXNvXHNQTY of DAP Technologies S1600 OHCI controllers running S1600 cameras using beta cables.

I don't know the details of their implementation, but I suppose they conform with the 1394 beta mode port specification. Which in turn means that their S1600 solution (and by extrapolation, their S3200 prototypes) comply with a maximum asynchronous packet payload of 4096 bytes. Citing IEEE 1394-2008:

Quoted 1 line

Quoted 1 line

Quoted 1 line

Table 16-18———Maximum payload size for Beta data packets

 Data rate | Maximum asynchronous payload size | Maximum isochronous payload
           |              (bytes)              |           (bytes)
 ----------+-----------------------------------+----------------------------
    S100   |                 512               |             1024
    S200   |                1024               |             2048
    S400   |                2048               |             4096
    S800   |                4096               |             8192
    S1600  |                4096               |            16384
    S3200  |                4096               |            32768

<<<

(Alpha mode payload limits are the same as the S100...S400 subset of beta mode. In IEEE 1394b-2002, the table number is 16-3.)

You can of course define registers (or better termed: buffers) which are larger than what can be atomically read or written, or atomically compared-swapped; IOW which are larger than what can be accessed in a single transaction, if such registers or buffers are useful. But if you particularly need a register which is just large enough to accommodate the largest possible inbound block write transaction which complies with IEEE 1394, and you don't know the peer's capability and the speeds of all intermediary cable hops, then fw_card.max_receive is the number that you need. Or you ignore the cards actual capability and just allocate 4096 bytes.