IOAT comments
Andy & Chris,
Sorry for the very very delayed response on your patch. Some comments below.
- Embedded chipsets have had DMA engines on them for a long time and having a single cross-platform API that can be used to offload standard kernel functions (memcpy, user_copy, etc) is a good starting point to make use of these. However, in addition to simple memory to memory copying, the engines on embedded devices often support memory <-> I/O space copying (accelereted memcpy_to_io/from_io).
What I would ideally like to see is an API that allows me to allocate a DMA channel between system memory and a device struct:
dma_client_alloc_chan(struct dma_client client*, struct device *dev);
The core would then search the DMA controller list, calling some function [(dma_device->device_supported(dev)?] to determine whether a controller can DMA to/from this device. Currently we have lots of CPU-specific DMA APIs in the embedded architectures and it would be nice to have well defined API that all drivers across all architectures could use.
Passing a NULL dev would signify that we just want to do mem <-> mem DMA.
- Make the various copy functions static inlines since they are just doing an extra function pointer dereference.
- The API currently supports only 1 client per DMA channel. I think a client should be able to ask for exclusive or non-exclusive usage of a DMA channel and the core can mark channels as unvailable when exclusive use is required. This could be useful in systems with just 1 DMA channel but multiple applications wanting to use it.
- Add an interface that takes scatter gather lists as both source and destination.
- DMA engine buffer alignment requirements? I've seen engines that handle any source/destination alignment and ones that handle only 32-bit or 64-bit aligned buffers. In the case of a transfer that is != alignment requirement of DMA engine, does the DMA device driver handle this or does the DMA core handle this?
- Can we get rid of the "async" in the various function names? I don't know of any HW that implements a synchronous DMA engine! It would sort of defeat the purpose. :)
- The user_dma code should be generic, not in net/ but in kernel/ or in drivers/dma as other subsystems and applications can probably make use of this funcionality.
~Deepak
Signature · 1 line
Deepak Saxena - [email protected] - http://www.plexity.net
Quoted 5 lines · Deepak Saxena
Andy & Chris,
Sorry for the very very delayed response on your patch. Some comments below.
Thanks for the feedback. I'm working on fixing up a few things and getting an updated set of patches out, so your timing's not bad at all.
Quoted 4 lines
What I would ideally like to see is an API that allows me to allocate a DMA channel between system memory and a device struct:
dma_client_alloc_chan(struct dma_client client*, struct device *dev);
I had something similar in my initial design, the I/OAT DMA engine can operate on any memory mapped region "downstream" from the PCI device. That's easy enough to add to the API, and then flush out the device matching code in the ioatdma driver later.
Quoted 2 lines
Make the various copy functions static inlines since they are just doing an extra function pointer dereference.
OK
Quoted 5 lines
- The API currently supports only 1 client per DMA channel. I think a client should be able to ask for exclusive or non-exclusive usage of a DMA channel and the core can mark channels as unvailable when exclusive use is required. This could be useful in systems with just 1 DMA channel but multiple applications wanting to use it.
We were trying to not share them for simplicity. But with the way we ended up using the DMA engine for TCP, we needed to get the locking right for requests from multiple threads anyway. Shared channel use shouldn't be hard to add, I'll look into it.
Quoted 2 lines
- Add an interface that takes scatter gather lists as both source and destination.
OK
Quoted 5 lines
- DMA engine buffer alignment requirements? I've seen engines that handle any source/destination alignment and ones that handle only 32-bit or 64-bit aligned buffers. In the case of a transfer that is != alignment requirement of DMA engine, does the DMA device driver handle this or does the DMA core handle this?
Any ideas of what this would look like?
Quoted 3 lines
- Can we get rid of the "async" in the various function names? I don't know of any HW that implements a synchronous DMA engine! It would
sort of defeat the purpose. :)
We tossed the async in after an early comment that the dma namespace wasn't unique enough, especially with the DMA mapping APIs already in place. Maybe hwdma? I'm open to suggestions.
Quoted 3 lines
- The
user_dmacode should be generic, not innet/but inkernel/or indrivers/dmaas other subsystems and applications can probably make use of this funcionality.
You're right. That file started out with dma_skb_copy_datagram_iovec, and dma_memcpy_[pg_]toiovec were created to support that. Anything not specifically tied to sk_buffs should be moved.
-Chris