Skip to contentSkip to conversation

generic device DMA implementation

RFC Adam J. Richter 3 participants 3 messages Dec 6, 2002

lore
Share & tools
lore
#0 lore raw
Quoted 7 lines · David Gibson
Quoted 6 lines · Adam J. Richter
Quoted 4 lines · David Gibson

Since, with James's approach you'd need a dma sync function (which might compile to NOP) in pretty much the same places you'd need map/sync calls, I don't see that it does make the source noticeably simpler.

Quoted 5 lines
Quoted 2 lines

Because then you don't have to have a branch for
case where the platform *does* support consistent memory.

Sorry, you're going to have to explain where this extra branch is, I don't see it.

	In linux-2.5.50/drivers/net/lasi_82596.c, the macros

CHECK_{WBACK,INV,WBACK_INV} have definitions like:

#define CHECK_WBACK(addr,len) \
do { if (!dma_consistent) dma_cache_wback((unsigned long)addr,len); } while (0)

These macros are even used in IO paths like i596_rx(). The
"if()" statement in each of these macros is the extra branch that disappears on most architectures under James's proposal.

[...]

Quoted 5 lines

What performance advantages of consistent memory? Can you name any non-fully-consistent platform where consistent memory is preferable when it is not strictly required? For, all the non-consistent platforms I'm aware of getting consistent memory means disabling the cache and therefore is to be avoided wherever it can be.

I believe that the cache synchronization operations for
nonconsistent memory are often expensive enough so that consistent memory is faster on many platforms for small reads and writes, such as dealing with control and status fields and hardware DMA lists. For example, pci_sync_single is 55 lines of C code in linux-2.5.50/arch/sparc64/kernel/pci_iommu.c.

Adam J. Richter     __     ______________   575 Oroville Road
[email protected]     \ /                  Milpitas, California 95035
+1 408 309-6081         | g g d r a s i l   United States of America

"Free Software For The Rest Of Us."

#1 lore raw
Quoted 15 lines · Adam J. Richter
Quoted 7 lines · David Gibson
Quoted 6 lines · Adam J. Richter
Quoted 4 lines · David Gibson

Since, with James's approach you'd need a dma sync function (which might compile to NOP) in pretty much the same places you'd need map/sync calls, I don't see that it does make the source noticeably simpler.

Quoted 5 lines
Quoted 2 lines

Because then you don't have to have a branch for
case where the platform *does* support consistent memory.

Sorry, you're going to have to explain where this extra branch is, I don't see it.

	In linux-2.5.50/drivers/net/lasi_82596.c, the macros
Quoted 8 lines

CHECK_{WBACK,INV,WBACK_INV} have definitions like:

#define CHECK_WBACK(addr,len) \
do { if (!dma_consistent) dma_cache_wback((unsigned long)addr,len); } while (0)

These macros are even used in IO paths like i596_rx(). The
"if()" statement in each of these macros is the extra branch that disappears on most architectures under James's proposal.

Erm... I have no problem with the macros that James's proposal would use to take away this branch - I would expect to use exactly the same ones. It's just the notion of "try to get consistent memory, but get me any old memory otherwise" that I'm not so convinced by.

In any case, on platforms where the dma_malloc() could really return either consistent or non-consistent memory, James's sync macros would have to have an equivalent branch within.

Quoted 7 lines

[...]

Quoted 5 lines

What performance advantages of consistent memory? Can you name any non-fully-consistent platform where consistent memory is preferable when it is not strictly required? For, all the non-consistent platforms I'm aware of getting consistent memory means disabling the cache and therefore is to be avoided wherever it can be.

Quoted 6 lines

I believe that the cache synchronization operations for
nonconsistent memory are often expensive enough so that consistent memory is faster on many platforms for small reads and writes, such as dealing with control and status fields and hardware DMA lists. For example, pci_sync_single is 55 lines of C code in linux-2.5.50/arch/sparc64/kernel/pci_iommu.c.

Hmm... fair enough. Ok, I can see the point of a fall back to non-consistent approach given that. So I guess the idea makes sense, so long as dma_malloc() (without the consistent flag) is taken to be "give me DMAable memory, consistent or not, whichever is cheaper for this platform" rather than "give me DMAable memory, consistent if possible". It was originally presented as the latter which misled me.

I think the change to the parameters which I suggested in a reply to James makes this a bit clearer.

Signature · 4 lines
David Gibson			| For every complex problem there is a
[email protected]	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson
#2 lore raw

I think it's a huge error to try and move the DMA stuff into the generic device interfaces _AND_ change semantics and arguments at the same time.

Each operation should be done in seperate steps.

Then, if you want to talk about changing semantics etc. there are more pressing needs (read as: real bugs) in the current DMA APIs that must be fixed before you add new "cool" features to the interfaces. For example, we have a "pci_dma_sync_*()" interface which changes ownership from the device back to the cpu, but we do not have the corollary which returns ownership of the DMA buffer back to the device. Basically, every networking device driver that recycles buffers using pci_dma_sync_*() to peak at the header but then gives the buffer back to the device is buggy for this reason.

Fix this before changing stuff.

I don't have any time to discuss this further so please do me a big favor and drop me from the CC: lists, I've been able to only lightly read the existing parts of this thread, if at all, so the postings will only hit /dev/null while I'm so busy right now.

Thanks.