Patch for UHCI driver (from kernel 2.6.6).
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 18 lines · David Brownell
Seems like the
dma_alloc_coherent()API spec can't be implemented on such machines then, since it's defined to returnmemory(*)such that:... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
Seems like the documentation should change to explain under what circumstances "coherent" memory will exhibit cache-incoherent behavior, and how to cope with that. (Then lots of drivers would need to change.)
OR ... maybe the bug is just that those PPC processors can't/shouldn't claim to implement that API. At which point all drivers relying on that API (including all the USB HCDs and many of the USB drivers) stop working.
Quoted 4 lines
- Dave
(*) DMA-API.txt uses two terms for this: "coherent" and "consistent".
DMA-mapping.txt only uses "consistent".
That text strikes me as rather ambiguous. Maybe it's intended to mean that a write by either side can be read immediately by the other side, and the values read will be the ones written (i.e., the read won't get stale data from some cache). It doesn't specify what happens to the other data bytes in the same cache line which _weren't_ written -- maybe they'll be messed up.
In other words, with "coherent" or "consistent" memory (there is some technical distinction between the two terms but I don't know what it is) you don't have to worry about reading stale data from a cache, but you might still have to worry about data unintentionally being overwritten with garbage.
Clearly this is not a tremendously useful guarantee, but I guess it's better than nothing.
Maybe someone on linux-mm can clarify things for the rest of us. For anyone interested, this thread started with:
http://marc.theaimsgroup.com/?l=linux-usb-devel&m=108728413809788&w=2
Alan Stern
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Alan Stern
Quoted 8 lines · David Brownell
Seems like the
dma_alloc_coherent()API spec can't be implemented on such machines then, since it's defined to returnmemory(*)such that:... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
Quoted 6 lines
Quoted 1 line
...
That text strikes me as rather ambiguous. ...
... It doesn't specify what happens to the other data
bytes in the same cache line which _weren't_ written -- maybe they'll be messed up.
Actually I thought it was quite explicit: "without having to worry about caching effects". What you described is clearly a caching effect: caused by caching. And maybe fixable by appropriate cache-flushing, or other cache-aware driver logic ... making it "worry about caching effects". Like the patch from Nicolas.
Maybe what we really need is patches to make USB switch to dma_alloc_noncoherent(), checking dma_is_consistent() to see whether a given QH/TD/ED/iTD/sITD/FSTN/... needs to be explicitly flushed from cpu cache before handover to HC.
- Dave
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines
Actually I thought it was quite explicit: "without having to worry about caching effects". What you described is clearly a caching effect: caused by caching.
Is it really a cache effect? Isn't it caused by the hc writing more bytes to memory than you expected? Now, it so happens that the number of bytes it writes is equal to the cache line size (or so it seems), but isn't that irrelevant?
Ciao,
Duncan.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 19 lines · David Brownell
Quoted 11 lines · Alan Stern
Quoted 8 lines · David Brownell
Seems like the
dma_alloc_coherent()API spec can't be implemented on such machines then, since it's defined to returnmemory(*)such that:... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
Quoted 6 lines
Quoted 1 line
...
That text strikes me as rather ambiguous. ...
... It doesn't specify what happens to the other data
bytes in the same cache line which _weren't_ written -- maybe they'll be messed up.
Quoted 6 lines
Actually I thought it was quite explicit: "without having to worry about caching effects". What you described is clearly a caching effect: caused by caching. And maybe fixable by appropriate cache-flushing, or other cache-aware driver logic ... making it "worry about caching effects". Like the patch from Nicolas.
No, you misunderstood what I wrote and misinterpreted the text. The text says:
... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
This means that when you read _the data that was written_ you don't have to worry about caching effects.
This does _not_ mean that when you read _other data stored nearby_ you don't have to worry about caching effects.
That is the problem Nicolas wants to solve. According to him, the problem occurs not when reading the data stored by the device but when reading data stored nearby by the CPU and mangled during the DMA transfer.
Now maybe the intention behind "consistent" or "coherent" mappings is that the memory really will behave as one would naively expect, and a write to one portion of a cache line won't mess up the contents of the remainder. But the text doesn't say this. That's why I said it is ambiguous.
Quoted 4 lines
Maybe what we really need is patches to make USB switch to
dma_alloc_noncoherent(), checkingdma_is_consistent()to see whether a given QH/TD/ED/iTD/sITD/FSTN/... needs to be explicitly flushed from cpu cache before handover to HC.
Will flushing the CPU cache really solve these problems? If the hardware that handles the DMA transfer always writes an entire cache line, and if it doesn't read the old contents before doing a partial write, then data stored in the same cache line as a DMA buffer is subject to overwriting whether it has been flushed or not.
Alan Stern
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 13 lines
Quoted 11 lines
Quoted 4 lines
That text strikes me as rather ambiguous. ...
... It doesn't specify what happens to the other data
bytes in the same cache line which _weren't_ written -- maybe they'll be messed up.Actually I thought it was quite explicit: "without having to worry about caching effects". What you described is clearly a caching effect: caused by caching. And maybe fixable by appropriate cache-flushing, or other cache-aware driver logic ... making it "worry about caching effects". Like the patch from Nicolas.
Quoted 7 lines
No, you misunderstood what I wrote and misinterpreted the text. The text says:
... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
This means that when you read
_thedata that waswritten_you don't have to worry about caching effects.
It doesn't limit the "without having to worry" to just the bytes written. And the rest of that API spec doesn't even suggest that there might be an issue there. I think you're trying to read things into that text that aren't there.
On the other hand, see dma_alloc_noncoherent() ... I hope you'll agree that the specification for "noncoherent" memory clearly expects those caching effects to exist. It specifically says that you need to understand cache line sharing issues to use the API correctly, and even exposes the cache line size.
It seems most likely to me that this particular PPC hardware can't implement dma_alloc_coherent(), and the code implementing that routine should be called dma_alloc_noncoherent() instead.
Quoted 7 lines
Quoted 4 lines
Maybe what we really need is patches to make USB switch to
dma_alloc_noncoherent(), checkingdma_is_consistent()to see whether a given QH/TD/ED/iTD/sITD/FSTN/... needs to be explicitly flushed from cpu cache before handover to HC.Will flushing the CPU cache really solve these problems? If the hardware
That's why we need PPC expertise applied here. Nicolas didn't seem to have answers for all the questions I was asking.
Quoted 4 lines
that handles the DMA transfer always writes an entire cache line, and if it doesn't read the old contents before doing a partial write, then data stored in the same cache line as a DMA buffer is subject to overwriting whether it has been flushed or not.
Addressed by using dma_alloc_noncoherent() and dma_cache_sync(); yes?
- Dave
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · David Brownell
Quoted 6 lines
... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
This means that when you read
_thedata that waswritten_you don't have to worry about caching effects.It doesn't limit the "without having to worry" to just the bytes written.
It's hard to say exactly what it means because it's ungrammatical. "A write ... can immediately be read..." -- what does it mean to read a write?
Quoted 3 lines
And the rest of that API spec doesn't even suggest that
there might be an issue there. I think you're trying to read things into that text that aren't there.
Maybe so. It's not worth spending more time on this point, anyhow.
Alan Stern
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 32 lines · Alan Stern
Quoted 18 lines · David Brownell
Seems like the
dma_alloc_coherent()API spec can't be implemented on such machines then, since it's defined to returnmemory(*)such that:... a write by either the device or the processor can immediately be read by the processor or device without having to worry about caching effects.
Seems like the documentation should change to explain under what circumstances "coherent" memory will exhibit cache-incoherent behavior, and how to cope with that. (Then lots of drivers would need to change.)
OR ... maybe the bug is just that those PPC processors can't/shouldn't claim to implement that API. At which point all drivers relying on that API (including all the USB HCDs and many of the USB drivers) stop working.
Quoted 4 lines
- Dave
(*) DMA-API.txt uses two terms for this: "coherent" and "consistent".
DMA-mapping.txt only uses "consistent".That text strikes me as rather ambiguous. Maybe it's intended to mean that a write by either side can be read immediately by the other side, and the values read will be the ones written (i.e., the read won't get stale data from some cache). It doesn't specify what happens to the other data bytes in the same cache line which _weren't_ written -- maybe they'll be messed up.
Quoted 2 lines
In other words, with "coherent" or "consistent" memory (there is some technical distinction between the two terms but I don't know what it is)
<quote>