Skip to contentSkip to conversation

About dma_sync_single_for_{cpu,device}

DISCUSSION [email protected] 3 participants 11 messages Jul 30, 2012 – Aug 1, 2012

lore
Share & tools
lore
#0 lore raw

Hi,

On our board we've got an MV78200 and a network device between which we xfer memory chunks via the ddram with an external dma controller. To handle these xfers we're using the dma API.

To tx a chunk of data from the SoC => network device, we :
- prepare a buffer with a leading header embedding a pattern,
- trigger the xfer and wait for an irq
// The device updates the pattern and then triggers an irq
- upon irq we check the pattern for the xfer completion

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	[...]

But this does not work (the buffer pattern does not reflect the ddram value).
On the other hand, the following works:

	[...]
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // RAM value
	[...]

Looking at

	dma-mapping.c:__dma_page_cpu_to_{dev,cpu}() and
	proc-feroceon.S: feroceon_dma_{,un}map_area

this behavior is not surprising.
The sync_for_cpu calls the unmap which just invalidates the outer cache while the sync_for_device invalidates both inner and outer.

It seems that:
- we need to invalidate after the RAM has been updated
- we need to invalidate with sync_single_for_device rather than sync_single_for_cpu to check the value

Is it correct ?

Maybe the following comment in dma-mapping.c explains the situation :
/*
* The DMA API is built upon the notion of "buffer ownership". A buffer
* is either exclusively owned by the CPU (and therefore may be accessed
* by it) or exclusively owned by the DMA device. These helper functions
* represent the transitions between these two ownership states. *
* Note, however, that on later ARMs, this notion does not work due to
* speculative prefetches. We model our approach on the assumption that
* the CPU does do speculative prefetches, which means we clean caches
* before transfers and delay cache invalidation until transfer completion. *
*/

Thanks for your input,

Regards,

Karl

#1 lore raw
Quoted 2 lines · On Mon, Jul 30, 2012 at 10:24:01PM +0200, [email protected] wrote:

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);

Sorry, I forgot this (invalidate):
dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);

Quoted 6 lines
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	[...]

Karl

#2 lore raw
In reply to [email protected]

Hi,

(This is an email originally addressed to the linux-kernel mailing-list.)

On our board we've got an MV78200 and a network device between which we xfer memory chunks via the ddram with an external dma controller. To handle these xfers we're using the dma API.

To tx a chunk of data from the SoC => network device, we :
- prepare a buffer with a leading header embedding a pattern,
- trigger the xfer and wait for an irq
// The device updates the pattern and then triggers an irq
- upon irq we check the pattern for the xfer completion

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
	dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	[...]

But this does not work (the buffer pattern does not reflect the ddram value).
On the other hand, the following works:

	[...]
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // RAM value
	[...]

Looking at

	dma-mapping.c:__dma_page_cpu_to_{dev,cpu}() and
	proc-feroceon.S: feroceon_dma_{,un}map_area

this behavior is not surprising.
The sync_for_cpu calls the unmap which just invalidates the outer cache while the sync_for_device invalidates both inner and outer.

It seems that:
- we need to invalidate after the RAM has been updated
- we need to invalidate with sync_single_for_device rather than sync_single_for_cpu to check the value

Is it correct ?

Maybe the following comment in dma-mapping.c explains the situation :
/*
* The DMA API is built upon the notion of "buffer ownership". A buffer
* is either exclusively owned by the CPU (and therefore may be accessed
* by it) or exclusively owned by the DMA device. These helper functions
* represent the transitions between these two ownership states. *
* Note, however, that on later ARMs, this notion does not work due to
* speculative prefetches. We model our approach on the assumption that
* the CPU does do speculative prefetches, which means we clean caches
* before transfers and delay cache invalidation until transfer completion. *
*/

Thanks for your input,

Regards,

Karl

#3 lore raw
Quoted 8 lines · Karl Beldan

To tx a chunk of data from the SoC => network device, we :
- prepare a buffer with a leading header embedding a pattern,
- trigger the xfer and wait for an irq
// The device updates the pattern and then triggers an irq
- upon irq we check the pattern for the xfer completion

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);

Of both the CPU and the device write to the buffer, you must use DMA_BIDIRECTIONAL.

Regards,
Clemens

#4 lore raw
Quoted 7 lines · Clemens Ladisch
Quoted 6 lines · Karl Beldan

To tx a chunk of data from the SoC => network device, we :
- prepare a buffer with a leading header embedding a pattern,
- trigger the xfer and wait for an irq
// The device updates the pattern and then triggers an irq
- upon irq we check the pattern for the xfer completion

Quoted 6 lines
Quoted 2 lines

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);

Of both the CPU and the device write to the buffer, you must use DMA_BIDIRECTIONAL.

Hi Clemens,

This does not work (tested) : seems to me BIDIRECTIONAL would just add invalidate, and invalidate before the ram has been updated, as stated, does not work.
In fact the immediately following sync_for_device is intended to cater for what DMA_BIDIRECTIONAL would provide (though it is not implementation agnostic), only invalidating a smaller address range.

Regards,

Karl

#5 lore raw
Quoted 11 lines · Karl Beldan
Quoted 10 lines · Clemens Ladisch
Quoted 8 lines · Karl Beldan

To tx a chunk of data from the SoC => network device, we :
- prepare a buffer with a leading header embedding a pattern,
- trigger the xfer and wait for an irq
// The device updates the pattern and then triggers an irq
- upon irq we check the pattern for the xfer completion

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
Quoted 6 lines
Quoted 2 lines

Of both the CPU and the device write to the buffer, you must use DMA_BIDIRECTIONAL.

This does not work (tested) : seems to me BIDIRECTIONAL would just add invalidate, and invalidate before the ram has been updated, as stated, does not work.

Please show the exact sequence of dma_* calls, and also show when and how the CPU and the device access the buffer.

Regards,
Clemens

#6 lore raw
Quoted 15 lines · Clemens Ladisch
Quoted 14 lines · Karl Beldan
Quoted 12 lines · Clemens Ladisch
Quoted 8 lines · Karl Beldan

To tx a chunk of data from the SoC => network device, we :
- prepare a buffer with a leading header embedding a pattern,
- trigger the xfer and wait for an irq
// The device updates the pattern and then triggers an irq
- upon irq we check the pattern for the xfer completion

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);

Of both the CPU and the device write to the buffer, you must use DMA_BIDIRECTIONAL.

Quoted 7 lines
Quoted 3 lines

This does not work (tested) : seems to me BIDIRECTIONAL would just add invalidate, and invalidate before the ram has been updated, as stated, does not work.

Please show the exact sequence of dma_* calls, and also show when and how the CPU and the device access the buffer.

Hmm, so I just spotted a line where we peek in the buffer after invalidating .. cannot believe I missed it .. so sorry for the noise .. now it's working.
I felt I would find the culprit right after posting ;) Thanks Clemens !

#7 lore raw
In reply to Karl Beldan
Quoted 9 lines · Karl Beldan

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
	dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	[...]

First point is that you clearly do not understand the DMA API at all. The DMA API has the idea of buffer ownership. Only the owner may access the buffer:

*CPU OWNS THE BUFFER*
dma_map_single()
*DEVICE OWNS THE BUFFER*
dma_sync_single_for_cpu()
*CPU OWNS THE BUFFER*
dma_sync_single_for_device()
*DEVICE OWNS THE BUFFER*
dma_unmap_single()
*CPU OWNS THE BUFFER*

So, there is absolutely no noeed what so ever to follow dma_map_single() with dma_sync_single_for_device().

Second point is that you should not change the 'direction' argument while a buffer is mapped.

Thirdly, enable DMA API debugging (DMA_API_DEBUG) to make sure you're using the DMA API correctly.

Fourthly, remember that the CPU deals with cache lines, and dirty cache lines may be written back in their _entirety_ - which means that data DMA'd from a device in the same cache line that you've modified via the CPU will not work (either the data in the cache line has to be invalidated and therefore the CPU update discarded, or the cache line has to be flushed back to RAM and the DMA'd data is overwritten.) Hence why the buffer ownership rules are extremely important.

The solution for this fourth point is to use coherent DMA memory for things like ring buffers and the like, which does not suffer from this.

Quoted 5 lines

Maybe the following comment in dma-mapping.c explains the situation :
/*
* The DMA API is built upon the notion of "buffer ownership". A buffer
* is either exclusively owned by the CPU (and therefore may be accessed
* by it) or exclusively owned by the DMA device. These helper functions

Quoted 8 lines

* represent the transitions between these two ownership states. *
* Note, however, that on later ARMs, this notion does not work due to
* speculative prefetches. We model our approach on the assumption that
* the CPU does do speculative prefetches, which means we clean caches
* before transfers and delay cache invalidation until transfer completion. *
*/

Even with that comment, the idea of buffer ownership must be preserved by drivers, and they must follow that rule. The fact that some ARM CPU do not respect the ownership entirely is worked around inside the DMA API and is of no interest to drivers. Feroceon is not one CPU which does this though.

#8 lore raw
Quoted 11 lines · Russell King - ARM Linux
Quoted 9 lines · Karl Beldan

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
	dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	[...]

Hi Russell,

Quoted 4 lines

First point is that you clearly do not understand the DMA API at all. The DMA API has the idea of buffer ownership. Only the owner may access the buffer:

Are you saying that this scenario does not work ?
We are taking some liberties with the DMA API, we're more using some of its funcs rather than _using_ it ;).
The question was not whether this was a proper usage of the API, but why that scenario would not lead to the expected results .. and now I've found the culprit peek I am happy.

[...]

Quoted 3 lines

So, there is absolutely no noeed what so ever to follow dma_map_single() with dma_sync_single_for_device().

Cleaning a wide address range while invalidating a small one ?

[...]

Quoted 8 lines

Fourthly, remember that the CPU deals with cache lines, and dirty cache lines may be written back in their _entirety_ - which means that data DMA'd from a device in the same cache line that you've modified via the CPU will not work (either the data in the cache line has to be invalidated and therefore the CPU update discarded, or the cache line has to be flushed back to RAM and the DMA'd data is overwritten.) Hence why the buffer ownership rules are extremely important.

Of course.

Quoted 3 lines

The solution for this fourth point is to use coherent DMA memory for things like ring buffers and the like, which does not suffer from this.

I might use something different in a not so distant future, yet, for the time being, this way of doing as its advantages.

[...]

Quoted 6 lines

Even with that comment, the idea of buffer ownership must be preserved by drivers, and they must follow that rule. The fact that some ARM CPU do not respect the ownership entirely is worked around inside the DMA API and is of no interest to drivers. Feroceon is not one CPU which does this though.

It was kind of a last resort explanation for a cache line filled out of the blue .. before I spotted the culprit peek this morning.

Thanks for your input,

Karl

#9 lore raw
Quoted 20 lines · Karl Beldan
Quoted 11 lines · Russell King - ARM Linux
Quoted 9 lines · Karl Beldan

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
	dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	[...]

Hi Russell,

Quoted 4 lines

First point is that you clearly do not understand the DMA API at all. The DMA API has the idea of buffer ownership. Only the owner may access the buffer:

Quoted 6 lines

Are you saying that this scenario does not work ?
We are taking some liberties with the DMA API, we're more using some of its funcs rather than _using_ it ;).
The question was not whether this was a proper usage of the API, but why that scenario would not lead to the expected results .. and now I've found the culprit peek I am happy.

If you abuse the API don't expect your stuff to work in future kernel versions.

It seems that the overall tone of your reply is "what we have now works, we don't care if it's correct, sod you."

Fine, I won't spend any more time on this. Just don't ever think about merging it into mainline, thanks.

#10 lore raw
Quoted 32 lines · Russell King - ARM Linux
Quoted 24 lines · Karl Beldan
Quoted 13 lines · Russell King - ARM Linux
Quoted 11 lines · Karl Beldan

I was expecting the following to work:

	addr = dma_map_single(dev, buffer, size, DMA_TO_DEVICE);
	dma_sync_single_for_device(dev, buffer, pattern_size,
DMA_FROM_DEVICE);
	dev_send(buffer);
	// wait for irq (don't peek in the buffer) ... got irq
	dma_sync_single_for_cpu(dev, buffer, pattern_size, DMA_FROM_DEVICE);
	if (!xfer_done(buffer)) // not RAM value
		dma_sync_single_for_device(dev, buffer, pattern_size,
DMA_FROM_DEVICE);
	[...]

Hi Russell,

Quoted 6 lines

First point is that you clearly do not understand the DMA API at all. The
DMA API has the idea of buffer ownership. Only the owner may access the
buffer:

Quoted 6 lines

Are you saying that this scenario does not work ?
We are taking some liberties with the DMA API, we're more using some of its funcs rather than _using_ it ;).
The question was not whether this was a proper usage of the API, but why that scenario would not lead to the expected results .. and now I've found the culprit peek I am happy.

Quoted 3 lines

If you abuse the API don't expect your stuff to work in future kernel versions.

Of course.

Quoted 3 lines

It seems that the overall tone of your reply is "what we have now works, we don't care if it's correct, sod you."

Not at all : {

Quoted 2 lines · Karl Beldan

I might use something different in a not so distant future, yet, for the time being, this way of doing as its advantages.

} and during this time I might stick to the API.
I am not at ease telling people how they should take things, especially if I asked for their help, all the more when they put efforts within the exchange while being expert on the matter, yet please, do not assume I did not care for your advices, which I deem of the most valuable, as, needless to say, do the community.

Quoted 3 lines

Fine, I won't spend any more time on this. Just don't ever think about merging it into mainline, thanks.

Merge submission while taking such liberties .. that I would not dare ;) this really was a down to the ground technical question not the start of a disguised start of a merging request.

I am sure that taking such liberties and feeling its limits before sticking to a super safe API is not a bad thing, e.g it might trigger easierly nasty hidden bugs, it is often beneficial to me at least.

Thanks for your feedback,

Karl