add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

RFC PATCH In review Marin Mitov 6 participants 43 messages Aug 19, 2010 – Oct 14, 2010

Subsystems (3)

Media (V4L/DVB) SuperH Driver core

lore

Share & tools

lore

Marin Mitov

author

2010-08-19 15:20 UTC #0 lore raw

RFC PATCH 1/1 add dma_reserve_coherent_memory()/dma_free_reserved_memory() API

Hi all,

struct device contains a member: struct dma_coherent_mem *dma_mem; to hold information for a piece of memory declared dma-coherent. Alternatively the same member could also be used to hold preallocated dma-coherent memory for latter per-device use.

This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()

Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.

Signed-off-by Marin Mitov <[email protected]>

======================================================================

drivers/base/dma-coherent.c +77 -0

@@ -93,6 +93,83 @@ void *dma_mark_declared_memory_occupied(
 EXPORT_SYMBOL(dma_mark_declared_memory_occupied);
 
 /**
+ * dma_reserve_coherent_memory() - reserve coherent memory for per-device use
+ *
+ * @dev:	device from which we allocate memory
+ * @size:	size of requested memory area in bytes
+ * @flags:	same as in dma_declare_coherent_memory()
+ *
+ * This function reserves coherent memory allocating it early (during probe())
+ * to support latter allocations from per-device coherent memory pools.
+ * For a given device one could use either dma_declare_coherent_memory() or
+ * dma_reserve_coherent_memory(), but not both, becase the result of these
+ * functions is stored in a single struct device member - dma_mem
+ *
+ * Returns DMA_MEMORY_MAP on success, or 0 if failed.
+ * (same as dma_declare_coherent_memory()
+ */
+int dma_reserve_coherent_memory(struct device *dev, size_t size, int flags)
+{
+	struct dma_coherent_mem *mem;
+	dma_addr_t dev_base;
+	int pages = size >> PAGE_SHIFT;
+	int bitmap_size = BITS_TO_LONGS(pages) * sizeof(long);
+
+	if ((flags & DMA_MEMORY_MAP) == 0)
+		goto out;
+	if (!size)
+		goto out;
+	if (dev->dma_mem)
+		goto out;
+
+	mem = kzalloc(sizeof(*mem), GFP_KERNEL);
+	if (!mem)
+		goto out;
+	mem->virt_base = dma_alloc_coherent(dev, size, &dev_base,
+							DT3155_COH_FLAGS);
+	if (!mem->virt_base)
+		goto err_alloc_coherent;
+	mem->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
+	if (!mem->bitmap)
+		goto err_bitmap;
+
+	mem->device_base = dev_base;
+	mem->size = pages;
+	mem->flags = flags;
+	dev->dma_mem = mem;
+	return DMA_MEMORY_MAP;
+
+err_bitmap:
+	dma_free_coherent(dev, size, mem->virt_base, dev_base);
+err_alloc_coherent:
+	kfree(mem);
+out:
+	return 0;
+}
+EXPORT_SYMBOL(dma_reserve_coherent_memory);
+
+/**
+ * dma_free_reserved_memory() - free the reserved dma-coherent memoty
+ *
+ * @dev:	device for which we free the dma-coherent memory
+ *
+ * same as dma_release_declared_memory()
+ */
+void dma_free_reserved_memory(struct device *dev)
+{
+	struct dma_coherent_mem *mem = dev->dma_mem;
+
+	if (!mem)
+		return;
+	dev->dma_mem = NULL;
+	dma_free_coherent(dev, mem->size << PAGE_SHIFT,
+					mem->virt_base, mem->device_base);
+	kfree(mem->bitmap);
+	kfree(mem);
+}
+EXPORT_SYMBOL(dma_free_reserved_memory);
+
+/**
  * dma_alloc_from_coherent() - try to allocate memory from the per-device coherent area
  *
  * @dev:	device from which we allocate memory

FUJITA Tomonori

2010-08-20 07:18 UTC #1 lore raw

Quoted 4 lines · On Thu, 19 Aug 2010 18:18:35 +0300 Marin Mitov <[email protected]> wrote:

struct device contains a member: struct dma_coherent_mem *dma_mem; to hold information for a piece of memory declared dma-coherent. Alternatively the same member could also be used to hold preallocated dma-coherent memory for latter per-device use.

I think that drivers/base/dma-coherent.c is for architectures to implement dma_alloc_coherent(). So using it for drivers doesn't look correct.

Quoted 5 lines

This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.

What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage, why dma_pool API (mm/dmapool.c) doesn't work?

Marin Mitov

author

2010-08-20 08:15 UTC #2 lore raw

Quoted 11 lines · FUJITA Tomonori

Quoted 4 lines · On Thu, 19 Aug 2010 18:18:35 +0300 Marin Mitov <[email protected]> wrote:
struct device contains a member: struct dma_coherent_mem *dma_mem; to hold information for a piece of memory declared dma-coherent. Alternatively the same member could also be used to hold preallocated dma-coherent memory for latter per-device use.
I think that drivers/base/dma-coherent.c is for architectures to implement dma_alloc_coherent(). So using it for drivers doesn't look correct.

It depends. Imagine your frame grabber has built-in RAM buffer on board just as the frame buffer RAM on graphics cards, defined in BAR. You can use dma_declare_coherent_memory()/dma_release_declared_memory() in your driver and then use dma_alloc_coherent()/dma_free_coherent() to allocate dma buffers from it and falling back transparently to system RAM when this local resource is exhausted.

Quoted 10 lines

Quoted 7 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,

Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.

Quoted 1 line

why dma_pool API (mm/dmapool.c) doesn't work?

I do not know why dma_pool API doesn't work for frame grabber buffers. May be they are too big ~400KB. I have tried dma_pool APIs without success some time ago, so I had to find some other way to solve my problem leading to the proposed dma_reserve_coherent_memory()/dma_free_reserved_memory().

Thanks.

Marin Mitov

FUJITA Tomonori

2010-08-20 08:35 UTC #3 lore raw

Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:

Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.

Hmm, you don't care about coherency? You just need contiguous memory?

Then, I prefer to invent the API to allocate contiguous memory. Coherent memory is precious on some arches.

Quoted 6 lines

Quoted 1 line
why dma_pool API (mm/dmapool.c) doesn't work?
I do not know why dma_pool API doesn't work for frame grabber buffers. May be they are too big ~400KB. I have tried dma_pool APIs without success some time ago, so I had to find some other way to solve my problem leading to the proposed dma_reserve_coherent_memory()/dma_free_reserved_memory().

I think that dma_pool API is for small coherent memory (smaller than PAGE_SIZE) so it might not work for you. However, the purpose of dma_pool API is exactly for what you want to do, creating a pool for coherent memory per device for drivers.

I don't see any reason why we can't extend the dma_pool API for your case. And it looks better to me rather than inventing the new API.

Marin Mitov

author

2010-08-20 11:51 UTC #4 lore raw

Quoted 9 lines · FUJITA Tomonori

Quoted 6 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 6 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.

Quoted 7 lines

Quoted 5 lines
Quoted 2 lines
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?

Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()

Quoted 3 lines

Then, I prefer to invent the API to allocate contiguous memory. Coherent memory is precious on some arches.

Sure, but in any case videobuf-dma-contig framework in drivers/media/video is already built around dma-coherent (nevertheless it is precious), so the two new functions are just a helpful extension to the existing use of dma-coherent memory.

In any case, as far as these two functions will be mainly used by media/video folks they could be added not to the drivers/base/dma-coherent.c (where I see their place), but to drivers/media/video/videobuf-dma-contig.c. In that case the disadvantage will be that if someone out of the media tree will need this functionality he(she) will need to compile media/videobuf-dma-contig.c

Quoted 11 lines

Quoted 8 lines
Quoted 3 lines
why dma_pool API (mm/dmapool.c) doesn't work?
I do not know why dma_pool API doesn't work for frame grabber buffers. May be they are too big ~400KB. I have tried dma_pool APIs without success some time ago, so I had to find some other way to solve my problem leading to the proposed dma_reserve_coherent_memory()/dma_free_reserved_memory().
I think that dma_pool API is for small coherent memory (smaller than PAGE_SIZE)

Yes.

Quoted 6 lines

so it might not work for you. However, the purpose of dma_pool API is exactly for what you want to do, creating a pool for coherent memory per device for drivers.
I don't see any reason why we can't extend the dma_pool API for your case. And it looks better to me rather than inventing the new API.

That will help. I will be happy if someone can do it. I am inpaciently waiting for alloc_huhepages()/free_hugepages() API - (transparent hugepages patches, may be) That also could be a solution for media/video folks with hardware that cannot do scatter/gatter. Another solution will be an IOMMU that could present a scattered user land buffer as contiguous dma address range (I have played in the past with AGP-GART without great success).

Thanks.

Marin Mitov

FUJITA Tomonori

2010-08-26 05:41 UTC #5 lore raw

Quoted 13 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:

Quoted 12 lines · FUJITA Tomonori
Quoted 9 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()

Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.

We need the proper API for contiguous memory. Seem that we could have something:

http://lkml.org/lkml/2010/8/20/167

Marin Mitov

author

2010-08-26 06:06 UTC #6 lore raw

Quoted 21 lines · FUJITA Tomonori

Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?

Quoted 7 lines

Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.

Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?

Thanks.

Marin Mitov

Quoted 6 lines

We need the proper API for contiguous memory. Seem that we could have something:
http://lkml.org/lkml/2010/8/20/167

FUJITA Tomonori

2010-08-26 06:24 UTC #7 lore raw

Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:

Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.

Quoted 5 lines

Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?

If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine, I think (that is, dt3155v4l driver is broken). There are already some workarounds for contigous memory in several drivers anyway.

We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.

Marin Mitov

author

2010-08-26 07:04 UTC #8 lore raw

Quoted 39 lines · FUJITA Tomonori

Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:
Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.
Quoted 5 lines
Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?

Quoted 4 lines

If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine,

Why, my understanding is that device->dma_mem is designed exactly for keeping some chunk of coherent memory for device's private use via dma_alloc_from_coherent() (and that is what dt3155v4l driver is using it for).

Quoted 1 line

I think (that is, dt3155v4l driver is broken).

If you mean that allocating some coherent memory (4MB in case of dt3155v4l) during pci probe() (during system booting) for device's latter use (that is dead for the rest of the system) you are right. But this gives me at least 8 full size buffers warranted for latter use. Without this hack the hardware will not work on strongly fragmented system. With this hack even if the system is strongly fragmented, this chunk of 4MB is available for use (though videobuf-dma-contig APIs and dma_alloc_from_coherent()) __transparently__ for users of videobuf-dma-contig (that is the gain - the transparency).

Quoted 2 lines

There are already some workarounds for
contigous memory in several drivers anyway.

Sure, can these workarounds be exposed as API for general use?

Thanks,

Marin Mitov

Quoted 4 lines

We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.

FUJITA Tomonori

2010-08-26 09:43 UTC #9 lore raw

Quoted 8 lines · On Thu, 26 Aug 2010 10:01:52 +0300 Marin Mitov <[email protected]> wrote:

Quoted 4 lines
If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine,
Why, my understanding is that device->dma_mem is designed exactly for keeping some chunk of coherent memory for device's private use via dma_alloc_from_coherent() (and that is what dt3155v4l driver is using it for).

I don't think so. device->dma_mem can be accessed only via the DMA-API. I think that the DMA-API says that dma_declare_coherent_memory declares coherent memory that can be access exclusively by a certain device. It's not for reserving coherent memory that can be used for any device for a device.

Anway, you don't need coherent memory. So using the API for coherent memory isn't a good idea.

Quoted 4 lines

Quoted 2 lines
There are already some workarounds for
contigous memory in several drivers anyway.
Sure, can these workarounds be exposed as API for general use?

I don't think that's a good idea. Adding temporary workaround to the generic API and removing it soon after that doesn't sound a good developing maner.

Marin Mitov

author

2010-08-26 10:16 UTC #10 lore raw

Quoted 8 lines · FUJITA Tomonori

Quoted 5 lines · On Thu, 26 Aug 2010 10:01:52 +0300 Marin Mitov <[email protected]> wrote:
Quoted 4 lines
If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine,

Quoted 8 lines

Quoted 3 lines
Why, my understanding is that device->dma_mem is designed exactly for keeping some chunk of coherent memory for device's private use via dma_alloc_from_coherent() (and that is what dt3155v4l driver is using it for).
I don't think so. device->dma_mem can be accessed only via the DMA-API. I think that the DMA-API says that dma_declare_coherent_memory declares coherent memory that can be access exclusively by a certain device.

Here I agree with you: "that can be access exclusively by a certain device"

Quoted 2 lines

It's not for reserving
coherent memory that can be used for any device for a device.

Here I disagree with you: "that can be used for any device for a device". Reserved coherent memory can be only and exclusively used by the __same__ device whose device->dma_mem is touched. No other devices are influenced because their device->dma_mem are NULL. and dma_alloc_from_coherent() is not invoked for them. That is why I think this hack is not dangerous. If some device driver decide to reserve some chunk of memory it is for its private use and no other device in the system is influenced.

Quoted 3 lines

Anway, you don't need coherent memory. So using the API for coherent memory isn't a good idea.

Here I agree with you, but for now we have no alternative in media/video framework.

Quoted 10 lines

Quoted 6 lines
Quoted 4 lines
There are already some workarounds for
contigous memory in several drivers anyway.
Sure, can these workarounds be exposed as API for general use?
I don't think that's a good idea. Adding temporary workaround to the generic API and removing it soon after that doesn't sound a good developing maner.

Yes, it is just a temporary solution. Just enhancing an existing temporary solution.

Thanks,

Marin Mitov

Quoted 1 line

Guennadi Liakhovetski

2010-08-26 09:06 UTC #11 lore raw

In reply to FUJITA Tomonori

Quoted 39 lines · FUJITA Tomonori

Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:
Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.
Quoted 5 lines
Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?

Quoted 6 lines

If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine, I think (that is, dt3155v4l driver is broken). There are already some workarounds for contigous memory in several drivers anyway.

No, this will not work - this API has to be used from board code and videobuf can be built modular.

Quoted 2 lines

We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.

We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series

http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

for 2.6.36.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Uwe Kleine-König

2010-08-26 09:17 UTC #12 lore raw

Hello,

Quoted 51 lines · Guennadi Liakhovetski

Quoted 39 lines · FUJITA Tomonori
Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:
Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.
Quoted 5 lines
Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?
Quoted 6 lines
If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine, I think (that is, dt3155v4l driver is broken). There are already some workarounds for contigous memory in several drivers anyway.
No, this will not work - this API has to be used from board code and videobuf can be built modular.

Quoted 8 lines

Quoted 2 lines
We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

this seems to be more mature to me. The original patch in this thread uses a symbol DT3155_COH_FLAGS which seems misplaced in generic code and doesn't put the new functions in a header.

Best regards
Uwe

Signature · 2 lines

Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

Marin Mitov

author

2010-08-26 10:18 UTC #13 lore raw

Quoted 57 lines · Uwe Kleine-K?nig

Hello,
Quoted 54 lines · Guennadi Liakhovetski
Quoted 39 lines · FUJITA Tomonori
Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:
Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.
Quoted 5 lines
Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?
Quoted 6 lines
If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine, I think (that is, dt3155v4l driver is broken). There are already some workarounds for contigous memory in several drivers anyway.
No, this will not work - this API has to be used from board code and videobuf can be built modular.
Quoted 2 lines
We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.

Quoted 8 lines

Quoted 5 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
this seems to be more mature to me. The original patch in this thread uses a symbol DT3155_COH_FLAGS which seems misplaced in generic code and doesn't put the new functions in a header.

You are right. DT3155_COH_FLAGS should be defined, and a declaration should be put in the headers.

But it is just RFC :-)

Marin Mitov

Quoted 5 lines

Best regards
Uwe

FUJITA Tomonori

2010-08-26 09:30 UTC #14 lore raw

In reply to Guennadi Liakhovetski

Quoted 54 lines · On Thu, 26 Aug 2010 11:06:20 +0200 (CEST) Guennadi Liakhovetski…

Quoted 39 lines · FUJITA Tomonori
Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:
Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.
Quoted 5 lines
Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?
Quoted 6 lines
If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine, I think (that is, dt3155v4l driver is broken). There are already some workarounds for contigous memory in several drivers anyway.
No, this will not work - this API has to be used from board code and videobuf can be built modular.
Quoted 2 lines
We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.

Quoted 7 lines

We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.

Why can't you revert a commit that causes the regression?

The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

Guennadi Liakhovetski

2010-08-26 09:45 UTC #15 lore raw

Quoted 1 line · FUJITA Tomonori

Why can't you revert a commit that causes the regression?

See this reply, and the complete thread too.

http://marc.info/?l=linux-sh&m=128130485208262&w=2

Quoted 7 lines

The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

Anyway, we need a way to fix the regression.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

FUJITA Tomonori

2010-08-26 09:51 UTC #16 lore raw

Quoted 16 lines · On Thu, 26 Aug 2010 11:45:58 +0200 (CEST) Guennadi Liakhovetski…

Quoted 1 line · FUJITA Tomonori
Why can't you revert a commit that causes the regression?
See this reply, and the complete thread too.
http://marc.info/?l=linux-sh&m=128130485208262&w=2
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

Quoted 1 line

Anyway, we need a way to fix the regression.

Needs to find a different way.

Russell King - ARM Linux

2010-08-26 17:49 UTC #17 lore raw

Quoted 19 lines · FUJITA Tomonori

Quoted 16 lines · On Thu, 26 Aug 2010 11:45:58 +0200 (CEST) Guennadi Liakhovetski…
Quoted 1 line · FUJITA Tomonori
Why can't you revert a commit that causes the regression?
See this reply, and the complete thread too.
http://marc.info/?l=linux-sh&m=128130485208262&w=2
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

Quoted 3 lines

Quoted 1 line
Anyway, we need a way to fix the regression.
Needs to find a different way.

No. ioremap on memory mapped by the kernel is just plain not permitted with ARMv6 and ARMv7 architectures.

It's not something you can say "oh, need to find another way" because there is _no_ software solution to having physical regions mapped multiple times with different attributes. It's an architectural restriction.

We can't unmap the kernel's memory mapping either, as I've already explained several times this month - and I'm getting frustrated at having to keep on explaining that point.

Just accept the plain fact that multiple mappings of the same physical regions have become illegal.

What we need is another alternative other than using ioremap on memory already mapped by the kernel - eg, by reserving a certain chunk of memory for this purpose at boot time which his _never_ mapped by the kernel, except via ioremap.

Marin Mitov

author

2010-08-26 18:32 UTC #18 lore raw

Quoted 20 lines · Russell King - ARM Linux

Quoted 19 lines · FUJITA Tomonori
Quoted 16 lines · On Thu, 26 Aug 2010 11:45:58 +0200 (CEST) Guennadi Liakhovetski…
Quoted 1 line · FUJITA Tomonori
Why can't you revert a commit that causes the regression?
See this reply, and the complete thread too.
http://marc.info/?l=linux-sh&m=128130485208262&w=2
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

Quoted 6 lines

Quoted 3 lines
Quoted 1 line
Anyway, we need a way to fix the regression.
Needs to find a different way.
No. ioremap on memory mapped by the kernel is just plain not permitted with ARMv6 and ARMv7 architectures.

Hi Russell,

Just because ioremap on memory mapped by the kernel is just plain not permitted
I have proposed a new pair of functions: dma_reserve_coherent_memory()/dma_free_reserved_memory()

http://lkml.org/lkml/2010/8/19/200

but it is not quite well accepted from the community. What is your opinion?

Thanks,

Marin Mitov

Quoted 21 lines

It's not something you can say "oh, need to find another way" because there is _no_ software solution to having physical regions mapped multiple times with different attributes. It's an architectural restriction.
We can't unmap the kernel's memory mapping either, as I've already explained several times this month - and I'm getting frustrated at having to keep on explaining that point.
Just accept the plain fact that multiple mappings of the same physical regions have become illegal.
What we need is another alternative other than using ioremap on memory already mapped by the kernel - eg, by reserving a certain chunk of memory for this purpose at boot time which his _never_ mapped by the kernel, except via ioremap.
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Uwe Kleine-König

2010-08-26 09:53 UTC #19 lore raw

In reply to FUJITA Tomonori

Quoted 67 lines · FUJITA Tomonori

Quoted 54 lines · On Thu, 26 Aug 2010 11:06:20 +0200 (CEST) Guennadi Liakhovetski…
Quoted 39 lines · FUJITA Tomonori
Quoted 30 lines · On Thu, 26 Aug 2010 09:04:14 +0300 Marin Mitov <[email protected]> wrote:
Quoted 21 lines · FUJITA Tomonori
Quoted 18 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 16 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Quoted 7 lines
Quoted 3 lines
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Then, we should avoid using coherent memory as I exaplained before. In addition, dma_alloc_coherent can't provide large enough contigous memory for some drivers so this patch doesn't help much.
Quoted 5 lines
Please, look at drivers/media/video/videobuf-dma-contig.c. Using coherent memory is inavoidable for now, there is no alternative for it for now. The two new functions, which I propose are just helpers for those of us who already use coherent memory (via videobuf-dma-contig API). May be adding these two functions to drivers/media/video/videobuf-dma-contig.c will be better solution?
Quoted 6 lines
If you add something to the videobuf-dma-contig API, that's fine by me because drivers/media/video/videobuf-dma-contig.c uses the own structure and plays with dma_alloc_coherent. As long as a driver doesn't touch device->dma_mem directly, it's fine, I think (that is, dt3155v4l driver is broken). There are already some workarounds for contigous memory in several drivers anyway.
No, this will not work - this API has to be used from board code and videobuf can be built modular.
Quoted 2 lines
We will have the proper API for contiguous memory. I don't think that adding such workaround to the DMA API is a good idea.
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?

Quoted 7 lines

The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.

And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.

Best regards
Uwe

Signature · 2 lines

Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

FUJITA Tomonori

2010-08-26 10:00 UTC #20 lore raw

Quoted 17 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…

Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.

Quoted 7 lines

The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.

How these drivers were able to work without hitting the hardware restriction?

Russell King - ARM Linux

2010-08-26 17:54 UTC #21 lore raw

Quoted 25 lines · FUJITA Tomonori

Quoted 22 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.

Quoted 4 lines

Quoted 2 lines
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
How these drivers were able to work without hitting the hardware restriction?

Well, OMAP processors have experienced lock-ups due to multiple mappings of memory, so the restriction in the architecture manual is for real.

But more the issue is that the behaviour you get from a region is _totally_ unpredictable (as the arch manual says). With the VIPT caches, they can be searched irrespective of whether the page tabkes indicate that it's supposed to be cached or not - which means you can still hit cache lines for an ioremap'd region.

And if you do, how do you know that the cached data is still valid - what if it's some critical data that results in corruption - how do you know whether that's happened or not? It might not even cause a kernel exception.

We have to adhere to the restrictions placed upon us by the architecture at hand, and if that means device drivers break, so be it - at least we get to know what needs to be fixed for these restrictions.

FUJITA Tomonori

2010-08-27 00:26 UTC #22 lore raw

Quoted 40 lines · On Thu, 26 Aug 2010 18:54:40 +0100 Russell King - ARM Linux…

Quoted 25 lines · FUJITA Tomonori
Quoted 22 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
Quoted 4 lines
Quoted 2 lines
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
How these drivers were able to work without hitting the hardware restriction?
Well, OMAP processors have experienced lock-ups due to multiple mappings of memory, so the restriction in the architecture manual is for real.
But more the issue is that the behaviour you get from a region is _totally_ unpredictable (as the arch manual says). With the VIPT caches, they can be searched irrespective of whether the page tabkes indicate that it's supposed to be cached or not - which means you can still hit cache lines for an ioremap'd region.

Quoted 8 lines

And if you do, how do you know that the cached data is still valid - what if it's some critical data that results in corruption - how do you know whether that's happened or not? It might not even cause a kernel exception.
We have to adhere to the restrictions placed upon us by the architecture at hand, and if that means device drivers break, so be it - at least we get to know what needs to be fixed for these restrictions.

I didn't say the commit is technically wrong. I simply meant that the commit broke some of working systems (so some complain, I guess).

As I wrote, the related DMA API wasn't changed in 2.6.36-rc1. It's not related with the regression at all. As long as nobody tries to extend the API wrongly after rc2, I have no complaint.

btw, Marin Mitov said that these drivers don't need coherent memory, they just want contiguous memory. Telling the page allocater to reserve some memory at boot time is enough, I guess.

Uwe Kleine-König

2010-08-27 04:41 UTC #23 lore raw

In reply to FUJITA Tomonori

Hello,

Quoted 25 lines · FUJITA Tomonori

Quoted 22 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.

Quoted 4 lines

Quoted 2 lines
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
How these drivers were able to work without hitting the hardware restriction?

In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code. And it's good to let developers that don't have the full range of machines supported by the kernel at hand notice when they introduce an incompatibility.

Best regards
Uwe

Signature · 2 lines

Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

FUJITA Tomonori

2010-08-27 05:00 UTC #24 lore raw

Quoted 31 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K?nig…

Hello,
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.

Quoted 6 lines

Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.

Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.

Uwe Kleine-König

2010-08-27 05:19 UTC #25 lore raw

Hey,

Quoted 38 lines · FUJITA Tomonori

Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K?nig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.

Quoted 4 lines

Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.

I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?

Best regards
Uwe

Signature · 2 lines

Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

FUJITA Tomonori

2010-08-27 05:57 UTC #26 lore raw

Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K?nig…

Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K?nig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.

Quoted 6 lines

I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?

No, I don't think so. We are talking about a regression, right?

On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.

Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way). In addition, the patch might break the current code. I really don't think that applying such patch after rc1 is senseble.

Uwe Kleine-König

2010-08-27 06:13 UTC #27 lore raw

Hello,

Quoted 64 lines · FUJITA Tomonori

Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K?nig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K?nig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K?nig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.

Quoted 4 lines

Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way). In addition, the patch might break the current code. I really don't think that applying such patch after rc1 is senseble.

So you suggest to revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 or at least restrict it to ARMv6+ and fix the problem during the next merge window? Russell?

Best regards
Uwe

Signature · 2 lines

Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

Marin Mitov

author

2010-08-27 06:23 UTC #28 lore raw

In reply to FUJITA Tomonori

Quoted 64 lines · FUJITA Tomonori

Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.

Quoted 4 lines

Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.

To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".

Thanks,

Marin Mitov

Quoted 3 lines

I really don't think that applying such patch after rc1 is senseble.

FUJITA Tomonori

2010-08-27 06:32 UTC #29 lore raw

Quoted 65 lines · On Fri, 27 Aug 2010 09:23:21 +0300 Marin Mitov <[email protected]> wrote:

Quoted 64 lines · FUJITA Tomonori
Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.

Quoted 8 lines

Quoted 4 lines
Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.
To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".

Looks like that the patch adds the new API that touches the exisitng code. It means the existing code could break. So the exsising API could break too.

http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

Uwe Kleine-König

2010-08-27 06:38 UTC #30 lore raw

Hello,

Quoted 77 lines · FUJITA Tomonori

Quoted 65 lines · On Fri, 27 Aug 2010 09:23:21 +0300 Marin Mitov <[email protected]> wrote:
Quoted 64 lines · FUJITA Tomonori
Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.
Quoted 8 lines
Quoted 4 lines
Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.
To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".

Quoted 5 lines

Looks like that the patch adds the new API that touches the exisitng code. It means the existing code could break. So the exsising API could break too.
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

I'm still trying to find out what you actually suggest we should do now. Maybe this is a request for a minimal "fix" without the cleanups Guennadi did? That is only patches 2(?), 4 and 5 of the series?

Best regards
Uwe

Signature · 2 lines

Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

Marin Mitov

author

2010-08-27 07:02 UTC #31 lore raw

In reply to FUJITA Tomonori

Quoted 87 lines · FUJITA Tomonori

Quoted 65 lines · On Fri, 27 Aug 2010 09:23:21 +0300 Marin Mitov <[email protected]> wrote:
Quoted 64 lines · FUJITA Tomonori
Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.
Quoted 8 lines
Quoted 4 lines
Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.
To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".
Looks like that the patch adds the new API that touches the exisitng code. It means the existing code could break. So the exsising API could break too.
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Marin Mitov

author

2010-08-28 06:14 UTC #32 lore raw

In reply to FUJITA Tomonori

Quoted 77 lines · FUJITA Tomonori

Quoted 65 lines · On Fri, 27 Aug 2010 09:23:21 +0300 Marin Mitov <[email protected]> wrote:
Quoted 64 lines · FUJITA Tomonori
Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.
Quoted 8 lines
Quoted 4 lines
Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.
To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".

Quoted 5 lines

Looks like that the patch adds the new API that touches the exisitng code. It means the existing code could break. So the exsising API could break too.
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

The above reference is not my patch. I am speaking for my patch:

http://lkml.org/lkml/2010/8/19/200

The only point my patch touches the existing code is struct device's member dma_mem and that is in condition you __use__ the new API, so you could decide yourself if it could break the current code. As far as one does not use the new API - nothing is touched, nothing can break. If one uses the new API, only the user can suffer if the new API have bugs.

Thanks,

Marin Mitov

FUJITA Tomonori

2010-08-28 07:10 UTC #33 lore raw

Quoted 82 lines · On Sat, 28 Aug 2010 09:14:25 +0300 Marin Mitov <[email protected]> wrote:

Quoted 81 lines · FUJITA Tomonori
Quoted 65 lines · On Fri, 27 Aug 2010 09:23:21 +0300 Marin Mitov <[email protected]> wrote:
Quoted 64 lines · FUJITA Tomonori
Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.
Quoted 8 lines
Quoted 4 lines
Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.
To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".
Looks like that the patch adds the new API that touches the exisitng code. It means the existing code could break. So the exsising API could break too.

Quoted 5 lines

Quoted 1 line
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
The above reference is not my patch. I am speaking for my patch:
http://lkml.org/lkml/2010/8/19/200

I think that I already NACK'ed the patch.

1) drivers/media/videobuf-dma-contig.c should not use dma_alloc_coherent. We shouldn't support the proposed API.

2) I don't think that the DMA API (drivers/base/dma-mapping.c) is not for creating "cache". Generally, the kernel uses "pool" concept for something like that.

IMHO, reverting the commit 309caa9cc6ff39d261264ec4ff10e29489afc8f8 temporary (or temporary disabling it for systems that had worked) is the most reasonable approach. I don't think that breaking systems that had worked is a good idea even if the patch does the right thing. I believe that we need to fix the broken solution (videobuf-dma-contig.c) before the commit.

Marin Mitov

author

2010-08-28 07:19 UTC #34 lore raw

Quoted 85 lines · FUJITA Tomonori

Quoted 82 lines · On Sat, 28 Aug 2010 09:14:25 +0300 Marin Mitov <[email protected]> wrote:
Quoted 81 lines · FUJITA Tomonori
Quoted 65 lines · On Fri, 27 Aug 2010 09:23:21 +0300 Marin Mitov <[email protected]> wrote:
Quoted 64 lines · FUJITA Tomonori
Quoted 45 lines · On Fri, 27 Aug 2010 07:19:07 +0200 Uwe Kleine-K$(D+S(Bnig…
Hey,
Quoted 38 lines · FUJITA Tomonori
Quoted 29 lines · On Fri, 27 Aug 2010 06:41:42 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 28 lines · FUJITA Tomonori
Quoted 24 lines · On Thu, 26 Aug 2010 11:53:11 +0200 Uwe Kleine-K$(D+S(Bnig…
Quoted 10 lines
Quoted 7 lines
We have currently a number of boards broken in the mainline. They must be fixed for 2.6.36. I don't think the mentioned API will do this for us. So, as I suggested earlier, we need either this or my patch series
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
for 2.6.36.
Why can't you revert a commit that causes the regression?
Quoted 7 lines
The related DMA API wasn't changed in 2.6.36-rc1. The DMA API is not responsible for the regression. And the patchset even exnteds the definition of the DMA API (dma_declare_coherent_memory). Such change shouldn't applied after rc1. I think that DMA-API.txt says that dma_declare_coherent_memory() handles coherent memory for a particular device. It's not for the API that reserves coherent memory that can be used for any device for a single device.
The patch that made the problem obvious for ARM is 309caa9cc6ff39d261264ec4ff10e29489afc8f8 aka v2.6.36-rc1~591^2~2^4~12. So this went in before v2.6.36-rc1. One of the "architectures which similar restrictions" is x86 BTW.
And no, we won't revert 309caa9cc6ff39d261264ec4ff10e29489afc8f8 as it addresses a hardware restriction.
Quoted 6 lines
Quoted 1 line
How these drivers were able to work without hitting the hardware restriction?
In my case the machine in question is an ARMv5, the hardware restriction is on ARMv6+ only. You could argue that so the breaking patch for arm should only break ARMv6, but I don't think this is sensible from a maintainers POV. We need an API that works independant of the machine that runs the code.
Quoted 4 lines
Agreed. But insisting that the DMA API needs to be extended wrongly after rc2 to fix the regression is not sensible too. The related DMA API wasn't changed in 2.6.36-rc1. The API isn't responsible for the regression at all.
Quoted 6 lines
I think this isn't about "responsiblity". Someone in arm-land found that the way dma memory allocation worked for some time doesn't work anymore on new generation chips. As pointing out this problem was expected to find some matches it was merged in the merge window. One such match is the current usage of the DMA API that doesn't currently offer a way to do it right, so it needs a patch, no?
No, I don't think so. We are talking about a regression, right?
On new generation chips, something often doesn't work (which have worked on old chips for some time). It's not a regresiion. I don't think that it's sensible to make large change (especially after rc1) to fix such issue. If you say that the DMA API doesn't work on new chips and proposes a patch for the next merge window, it's sensible, I suppose.
Quoted 8 lines
Quoted 4 lines
Btw, the patch isn't a fix for the DMA API. It tries to extend the DMA API (and IMO in the wrong way).
In addition, the patch might break the
current code.
To "break the current code" is simply not possible. Sorry to oppose. As you have written it "extend the DMA API", so if you do not use the new API (and no current code is using it) you cannot "break the current code".
Looks like that the patch adds the new API that touches the exisitng code. It means the existing code could break. So the exsising API could break too.

Quoted 7 lines

Quoted 5 lines
Quoted 1 line
http://thread.gmane.org/gmane.linux.ports.sh.devel/8595
The above reference is not my patch. I am speaking for my patch:
http://lkml.org/lkml/2010/8/19/200
I think that I already NACK'ed the patch.

OK.

Thanks,

Marin Mitov

Quoted 19 lines

1) drivers/media/videobuf-dma-contig.c should not use dma_alloc_coherent. We shouldn't support the proposed API.
2) I don't think that the DMA API (drivers/base/dma-mapping.c) is not for creating "cache". Generally, the kernel uses "pool" concept for something like that.
IMHO, reverting the commit 309caa9cc6ff39d261264ec4ff10e29489afc8f8 temporary (or temporary disabling it for systems that had worked) is the most reasonable approach. I don't think that breaking systems that had worked is a good idea even if the patch does the right thing. I believe that we need to fix the broken solution (videobuf-dma-contig.c) before the commit.
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

FUJITA Tomonori

2010-10-10 14:08 UTC #35 lore raw

In reply to Marin Mitov

Quoted 13 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:

Quoted 12 lines · FUJITA Tomonori
Quoted 9 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()

Anyone is working on this?

KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.

The approach enables us to solve this issue without adding any new API.

Marin Mitov

author

2010-10-10 15:01 UTC #36 lore raw

Quoted 19 lines · FUJITA Tomonori

Quoted 16 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 15 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.

Quoted 7 lines

Quoted 5 lines
Quoted 1 line
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Anyone is working on this?

I am not, sorry.

Quoted 4 lines

KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.

I am waiting for the new videobuf2 framework to become part of the kernel. Then KAMEZAWA's improvements can help.

Marin Mitov

Quoted 4 lines

The approach enables us to solve this issue without adding any new API.

Guennadi Liakhovetski

2010-10-10 18:21 UTC #37 lore raw

Quoted 31 lines · Marin Mitov

Quoted 19 lines · FUJITA Tomonori
Quoted 16 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 15 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Anyone is working on this?
I am not, sorry.
Quoted 1 line

Quoted 6 lines

Quoted 3 lines
KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.
I am waiting for the new videobuf2 framework to become part of the kernel. Then KAMEZAWA's improvements can help.

You probably have seen this related thread:
http://marc.info/?t=128644473600004&r=1&w=2

Thanks
Guennadi

Quoted 11 lines

Marin Mitov
Quoted 4 lines
The approach enables us to solve this issue without adding any new API.
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Marin Mitov

author

2010-10-10 18:53 UTC #38 lore raw

Quoted 37 lines · Guennadi Liakhovetski

Quoted 35 lines · Marin Mitov
Quoted 19 lines · FUJITA Tomonori
Quoted 16 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 15 lines · FUJITA Tomonori
Quoted 11 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Anyone is working on this?
I am not, sorry.
Quoted 4 lines
KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.

Quoted 5 lines

Quoted 2 lines
I am waiting for the new videobuf2 framework to become part of the kernel. Then KAMEZAWA's improvements can help.
You probably have seen this related thread:
http://marc.info/?t=128644473600004&r=1&w=2

Thanks.

Marin Mitov

Quoted 21 lines

Thanks
Guennadi
Quoted 11 lines
Marin Mitov
Quoted 4 lines
The approach enables us to solve this issue without adding any new API.
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

KAMEZAWA Hiroyuki

2010-10-13 08:10 UTC #39 lore raw

In reply to FUJITA Tomonori

Quoted 25 lines · On Sun, 10 Oct 2010 23:08:22 +0900 FUJITA Tomonori…

Quoted 13 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 12 lines · FUJITA Tomonori
Quoted 9 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()

Quoted 6 lines

Anyone is working on this?
KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.

I said I do make an effort ;)
New one here.

http://lkml.org/lkml/2010/10/12/421

Thanks,
-Kame

Marin Mitov

author

2010-10-13 16:43 UTC #40 lore raw

Quoted 30 lines · KAMEZAWA Hiroyuki

Quoted 27 lines · On Sun, 10 Oct 2010 23:08:22 +0900 FUJITA Tomonori…
Quoted 13 lines · On Fri, 20 Aug 2010 14:50:12 +0300 Marin Mitov <[email protected]> wrote:
Quoted 12 lines · FUJITA Tomonori
Quoted 9 lines · On Fri, 20 Aug 2010 11:13:45 +0300 Marin Mitov <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.
What you guys exactly want to do? If you just want to pre-allocate coherent memory for latter usage,
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Yes, just to preallocate not coherent, but rather contiguous memory for latter usage. We use coherent memory because it turns out to be contiguous.
Hmm, you don't care about coherency? You just need contiguous memory?
Yes. We just need contiguous memory. Coherency is important as far as when dma transfer finishes user land is able to see the new data. Could be done by something like dma_{,un}map_single()
Anyone is working on this?

Quoted 8 lines

Quoted 4 lines
KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.
I said I do make an effort ;)
New one here.
http://lkml.org/lkml/2010/10/12/421

I like the patch. The possibility to allocate a contiguous chunk of memory (or few of them) is what I need. The next step will be to get a dma handle (for dma transfers to/from) and then mmap them to user space.

Thanks.

Marin Mitov

Quoted 4 lines

Thanks,
-Kame

FUJITA Tomonori

2010-10-14 07:17 UTC #41 lore raw

Quoted 8 lines · On Wed, 13 Oct 2010 19:42:56 +0300 Marin Mitov <[email protected]> wrote:

Quoted 8 lines
Quoted 4 lines
KAMEZAWA posted a patch to improve the generic page allocator to allocate physically contiguous memory. He said that he can push it into mainline.
I said I do make an effort ;)
New one here.
http://lkml.org/lkml/2010/10/12/421

Kamezawa, Thanks a lot!!

Quoted 3 lines

I like the patch. The possibility to allocate a contiguous chunk of memory (or few of them) is what I need. The next step wilfl be to get a dma handle (for dma transfers to/from) and then mmap them to user space.

Let's help him to push this patch to upstream first. The next step is a different issue (and the dma stuff isn't even a problem; we can handle it with the current API).

Guennadi Liakhovetski

2010-08-20 20:05 UTC #42 lore raw

In reply to Marin Mitov

Quoted 7 lines · Marin Mitov

Hi all,
struct device contains a member: struct dma_coherent_mem *dma_mem; to hold information for a piece of memory declared dma-coherent. Alternatively the same member could also be used to hold preallocated dma-coherent memory for latter per-device use.

Quoted 5 lines

This tric is already used in drivers/staging/dt3155v4l.c dt3155_alloc_coherent()/dt3155_free_coherent()
Here proposed for general use by popular demand from video4linux folks. Helps for videobuf-dma-contig framework.

Ok, so, we've got two solutions to this problem submitted on the same day;) Following this thread:

http://marc.info/?t=128128236400002&r=1&w=2

on the ARM Linux kernel ML, I submitted a patch series

http://thread.gmane.org/gmane.linux.ports.sh.devel/8595

with a couple of fixes and improvements, the actual new API and a use example. My approach is slightly different, in that instead of requiring drivers to issue two calls - one to reserve RAM (usually dma_alloc_coherent()) and one to assign it to a device, my patch follows the suggestion from Russell King from the first thread and unites these two operations. So, now we have a choice;) Unfortunately, these two patch series went to orthogonal sets of recepients, I'm trying to fix this by adding a couple of CC entries.

Thanks
Guennadi

Quoted 91 lines

Signed-off-by Marin Mitov <[email protected]>

======================================================================

drivers/base/dma-coherent.c +77 -1

@@ -93,6 +93,83 @@ void *dma_mark_declared_memory_occupied(
 EXPORT_SYMBOL(dma_mark_declared_memory_occupied);
 
 /**
+ * dma_reserve_coherent_memory() - reserve coherent memory for per-device use
+ *
+ * @dev:	device from which we allocate memory
+ * @size:	size of requested memory area in bytes
+ * @flags:	same as in dma_declare_coherent_memory()
+ *
+ * This function reserves coherent memory allocating it early (during probe())
+ * to support latter allocations from per-device coherent memory pools.
+ * For a given device one could use either dma_declare_coherent_memory() or
+ * dma_reserve_coherent_memory(), but not both, becase the result of these
+ * functions is stored in a single struct device member - dma_mem
+ *
+ * Returns DMA_MEMORY_MAP on success, or 0 if failed.
+ * (same as dma_declare_coherent_memory()
+ */
+int dma_reserve_coherent_memory(struct device *dev, size_t size, int flags)
+{
+	struct dma_coherent_mem *mem;
+	dma_addr_t dev_base;
+	int pages = size >> PAGE_SHIFT;
+	int bitmap_size = BITS_TO_LONGS(pages) * sizeof(long);
+
+	if ((flags & DMA_MEMORY_MAP) == 0)
+		goto out;
+	if (!size)
+		goto out;
+	if (dev->dma_mem)
+		goto out;
+
+	mem = kzalloc(sizeof(*mem), GFP_KERNEL);
+	if (!mem)
+		goto out;
+	mem->virt_base = dma_alloc_coherent(dev, size, &dev_base,
+							DT3155_COH_FLAGS);
+	if (!mem->virt_base)
+		goto err_alloc_coherent;
+	mem->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
+	if (!mem->bitmap)
+		goto err_bitmap;
+
+	mem->device_base = dev_base;
+	mem->size = pages;
+	mem->flags = flags;
+	dev->dma_mem = mem;
+	return DMA_MEMORY_MAP;
+
+err_bitmap:
+	dma_free_coherent(dev, size, mem->virt_base, dev_base);
+err_alloc_coherent:
+	kfree(mem);
+out:
+	return 0;
+}
+EXPORT_SYMBOL(dma_reserve_coherent_memory);
+
+/**
+ * dma_free_reserved_memory() - free the reserved dma-coherent memoty
+ *
+ * @dev:	device for which we free the dma-coherent memory
+ *
+ * same as dma_release_declared_memory()
+ */
+void dma_free_reserved_memory(struct device *dev)
+{
+	struct dma_coherent_mem *mem = dev->dma_mem;
+
+	if (!mem)
+		return;
+	dev->dma_mem = NULL;
+	dma_free_coherent(dev, mem->size << PAGE_SHIFT,
+					mem->virt_base, mem->device_base);
+	kfree(mem->bitmap);
+	kfree(mem);
+}
+EXPORT_SYMBOL(dma_free_reserved_memory);
+
+/**
  * dma_alloc_from_coherent() - try to allocate memory from the per-device coherent area
  *
  * @dev:	device from which we allocate memory
--

Quoted 5 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/