Skip to contentSkip to conversation

revert dma-mapping and vmap API abuse in qcom_q6v5_mss

PATCH In review Christoph Hellwig 3 participants 10 messages Dec 23, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Hi all,

this series reverts a completely broken commit to qcom_q6v5_mss the abuses the dma-mapping and vmap APIs in multiple, and mostly clearly documented ways, and then adds a patch to document and reject the so far undocumented API abuse.

Diffstat:

 drivers/remoteproc/qcom_q6v5_mss.c |   38 +++++--------------------------------
 mm/vmalloc.c                       |    3 ++
 2 files changed, 9 insertions(+), 32 deletions(-)
#1 lore raw

PATCH 1/2 Revert "remoteproc: qcom_q6v5_mss: map/unmap metadata region before/after use"

This reverts commit fc156629b23a21181e473e60341e3a78af25a1d4.

This commit manages to do three API violations at once:

- dereference the return value of dma_alloc_attrs with the DMA_ATTR_NO_KERNEL_MAPPING mapping, which is clearly forbidden and will do the wrong thing on various dma mapping implementations. The fact that dma-direct uses a struct page as a cookie is an undocumented implementation detail
- include dma-map-ops.h and use pgprot_dmacoherent despite a clear comment documenting that this is not acceptable
- use of the VM_DMA_COHERENT for something that is not the dma-mapping code
- use of VM_FLUSH_RESET_PERMS for vmap, while it is only supported for vmalloc

---
 drivers/remoteproc/qcom_q6v5_mss.c | 38 +++++-------------------------
 1 file changed, 6 insertions(+), 32 deletions(-)
drivers/remoteproc/qcom_q6v5_mss.c +6 -32
index fddb63cffee078..a8b141db4de63f 100644
@@ -10,7 +10,6 @@
 #include <linux/clk.h>
 #include <linux/delay.h>
 #include <linux/devcoredump.h>
-#include <linux/dma-map-ops.h>
 #include <linux/dma-mapping.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
@@ -933,52 +932,27 @@ static void q6v5proc_halt_axi_port(struct q6v5 *qproc,
 static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw,
 				const char *fw_name)
 {
-	unsigned long dma_attrs = DMA_ATTR_FORCE_CONTIGUOUS | DMA_ATTR_NO_KERNEL_MAPPING;
-	unsigned long flags = VM_DMA_COHERENT | VM_FLUSH_RESET_PERMS;
-	struct page **pages;
-	struct page *page;
+	unsigned long dma_attrs = DMA_ATTR_FORCE_CONTIGUOUS;
 	dma_addr_t phys;
 	void *metadata;
 	int mdata_perm;
 	int xferop_ret;
 	size_t size;
-	void *vaddr;
-	int count;
+	void *ptr;
 	int ret;
-	int i;
 
 	metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev);
 	if (IS_ERR(metadata))
 		return PTR_ERR(metadata);
 
-	page = dma_alloc_attrs(qproc->dev, size, &phys, GFP_KERNEL, dma_attrs);
-	if (!page) {
+	ptr = dma_alloc_attrs(qproc->dev, size, &phys, GFP_KERNEL, dma_attrs);
+	if (!ptr) {
 		kfree(metadata);
 		dev_err(qproc->dev, "failed to allocate mdt buffer\n");
 		return -ENOMEM;
 	}
 
-	count = PAGE_ALIGN(size) >> PAGE_SHIFT;
-	pages = kmalloc_array(count, sizeof(struct page *), GFP_KERNEL);
-	if (!pages) {
-		ret = -ENOMEM;
-		goto free_dma_attrs;
-	}
-
-	for (i = 0; i < count; i++)
-		pages[i] = nth_page(page, i);
-
-	vaddr = vmap(pages, count, flags, pgprot_dmacoherent(PAGE_KERNEL));
-	kfree(pages);
-	if (!vaddr) {
-		dev_err(qproc->dev, "unable to map memory region: %pa+%zx\n", &phys, size);
-		ret = -EBUSY;
-		goto free_dma_attrs;
-	}
-
-	memcpy(vaddr, metadata, size);
-
-	vunmap(vaddr);
+	memcpy(ptr, metadata, size);
 
 	/* Hypervisor mapping to access metadata by modem */
 	mdata_perm = BIT(QCOM_SCM_VMID_HLOS);
@@ -1008,7 +982,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw,
 			 "mdt buffer not reclaimed system may become unstable\n");
 
 free_dma_attrs:
-	dma_free_attrs(qproc->dev, size, page, phys, dma_attrs);
+	dma_free_attrs(qproc->dev, size, ptr, phys, dma_attrs);
 	kfree(metadata);
 
 	return ret < 0 ? ret : 0;
Signature · 1 line
2.35.1
Acked-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] Revert "remoteproc: qcom_q6v5_mss: map/unmap metadata region before/after use"

Quoted 16 lines · Christoph Hellwig

This reverts commit fc156629b23a21181e473e60341e3a78af25a1d4.

This commit manages to do three API violations at once:

- dereference the return value of dma_alloc_attrs with the DMA_ATTR_NO_KERNEL_MAPPING mapping, which is clearly forbidden and will do the wrong thing on various dma mapping implementations. The fact that dma-direct uses a struct page as a cookie is an undocumented implementation detail
- include dma-map-ops.h and use pgprot_dmacoherent despite a clear comment documenting that this is not acceptable
- use of the VM_DMA_COHERENT for something that is not the dma-mapping code
- use of VM_FLUSH_RESET_PERMS for vmap, while it is only supported for vmalloc

Quoted 1 line

Thanks,
Mani

Quoted 86 lines
---
 drivers/remoteproc/qcom_q6v5_mss.c | 38 +++++-------------------------
 1 file changed, 6 insertions(+), 32 deletions(-)
drivers/remoteproc/qcom_q6v5_mss.c +6 -32
index fddb63cffee078..a8b141db4de63f 100644
@@ -10,7 +10,6 @@
 #include <linux/clk.h>
 #include <linux/delay.h>
 #include <linux/devcoredump.h>
-#include <linux/dma-map-ops.h>
 #include <linux/dma-mapping.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
@@ -933,52 +932,27 @@ static void q6v5proc_halt_axi_port(struct q6v5 *qproc,
 static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw,
 				const char *fw_name)
 {
-	unsigned long dma_attrs = DMA_ATTR_FORCE_CONTIGUOUS | DMA_ATTR_NO_KERNEL_MAPPING;
-	unsigned long flags = VM_DMA_COHERENT | VM_FLUSH_RESET_PERMS;
-	struct page **pages;
-	struct page *page;
+	unsigned long dma_attrs = DMA_ATTR_FORCE_CONTIGUOUS;
 	dma_addr_t phys;
 	void *metadata;
 	int mdata_perm;
 	int xferop_ret;
 	size_t size;
-	void *vaddr;
-	int count;
+	void *ptr;
 	int ret;
-	int i;
 
 	metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev);
 	if (IS_ERR(metadata))
 		return PTR_ERR(metadata);
 
-	page = dma_alloc_attrs(qproc->dev, size, &phys, GFP_KERNEL, dma_attrs);
-	if (!page) {
+	ptr = dma_alloc_attrs(qproc->dev, size, &phys, GFP_KERNEL, dma_attrs);
+	if (!ptr) {
 		kfree(metadata);
 		dev_err(qproc->dev, "failed to allocate mdt buffer\n");
 		return -ENOMEM;
 	}
 
-	count = PAGE_ALIGN(size) >> PAGE_SHIFT;
-	pages = kmalloc_array(count, sizeof(struct page *), GFP_KERNEL);
-	if (!pages) {
-		ret = -ENOMEM;
-		goto free_dma_attrs;
-	}
-
-	for (i = 0; i < count; i++)
-		pages[i] = nth_page(page, i);
-
-	vaddr = vmap(pages, count, flags, pgprot_dmacoherent(PAGE_KERNEL));
-	kfree(pages);
-	if (!vaddr) {
-		dev_err(qproc->dev, "unable to map memory region: %pa+%zx\n", &phys, size);
-		ret = -EBUSY;
-		goto free_dma_attrs;
-	}
-
-	memcpy(vaddr, metadata, size);
-
-	vunmap(vaddr);
+	memcpy(ptr, metadata, size);
 
 	/* Hypervisor mapping to access metadata by modem */
 	mdata_perm = BIT(QCOM_SCM_VMID_HLOS);
@@ -1008,7 +982,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw,
 			 "mdt buffer not reclaimed system may become unstable\n");
 
 free_dma_attrs:
-	dma_free_attrs(qproc->dev, size, page, phys, dma_attrs);
+	dma_free_attrs(qproc->dev, size, ptr, phys, dma_attrs);
 	kfree(metadata);
 
 	return ret < 0 ? ret : 0;
Signature · 1 line
2.35.1
Signature · 1 line
மணிவண்ணன் சதாசிவம்
#3 lore raw

Re: [PATCH 1/2] Revert "remoteproc: qcom_q6v5_mss: map/unmap metadata region before/after use"

Quoted 1 line · Manivannan Sadhasivam

Btw, if the hardware really does not like a kernel mapping, the right way is to just keep using the normal dma allocator, but make sure that there shared-dma-pool with the no-map property for the device.

#4 lore raw

Re: [PATCH 1/2] Revert "remoteproc: qcom_q6v5_mss: map/unmap metadata region before/after use"

Quoted 7 lines · Christoph Hellwig
Quoted 1 line · Manivannan Sadhasivam

Btw, if the hardware really does not like a kernel mapping, the right way is to just keep using the normal dma allocator, but make sure that there shared-dma-pool with the no-map property for the device.

Sibi posted a series that uses a separate no-map carveout for this usecase:
https://lore.kernel.org/lkml/[email protected]/

But that doesn't use dma allocator with shared-dma-pool.

Thanks,
Mani

Signature · 1 line
மணிவண்ணன் சதாசிவம்
#5 lore raw

Re: [PATCH 1/2] Revert "remoteproc: qcom_q6v5_mss: map/unmap metadata region before/after use"

Quoted 11 lines · Manivannan Sadhasivam
Quoted 7 lines · Christoph Hellwig
Quoted 1 line · Manivannan Sadhasivam

Btw, if the hardware really does not like a kernel mapping, the right way is to just keep using the normal dma allocator, but make sure that there shared-dma-pool with the no-map property for the device.

Sibi posted a series that uses a separate no-map carveout for this usecase:
https://lore.kernel.org/lkml/[email protected]/

Oh, I've missed that entire thread. I actually stumbled over this today while finding it during a vmap audit..

#6 lore raw

PATCH 2/2 vmalloc: reject vmap with VM_FLUSH_RESET_PERMS

In reply to Christoph Hellwig

VM_FLUSH_RESET_PERMS is just for use with vmalloc as it is tied to freeing the underlying pages.

---
 mm/vmalloc.c | 3 +++
 1 file changed, 3 insertions(+)
mm/vmalloc.c +3 -0
index 9e30f0b3920325..88a644cde9fb12 100644
@@ -2849,6 +2849,9 @@ void *vmap(struct page **pages, unsigned int count,
 
 	might_sleep();
 
+	if (WARN_ON_ONCE(flags & VM_FLUSH_RESET_PERMS))
+		return NULL;
+
 	/*
 	 * Your top guard is someone else's bottom guard. Not having a top
 	 * guard compromises someone else's mappings too.
Signature · 1 line
2.35.1
#7 lore raw

Re: [PATCH 2/2] vmalloc: reject vmap with VM_FLUSH_RESET_PERMS

Quoted 13 lines · Christoph Hellwig

VM_FLUSH_RESET_PERMS is just for use with vmalloc as it is tied to freeing the underlying pages.

---
 mm/vmalloc.c | 3 +++
 1 file changed, 3 insertions(+)
mm/vmalloc.c +0 -0
index 9e30f0b3920325..88a644cde9fb12 100644

mm/vmalloc.c void *vmap(struct page **pages, unsigned int count, · line 2854

Quoted 6 lines
mm/vmalloc.c +3 -0
@@ -2849,6 +2849,9 @@ void *vmap(struct page **pages, unsigned int count,
 
 	might_sleep();
 
+	if (WARN_ON_ONCE(flags & VM_FLUSH_RESET_PERMS))
+		return NULL;
+

Might it be worth adding a specific vmap mask that explicitly indicates what flags are permissible on vmap()? Then this could become e.g.:-

	if (WARN_ON_ONCE(flags & ~VM_VMAP_PERMITTED_MASK))
		return NULL;

And would be self-documenting as to why we are disallowing flags (i.e. they are not part of the permitted vmap mask).

#8 lore raw

Re: [PATCH 2/2] vmalloc: reject vmap with VM_FLUSH_RESET_PERMS

Quoted 8 lines · Lorenzo Stoakes

Might it be worth adding a specific vmap mask that explicitly indicates what flags are permissible on vmap()? Then this could become e.g.:-

	if (WARN_ON_ONCE(flags & ~VM_VMAP_PERMITTED_MASK))
		return NULL;

And would be self-documenting as to why we are disallowing flags (i.e. they are not part of the permitted vmap mask).

That's probably a good idea. It might need some time to audit for use of all the flags, though.

#9 lore raw

Re: [PATCH 2/2] vmalloc: reject vmap with VM_FLUSH_RESET_PERMS

Quoted 4 lines · Christoph Hellwig
Quoted 3 lines · Lorenzo Stoakes

Might it be worth adding a specific vmap mask that explicitly indicates what flags are permissible on vmap()? Then this could become e.g.:-

Quoted 8 lines
Quoted 5 lines
	if (WARN_ON_ONCE(flags & ~VM_VMAP_PERMITTED_MASK))
		return NULL;

And would be self-documenting as to why we are disallowing flags (i.e. they are not part of the permitted vmap mask).

That's probably a good idea. It might need some time to audit for use of all the flags, though.

Perhaps leave that for a later patch (I could take a look as well), but in the meantime might be worth adding a quick comment here indicating why the flag is prohibited?