Skip to contentSkip to conversation

Gen2 IPMMU and DU prototype

PATCH In review Magnus Damm 2 participants 4 messages Sep 30, 2015 – Oct 2, 2015

Subsystems (8)
lore
Share & tools
lore

Magnus Damm

author maintainer
#0 lore raw

PATCH 1/1 Gen2 IPMMU and DU prototype

From: Magnus Damm <[email protected]>

Here is a simple hack to enable the IPMMU on R-Car Gen2.

The VGA port on r8a7790 Lager may be driven by the DU and the IPMMU via the modetest utility to output some test image.

Actually the r8a7790 DU device is connected to the IPMMU using two separate uTLBs, but the DU driver is currently supporting all DU instances using a single device. To make testing easily the r8a7790 DU driver code is with this patch hacked to just cover a single DU instance with a single IPMMU uTLB.

The map/umap debug printk() will show you if the IPMMU is used or not. Another way could be to XOR the physical address in the map routine to adjust the test image.

Several sources have told me that the DU on Lager is not working as expected. With this hack (with or without IPMMU) at least the VGA output seems fine. So it would be good to look into HDMI and LVDS as well as unmodified VGA support for the DU on the Lager board.

Also, we need to figure out how to modify the DU driver to support multiple uTLBs.

Not for upstream merge.

Not-Yet-Signed-off-by Magnus Damm <[email protected]>

---

 arch/arm/boot/dts/r8a7790.dtsi        |    3 ++-
 arch/arm/boot/dts/r8a7791.dtsi        |    3 ++-
 arch/arm/mm/dma-mapping.c             |    4 ++--
 drivers/gpu/drm/rcar-du/rcar_du_drv.c |    6 +++---
 drivers/iommu/ipmmu-vmsa.c            |    8 ++++++--
 drivers/of/device.c                   |    6 +++---
 6 files changed, 18 insertions(+), 12 deletions(-)
arch/arm/boot/dts/r8a7790.dtsi +2 -1
@@ -882,6 +882,7 @@
 			 <&mstp7_clks R8A7790_CLK_LVDS0>,
 			 <&mstp7_clks R8A7790_CLK_LVDS1>;
 		clock-names = "du.0", "du.1", "du.2", "lvds.0", "lvds.1";
+		iommus = <&ipmmu_mx 15>;
 		status = "disabled";
 
 		ports {
@@ -1805,7 +1806,7 @@
 		interrupts = <0 222 IRQ_TYPE_LEVEL_HIGH>,
 			     <0 221 IRQ_TYPE_LEVEL_HIGH>;
 		#iommu-cells = <1>;
-		status = "disabled";
+		status = "okay";
 	};
 
 	ipmmu_rt: mmu@ffc80000 {
arch/arm/boot/dts/r8a7791.dtsi +2 -1
@@ -919,6 +919,7 @@
 			 <&mstp7_clks R8A7791_CLK_DU1>,
 			 <&mstp7_clks R8A7791_CLK_LVDS0>;
 		clock-names = "du.0", "du.1", "lvds.0";
+		iommus = <&ipmmu_mx 15>;
 		status = "disabled";
 
 		ports {
@@ -1618,7 +1619,7 @@
 		interrupts = <0 222 IRQ_TYPE_LEVEL_HIGH>,
 			     <0 221 IRQ_TYPE_LEVEL_HIGH>;
 		#iommu-cells = <1>;
-		status = "disabled";
+		status = "okay";
 	};
 
 	ipmmu_rt: mmu@ffc80000 {
arch/arm/mm/dma-mapping.c +2 -2
@@ -1998,7 +1998,7 @@ static int __arm_iommu_attach_device(str
 	kref_get(&mapping->kref);
 	to_dma_iommu_mapping(dev) = mapping;
 
-	pr_debug("Attached IOMMU controller to %s device.\n", dev_name(dev));
+	pr_info("Attached IOMMU controller to %s device.\n", dev_name(dev));
 	return 0;
 }
 
@@ -2043,7 +2043,7 @@ static void __arm_iommu_detach_device(st
 	kref_put(&mapping->kref, release_iommu_mapping);
 	to_dma_iommu_mapping(dev) = NULL;
 
-	pr_debug("Detached IOMMU controller from %s device.\n", dev_name(dev));
+	pr_info("Detached IOMMU controller from %s device.\n", dev_name(dev));
 }
 
 /**
drivers/gpu/drm/rcar-du/rcar_du_drv.c +3 -3
@@ -60,13 +60,13 @@ static const struct rcar_du_device_info
 	.features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK
 		  | RCAR_DU_FEATURE_EXT_CTRL_REGS,
 	.quirks = RCAR_DU_QUIRK_ALIGN_128B | RCAR_DU_QUIRK_LVDS_LANES,
-	.num_crtcs = 3,
+	.num_crtcs = 2,
 	.routes = {
 		/* R8A7790 has one RGB output, two LVDS outputs and one
 		 * (currently unsupported) TCON output.
 		 */
 		[RCAR_DU_OUTPUT_DPAD0] = {
-			.possible_crtcs = BIT(2) | BIT(1) | BIT(0),
+			.possible_crtcs = BIT(1) | BIT(0),
 			.encoder_type = DRM_MODE_ENCODER_NONE,
 			.port = 0,
 		},
@@ -76,7 +76,7 @@ static const struct rcar_du_device_info
 			.port = 1,
 		},
 		[RCAR_DU_OUTPUT_LVDS1] = {
-			.possible_crtcs = BIT(2) | BIT(1),
+			.possible_crtcs = BIT(1),
 			.encoder_type = DRM_MODE_ENCODER_LVDS,
 			.port = 2,
 		},
drivers/iommu/ipmmu-vmsa.c +6 -2
@@ -7,7 +7,7 @@
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation; version 2 of the License.
  */
-
+#define DEBUG
 #include <linux/delay.h>
 #include <linux/dma-mapping.h>
 #include <linux/err.h>
@@ -544,9 +544,11 @@ static int ipmmu_map(struct iommu_domain
 {
 	struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
 
+	printk("xxx map 0x%08lx <-> %pad %zu\n", iova, &paddr, size);
+
 	if (!domain)
 		return -ENODEV;
-
+	
 	return domain->iop->map(domain->iop, iova, paddr, size, prot);
 }
 
@@ -555,6 +557,8 @@ static size_t ipmmu_unmap(struct iommu_d
 {
 	struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
 
+	printk("xxx unmap 0x%08lx %zu\n", iova, size);
+
 	return domain->iop->unmap(domain->iop, iova, size);
 }
 
drivers/of/device.c +3 -3
@@ -124,7 +124,7 @@ void of_dma_configure(struct device *dev
 			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
 			return;
 		}
-		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
+		dev_info(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
 	dev->dma_pfn_offset = offset;
@@ -139,11 +139,11 @@ void of_dma_configure(struct device *dev
 			     DMA_BIT_MASK(ilog2(dma_addr + size)));
 
 	coherent = of_dma_is_coherent(np);
-	dev_dbg(dev, "device is%sdma coherent\n",
+	dev_info(dev, "device is%sdma coherent\n",
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np);
-	dev_dbg(dev, "device is%sbehind an iommu\n",
+	dev_info(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
#1 lore raw

Hi Magnus,

Thank you for the patch.

Quoted 8 lines · On Wednesday 30 September 2015 23:22:42 Magnus Damm wrote:

From: Magnus Damm <[email protected]>

Here is a simple hack to enable the IPMMU on R-Car Gen2.

The VGA port on r8a7790 Lager may be driven by the DU and the IPMMU via the modetest utility to output some test image.

Quoted 6 lines

Actually the r8a7790 DU device is connected to the IPMMU using two separate uTLBs, but the DU driver is currently supporting all DU instances using a single device. To make testing easily the r8a7790 DU driver code is with this patch hacked to just cover a single DU instance with a single IPMMU uTLB.

You can reference multiple uTLBs in the DT ipmmus property. The ipmmu-vmsa driver currently requires all uTLBs to belong to the same IPMMU instance, which shouldn't be an issue here.

drivers/gpu/drm/rcar-du/rcar_du_drv.c

Quoted 86 lines

The map/umap debug printk() will show you if the IPMMU is used or not. Another way could be to XOR the physical address in the map routine to adjust the test image.

Several sources have told me that the DU on Lager is not working as expected. With this hack (with or without IPMMU) at least the VGA output seems fine. So it would be good to look into HDMI and LVDS as well as unmodified VGA support for the DU on the Lager board.

Also, we need to figure out how to modify the DU driver to support multiple uTLBs.

Not for upstream merge.

Not-Yet-Signed-off-by Magnus Damm <[email protected]>

---

 arch/arm/boot/dts/r8a7790.dtsi        |    3 ++-
 arch/arm/boot/dts/r8a7791.dtsi        |    3 ++-
 arch/arm/mm/dma-mapping.c             |    4 ++--
 drivers/gpu/drm/rcar-du/rcar_du_drv.c |    6 +++---
 drivers/iommu/ipmmu-vmsa.c            |    8 ++++++--
 drivers/of/device.c                   |    6 +++---
 6 files changed, 18 insertions(+), 12 deletions(-)
arch/arm/boot/dts/r8a7790.dtsi +2 -1
@@ -882,6 +882,7 @@
 			 <&mstp7_clks R8A7790_CLK_LVDS0>,
 			 <&mstp7_clks R8A7790_CLK_LVDS1>;
 		clock-names = "du.0", "du.1", "du.2", "lvds.0", "lvds.1";
+		iommus = <&ipmmu_mx 15>;
 		status = "disabled";
 
 		ports {
@@ -1805,7 +1806,7 @@
 		interrupts = <0 222 IRQ_TYPE_LEVEL_HIGH>,
 			     <0 221 IRQ_TYPE_LEVEL_HIGH>;
 		#iommu-cells = <1>;
-		status = "disabled";
+		status = "okay";
 	};
 
 	ipmmu_rt: mmu@ffc80000 {
arch/arm/boot/dts/r8a7791.dtsi +2 -1
@@ -919,6 +919,7 @@
 			 <&mstp7_clks R8A7791_CLK_DU1>,
 			 <&mstp7_clks R8A7791_CLK_LVDS0>;
 		clock-names = "du.0", "du.1", "lvds.0";
+		iommus = <&ipmmu_mx 15>;
 		status = "disabled";
 
 		ports {
@@ -1618,7 +1619,7 @@
 		interrupts = <0 222 IRQ_TYPE_LEVEL_HIGH>,
 			     <0 221 IRQ_TYPE_LEVEL_HIGH>;
 		#iommu-cells = <1>;
-		status = "disabled";
+		status = "okay";
 	};
 
 	ipmmu_rt: mmu@ffc80000 {
arch/arm/mm/dma-mapping.c +2 -2
@@ -1998,7 +1998,7 @@ static int __arm_iommu_attach_device(str
 	kref_get(&mapping->kref);
 	to_dma_iommu_mapping(dev) = mapping;
 
-	pr_debug("Attached IOMMU controller to %s device.\n", dev_name(dev));
+	pr_info("Attached IOMMU controller to %s device.\n", dev_name(dev));
 	return 0;
 }
 
@@ -2043,7 +2043,7 @@ static void __arm_iommu_detach_device(st
 	kref_put(&mapping->kref, release_iommu_mapping);
 	to_dma_iommu_mapping(dev) = NULL;
 
-	pr_debug("Detached IOMMU controller from %s device.\n", dev_name(dev));
+	pr_info("Detached IOMMU controller from %s device.\n", dev_name(dev));
 }
 
 /**
drivers/gpu/drm/rcar-du/rcar_du_drv.c +0 -0

15:05:31.520513000

Quoted 85 lines

+0900 @@ -60,13 +60,13 @@ static const struct rcar_du_device_info

drivers/gpu/drm/rcar-du/rcar_du_drv.c +3 -3
 	.features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK
 
 		  | RCAR_DU_FEATURE_EXT_CTRL_REGS,
 
 	.quirks = RCAR_DU_QUIRK_ALIGN_128B | RCAR_DU_QUIRK_LVDS_LANES,
-	.num_crtcs = 3,
+	.num_crtcs = 2,
 	.routes = {
 		/* R8A7790 has one RGB output, two LVDS outputs and one
 		 * (currently unsupported) TCON output.
 		 */
 		[RCAR_DU_OUTPUT_DPAD0] = {
-			.possible_crtcs = BIT(2) | BIT(1) | BIT(0),
+			.possible_crtcs = BIT(1) | BIT(0),
 			.encoder_type = DRM_MODE_ENCODER_NONE,
 			.port = 0,
 		},
@@ -76,7 +76,7 @@ static const struct rcar_du_device_info
 			.port = 1,
 		},
 		[RCAR_DU_OUTPUT_LVDS1] = {
-			.possible_crtcs = BIT(2) | BIT(1),
+			.possible_crtcs = BIT(1),
 			.encoder_type = DRM_MODE_ENCODER_LVDS,
 			.port = 2,
 		},
drivers/iommu/ipmmu-vmsa.c +6 -2
@@ -7,7 +7,7 @@
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation; version 2 of the License.
  */
-
+#define DEBUG
 #include <linux/delay.h>
 #include <linux/dma-mapping.h>
 #include <linux/err.h>
@@ -544,9 +544,11 @@ static int ipmmu_map(struct iommu_domain
 {
 	struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
 
+	printk("xxx map 0x%08lx <-> %pad %zu\n", iova, &paddr, size);
+
 	if (!domain)
 		return -ENODEV;
-
+
 	return domain->iop->map(domain->iop, iova, paddr, size, prot);
 }
 
@@ -555,6 +557,8 @@ static size_t ipmmu_unmap(struct iommu_d
 {
 	struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
 
+	printk("xxx unmap 0x%08lx %zu\n", iova, size);
+
 	return domain->iop->unmap(domain->iop, iova, size);
 }
 
drivers/of/device.c +3 -3
@@ -124,7 +124,7 @@ void of_dma_configure(struct device *dev
 			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
 			return;
 		}
-		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
+		dev_info(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
 	dev->dma_pfn_offset = offset;
@@ -139,11 +139,11 @@ void of_dma_configure(struct device *dev
 			     DMA_BIT_MASK(ilog2(dma_addr + size)));
 
 	coherent = of_dma_is_coherent(np);
-	dev_dbg(dev, "device is%sdma coherent\n",
+	dev_info(dev, "device is%sdma coherent\n",
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np);
-	dev_dbg(dev, "device is%sbehind an iommu\n",
+	dev_info(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#2 lore raw
In reply to Magnus Damm

Hi Laurent,

Quoted 3 lines · Laurent Pinchart

Hi Magnus,

Thank you for the patch.

Thanks for feedback!!

Quoted 16 lines · On Wednesday 30 September 2015 23:22:42 Magnus Damm wrote:
Quoted 8 lines

From: Magnus Damm <[email protected]>

Here is a simple hack to enable the IPMMU on R-Car Gen2.

The VGA port on r8a7790 Lager may be driven by the DU and the IPMMU via the modetest utility to output some test image.

Quoted 6 lines

Actually the r8a7790 DU device is connected to the IPMMU using two separate uTLBs, but the DU driver is currently supporting all DU instances using a single device. To make testing easily the r8a7790 DU driver code is with this patch hacked to just cover a single DU instance with a single IPMMU uTLB.

Quoted 3 lines

You can reference multiple uTLBs in the DT ipmmus property. The ipmmu-vmsa driver currently requires all uTLBs to belong to the same IPMMU instance, which shouldn't be an issue here.

Sure, from a DT property point of view it seems possible to assign multiple uTLBs to the DU node. I would like us to start integrating the IPMMU on Gen2 and passing multiple uTLBs in case of r8a7790 must be the right way to describe the hardware.

When it comes to the DU driver and memory management, from my point of view Gen2 also seems to pass a single device pointer passed to dma_alloc_writecombine(). I may be wrong, but to support multiple uTLBs it looks like we need to pass different device pointers to dma_alloc_writecombine() - unless some index is also passed how can the IPMMU code otherwise know which uTLB to use?

So it looks like the device pointer issue both affects Gen2 and Gen3. Of course I wish that wasn't the case, so please prove me wrong. =)

Cheers,

/ magnus

#3 lore raw
In reply to Magnus Damm

Hi Magnus,

Quoted 33 lines · On Friday 02 October 2015 12:12:22 Magnus Damm wrote:
Quoted 16 lines · Laurent Pinchart
Quoted 8 lines · On Wednesday 30 September 2015 23:22:42 Magnus Damm wrote:

From: Magnus Damm <[email protected]>

Here is a simple hack to enable the IPMMU on R-Car Gen2.

The VGA port on r8a7790 Lager may be driven by the DU and the IPMMU via the modetest utility to output some test image.

Quoted 6 lines

Actually the r8a7790 DU device is connected to the IPMMU using two separate uTLBs, but the DU driver is currently supporting all DU instances using a single device. To make testing easily the r8a7790 DU driver code is with this patch hacked to just cover a single DU instance with a single IPMMU uTLB.

Quoted 3 lines

You can reference multiple uTLBs in the DT ipmmus property. The ipmmu-vmsa driver currently requires all uTLBs to belong to the same IPMMU instance, which shouldn't be an issue here.

Sure, from a DT property point of view it seems possible to assign multiple uTLBs to the DU node. I would like us to start integrating the IPMMU on Gen2 and passing multiple uTLBs in case of r8a7790 must be the right way to describe the hardware.

When it comes to the DU driver and memory management, from my point of view Gen2 also seems to pass a single device pointer passed to dma_alloc_writecombine(). I may be wrong, but to support multiple uTLBs it looks like we need to pass different device pointers to dma_alloc_writecombine() - unless some index is also passed how can the IPMMU code otherwise know which uTLB to use?

Quoted 2 lines

So it looks like the device pointer issue both affects Gen2 and Gen3. Of course I wish that wasn't the case, so please prove me wrong. =)

The IPMMU driver currently enables all uTLBs associated with a bus master device. As the driver uses a single page table for all uTLBs we shouldn't have any issue on Gen2. Problems will of course appear if/when we start using multiple page tables for a single IPMMU instance.

Signature · 3 lines
Regards,

Laurent Pinchart