iommu: sun50i: Add Allwinner H616 support
Subsystems (3)
v2 cover iommu: sun50i: Add Allwinner H616 support
Version two of this series adds a check that no physical address larger than 4GB makes it into the PTEs: the map_pages() function returns an error and prints a warning into dmesg to give users a clue why this failed. I haven't tested whether this really happens, or whether the 32-bit DMA mask of the master devices already prevents this. In the worst case this might fail on devices with 4GB of DRAM, but would always work on smaller devices, which are arguably under bigger pressure to find contiguous PA ranges. Changelog below.
===========================================
The Allwinner H616 contains an IOMMU almost compatible to the one used in the H6. The differing default reset value of the bypass register makes the two technically incompatible, so use a new DT compatible string to be on the safe side.
The required driver changes can be applied to both variants, so the driver is ignorant of the differences between the two for now.
Change the driver to cope with the new variant in patch 1/5 and 2/5, then apply the required devicetree and binding changes in the remaining patches.
I could just verify that the driver probes and allocates the page table from below 4 GB. Others have verified that the driver works with the (not yet upstream) video decoder engine, but more tests are surely welcome.
Cheers,
Andre.
Changelog v1 .. v2:
- return error for too large PA in map_pages()
- add Krzysztof's ACK to the binding patch
Andre Przywara (4):
iommu: sun50i: allocate page tables from below 4 GiB
dt-bindings: iommu: add new compatible strings
iommu: sun50i: Add H616 compatible string
arm64: dts: allwinner: h616: add IOMMU node
Jernej Skrabec (1):
iommu: sun50i: clear bypass register
.../bindings/iommu/allwinner,sun50i-h6-iommu.yaml | 7 ++++++- arch/arm64/boot/dts/allwinner/sun50i-h616.dtsi | 9 +++++++++ drivers/iommu/sun50i-iommu.c | 15 +++++++++++++-- 3 files changed, 28 insertions(+), 3 deletions(-)
Signature · 1 line
2.39.4
PATCH v2 1/5 iommu: sun50i: clear bypass register
From: Jernej Skrabec <[email protected]>
The Allwinner H6 IOMMU has a bypass register, which allows to circumvent the page tables for each possible master. The reset value for this register is 0, which disables the bypass.
The Allwinner H616 IOMMU resets this register to 0x7f, which activates the bypass for all masters, which is not what we want.
Always clear this register to 0, to enforce the usage of page tables, and make this driver compatible with the H616 in this respect.
--- drivers/iommu/sun50i-iommu.c | 1 + 1 file changed, 1 insertion(+)
drivers/iommu/sun50i-iommu.c +1 -0
@@ -452,6 +452,7 @@ static int sun50i_iommu_enable(struct sun50i_iommu *iommu)IOMMU_TLB_PREFETCH_MASTER_ENABLE (3 ) |IOMMU_TLB_PREFETCH_MASTER_ENABLE (4 ) | IOMMU_TLB_PREFETCH_MASTER_ENABLE(5)); +iommu_write (iommu,IOMMU_BYPASS_REG ,0 );iommu_write (iommu,IOMMU_INT_ENABLE_REG ,IOMMU_INT_MASK ); iommu_write(iommu, IOMMU_DM_AUT_CTRL_REG(SUN50I_IOMMU_ACI_NONE), IOMMU_DM_AUT_CTRL_RD_UNAVAIL(SUN50I_IOMMU_ACI_NONE, 0) |
Signature · 1 line
2.39.4
Re: [PATCH v2 1/5] iommu: sun50i: clear bypass register
Quoted 9 lines · Andre Przywara
From: Jernej Skrabec <[email protected]>
The Allwinner H6 IOMMU has a bypass register, which allows to circumvent the page tables for each possible master. The reset value for this register is 0, which disables the bypass.
The Allwinner H616 IOMMU resets this register to0x7f, which activates the bypass for all masters, which is not what we want.
Quoted 5 lines
Always clear this register to 0, to enforce the usage of page tables, and make this driver compatible with the H616 in this respect.
Signed-off-by Jernej Skrabec <[email protected]>Signed-off-by Andre Przywara <[email protected]>
Quoted 19 lines
--- drivers/iommu/sun50i-iommu.c | 1 + 1 file changed, 1 insertion(+)drivers/iommu/sun50i-iommu.c +1
-1@@ -452,6 +452,7 @@ static int sun50i_iommu_enable(struct sun50i_iommu *iommu)IOMMU_TLB_PREFETCH_MASTER_ENABLE (3 ) |IOMMU_TLB_PREFETCH_MASTER_ENABLE (4 ) | IOMMU_TLB_PREFETCH_MASTER_ENABLE(5)); +iommu_write (iommu,IOMMU_BYPASS_REG ,0 );iommu_write (iommu,IOMMU_INT_ENABLE_REG ,IOMMU_INT_MASK ); iommu_write(iommu, IOMMU_DM_AUT_CTRL_REG(SUN50I_IOMMU_ACI_NONE), IOMMU_DM_AUT_CTRL_RD_UNAVAIL(SUN50I_IOMMU_ACI_NONE, 0) | --2.39.4
PATCH v2 2/5 iommu: sun50i: allocate page tables from below 4 GiB
In reply to Andre PrzywaraThe Allwinner IOMMU is a strict 32-bit device, with its input addresses, the page table root pointer as well as both level's page tables and also the target addresses all required to be below 4GB.
The Allwinner H6 SoC only supports 32-bit worth of physical addresses anyway, so this isn't a problem so far, but the H616 and later SoCs extend the PA space beyond 32 bit to accommodate more DRAM.
To make sure we stay within the 32-bit PA range required by the IOMMU, force the memory for the page tables to come from below 4GB. by using allocations with the DMA32 flag.
Also reject any attempt to map target addresses beyond 4GB, and print a warning to give users a hint while this fails.
--- drivers/iommu/sun50i-iommu.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/iommu/sun50i-iommu.c +11 -2
@@ -602,6 +602,14 @@ static int sun50i_iommu_map(struct iommu_domain *domain, unsigned long iova,u32 *page_table, *pte_addr;int ret =0 ; +/* the IOMMU can only handle 32-bit addresses, both input and output */ +if ((uint64_t )paddr >>32 ) { + ret = -EINVAL ; +dev_warn_once (iommu->dev, +"attempt to map address beyond 4GB\n" ); + goto out; + } + page_table =sun50i_dte_get_page_table (sun50i_domain, iova, gfp);if (IS_ERR (page_table)) { ret =PTR_ERR (page_table);
@@ -682,7 +690,8 @@ sun50i_iommu_domain_alloc_paging(struct device *dev)if (!sun50i_domain)return NULL ; - sun50i_domain->dt =iommu_alloc_pages (GFP_KERNEL ,get_order (DT_SIZE )); + sun50i_domain->dt =iommu_alloc_pages (GFP_KERNEL |GFP_DMA32 , +get_order (DT_SIZE ));if (!sun50i_domain->dt) goto err_free_domain;
@@ -997,7 +1006,7 @@ static int sun50i_iommu_probe(struct platform_device *pdev) iommu->pt_pool =kmem_cache_create (dev_name (&pdev->dev),PT_SIZE ,PT_SIZE , -SLAB_HWCACHE_ALIGN , +SLAB_HWCACHE_ALIGN |SLAB_CACHE_DMA32 ,NULL );if (!iommu->pt_pool)return -ENOMEM ;
Signature · 1 line
2.39.4
Re: [PATCH v2 2/5] iommu: sun50i: allocate page tables from below 4 GiB
Quoted 13 lines · Andre Przywara
The Allwinner IOMMU is a strict 32-bit device, with its input addresses, the page table root pointer as well as both level's page tables and also the target addresses all required to be below 4GB.
The Allwinner H6 SoC only supports 32-bit worth of physical addresses anyway, so this isn't a problem so far, but the H616 and later SoCs extend the PA space beyond 32 bit to accommodate more DRAM.
To make sure we stay within the 32-bit PA range required by the IOMMU, force the memory for the page tables to come from below 4GB. by using allocations with the DMA32 flag.
Also reject any attempt to map target addresses beyond 4GB, and print a warning to give users a hint while this fails.
Quoted 1 line
Signed-off-by Andre Przywara <[email protected]>
Quoted 45 lines
--- drivers/iommu/sun50i-iommu.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)drivers/iommu/sun50i-iommu.c +11
-3@@ -602,6 +602,14 @@ static int sun50i_iommu_map(struct iommu_domain *domain, unsigned long iova,u32 *page_table, *pte_addr;int ret =0 ; +/* the IOMMU can only handle 32-bit addresses, both input and output */ +if ((uint64_t )paddr >>32 ) { + ret = -EINVAL ; +dev_warn_once (iommu->dev, +"attempt to map address beyond 4GB\n" ); + goto out; + } + page_table =sun50i_dte_get_page_table (sun50i_domain, iova, gfp);if (IS_ERR (page_table)) { ret =PTR_ERR (page_table);@@ -682,7 +690,8 @@ sun50i_iommu_domain_alloc_paging(struct device *dev)if (!sun50i_domain)return NULL ; - sun50i_domain->dt =iommu_alloc_pages (GFP_KERNEL ,get_order (DT_SIZE )); + sun50i_domain->dt =iommu_alloc_pages (GFP_KERNEL |GFP_DMA32 , +get_order (DT_SIZE ));if (!sun50i_domain->dt) goto err_free_domain;@@ -997,7 +1006,7 @@ static int sun50i_iommu_probe(struct platform_device *pdev) iommu->pt_pool =kmem_cache_create (dev_name (&pdev->dev),PT_SIZE ,PT_SIZE , -SLAB_HWCACHE_ALIGN , +SLAB_HWCACHE_ALIGN |SLAB_CACHE_DMA32 ,NULL );if (!iommu->pt_pool)return -ENOMEM ; --2.39.4
Re: [PATCH v2 2/5] iommu: sun50i: allocate page tables from below 4 GiB
In reply to Andre PrzywaraQuoted 22 lines · Andre Przywara
The Allwinner IOMMU is a strict 32-bit device, with its input addresses, the page table root pointer as well as both level's page tables and also the target addresses all required to be below 4GB.
The Allwinner H6 SoC only supports 32-bit worth of physical addresses anyway, so this isn't a problem so far, but the H616 and later SoCs extend the PA space beyond 32 bit to accommodate more DRAM.
To make sure we stay within the 32-bit PA range required by the IOMMU, force the memory for the page tables to come from below 4GB. by using allocations with the DMA32 flag.
Also reject any attempt to map target addresses beyond 4GB, and print a warning to give users a hint while this fails.Signed-off-by Andre Przywara <[email protected]>--- drivers/iommu/sun50i-iommu.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)drivers/iommu/sun50i-iommu.c +0
-0
drivers/iommu/sun50i-iommu.c static int sun50i_iommu_map(struct iommu_domain *domain, unsigned long iova, · line 609
Quoted 8 lines
drivers/iommu/sun50i-iommu.c +5
-0@@ -602,6 +602,14 @@ static int sun50i_iommu_map(struct iommu_domain *domain, unsigned long iova,u32 *page_table, *pte_addr;int ret =0 ; +/* the IOMMU can only handle 32-bit addresses, both input and output */ +if ((uint64_t )paddr >>32 ) { + ret = -EINVAL ; +dev_warn_once (iommu->dev, +"attempt to map address beyond 4GB\n" );
A133 uses the same IOMMU block and works fine when claimed by the display engine under 1GB RAM. When used with 4GB,
root@helper-board-a133:~# head /proc/meminfo MemTotal: 4021212 kB
root@helper-board-a133:~# dmesg | grep -i -e drm -e iommu [ 0.465613] iommu: Default domain type: Translated [ 0.470892] iommu: DMA domain TLB invalidation policy: strict mode [ 0.914097] sun8i-mixer 6100000.mixer: Adding to iommu group 0 [ 1.092400] sun4i-drm display-engine: Adding to iommu group 0 [ 1.100044] sun4i-drm display-engine: bound 6100000.mixer (ops 0xffff800080b05dc0) [ 1.108399] sun4i-drm display-engine: bound 6510000.tcon-top (ops 0xffff800080b0a628) [ 1.116694] sun4i-drm display-engine: bound 6511000.lcd-controller (ops 0xffff800080b04220) [ 1.126415] [drm] Initialized sun4i-drm 1.0.0 for display-engine on minor 0 [ 1.135110] sun50i-iommu 30f0000.iommu: attempt to map address beyond 4GB [ 1.142559] sun4i-drm display-engine: [drm] *ERROR* fbdev-dma: Failed to setup generic emulation (ret=-12) [ 1.154210] sun4i-drm display-engine: [drm] *ERROR* fbdev-dma: Failed to setup generic emulation (ret=-12)
allocation fails. Limiting the memory mem=2G to the kernel works.
Thanks,
Parthiban
Quoted 25 lines
drivers/iommu/sun50i-iommu.c +6
-2+ goto out; + } + page_table =sun50i_dte_get_page_table (sun50i_domain, iova, gfp);if (IS_ERR (page_table)) { ret =PTR_ERR (page_table);@@ -682,7 +690,8 @@ sun50i_iommu_domain_alloc_paging(struct device *dev)if (!sun50i_domain)return NULL ; - sun50i_domain->dt =iommu_alloc_pages (GFP_KERNEL ,get_order (DT_SIZE )); + sun50i_domain->dt =iommu_alloc_pages (GFP_KERNEL |GFP_DMA32 , +get_order (DT_SIZE ));if (!sun50i_domain->dt) goto err_free_domain;@@ -997,7 +1006,7 @@ static int sun50i_iommu_probe(struct platform_device *pdev) iommu->pt_pool =kmem_cache_create (dev_name (&pdev->dev),PT_SIZE ,PT_SIZE , -SLAB_HWCACHE_ALIGN , +SLAB_HWCACHE_ALIGN |SLAB_CACHE_DMA32 ,NULL );if (!iommu->pt_pool)return -ENOMEM ;
PATCH v2 3/5 dt-bindings: iommu: add new compatible strings
In reply to Andre PrzywaraThe Allwinner H616 and A523 contain IOMMU IP very similar to the H6, but use a different reset value for the bypass register, which makes them strictly speaking incompatible.
Add a new compatible string for the H616, and a version for the A523, falling back to the H616.
--- .../bindings/iommu/allwinner,sun50i-h6-iommu.yaml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
Documentation/devicetree/bindings/iommu/allwinner,sun50i-h6-iommu.yaml +6 -1
@@ -17,7 +17,12 @@ properties: The content of the cell is the master ID. compatible: - const: allwinner,sun50i-h6-iommu + oneOf: + - const: allwinner,sun50i-h6-iommu + - const: allwinner,sun50i-h616-iommu + - items: + - const: allwinner,sun55i-a523-iommu + - const: allwinner,sun50i-h616-iommu reg: maxItems: 1
Signature · 1 line
2.39.4
Re: [PATCH v2 3/5] dt-bindings: iommu: add new compatible strings
Quoted 10 lines · Andre Przywara
The Allwinner H616 and A523 contain IOMMU IP very similar to the H6, but use a different reset value for the bypass register, which makes them strictly speaking incompatible.
Add a new compatible string for the H616, and a version for the A523, falling back to the H616.
Signed-off-by Andre Przywara <[email protected]>
Quoted 26 lines
--- .../bindings/iommu/allwinner,sun50i-h6-iommu.yaml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)Documentation/devicetree/bindings/iommu/allwinner,sun50i-h6-iommu.yaml +6
-2@@ -17,7 +17,12 @@ properties: The content of the cell is the master ID. compatible: - const: allwinner,sun50i-h6-iommu + oneOf: + - const: allwinner,sun50i-h6-iommu + - const: allwinner,sun50i-h616-iommu + - items: + - const: allwinner,sun55i-a523-iommu + - const: allwinner,sun50i-h616-iommu reg: maxItems: 1 --2.39.4
PATCH v2 4/5 iommu: sun50i: Add H616 compatible string
In reply to Andre PrzywaraThe IOMMU IP in the Allwinner H616 SoC is *almost* compatible to the H6, but uses a different reset value for the bypass register, and adds some more registers.
While a driver *can* be written to support both variants (which we in fact do), the hardware itself is not fully compatible, so we require a separate compatible string.
Add the new compatible string to the list, but without changing the behaviour, since the driver already supports both variants.
--- drivers/iommu/sun50i-iommu.c | 1 + 1 file changed, 1 insertion(+)
drivers/iommu/sun50i-iommu.c +1 -0
@@ -1067,6 +1067,7 @@ static int sun50i_iommu_probe(struct platform_device *pdev)static const struct of_device_id sun50i_iommu_dt[] = { { .compatible ="allwinner,sun50i-h6-iommu" , }, + { .compatible ="allwinner,sun50i-h616-iommu" , }, {/* sentinel */ }, };MODULE_DEVICE_TABLE (of, sun50i_iommu_dt);
Signature · 1 line
2.39.4
Re: [PATCH v2 4/5] iommu: sun50i: Add H616 compatible string
Quoted 5 lines · Andre Przywara
The IOMMU IP in the Allwinner H616 SoC is *almost* compatible to the H6, but uses a different reset value for the bypass register, and adds some more registers.
Quoted 8 lines
While a driver *can* be written to support both variants (which we in fact do), the hardware itself is not fully compatible, so we require a separate compatible string.
Add the new compatible string to the list, but without changing the behaviour, since the driver already supports both variants.
Signed-off-by Andre Przywara <[email protected]>
Quoted 19 lines
--- drivers/iommu/sun50i-iommu.c | 1 + 1 file changed, 1 insertion(+)drivers/iommu/sun50i-iommu.c +1
-1@@ -1067,6 +1067,7 @@ static int sun50i_iommu_probe(struct platform_device *pdev)static const struct of_device_id sun50i_iommu_dt[] = { { .compatible ="allwinner,sun50i-h6-iommu" , }, + { .compatible ="allwinner,sun50i-h616-iommu" , }, {/* sentinel */ }, };MODULE_DEVICE_TABLE (of, sun50i_iommu_dt); --2.39.4
PATCH v2 5/5 arm64: dts: allwinner: h616: add IOMMU node
In reply to Andre PrzywaraThe Allwinner H616 contains a scatter-gather IOMMU connected to some video related devices. It's almost compatible to the one used in the H6, though with minor incompatibilities.
Add the DT node describing its resources, so that devices like the video or display engine can connect to it.
--- arch/arm64/boot/dts/allwinner/sun50i-h616.dtsi | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/arm64/boot/dts/allwinner/sun50i-h616.dtsi +9 -0
@@ -305,6 +305,15 @@ x32clk_fanout_pin: x32clk-fanout-pin { }; }; + iommu: iommu@30f0000 { + compatible = "allwinner,sun50i-h616-iommu"; + reg = <0x030f0000 0x10000>; + interrupts = <GIC_SPI 61 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&ccu CLK_BUS_IOMMU>; + resets = <&ccu RST_BUS_IOMMU>; + #iommu-cells = <1>; + }; + gic: interrupt-controller@3021000 { compatible = "arm,gic-400"; reg = <0x03021000 0x1000>,
Signature · 1 line
2.39.4
Re: [PATCH v2 5/5] arm64: dts: allwinner: h616: add IOMMU node
Hi Andre,
Quoted 6 lines · Andre Przywara
The Allwinner H616 contains a scatter-gather IOMMU connected to some video related devices. It's almost compatible to the one used in the H6, though with minor incompatibilities.
Add the DT node describing its resources, so that devices like the video or display engine can connect to it.
Thanks, tested working with the display engine on an Allwinner H700 device (Anbernic RG35XX Plus).
[ 0.038060] iommu: Default domain type: Translated [ 0.038080] iommu: DMA domain TLB invalidation policy: strict mode ... [ 0.175075] sun8i-mixer 1100000.mixer: Adding to iommu group 0
Regards,
Ryan
Quoted 4 lines · Andre Przywara
.../bindings/iommu/allwinner,sun50i-h6-iommu.yaml | 7 ++++++- arch/arm64/boot/dts/allwinner/sun50i-h616.dtsi | 9 +++++++++ drivers/iommu/sun50i-iommu.c | 15 +++++++++++++-- 3 files changed, 28 insertions(+), 3 deletions(-)
Applied, thanks.
Hi Joerg,
Quoted 8 lines · Joerg Roedel
Quoted 4 lines · Andre Przywara
.../bindings/iommu/allwinner,sun50i-h6-iommu.yaml | 7 ++++++- arch/arm64/boot/dts/allwinner/sun50i-h616.dtsi | 9 +++++++++ drivers/iommu/sun50i-iommu.c | 15 +++++++++++++-- 3 files changed, 28 insertions(+), 3 deletions(-)Applied, thanks.
Could you back out the last patch? I'd like to take it through the sunxi tree, which already has other H616 device tree patches.
Thanks
ChenYu
Joerg Roedel
maintainerQuoted 2 lines · Chen-Yu Tsai
Could you back out the last patch? I'd like to take it through the sunxi tree, which already has other H616 device tree patches.
Done. Removed "arm64: dts: allwinner: h616: add IOMMU node" from the IOMMU tree.
Regards,
Joerg
Re: (subset) [PATCH v2 0/5] iommu: sun50i: Add Allwinner H616 support
In reply to Andre PrzywaraFrom: Chen-Yu Tsai <[email protected]>
Quoted 11 lines · Andre Przywara
Version two of this series adds a check that no physical address larger than 4GB makes it into the PTEs: the
map_pages()function returns an error and prints a warning into dmesg to give users a clue why this failed. I haven't tested whether this really happens, or whether the 32-bit DMA mask of the master devices already prevents this. In the worst case this might fail on devices with 4GB of DRAM, but would always work on smaller devices, which are arguably under bigger pressure to find contiguous PA ranges. Changelog below.
===========================================[...]
Applied to sunxi/dt-for-6.11 in sunxi/linux.git, thanks!
[5/5] arm64: dts: allwinner: h616: add IOMMU node
https://git.kernel.org/sunxi/linux/c/0c85e2e377c3
FTR, I moved the IOMMU device node after the GIC node so the nodes are properly ordered.
Best regards,
Signature · 1 line
Chen-Yu Tsai <[email protected]>