Skip to contentSkip to conversation

Add system mmu support for Armada-806

PATCH Applied Tomasz Nowicki 6 participants 24 messages Jul 15, 2020 – Oct 23, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v4 cover Add system mmu support for Armada-806

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum

    #582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806

    SMMU-500
 Documentation/arm64/silicon-errata.rst        |  3 ++
 .../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
 arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
 arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
 arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
 drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
 drivers/iommu/arm-smmu.c                      | 11 +++--
 7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v4 1/4 iommu/arm-smmu: Call configuration impl hook before consuming features

'cfg_probe' hook is called at the very end of configuration probing procedure and therefore features override and workaround may become complex like for ID register fixups. In preparation for adding Marvell errata move 'cfg_probe' a bit earlier to have chance to adjust the detected features before we start consuming them.

Since the Cavium quirk (the only user) does not alter features it is safe to do so.

---
 drivers/iommu/arm-smmu.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
drivers/iommu/arm-smmu.c +7 -4
index 243bc4cb2705..19f906de6420 100644
@@ -1728,7 +1728,7 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	unsigned int size;
 	u32 id;
 	bool cttw_reg, cttw_fw = smmu->features & ARM_SMMU_FEAT_COHERENT_WALK;
-	int i;
+	int i, ret;
 
 	dev_notice(smmu->dev, "probing hardware configuration...\n");
 	dev_notice(smmu->dev, "SMMUv%d with:\n",
@@ -1891,6 +1891,12 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 			smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K;
 	}
 
+	if (smmu->impl && smmu->impl->cfg_probe) {
+		ret = smmu->impl->cfg_probe(smmu);
+		if (ret)
+			return ret;
+	}
+
 	/* Now we've corralled the various formats, what'll it do? */
 	if (smmu->features & ARM_SMMU_FEAT_FMT_AARCH32_S)
 		smmu->pgsize_bitmap |= SZ_4K | SZ_64K | SZ_1M | SZ_16M;
@@ -1918,9 +1924,6 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 		dev_notice(smmu->dev, "\tStage-2: %lu-bit IPA -> %lu-bit PA\n",
 			   smmu->ipa_size, smmu->pa_size);
 
-	if (smmu->impl && smmu->impl->cfg_probe)
-		return smmu->impl->cfg_probe(smmu);
-
 	return 0;
 }
 
Signature · 1 line
2.17.1
reviewer Reviewed-by on v4 1/4
#2 lore raw

Re: [PATCH v4 1/4] iommu/arm-smmu: Call configuration impl hook before consuming features

Quoted 8 lines · Tomasz Nowicki

'cfg_probe' hook is called at the very end of configuration probing procedure and therefore features override and workaround may become complex like for ID register fixups. In preparation for adding Marvell errata move 'cfg_probe' a bit earlier to have chance to adjust the detected features before we start consuming them.

Since the Cavium quirk (the only user) does not alter features it is safe to do so.

Sorry for the confusion of failing to match my own intent in the first place ;)

Quoted 43 lines
---
  drivers/iommu/arm-smmu.c | 11 +++++++----
  1 file changed, 7 insertions(+), 4 deletions(-)
drivers/iommu/arm-smmu.c +7 -4
index 243bc4cb2705..19f906de6420 100644
@@ -1728,7 +1728,7 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
  	unsigned int size;
  	u32 id;
  	bool cttw_reg, cttw_fw = smmu->features & ARM_SMMU_FEAT_COHERENT_WALK;
-	int i;
+	int i, ret;
 
  	dev_notice(smmu->dev, "probing hardware configuration...\n");
  	dev_notice(smmu->dev, "SMMUv%d with:\n",
@@ -1891,6 +1891,12 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
  			smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K;
  	}
 
+	if (smmu->impl && smmu->impl->cfg_probe) {
+		ret = smmu->impl->cfg_probe(smmu);
+		if (ret)
+			return ret;
+	}
+
  	/* Now we've corralled the various formats, what'll it do? */
  	if (smmu->features & ARM_SMMU_FEAT_FMT_AARCH32_S)
  		smmu->pgsize_bitmap |= SZ_4K | SZ_64K | SZ_1M | SZ_16M;
@@ -1918,9 +1924,6 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
  		dev_notice(smmu->dev, "\tStage-2: %lu-bit IPA -> %lu-bit PA\n",
  			   smmu->ipa_size, smmu->pa_size);
 
-	if (smmu->impl && smmu->impl->cfg_probe)
-		return smmu->impl->cfg_probe(smmu);
-
  	return 0;
  }
 
#3 lore raw

PATCH v4 2/4 iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743

In reply to Tomasz Nowicki

From: Hanna Hawa <[email protected]>

Due to erratum #582743, the Marvell Armada-AP806 can't access 64bit to ARM SMMUv2 registers.

Provide implementation relevant hooks:
- split the writeq/readq to two accesses of writel/readl.
- mask the MMU_IDR2.PTFSv8 fields to not use AArch64 format (but only AARCH32_L) since with AArch64 format 32 bits access is not supported.

Note that most 64-bit registers like TTBRn can be accessed as two 32-bit halves without issue, and AArch32 format ensures that the register writes which must be atomic (for TLBI etc.) need only be 32-bit.

---
 Documentation/arm64/silicon-errata.rst |  3 ++
 drivers/iommu/arm-smmu-impl.c          | 45 ++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)
Documentation/arm64/silicon-errata.rst +3 -0
index 936cf2a59ca4..157214d3abe1 100644
@@ -125,6 +125,9 @@ stable kernels.
 | Cavium         | ThunderX2 Core  | #219            | CAVIUM_TX2_ERRATUM_219      |
 +----------------+-----------------+-----------------+-----------------------------+
 +----------------+-----------------+-----------------+-----------------------------+
+| Marvell        | ARM-MMU-500     | #582743         | N/A                         |
++----------------+-----------------+-----------------+-----------------------------+
++----------------+-----------------+-----------------+-----------------------------+
 | Freescale/NXP  | LS2080A/LS1043A | A-008585        | FSL_ERRATUM_A008585         |
 +----------------+-----------------+-----------------+-----------------------------+
 +----------------+-----------------+-----------------+-----------------------------+
drivers/iommu/arm-smmu-impl.c +45 -0

56 lines not on this page · raw message

Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
reviewer Reviewed-by on v4 2/4
#4 lore raw

Re: [PATCH v4 2/4] iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743

Quoted 5 lines · Tomasz Nowicki

From: Hanna Hawa <[email protected]>

Due to erratum #582743, the Marvell Armada-AP806 can't access 64bit to ARM SMMUv2 registers.

Quoted 8 lines

Provide implementation relevant hooks:
- split the writeq/readq to two accesses of writel/readl.
- mask the MMU_IDR2.PTFSv8 fields to not use AArch64 format (but only AARCH32_L) since with AArch64 format 32 bits access is not supported.

Note that most 64-bit registers like TTBRn can be accessed as two 32-bit halves without issue, and AArch32 format ensures that the register writes which must be atomic (for TLBI etc.) need only be 32-bit.

Thanks Tomasz, this has ended up as clean as I'd hoped it could, and there's still room to come back and play more complicated games later if a real need for AARCH64_64K at stage 2 crops up.

Quoted 13 lines
---
  Documentation/arm64/silicon-errata.rst |  3 ++
  drivers/iommu/arm-smmu-impl.c          | 45 ++++++++++++++++++++++++++
  2 files changed, 48 insertions(+)
Documentation/arm64/silicon-errata.rst +0 -0
index 936cf2a59ca4..157214d3abe1 100644

Documentation/arm64/silicon-errata.rst stable kernels. · line 130

Quoted 6 lines
Documentation/arm64/silicon-errata.rst +3 -0
@@ -125,6 +125,9 @@ stable kernels.
  | Cavium         | ThunderX2 Core  | #219            | CAVIUM_TX2_ERRATUM_219      |
  +----------------+-----------------+-----------------+-----------------------------+
  +----------------+-----------------+-----------------+-----------------------------+
+| Marvell        | ARM-MMU-500     | #582743         | N/A                         |
++----------------+-----------------+-----------------+-----------------------------+
++----------------+-----------------+-----------------+-----------------------------+

And in case anyone feels like nit-picking the order here, I think the current respective corporate structures perfectly justify "Marvell" sorting alphabetically before "NXP", to be next to "Cavium" :D

Robin.

Quoted 66 lines
Documentation/arm64/silicon-errata.rst +0 -0
  | Freescale/NXP  | LS2080A/LS1043A | A-008585        | FSL_ERRATUM_A008585         |
  +----------------+-----------------+-----------------+-----------------------------+
  +----------------+-----------------+-----------------+-----------------------------+
drivers/iommu/arm-smmu-impl.c +45 -0
index c75b9d957b70..59422cb92488 100644
@@ -147,6 +147,48 @@ static const struct arm_smmu_impl arm_mmu500_impl = {
  	.reset = arm_mmu500_reset,
  };
 
+static u64 mrvl_mmu500_readq(struct arm_smmu_device *smmu, int page, int off)
+{
+	/*
+	 * Marvell Armada-AP806 erratum #582743.
+	 * Split all the readq to double readl
+	 */
+	return hi_lo_readq_relaxed(arm_smmu_page(smmu, page) + off);
+}
+
+static void mrvl_mmu500_writeq(struct arm_smmu_device *smmu, int page, int off,
+			       u64 val)
+{
+	/*
+	 * Marvell Armada-AP806 erratum #582743.
+	 * Split all the writeq to double writel
+	 */
+	hi_lo_writeq_relaxed(val, arm_smmu_page(smmu, page) + off);
+}
+
+static int mrvl_mmu500_cfg_probe(struct arm_smmu_device *smmu)
+{
+
+	/*
+	 * Armada-AP806 erratum #582743.
+	 * Hide the SMMU_IDR2.PTFSv8 fields to sidestep the AArch64
+	 * formats altogether and allow using 32 bits access on the
+	 * interconnect.
+	 */
+	smmu->features &= ~(ARM_SMMU_FEAT_FMT_AARCH64_4K |
+			    ARM_SMMU_FEAT_FMT_AARCH64_16K |
+			    ARM_SMMU_FEAT_FMT_AARCH64_64K);
+
+	return 0;
+}
+
+static const struct arm_smmu_impl mrvl_mmu500_impl = {
+	.read_reg64 = mrvl_mmu500_readq,
+	.write_reg64 = mrvl_mmu500_writeq,
+	.cfg_probe = mrvl_mmu500_cfg_probe,
+	.reset = arm_mmu500_reset,
+};
+
 
  struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)
  {
@@ -175,5 +217,8 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)
  	    of_device_is_compatible(np, "qcom,sc7180-smmu-500"))
  		return qcom_smmu_impl_init(smmu);
 
+	if (of_device_is_compatible(np, "marvell,ap806-smmu-500"))
+		smmu->impl = &mrvl_mmu500_impl;
+
  	return smmu;
  }
#5 lore raw

Re: [PATCH v4 2/4] iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743

Quoted 11 lines · Robin Murphy
Quoted 10 lines · Tomasz Nowicki

From: Hanna Hawa <[email protected]>

Due to erratum #582743, the Marvell Armada-AP806 can't access 64bit to ARM SMMUv2 registers.

Provide implementation relevant hooks:
- split the writeq/readq to two accesses of writel/readl.
- mask the MMU_IDR2.PTFSv8 fields to not use AArch64 format (but only AARCH32_L) since with AArch64 format 32 bits access is not supported.

Quoted 5 lines
Quoted 3 lines

Note that most 64-bit registers like TTBRn can be accessed as two 32-bit halves without issue, and AArch32 format ensures that the register writes which must be atomic (for TLBI etc.) need only be 32-bit.

Thanks Tomasz, this has ended up as clean as I'd hoped it could, and there's still room to come back and play more complicated games later if a real need for AARCH64_64K at stage 2 crops up.

Based on your implementation infrastructure rework, indeed the code looks much cleaner :)

Quoted 3 lines

Thanks!

Tomasz

#6 lore raw

PATCH v4 3/4 dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500

In reply to Tomasz Nowicki

Add specific compatible string for Marvell usage due to errata of accessing 64bits registers of ARM SMMU, in AP806.

AP806 SoC uses the generic ARM-MMU500, and there's no specific implementation of Marvell, this compatible is used for errata only.

---
 Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 4 ++++
 1 file changed, 4 insertions(+)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +4 -0
index d7ceb4c34423..156b38924a00 100644
@@ -38,6 +38,10 @@ properties:
               - qcom,sc7180-smmu-500
               - qcom,sdm845-smmu-500
           - const: arm,mmu-500
+      - description: Marvell SoCs implementing "arm,mmu-500"
+        items:
+          - const: marvell,ap806-smmu-500
+          - const: arm,mmu-500
       - items:
           - const: arm,mmu-500
           - const: arm,smmu-v2
Signature · 1 line
2.17.1
reviewer Reviewed-by on v4 3/4
#7 lore raw

Re: [PATCH v4 3/4] dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500

Quoted 5 lines · Tomasz Nowicki

Add specific compatible string for Marvell usage due to errata of accessing 64bits registers of ARM SMMU, in AP806.

AP806 SoC uses the generic ARM-MMU500, and there's no specific implementation of Marvell, this compatible is used for errata only.

Presumably Will can pick up these first 3 patches for 5.9 and #4 can go via arm-soc.

Robin.

Quoted 24 lines
---
  Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 4 ++++
  1 file changed, 4 insertions(+)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +4 -0
index d7ceb4c34423..156b38924a00 100644
@@ -38,6 +38,10 @@ properties:
                - qcom,sc7180-smmu-500
                - qcom,sdm845-smmu-500
            - const: arm,mmu-500
+      - description: Marvell SoCs implementing "arm,mmu-500"
+        items:
+          - const: marvell,ap806-smmu-500
+          - const: arm,mmu-500
        - items:
            - const: arm,mmu-500
            - const: arm,smmu-v2
#8 lore raw

Re: [PATCH v4 3/4] dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500

Quoted 10 lines · Robin Murphy
Quoted 5 lines · Tomasz Nowicki

Add specific compatible string for Marvell usage due to errata of accessing 64bits registers of ARM SMMU, in AP806.

AP806 SoC uses the generic ARM-MMU500, and there's no specific implementation of Marvell, this compatible is used for errata only.

Presumably Will can pick up these first 3 patches for 5.9 and #4 can go via arm-soc.

Thanks Robin for review and valuable comments.

Tomasz

#9 lore raw

PATCH v4 4/4 arm64: dts: marvell: add SMMU support

In reply to Tomasz Nowicki

From: Marcin Wojtas <[email protected]>

Add IOMMU node for Marvell AP806 based SoCs together with platform and PCI device Stream ID mapping.

---
 arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 +++++++++++++
 arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++++
 arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 +++++++++
 3 files changed, 86 insertions(+)
arch/arm64/boot/dts/marvell/armada-7040.dtsi +28 -0
index 47247215770d..7a3198cd7a07 100644
@@ -14,3 +14,31 @@
 	compatible = "marvell,armada7040", "marvell,armada-ap806-quad",
 		     "marvell,armada-ap806";
 };
+
+&smmu {
+	status = "okay";
+};
+
+&cp0_pcie0 {
+	iommu-map =
+		<0x0   &smmu 0x480 0x20>,
+		<0x100 &smmu 0x4a0 0x20>,
+		<0x200 &smmu 0x4c0 0x20>;
+	iommu-map-mask = <0x031f>;
+};
+
+&cp0_sata0 {
+	iommus = <&smmu 0x444>;
+};
+
+&cp0_sdhci0 {
+	iommus = <&smmu 0x445>;
+};
+
+&cp0_usb3_0 {
+	iommus = <&smmu 0x440>;
+};
+
+&cp0_usb3_1 {
+	iommus = <&smmu 0x441>;
+};
arch/arm64/boot/dts/marvell/armada-8040.dtsi +40 -0
index 7699b19224c2..79e8ce59baa8 100644
@@ -15,6 +15,18 @@
 		     "marvell,armada-ap806";
 };
 
+&smmu {
+	status = "okay";
+};
+
+&cp0_pcie0 {
+	iommu-map =
+		<0x0   &smmu 0x480 0x20>,
+		<0x100 &smmu 0x4a0 0x20>,
+		<0x200 &smmu 0x4c0 0x20>;
+	iommu-map-mask = <0x031f>;
+};
+
 /* The RTC requires external oscillator. But on Aramda 80x0, the RTC clock
  * in CP master is not connected (by package) to the oscillator. So
  * disable it. However, the RTC clock in CP slave is connected to the
@@ -23,3 +35,31 @@
 &cp0_rtc {
 	status = "disabled";
 };
+
+&cp0_sata0 {
+	iommus = <&smmu 0x444>;
+};
+
+&cp0_sdhci0 {
+	iommus = <&smmu 0x445>;
+};
+
+&cp0_usb3_0 {
+	iommus = <&smmu 0x440>;
+};
+
+&cp0_usb3_1 {
+	iommus = <&smmu 0x441>;
+};
+
+&cp1_sata0 {
+	iommus = <&smmu 0x454>;
+};
+
+&cp1_usb3_0 {
+	iommus = <&smmu 0x450>;
+};
+
+&cp1_usb3_1 {
+	iommus = <&smmu 0x451>;
+};
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi +18 -0
index 7f9b9a647717..12e477f1aeb9 100644
@@ -56,6 +56,24 @@
 			compatible = "simple-bus";
 			ranges = <0x0 0x0 0xf0000000 0x1000000>;
 
+			smmu: iommu@5000000 {
+				compatible = "marvell,ap806-smmu-500", "arm,mmu-500";
+				reg = <0x100000 0x100000>;
+				dma-coherent;
+				#iommu-cells = <1>;
+				#global-interrupts = <1>;
+				interrupts = <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;
+				status = "disabled";
+			};
+
 			gic: interrupt-controller@210000 {
 				compatible = "arm,gic-400";
 				#interrupt-cells = <3>;
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
maintainer
#10 lore raw

Re: [PATCH v4 4/4] arm64: dts: marvell: add SMMU support

Quoted 7 lines · Tomasz Nowicki

From: Marcin Wojtas <[email protected]>

Add IOMMU node for Marvell AP806 based SoCs together with platform and PCI device Stream ID mapping.

Applied on mvebu/dt64

Thanks,

Gregory

Quoted 129 lines
---
 arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 +++++++++++++
 arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++++
 arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 +++++++++
 3 files changed, 86 insertions(+)
arch/arm64/boot/dts/marvell/armada-7040.dtsi +28 -0
index 47247215770d..7a3198cd7a07 100644
@@ -14,3 +14,31 @@
 	compatible = "marvell,armada7040", "marvell,armada-ap806-quad",
 		     "marvell,armada-ap806";
 };
+
+&smmu {
+	status = "okay";
+};
+
+&cp0_pcie0 {
+	iommu-map =
+		<0x0   &smmu 0x480 0x20>,
+		<0x100 &smmu 0x4a0 0x20>,
+		<0x200 &smmu 0x4c0 0x20>;
+	iommu-map-mask = <0x031f>;
+};
+
+&cp0_sata0 {
+	iommus = <&smmu 0x444>;
+};
+
+&cp0_sdhci0 {
+	iommus = <&smmu 0x445>;
+};
+
+&cp0_usb3_0 {
+	iommus = <&smmu 0x440>;
+};
+
+&cp0_usb3_1 {
+	iommus = <&smmu 0x441>;
+};
arch/arm64/boot/dts/marvell/armada-8040.dtsi +40 -0
index 7699b19224c2..79e8ce59baa8 100644
@@ -15,6 +15,18 @@
 		     "marvell,armada-ap806";
 };
 
+&smmu {
+	status = "okay";
+};
+
+&cp0_pcie0 {
+	iommu-map =
+		<0x0   &smmu 0x480 0x20>,
+		<0x100 &smmu 0x4a0 0x20>,
+		<0x200 &smmu 0x4c0 0x20>;
+	iommu-map-mask = <0x031f>;
+};
+
 /* The RTC requires external oscillator. But on Aramda 80x0, the RTC clock
  * in CP master is not connected (by package) to the oscillator. So
  * disable it. However, the RTC clock in CP slave is connected to the
@@ -23,3 +35,31 @@
 &cp0_rtc {
 	status = "disabled";
 };
+
+&cp0_sata0 {
+	iommus = <&smmu 0x444>;
+};
+
+&cp0_sdhci0 {
+	iommus = <&smmu 0x445>;
+};
+
+&cp0_usb3_0 {
+	iommus = <&smmu 0x440>;
+};
+
+&cp0_usb3_1 {
+	iommus = <&smmu 0x441>;
+};
+
+&cp1_sata0 {
+	iommus = <&smmu 0x454>;
+};
+
+&cp1_usb3_0 {
+	iommus = <&smmu 0x450>;
+};
+
+&cp1_usb3_1 {
+	iommus = <&smmu 0x451>;
+};
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi +18 -0
index 7f9b9a647717..12e477f1aeb9 100644
@@ -56,6 +56,24 @@
 			compatible = "simple-bus";
 			ranges = <0x0 0x0 0xf0000000 0x1000000>;
 
+			smmu: iommu@5000000 {
+				compatible = "marvell,ap806-smmu-500", "arm,mmu-500";
+				reg = <0x100000 0x100000>;
+				dma-coherent;
+				#iommu-cells = <1>;
+				#global-interrupts = <1>;
+				interrupts = <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+					     <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;
+				status = "disabled";
+			};
+
 			gic: interrupt-controller@210000 {
 				compatible = "arm,gic-400";
 				#interrupt-cells = <3>;
Signature · 1 line
2.17.1
Signature · 3 lines
Gregory Clement, Bootlin
Embedded Linux and Kernel engineering
http://bootlin.com
maintainer Applied
#11 lore raw
In reply to Tomasz Nowicki
Quoted 8 lines · Tomasz Nowicki

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

[...]

Applied to will (for-joerg/arm-smmu/updates), thanks!

[1/3] iommu/arm-smmu: Call configuration impl hook before consuming features

      https://git.kernel.org/will/c/6a79a5a3842b
[2/3] iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743
      https://git.kernel.org/will/c/f2d9848aeb9f
[3/3] dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500
      https://git.kernel.org/will/c/e85e84d19b9d

Cheers,

Signature · 5 lines
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev
maintainer
#12 lore raw
Quoted 10 lines · Will Deacon
Quoted 8 lines · Tomasz Nowicki

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

[...]

Quoted 8 lines

Applied to will (for-joerg/arm-smmu/updates), thanks!

[1/3] iommu/arm-smmu: Call configuration impl hook before consuming features

      https://git.kernel.org/will/c/6a79a5a3842b
[2/3] iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743
      https://git.kernel.org/will/c/f2d9848aeb9f
[3/3] dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500
      https://git.kernel.org/will/c/e85e84d19b9d

(note that I left patch 4 for arm-soc, as that's just updating .dts files)

Will

#13 lore raw

czw., 16 lip 2020 o 14:02 Will Deacon <[email protected]> napisał(a):

Quoted 14 lines · Will Deacon
Quoted 12 lines · Tomasz Nowicki
Quoted 8 lines

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

[...]

Applied to will (for-joerg/arm-smmu/updates), thanks!

Quoted 9 lines
Quoted 6 lines

[1/3] iommu/arm-smmu: Call configuration impl hook before consuming features

      https://git.kernel.org/will/c/6a79a5a3842b
[2/3] iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743
      https://git.kernel.org/will/c/f2d9848aeb9f
[3/3] dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500
      https://git.kernel.org/will/c/e85e84d19b9d

(note that I left patch 4 for arm-soc, as that's just updating .dts files)

Hi Gregory,

Can you please help with the review/merge of patch #4?

Best regards,
Marcin

maintainer
#14 lore raw

Hello,

Quoted 22 lines

czw., 16 lip 2020 o 14:02 Will Deacon <[email protected]> napisał(a):

Quoted 21 lines · Will Deacon
Quoted 18 lines · Tomasz Nowicki
Quoted 8 lines

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

[...]

Applied to will (for-joerg/arm-smmu/updates), thanks!

[1/3] iommu/arm-smmu: Call configuration impl hook before consuming features

      https://git.kernel.org/will/c/6a79a5a3842b
[2/3] iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743
      https://git.kernel.org/will/c/f2d9848aeb9f
[3/3] dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500
      https://git.kernel.org/will/c/e85e84d19b9d
Quoted 6 lines
Quoted 2 lines

(note that I left patch 4 for arm-soc, as that's just updating .dts files)

Hi Gregory,

Can you please help with the review/merge of patch #4?

Sure!

I've followed the series since the v1 even if I didn't commetn and I am happy that it finally managed to be merged. I can now remove it from my TODO list! :)

Gregory

Quoted 3 lines

Best regards,
Marcin

Signature · 3 lines
Gregory Clement, Bootlin
Embedded Linux and Kernel engineering
http://bootlin.com
#15 lore raw
In reply to Tomasz Nowicki

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 18 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum
#582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806
SMMU-500

Quoted 17 lines
Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#16 lore raw

Hi Denis,

Thank you for your report.

wt., 6 paź 2020 o 17:17 Denis Odintsov <[email protected]> napisał(a):

Quoted 21 lines

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

Quoted 5 lines

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

According to my knowledge and the docs AP IO devices cannot be virtualized, only ones connected via CP110/CP115. We'd need to check what should be done in such configuration and get back to you.

Quoted 46 lines

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)
Quoted 1 line

Thank you!

The PCIE was validated when booting from edk2 + using pci-host-generic driver and standard intel NIC. Not sure if it makes any difference vs the Designware driver ("marvell,armada8k-pcie"), but we need to double-check that.

Best regards,
Marcin

Quoted 37 lines
Quoted 36 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum
#582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806
SMMU-500

Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)

--
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

reviewer
#17 lore raw
In reply to Denis Odintsov
Quoted 20 lines · Denis Odintsov

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

Quoted 5 lines

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

FWIW the SMMU tells you the offending unmatched Stream ID, so if faults can reasonably be correlated with a particular device making accesses, you can effectively discover the Stream ID assignment by trial and error. Often that can be easier than trying to find formal documentation anyway ;)

Quoted 32 lines

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
Quoted 1 line

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

Does forcing ath10k to use legacy interrupts rather than MSIs make a difference?

Judging by the DT it looks like MSIs ought to be targeting the GICv2M widget, but if things somehow end up trying to use the PCIe controller's internal MSI doorbell (upstream of SMMU translation) instead, then that might account for general interrupt-related weirdness.

Robin.

Quoted 49 lines
[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 35 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum

    #582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806

    SMMU-500
Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#18 lore raw

Hi,

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

Does forcing ath10k to use legacy interrupts rather than MSIs make a difference?

Judging by the DT it looks like MSIs ought to be targeting the GICv2M widget, but if things somehow end up trying to use the PCIe controller's internal MSI doorbell (upstream of SMMU translation) instead, then that might account for general interrupt-related weirdness.

Robin.

Frankly speaking you quickly overcome here my knowledge depth, this is already way far from what I understand about PCI devices. But I tried my best to try it out what you suggested. putting 0 to /sys/bus/pci/devices/0000\:01\:00.0/msi_bus (bus of ath10k) and reloading the driver didn't make a difference with those almost same messages:

[  103.245287] ath10k_pci 0000:01:00.0: MSI/MSI-X disallowed for future drivers
[  145.938562] ath10k_pci 0000:01:00.0: pci irq legacy oper_irq_mode 1 irq_mode 0 reset_mode 0
[  146.053590] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[  146.161637] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[  146.269515] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[  146.453633] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[  146.561589] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[  146.669550] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[  146.753456] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[  146.760129] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[  146.766695] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[  146.773196] ath10k_pci 0000:01:00.0: could not probe fw (-16)

lspci -v says
Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit-

So it seems it does what you suggested, disabled the MSI. With not much differences unfortunately. =(

I also compared lscpi output also with non-SMMU-dts (with which ath10k works) and SMMU-dts, and there is a difference I guess, I'm not sure how affecting it is. On non-SMMU dts host controller (served by pcieport driver) IRQ is 37 and ath10k IRQ is 82:

00:00.0 PCI bridge: Marvell Technology Group Ltd. Device 0110 (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 37
...
01:00.0 Network controller: Qualcomm Atheros QCA986x/988x 802.11ac Wireless Network Adapter
Flags: bus master, fast devsel, latency 0, IRQ 82

On SMMU dt's though both IRQ's are same and are 38:

00:00.0 PCI bridge: Marvell Technology Group Ltd. Device 0110 (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 38
...
01:00.0 Network controller: Qualcomm Atheros QCA986x/988x 802.11ac Wireless Network Adapter
Flags: bus master, fast devsel, latency 0, IRQ 38

[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!
v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum
#582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806
SMMU-500

Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)

--
2.17.1

_______________________________________________
iommu mailing list [email protected]<mailto:[email protected]>
https://lists.linuxfoundation.org/mailman/listinfo/iommu

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#19 lore raw
In reply to Robin Murphy
Quoted 114 lines

Am 13.10.2020 um 15:08 schrieb Robin Murphy <[email protected]>:

Quoted 18 lines · Denis Odintsov

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

Quoted 5 lines

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

FWIW the SMMU tells you the offending unmatched Stream ID, so if faults can reasonably be correlated with a particular device making accesses, you can effectively discover the Stream ID assignment by trial and error. Often that can be easier than trying to find formal documentation anyway ;)

Quoted 30 lines

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
Quoted 1 line

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

Does forcing ath10k to use legacy interrupts rather than MSIs make a difference?

Judging by the DT it looks like MSIs ought to be targeting the GICv2M widget, but if things somehow end up trying to use the PCIe controller's internal MSI doorbell (upstream of SMMU translation) instead, then that might account for general interrupt-related weirdness.

Robin.

Quoted 45 lines
[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 34 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum
#582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806
SMMU-500

Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#20 lore raw
In reply to Denis Odintsov

Hi Denis,

Sorry for late response, we had to check few things. Please see comments inline.

Quoted 20 lines · Denis Odintsov

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

Quoted 8 lines

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

Firmware has to configure and assign device StreamIDs. Most of the devices are configured properly and supported in public FW. However, for both these cases (ap_sdhci0 and PCIe) some extra (u-boot/UEFI/ATF) patches are required which are not available yet. Sorry we let that happen.

Since we have dependency on custom FW and we cannot enforce people to patch their FW we will send the follow up fix patch (v5.9+) and revert respective DTS changes.

The most important Armada-806 SMMU driver enhancements were merged so people who still willing to use SMMU need to provide proper DTB and use ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=n (or via kernel command line) with extra cautious.

Thanks,
Tomasz

Quoted 81 lines
[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 35 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum

    #582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806

    SMMU-500
Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
reviewer
#21 lore raw
Quoted 31 lines · Tomasz Nowicki

Hi Denis,

Sorry for late response, we had to check few things. Please see comments inline.

Quoted 26 lines · Denis Odintsov

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

Quoted 6 lines
Quoted 2 lines

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

Firmware has to configure and assign device StreamIDs. Most of the devices are configured properly and supported in public FW. However, for both these cases (ap_sdhci0 and PCIe) some extra (u-boot/UEFI/ATF) patches are required which are not available yet. Sorry we let that happen.

Since we have dependency on custom FW and we cannot enforce people to patch their FW we will send the follow up fix patch (v5.9+) and revert respective DTS changes.

Note that it should be sufficient to simply keep the SMMU node disabled, rather than fully revert everything. For example, the PCIe SMMU for Arm Juno boards has been in that state for a long time - there are reasons why it isn't (yet) 100% usable for everyone, but it can easily be enabled locally for development (as I do).

Robin.

Quoted 90 lines

The most important Armada-806 SMMU driver enhancements were merged so people who still willing to use SMMU need to provide proper DTB and use ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=n (or via kernel command line) with extra cautious.

Thanks,
Tomasz

Quoted 85 lines

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000 [    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref] [    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit] [    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref] [    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref] [    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr:
-16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16 [    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16) [    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 38 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
  iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743

Marcin Wojtas (1):
  arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
  iommu/arm-smmu: Call configuration impl hook before consuming features
  dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500

Documentation/arm64/silicon-errata.rst        |  3 ++ .../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++ arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++ arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++ arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++ drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++ drivers/iommu/arm-smmu.c                      | 11 +++--

7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#22 lore raw
Quoted 32 lines · Robin Murphy
Quoted 31 lines · Tomasz Nowicki

Hi Denis,

Sorry for late response, we had to check few things. Please see comments inline.

Quoted 26 lines · Denis Odintsov

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card. It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

Firmware has to configure and assign device StreamIDs. Most of the devices are configured properly and supported in public FW. However, for both these cases (ap_sdhci0 and PCIe) some extra (u-boot/UEFI/ATF) patches are required which are not available yet. Sorry we let that happen.

Since we have dependency on custom FW and we cannot enforce people to patch their FW we will send the follow up fix patch (v5.9+) and revert respective DTS changes.

Note that it should be sufficient to simply keep the SMMU node disabled, rather than fully revert everything. For example, the PCIe SMMU for Arm Juno boards has been in that state for a long time - there are reasons why it isn't (yet) 100% usable for everyone, but it can easily be enabled locally for development (as I do).

Actually that was our plan :) but then we decided to keep DTS clean if something is not used. Your reasoning, however, does make sense and we will go for it.

Thanks,
Tomasz

Quoted 90 lines
Quoted 90 lines

The most important Armada-806 SMMU driver enhancements were merged so people who still willing to use SMMU need to provide proper DTB and use ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=n (or via kernel command line) with extra cautious.

Thanks,
Tomasz

Quoted 84 lines

[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000 [    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref] [    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit] [    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref] [    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref] [    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0

up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37

[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16 [    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16 [    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16 [    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16) [    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 38 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
  iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum #582743

Marcin Wojtas (1):
  arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
  iommu/arm-smmu: Call configuration impl hook before consuming features
  dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806 SMMU-500

Documentation/arm64/silicon-errata.rst        |  3 ++ .../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++ arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++ arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++ arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++ drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++ drivers/iommu/arm-smmu.c                      | 11 +++--

7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#23 lore raw
In reply to Tomasz Nowicki

Hi,

Quoted 33 lines

Am 23.10.2020 um 14:19 schrieb Tomasz Nowicki <[email protected]>:

Hi Denis,

Sorry for late response, we had to check few things. Please see comments inline.

Quoted 18 lines · Denis Odintsov

Hi,

Quoted 9 lines

Am 15.07.2020 um 09:06 schrieb Tomasz Nowicki <[email protected]>:

The series is meant to support SMMU for AP806 and a workaround for accessing ARM SMMU 64bit registers is the gist of it.

For the record, AP-806 can't access SMMU registers with 64bit width. This patches split the readq/writeq into two 32bit accesses instead and update DT bindings.

Quoted 8 lines

The series was successfully tested on a vanilla v5.8-rc3 kernel and Intel e1000e PCIe NIC. The same for platform devices like SATA and USB.

For reference, previous versions are listed below:
V1: https://lkml.org/lkml/2018/10/15/373
V2: https://lkml.org/lkml/2019/7/11/426
V3: https://lkml.org/lkml/2020/7/2/1114

Quoted 7 lines

1) After enabling SMMU on Armada 8040, and ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=y by default in kernel since 954a03be033c7cef80ddc232e7cbdb17df735663, internal eMMC is prevented from being initialised (as there is no iommus property for ap_sdhci0)
Disabling "Disable bypass by default" make it work, but the patch highly suggest doing it properly.
I wasn't able to find correct path for ap_sdhci for iommus in any publicly available documentation,
would be highly appreciated addressed properly, thank you!
2) Second issue I got (btw I have ClearFog GT 8k armada-8040 based board) is mpci ath10k card.
It is found, it is enumerated, it is visible in lspci, but it fails to be initialised. Here is the log:

Quoted 6 lines

Firmware has to configure and assign device StreamIDs. Most of the devices are configured properly and supported in public FW. However, for both these cases (ap_sdhci0 and PCIe) some extra (u-boot/UEFI/ATF) patches are required which are not available yet. Sorry we let that happen.

Since we have dependency on custom FW and we cannot enforce people to patch their FW we will send the follow up fix patch (v5.9+) and revert respective DTS changes.

The most important Armada-806 SMMU driver enhancements were merged so people who still willing to use SMMU need to provide proper DTB and use ARM_SMMU_DISABLE_BYPASS_BY_DEFAUL=n (or via kernel command line) with extra cautious.

Thank you very much for the reply, I'm a mere computer enthusiast who like to progress with the technology and keep up with the software. There was no harm at all with all those changes, and I see how they all are planned for a good reason.
I'm looking forward for that patches and further progress, and thank you for your work.

Denis.

Quoted 77 lines

Thanks,
Tomasz

Quoted 74 lines
[    1.743754] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    1.751116] armada8k-pcie f2600000.pcie:      MEM 0x00f6000000..0x00f6efffff -> 0x00f6000000
[    1.964690] armada8k-pcie f2600000.pcie: Link up
[    1.969379] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    1.976026] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.981537] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    1.988462] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    1.994504] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.000843] pci 0000:00:00.0: supports D1 D2
[    2.005132] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.011853] pci 0000:01:00.0: [168c:003c] type 00 class 0x028000
[    2.018001] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    2.025002] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.032111] pci 0000:01:00.0: supports D1 D2
[    2.049409] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6000000-0xf61fffff]
[    2.056322] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6200000-0xf62fffff]
[    2.063142] pci 0000:00:00.0: BAR 15: assigned [mem 0xf6300000-0xf63fffff pref]
[    2.070484] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6000000-0xf61fffff 64bit]
[    2.077880] pci 0000:01:00.0: BAR 6: assigned [mem 0xf6300000-0xf630ffff pref]
[    2.085135] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.090384] pci 0000:00:00.0:   bridge window [mem 0xf6000000-0xf61fffff]
[    2.097202] pci 0000:00:00.0:   bridge window [mem 0xf6300000-0xf63fffff pref]
[    2.104539] pcieport 0000:00:00.0: Adding to iommu group 4
[    2.110232] pcieport 0000:00:00.0: PME: Signaling with IRQ 38
[    2.116141] pcieport 0000:00:00.0: AER: enabled with IRQ 38
[    8.131135] ath10k_pci 0000:01:00.0: Adding to iommu group 4
[    8.131874] ath10k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[    8.132203] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
up to that point the log is the same as without SMMU enabled, except "Adding to iommu group N" lines, and IRQ being 37
[    8.221328] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.313362] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.409373] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.553433] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.641370] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.737979] ath10k_pci 0000:01:00.0: failed to poke copy engine: -16
[    8.807356] ath10k_pci 0000:01:00.0: Failed to get pcie state addr: -16
[    8.814032] ath10k_pci 0000:01:00.0: failed to setup init config: -16
[    8.820605] ath10k_pci 0000:01:00.0: could not power on hif bus (-16)
[    8.827111] ath10k_pci 0000:01:00.0: could not probe fw (-16)

Thank you!

Quoted 34 lines

v3 -> v4
- call cfg_probe() impl hook a bit earlier which simplifies errata handling
- use hi_lo_readq_relaxed() and hi_lo_writeq_relaxed() for register accessors
- keep SMMU status disabled by default and enable where possible (DTS changes)
- commit logs improvements and other minor fixes

Hanna Hawa (1):
iommu/arm-smmu: Workaround for Marvell Armada-AP806 SoC erratum
#582743

Marcin Wojtas (1):
arm64: dts: marvell: add SMMU support

Tomasz Nowicki (2):
iommu/arm-smmu: Call configuration impl hook before consuming features
dt-bindings: arm-smmu: add compatible string for Marvell Armada-AP806
SMMU-500

Documentation/arm64/silicon-errata.rst        |  3 ++
.../devicetree/bindings/iommu/arm,smmu.yaml   |  4 ++
arch/arm64/boot/dts/marvell/armada-7040.dtsi  | 28 ++++++++++++
arch/arm64/boot/dts/marvell/armada-8040.dtsi  | 40 +++++++++++++++++
arch/arm64/boot/dts/marvell/armada-ap80x.dtsi | 18 ++++++++
drivers/iommu/arm-smmu-impl.c                 | 45 +++++++++++++++++++
drivers/iommu/arm-smmu.c                      | 11 +++--
7 files changed, 145 insertions(+), 4 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu