Skip to contentSkip to conversation

Request direct mapping for modem firmware subdevice

PATCH Superseded Sibi Sankar 5 participants 13 messages Mar 9, 2020 – Mar 23, 2020

Subsystems (4)
Device tree bindings Arm Qualcomm mailing list IOMMU Remote processor (remoteproc)

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Sibi Sankar

author
#0 lore raw

cover Request direct mapping for modem firmware subdevice

The Q6 modem sub-system has direct access to DDR through memnoc and an indirect access routed through a SMMU which MSS CE (crypto engine sub-component of MSS) uses during out of reset sequence. Request direct mapping for the modem-firmware subdevice since smmu is not expected to provide access control/translation for these SIDs (sandboxing of the modem is achieved through XPUs engaged using SMC calls).

Sibi Sankar (3):
iommu: Export "iommu_request_dm_for_dev"
dt-bindings: remoteproc: qcom: Add modem-firmware bindings
remoteproc: qcom_q6v5_mss: Request direct mapping for firmware
subdevice

 .../bindings/remoteproc/qcom,q6v5.txt         |  4 ++
 drivers/iommu/iommu.c                         |  1 +
 drivers/remoteproc/qcom_q6v5_mss.c            | 68 +++++++++++++++++++
 3 files changed, 73 insertions(+)
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
author
#1 lore raw

PATCH 1/3 iommu: Export "iommu_request_dm_for_dev"

Export "iommu_request_dm_for_dev" to allow for modular builds of drivers requesting for direct mapping.

Signed-off-by Sibi Sankar <[email protected]>
---
 drivers/iommu/iommu.c | 1 +
 1 file changed, 1 insertion(+)
drivers/iommu/iommu.c +1 -0
index 3e3528436e0b2..06a36230fe22d 100644
@@ -2353,6 +2353,7 @@ int iommu_request_dm_for_dev(struct device *dev)
 {
 	return request_default_domain_for_dev(dev, IOMMU_DOMAIN_IDENTITY);
 }
+EXPORT_SYMBOL_GPL(iommu_request_dm_for_dev);
 
 /* Request that a device can't be direct mapped by the IOMMU */
 int iommu_request_dma_domain_for_dev(struct device *dev)
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
author
#2 lore raw

PATCH 2/3 dt-bindings: remoteproc: qcom: Add modem-firmware bindings

In reply to Sibi Sankar

Introduce modem-firmware binding to allow Q6 modem to boot on platforms which do not have trustZone.

Signed-off-by Sibi Sankar <[email protected]>
---
 Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt | 4 ++++
 1 file changed, 4 insertions(+)
Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt +4 -0
index 88dfa3fc15f78..b1715bc90fb05 100644
@@ -198,6 +198,10 @@ The Hexagon node may also have an subnode named either "smd-edge" or
 related to the Hexagon.  See ../soc/qcom/qcom,smd.txt and
 ../soc/qcom/qcom,glink.txt for details on how to describe these.
 
+The Hexagon node must contain a subnode named "modem-firmware" on
+platforms which do not have TrustZone. The sub-node should have the
+iommu property as described in ../iommu/iommu.txt.
+
 = EXAMPLE
 The following example describes the resources needed to boot control the
 Hexagon, as it is found on MSM8974 boards.
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
author
#3 lore raw

PATCH 3/3 remoteproc: qcom_q6v5_mss: Request direct mapping for firmware subdevice

In reply to Sibi Sankar

The Q6 modem sub-system has direct access to DDR through memnoc and an indirect access routed through a SMMU which MSS CE (crypto engine sub-component of MSS) uses during out of reset sequence. Request direct mapping for the modem-firmware subdevice since smmu is not expected to provide access control/translation for these SIDs (sandboxing of the modem is achieved through XPUs engaged using SMC calls).

Signed-off-by Sibi Sankar <[email protected]>
---
 drivers/remoteproc/qcom_q6v5_mss.c | 68 ++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
drivers/remoteproc/qcom_q6v5_mss.c +68 -0
index d7667418a62f4..ceb7f71dd17df 100644
@@ -10,6 +10,7 @@
 #include <linux/clk.h>
 #include <linux/delay.h>
 #include <linux/dma-mapping.h>
+#include <linux/iommu.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
 #include <linux/mfd/syscon.h>

@@ -202,6 +203,7 @@ struct q6v5 {
 	struct qcom_rproc_subdev smd_subdev;
 	struct qcom_rproc_ssr ssr_subdev;
 	struct qcom_sysmon *sysmon;
+	struct device *fw_subdev;
 	bool need_mem_protection;
 	bool has_alt_reset;
 	bool has_halt_nav;

@@ -378,6 +380,67 @@ static void q6v5_pds_disable(struct q6v5 *qproc, struct device **pds,
 	}
 }
 
+static int qcom_init_firmware(struct q6v5 *qproc)
+{
+	struct platform_device_info info;
+	struct platform_device *pdev;
+	struct device_node *np;
+	int ret;
+
+	np = of_get_child_by_name(qproc->dev->of_node, "modem-firmware");
+	if (!np)
+		return 0;
+
+	memset(&info, 0, sizeof(info));
+	info.fwnode = &np->fwnode;
+	info.parent = qproc->dev;
+	info.name = np->name;
+
+	pdev = platform_device_register_full(&info);
+	if (IS_ERR(pdev)) {
+		of_node_put(np);
+		return PTR_ERR(pdev);
+	}
+
+	pdev->dev.of_node = np;
+	ret = of_dma_configure(&pdev->dev, np, true);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to configure DMA\n");
+		goto err_unregister;
+	}
+
+	ret = iommu_request_dm_for_dev(&pdev->dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to request direct mapping\n");
+		goto err_unregister;
+	}
+
+	qproc->fw_subdev = &pdev->dev;
+	of_node_put(np);
+	return 0;
+
+err_unregister:
+	platform_device_unregister(pdev);
+	of_node_put(np);
+	return ret;
+}
+
+static void qcom_remove_firmware(struct q6v5 *qproc)
+{
+	struct iommu_domain *iommu;
+
+	if (!qproc->fw_subdev)
+		return;
+
+	iommu = iommu_get_domain_for_dev(qproc->fw_subdev);
+	if (!iommu)
+		return;
+
+	iommu_detach_device(iommu, qproc->fw_subdev);
+	iommu_domain_free(iommu);
+	platform_device_unregister(to_platform_device(qproc->fw_subdev));
+}
+
 static int q6v5_xfer_mem_ownership(struct q6v5 *qproc, int *current_perm,
 				   bool local, bool remote, phys_addr_t addr,
 				   size_t size)

@@ -1722,6 +1785,10 @@ static int q6v5_probe(struct platform_device *pdev)
 	if (ret)
 		goto detach_proxy_pds;
 
+	ret = qcom_init_firmware(qproc);
+	if (ret)
+		goto detach_proxy_pds;
+
 	qproc->mpss_perm = BIT(QCOM_SCM_VMID_HLOS);
 	qproc->mba_perm = BIT(QCOM_SCM_VMID_HLOS);
 	qcom_add_glink_subdev(rproc, &qproc->glink_subdev);

@@ -1759,6 +1826,7 @@ static int q6v5_remove(struct platform_device *pdev)
 	qcom_remove_glink_subdev(qproc->rproc, &qproc->glink_subdev);
 	qcom_remove_smd_subdev(qproc->rproc, &qproc->smd_subdev);
 	qcom_remove_ssr_subdev(qproc->rproc, &qproc->ssr_subdev);
+	qcom_remove_firmware(qproc);
 
 	q6v5_pds_detach(qproc, qproc->active_pds, qproc->active_pd_count);
 	q6v5_pds_detach(qproc, qproc->proxy_pds, qproc->proxy_pd_count);
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
maintainer
#4 lore raw
In reply to Sibi Sankar
Quoted 6 lines · Sibi Sankar

The Q6 modem sub-system has direct access to DDR through memnoc and an indirect access routed through a SMMU which MSS CE (crypto engine sub-component of MSS) uses during out of reset sequence. Request direct mapping for the modem-firmware subdevice since smmu is not expected to provide access control/translation for these SIDs (sandboxing of the modem is achieved through XPUs engaged using SMC calls).

So the DMA accesses are initiated by the firmware and need to be direct mapped? Which memory region do they access?

Regards,

Joerg

author
#5 lore raw

Hey Joerg,
Thanks for taking time to review
the series!

Quoted 10 lines · Joerg Roedel
Quoted 6 lines · Sibi Sankar

The Q6 modem sub-system has direct access to DDR through memnoc and an indirect access routed through a SMMU which MSS CE (crypto engine sub-component of MSS) uses during out of reset sequence. Request direct mapping for the modem-firmware subdevice since smmu is not expected to provide access control/translation for these SIDs (sandboxing of the modem is achieved through XPUs engaged using SMC calls).

So the DMA accesses are initiated by the firmware and need to be direct mapped? Which memory region do they access?

The accesses are initiated by the firmware and they access modem reserved regions. However as explained in ^^ any accesses outside the region will result in a violation and is controlled through XPUs (protection units).

With ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT y
the firmware SIDs will explicitly need to
be directly mapped to avoid observing
global faults in the absence of secure
firmware (TrustZone) programming the modem
SIDs as S2CR-Bypass.

Quoted 4 lines

Regards,

Joerg

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
maintainer
#6 lore raw
Quoted 5 lines · Sibi Sankar

The accesses are initiated by the firmware and they access modem reserved regions. However as explained in ^^ any accesses outside the region will result in a violation and is controlled through XPUs (protection units).

Okay, this sounds like a case for arm_smmu_get_resv_region(). It should return an entry for the reserved memory region the firmware needs to access, so that generic iommu can setup this mapping.

Note that it should return that entry only for your device, not for all devices. Maybe there is a property in DT or IORT you can set to transport this information into the arm-smmu driver.

This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or Unity-mapped ranges in the AMD-Vi IOMMU.

Regards,

Joerg

reviewer
#7 lore raw
Quoted 11 lines · Joerg Roedel
Quoted 5 lines · Sibi Sankar

The accesses are initiated by the firmware and they access modem reserved regions. However as explained in ^^ any accesses outside the region will result in a violation and is controlled through XPUs (protection units).

Okay, this sounds like a case for arm_smmu_get_resv_region(). It should return an entry for the reserved memory region the firmware needs to access, so that generic iommu can setup this mapping.

Quoted 6 lines

Note that it should return that entry only for your device, not for all devices. Maybe there is a property in DT or IORT you can set to transport this information into the arm-smmu driver.

This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or Unity-mapped ranges in the AMD-Vi IOMMU.

Yup, a way to describe boot-time memory regions in IORT is in the process of being specced out; the first attempt at an equivalent for DT is here:

https://lore.kernel.org/linux-iommu/[email protected]/

If that's not enough and the SMMU still needs to treat certain Stream IDs specially because they may be untranslatable (due to having direct access to memory as a side-channel), then that should be handled in the SoC-specific corner of the SMMU driver, not delegated to individual endpoint drivers.

Robin.

#8 lore raw

Hi Robin,

Quoted 23 lines · Robin Murphy
Quoted 11 lines · Joerg Roedel
Quoted 5 lines · Sibi Sankar

The accesses are initiated by the firmware and they access modem reserved regions. However as explained in ^^ any accesses outside the region will result in a violation and is controlled through XPUs (protection units).

Okay, this sounds like a case for arm_smmu_get_resv_region(). It should return an entry for the reserved memory region the firmware needs to access, so that generic iommu can setup this mapping.

Quoted 6 lines

Note that it should return that entry only for your device, not for all devices. Maybe there is a property in DT or IORT you can set to transport this information into the arm-smmu driver.

This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or Unity-mapped ranges in the AMD-Vi IOMMU.

Yup, a way to describe boot-time memory regions in IORT is in the process of being specced out; the first attempt at an equivalent for DT is here:

Quoted 8 lines

https://lore.kernel.org/linux-iommu/[email protected]/

If that's not enough and the SMMU still needs to treat certain Stream IDs specially because they may be untranslatable (due to having direct access to memory as a side-channel), then that should be handled in the SoC-specific corner of the SMMU driver, not delegated to individual endpoint drivers.

Are you talking about this one for SoC specific change - https://lore.kernel.org/patchwork/patch/1183530/

Thanks,
Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
reviewer
#9 lore raw
Quoted 29 lines · Sai Prakash Ranjan

Hi Robin,

Quoted 26 lines · Robin Murphy
Quoted 11 lines · Joerg Roedel
Quoted 5 lines · Sibi Sankar

The accesses are initiated by the firmware and they access modem reserved regions. However as explained in ^^ any accesses outside the region will result in a violation and is controlled through XPUs (protection units).

Okay, this sounds like a case for arm_smmu_get_resv_region(). It should return an entry for the reserved memory region the firmware needs to access, so that generic iommu can setup this mapping.

Quoted 6 lines

Note that it should return that entry only for your device, not for all devices. Maybe there is a property in DT or IORT you can set to transport this information into the arm-smmu driver.

This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or Unity-mapped ranges in the AMD-Vi IOMMU.

Yup, a way to describe boot-time memory regions in IORT is in the process of being specced out; the first attempt at an equivalent for DT is here:

https://lore.kernel.org/linux-iommu/[email protected]/

Quoted 8 lines
Quoted 6 lines

If that's not enough and the SMMU still needs to treat certain Stream IDs specially because they may be untranslatable (due to having direct access to memory as a side-channel), then that should be handled in the SoC-specific corner of the SMMU driver, not delegated to individual endpoint drivers.

Are you talking about this one for SoC specific change - https://lore.kernel.org/patchwork/patch/1183530/

Exactly - this particular wheel needs no reinventing at all.

[ I guess I should go review those patches properly... :) ]

Robin.

#10 lore raw

Hi Robin,

Quoted 10 lines · Robin Murphy
Quoted 4 lines · Sai Prakash Ranjan

Hi Robin,

Are you talking about this one for SoC specific change - https://lore.kernel.org/patchwork/patch/1183530/

Exactly - this particular wheel needs no reinventing at all.

[ I guess I should go review those patches properly... :) ]

It would be great if you could review the patch - https://lore.kernel.org/patchwork/patch/1183530/
Sibi has posted a v2 of this series based on that patch.

Thanks,
Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
#11 lore raw
In reply to Sibi Sankar
Quoted 6 lines · Sibi Sankar

The Q6 modem sub-system has direct access to DDR through memnoc and an indirect access routed through a SMMU which MSS CE (crypto engine sub-component of MSS) uses during out of reset sequence. Request direct mapping for the modem-firmware subdevice since smmu is not expected to provide access control/translation for these SIDs (sandboxing of the modem is achieved through XPUs engaged using SMC calls).

Please fix your device tree so that the device isn't bound to an IOMMU.

author
#12 lore raw

Hey Christoph,
Thanks for taking time to review
the series.

Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Sibi Sankar

The Q6 modem sub-system has direct access to DDR through memnoc and an indirect access routed through a SMMU which MSS CE (crypto engine sub-component of MSS) uses during out of reset sequence. Request direct mapping for the modem-firmware subdevice since smmu is not expected to provide access control/translation for these SIDs (sandboxing of the modem is achieved through XPUs engaged using SMC calls).

Please fix your device tree so that the device isn't bound to an IOMMU.

the bindings proposed in the series
would add a sub-device with an iommu
property.

modem_pil: remoteproc@xxxxx {
...

   modem-firmware {
        iommus = <&apps_smmu 0x460 0x1>;

};
...
};

Remoteproc device will not have a iommu
property but modem-firmware sub-device
will.

With ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT y,
we would want to configure the SID either
in direct mapping or bypass (either will
do since protection is achieved through
other means)

https://lore.kernel.org/lkml/[email protected]/

Currently the restructuring is trending
towards whats discussed in the ^^ thread.
i.e either direct mapping/bypass will be
done in the SoC specific corner of the
SMMU driver.

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.