Skip to contentSkip to conversation

iommu/omap: misc fixes

PATCH In review Tero Kristo via iommu 2 participants 12 messages Aug 7, 2019 – Aug 9, 2019

Subsystems (4)
lore
Share & tools
lore
#1 lore raw

PATCH 1/8 iommu/omap: fix boot issue on remoteprocs with AMMU/Unicache

From: Suman Anna <[email protected]>

Support has been added to the OMAP IOMMU driver to fix a boot hang issue on OMAP remoteprocs with AMMU/Unicache, caused by an improper AMMU/Unicache state upon initial deassertion of the processor reset. The issue is described in detail in the next three paragraphs.

All the Cortex M3/M4 IPU processor subsystems in OMAP SoCs have a AMMU/Unicache IP that dictates the memory attributes for addresses seen by the processor cores. The AMMU/Unicache is configured/enabled by the SCACHE_CONFIG.BYPASS bit - a value of 1 enables the cache and mandates all addresses accessed by M3/M4 be defined in the AMMU. This bit is not programmable from the host processor. The M3/M4 boot sequence starts out with the AMMU/Unicache in disabled state, and SYS/BIOS programs the AMMU regions and enables the Unicache during one of its initial boot steps. This SCACHE_CONFIG.BYPASS bit is however enabled by default whenever a RET reset is applied to the IP, irrespective of whether it was previously enabled or not. The AMMU registers lose their context whenever this reset is applied. The reset is effective as long as the MMU portion of the subsystem is enabled and clocked. This behavior is common to all the IPU and DSP subsystems that have an AMMU/Unicache.

The IPU boot sequence involves enabling and programming the MMU, and loading the processor and releasing the reset(s) for the processor. The PM setup code currently sets the target state for most of the power domains to RET. The L2 MMU can be enabled, programmed and accessed properly just fine with the domain in hardware supervised mode, while the power domain goes through a RET->ON->RET transition during the programming sequence. However, the ON->RET transition asserts a RET reset, and the SCACHE_CONFIG.BYPASS bit gets auto-set. An AMMU fault is thrown immediately when the M3/M4 core's reset is released since the first instruction address itself will not be defined in any valid AMMU regions. The ON->RET transition happens automatically on the power domain after enabling the iommu due to the hardware supervised mode.

This patch adds and invokes the .set_pwrdm_constraint pdata ops, if present, during the OMAP IOMMU enable and disable functions to resolve the above boot hang issue. The ops will allow to invoke a mach-omap2 layer API pwrdm_set_next_pwrst() in a multi-arch kernel environment. The ops also returns the current power domain state while enforcing the constraint so that the driver can store it and use it to set back the power domain state while releasing the constraint. The pdata ops implementation restricts the target power domain to ON during enable, and back to the original power domain state during disable, and thereby eliminating the conditions for the boot issue. The implementation is effective only when the original power domain state is either RET or OFF, and is a no-op when it is ON or INACTIVE.

The .set_pwrdm_constraint ops need to be plugged in pdata-quirks for the affected remote processors to be able to boot properly.

Note that the current issue is seen only on kernels with the affected power domains programmed to enter RET. For eg., IPU1 on DRA7xx is in a separate domain and is susceptible to this bug, while the IPU2 subsystem is within CORE power domain, and CORE RET is not supported on this SoC. IPUs on OMAP4 and OMAP5 are also susceptible since they are in CORE power domain, and CORE RET is a valid power target on these SoCs.

---
 arch/arm/mach-omap2/Makefile             |  2 ++
 arch/arm/mach-omap2/omap-iommu.c         | 51 ++++++++++++++++++++++++++++++++
 drivers/iommu/omap-iommu.c               | 17 +++++++++++
 drivers/iommu/omap-iommu.h               |  2 ++
 include/linux/platform_data/iommu-omap.h |  2 ++
 5 files changed, 74 insertions(+)
 create mode 100644 arch/arm/mach-omap2/omap-iommu.c
arch/arm/mach-omap2/Makefile +2 -0
index 6006505..d4f11c5 100644
@@ -229,3 +229,5 @@ include/generated/ti-pm-asm-offsets.h: arch/arm/mach-omap2/pm-asm-offsets.s FORC
 $(obj)/sleep33xx.o $(obj)/sleep43xx.o: include/generated/ti-pm-asm-offsets.h
 
 targets += pm-asm-offsets.s
+
+obj-$(CONFIG_OMAP_IOMMU)		+= omap-iommu.o
arch/arm/mach-omap2/omap-iommu.c new file +43 -0

43 lines not on this page · raw message

drivers/iommu/omap-iommu.c +17 -0
index dfb961d..84b99d5 100644
@@ -190,6 +190,14 @@ static int iommu_enable(struct omap_iommu *obj)
 	struct platform_device *pdev = to_platform_device(obj->dev);
 	struct iommu_platform_data *pdata = dev_get_platdata(&pdev->dev);
 
+	if (pdata && pdata->set_pwrdm_constraint) {
+		err = pdata->set_pwrdm_constraint(pdev, true, &obj->pwrst);
+		if (err) {
+			dev_warn(obj->dev, "pwrdm_constraint failed to be set, status = %d\n",
+				 err);
+		}
+	}
+
 	if (pdata && pdata->deassert_reset) {
 		err = pdata->deassert_reset(pdev, pdata->reset_name);
 		if (err) {
@@ -209,6 +217,7 @@ static void iommu_disable(struct omap_iommu *obj)
 {
 	struct platform_device *pdev = to_platform_device(obj->dev);
 	struct iommu_platform_data *pdata = dev_get_platdata(&pdev->dev);
+	int ret;
 
 	omap2_iommu_disable(obj);
 
@@ -216,6 +225,14 @@ static void iommu_disable(struct omap_iommu *obj)
 
 	if (pdata && pdata->assert_reset)
 		pdata->assert_reset(pdev, pdata->reset_name);
+
+	if (pdata && pdata->set_pwrdm_constraint) {
+		ret = pdata->set_pwrdm_constraint(pdev, false, &obj->pwrst);
+		if (ret) {
+			dev_warn(obj->dev, "pwrdm_constraint failed to be reset, status = %d\n",
+				 ret);
+		}
+	}
 }
 
 /*
drivers/iommu/omap-iommu.h +2 -0
index 09968a0..aac1ca6 100644
@@ -78,6 +78,8 @@ struct omap_iommu {
 
 	struct iommu_device iommu;
 	struct iommu_group *group;
+
+	u8 pwrst;
 };
 
 /**
include/linux/platform_data/iommu-omap.h +2 -0
index 44d913a..1ed6026 100644
@@ -13,4 +13,6 @@ struct iommu_platform_data {
 	const char *reset_name;
 	int (*assert_reset)(struct platform_device *pdev, const char *name);
 	int (*deassert_reset)(struct platform_device *pdev, const char *name);
+	int (*set_pwrdm_constraint)(struct platform_device *pdev, bool request,
+				    u8 *pwrst);
 };
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#2 lore raw

PATCH 2/8 iommu/omap: add pdata ops for omap_device_enable/idle

In reply to Tero Kristo via iommu

From: Suman Anna <[email protected]>

Add two new platform data ops to allow the OMAP iommu driver to be able to invoke the omap_device_enable and omap_device_idle from within the driver. These are being added to streamline the sequence between managing the hard reset lines and the clocks during the suspend path, as the default device pm_domain callback sequences in omap_device layer are not conducive for the OMAP IOMMU driver.

This could have been done by expanding the existing pdata ops for reset management (like in the OMAP remoteproc driver), but this was chosen to avoid adding additional code in the separate file in the mach-omap2 layer.

---
 include/linux/platform_data/iommu-omap.h | 2 ++
 1 file changed, 2 insertions(+)
include/linux/platform_data/iommu-omap.h +2 -0
index 1ed6026..8474a02 100644
@@ -13,6 +13,8 @@ struct iommu_platform_data {
 	const char *reset_name;
 	int (*assert_reset)(struct platform_device *pdev, const char *name);
 	int (*deassert_reset)(struct platform_device *pdev, const char *name);
+	int (*device_enable)(struct platform_device *pdev);
+	int (*device_idle)(struct platform_device *pdev);
 	int (*set_pwrdm_constraint)(struct platform_device *pdev, bool request,
 				    u8 *pwrst);
 };
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#3 lore raw

PATCH 3/8 iommu/omap: streamline enable/disable through runtime pm callbacks

In reply to Tero Kristo via iommu

From: Suman Anna <[email protected]>

The OMAP IOMMU devices are typically present within the respective client processor subsystem and have their own dedicated hard-reset line. Enabling an IOMMU requires the reset line to be deasserted and the clocks to be enabled before programming the necessary IOMMU registers. The IOMMU disable sequence follow the reverse order of enabling. The OMAP IOMMU driver programs the reset lines through pdata ops to invoke the omap_device_assert/deassert_hardreset API. The clocks are managed through the pm_runtime framework, and the callbacks associated with the device's pm_domain, implemented in the omap_device layer.

Streamline the enable and disable sequences in the OMAP IOMMU driver by implementing all the above operations within the runtime pm callbacks. All the OMAP devices have device pm_domain callbacks plugged in the omap_device layer for automatic runtime management of the clocks. Invoking the reset management functions within the runtime pm callbacks in OMAP IOMMU driver therefore requires that the default device's pm domain callbacks in the omap_device layer be reset, as the ordering sequence for managing the reset lines and clocks from the pm_domain callbacks don't gel well with the implementation in the IOMMU driver callbacks. The omap_device_enable/omap_device_idle functions are invoked through the newly added pdata ops.

Consolidating all the device management sequences within the runtime pm callbacks allows the driver to easily support both system suspend/resume and runtime suspend/resume using common code.

---
 drivers/iommu/omap-iommu.c | 139 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 99 insertions(+), 40 deletions(-)
drivers/iommu/omap-iommu.c +99 -40
index 84b99d5..fbceae3 100644
@@ -186,53 +186,18 @@ static void omap2_iommu_disable(struct omap_iommu *obj)
 
 static int iommu_enable(struct omap_iommu *obj)
 {
-	int err;
-	struct platform_device *pdev = to_platform_device(obj->dev);
-	struct iommu_platform_data *pdata = dev_get_platdata(&pdev->dev);
-
-	if (pdata && pdata->set_pwrdm_constraint) {
-		err = pdata->set_pwrdm_constraint(pdev, true, &obj->pwrst);
-		if (err) {
-			dev_warn(obj->dev, "pwrdm_constraint failed to be set, status = %d\n",
-				 err);
-		}
-	}
-
-	if (pdata && pdata->deassert_reset) {
-		err = pdata->deassert_reset(pdev, pdata->reset_name);
-		if (err) {
-			dev_err(obj->dev, "deassert_reset failed: %d\n", err);
-			return err;
-		}
-	}
-
-	pm_runtime_get_sync(obj->dev);
+	int ret;
 
-	err = omap2_iommu_enable(obj);
+	ret = pm_runtime_get_sync(obj->dev);
+	if (ret < 0)
+		pm_runtime_put_noidle(obj->dev);
 
-	return err;
+	return ret < 0 ? ret : 0;
 }
 
 static void iommu_disable(struct omap_iommu *obj)
 {
-	struct platform_device *pdev = to_platform_device(obj->dev);
-	struct iommu_platform_data *pdata = dev_get_platdata(&pdev->dev);
-	int ret;
-
-	omap2_iommu_disable(obj);
-
 	pm_runtime_put_sync(obj->dev);
-
-	if (pdata && pdata->assert_reset)
-		pdata->assert_reset(pdev, pdata->reset_name);
-
-	if (pdata && pdata->set_pwrdm_constraint) {
-		ret = pdata->set_pwrdm_constraint(pdev, false, &obj->pwrst);
-		if (ret) {
-			dev_warn(obj->dev, "pwrdm_constraint failed to be reset, status = %d\n",
-				 ret);
-		}
-	}
 }
 
 /*
@@ -927,6 +892,85 @@ static void omap_iommu_detach(struct omap_iommu *obj)
 	dev_dbg(obj->dev, "%s: %s\n", __func__, obj->name);
 }
 
+/**
+ * omap_iommu_runtime_suspend - disable an iommu device
+ * @dev:	iommu device
+ *
+ * This function performs all that is necessary to disable an
+ * IOMMU device, either during final detachment from a client
+ * device, or during system/runtime suspend of the device. This
+ * includes programming all the appropriate IOMMU registers, and
+ * managing the associated omap_hwmod's state and the device's
+ * reset line.
+ **/
+static int omap_iommu_runtime_suspend(struct device *dev)
+{
+	struct platform_device *pdev = to_platform_device(dev);
+	struct iommu_platform_data *pdata = dev_get_platdata(dev);
+	struct omap_iommu *obj = to_iommu(dev);
+	int ret;
+
+	omap2_iommu_disable(obj);
+
+	if (pdata && pdata->device_idle)
+		pdata->device_idle(pdev);
+
+	if (pdata && pdata->assert_reset)
+		pdata->assert_reset(pdev, pdata->reset_name);
+
+	if (pdata && pdata->set_pwrdm_constraint) {
+		ret = pdata->set_pwrdm_constraint(pdev, false, &obj->pwrst);
+		if (ret) {
+			dev_warn(obj->dev, "pwrdm_constraint failed to be reset, status = %d\n",
+				 ret);
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * omap_iommu_runtime_resume - enable an iommu device
+ * @dev:	iommu device
+ *
+ * This function performs all that is necessary to enable an
+ * IOMMU device, either during initial attachment to a client
+ * device, or during system/runtime resume of the device. This
+ * includes programming all the appropriate IOMMU registers, and
+ * managing the associated omap_hwmod's state and the device's
+ * reset line.
+ **/
+static int omap_iommu_runtime_resume(struct device *dev)
+{
+	struct platform_device *pdev = to_platform_device(dev);
+	struct iommu_platform_data *pdata = dev_get_platdata(dev);
+	struct omap_iommu *obj = to_iommu(dev);
+	int ret = 0;
+
+	if (pdata && pdata->set_pwrdm_constraint) {
+		ret = pdata->set_pwrdm_constraint(pdev, true, &obj->pwrst);
+		if (ret) {
+			dev_warn(obj->dev, "pwrdm_constraint failed to be set, status = %d\n",
+				 ret);
+		}
+	}
+
+	if (pdata && pdata->deassert_reset) {
+		ret = pdata->deassert_reset(pdev, pdata->reset_name);
+		if (ret) {
+			dev_err(dev, "deassert_reset failed: %d\n", ret);
+			return ret;
+		}
+	}
+
+	if (pdata && pdata->device_enable)
+		pdata->device_enable(pdev);
+
+	ret = omap2_iommu_enable(obj);
+
+	return ret;
+}
+
 static bool omap_iommu_can_register(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node;
@@ -1001,6 +1045,15 @@ static int omap_iommu_probe(struct platform_device *pdev)
 	if (!obj)
 		return -ENOMEM;
 
+	/*
+	 * self-manage the ordering dependencies between omap_device_enable/idle
+	 * and omap_device_assert/deassert_hardreset API
+	 */
+	if (pdev->dev.pm_domain) {
+		dev_dbg(&pdev->dev, "device pm_domain is being reset\n");
+		pdev->dev.pm_domain = NULL;
+	}
+
 	obj->name = dev_name(&pdev->dev);
 	obj->nr_tlb_entries = 32;
 	err = of_property_read_u32(of, "ti,#tlb-entries", &obj->nr_tlb_entries);
@@ -1089,6 +1142,11 @@ static int omap_iommu_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct dev_pm_ops omap_iommu_pm_ops = {
+	SET_RUNTIME_PM_OPS(omap_iommu_runtime_suspend,
+			   omap_iommu_runtime_resume, NULL)
+};
+
 static const struct of_device_id omap_iommu_of_match[] = {
 	{ .compatible = "ti,omap2-iommu" },
 	{ .compatible = "ti,omap4-iommu" },
@@ -1102,6 +1160,7 @@ static int omap_iommu_remove(struct platform_device *pdev)
 	.remove	= omap_iommu_remove,
 	.driver	= {
 		.name	= "omap-iommu",
+		.pm	= &omap_iommu_pm_ops,
 		.of_match_table = of_match_ptr(omap_iommu_of_match),
 	},
 };
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#4 lore raw

PATCH 4/8 iommu/omap: add logic to save/restore locked TLBs

In reply to Tero Kristo via iommu

From: Suman Anna <[email protected]>

The MMUs provide a mechanism to lock TLB entries to avoid eviction and fetching of frequently used page table entries. These TLBs lose context when the MMUs are turned OFF. Add the logic to save and restore these locked TLBS during suspend and resume respectively. There are no locked TLBs during initial power ON, and they need not be saved during final shutdown.

---
 drivers/iommu/omap-iommu.c | 61 +++++++++++++++++++++++++++++++++++++++++++---
 drivers/iommu/omap-iommu.h |  3 +++
 2 files changed, 61 insertions(+), 3 deletions(-)
drivers/iommu/omap-iommu.c +58 -3
index fbceae3..51bf9d7 100644
@@ -883,15 +883,55 @@ static void omap_iommu_detach(struct omap_iommu *obj)
 
 	dma_unmap_single(obj->dev, obj->pd_dma, IOPGD_TABLE_SIZE,
 			 DMA_TO_DEVICE);
-	iommu_disable(obj);
 	obj->pd_dma = 0;
 	obj->iopgd = NULL;
+	iommu_disable(obj);
 
 	spin_unlock(&obj->iommu_lock);
 
 	dev_dbg(obj->dev, "%s: %s\n", __func__, obj->name);
 }
 
+static void omap_iommu_save_tlb_entries(struct omap_iommu *obj)
+{
+	struct iotlb_lock lock;
+	struct cr_regs cr;
+	struct cr_regs *tmp;
+	int i;
+
+	/* check if there are any locked tlbs to save */
+	iotlb_lock_get(obj, &lock);
+	obj->num_cr_ctx = lock.base;
+	if (!obj->num_cr_ctx)
+		return;
+
+	tmp = obj->cr_ctx;
+	for_each_iotlb_cr(obj, obj->num_cr_ctx, i, cr)
+		* tmp++ = cr;
+}
+
+static void omap_iommu_restore_tlb_entries(struct omap_iommu *obj)
+{
+	struct iotlb_lock l;
+	struct cr_regs *tmp;
+	int i;
+
+	/* no locked tlbs to restore */
+	if (!obj->num_cr_ctx)
+		return;
+
+	l.base = 0;
+	tmp = obj->cr_ctx;
+	for (i = 0; i < obj->num_cr_ctx; i++, tmp++) {
+		l.vict = i;
+		iotlb_lock_set(obj, &l);
+		iotlb_load_cr(obj, tmp);
+	}
+	l.base = obj->num_cr_ctx;
+	l.vict = i;
+	iotlb_lock_set(obj, &l);
+}
+
 /**
  * omap_iommu_runtime_suspend - disable an iommu device
  * @dev:	iommu device
@@ -901,7 +941,8 @@ static void omap_iommu_detach(struct omap_iommu *obj)
  * device, or during system/runtime suspend of the device. This
  * includes programming all the appropriate IOMMU registers, and
  * managing the associated omap_hwmod's state and the device's
- * reset line.
+ * reset line. This function also saves the context of any
+ * locked TLBs if suspending.
  **/
 static int omap_iommu_runtime_suspend(struct device *dev)
 {
@@ -910,6 +951,10 @@ static int omap_iommu_runtime_suspend(struct device *dev)
 	struct omap_iommu *obj = to_iommu(dev);
 	int ret;
 
+	/* save the TLBs only during suspend, and not for power down */
+	if (obj->domain && obj->iopgd)
+		omap_iommu_save_tlb_entries(obj);
+
 	omap2_iommu_disable(obj);
 
 	if (pdata && pdata->device_idle)
@@ -938,7 +983,8 @@ static int omap_iommu_runtime_suspend(struct device *dev)
  * device, or during system/runtime resume of the device. This
  * includes programming all the appropriate IOMMU registers, and
  * managing the associated omap_hwmod's state and the device's
- * reset line.
+ * reset line. The function also restores any locked TLBs if
+ * resuming after a suspend.
  **/
 static int omap_iommu_runtime_resume(struct device *dev)
 {
@@ -966,6 +1012,10 @@ static int omap_iommu_runtime_resume(struct device *dev)
 	if (pdata && pdata->device_enable)
 		pdata->device_enable(pdev);
 
+	/* restore the TLBs only during resume, and not for power up */
+	if (obj->domain)
+		omap_iommu_restore_tlb_entries(obj);
+
 	ret = omap2_iommu_enable(obj);
 
 	return ret;
@@ -1066,6 +1116,11 @@ static int omap_iommu_probe(struct platform_device *pdev)
 
 	obj->dev = &pdev->dev;
 	obj->ctx = (void *)obj + sizeof(*obj);
+	obj->cr_ctx = devm_kzalloc(&pdev->dev,
+				   sizeof(*obj->cr_ctx) * obj->nr_tlb_entries,
+				   GFP_KERNEL);
+	if (!obj->cr_ctx)
+		return -ENOMEM;
 
 	spin_lock_init(&obj->iommu_lock);
 	spin_lock_init(&obj->page_table_lock);
drivers/iommu/omap-iommu.h +3 -0
index aac1ca6..1d15aa8 100644
@@ -73,6 +73,9 @@ struct omap_iommu {
 
 	void *ctx; /* iommu context: registres saved area */
 
+	struct cr_regs *cr_ctx;
+	u32 num_cr_ctx;
+
 	int has_bus_err_back;
 	u32 id;
 
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#5 lore raw

PATCH 5/8 iommu/omap: Add system suspend/resume support

In reply to Tero Kristo via iommu

From: Suman Anna <[email protected]>

The MMU registers for the remote processors lose their context in Open Switch Retention (OSWR) or device OFF modes. Hence, the context of the IOMMU needs to be saved before it is put into any of these lower power state (OSWR/OFF) and restored before it is powered up to ON again. The IOMMUs need to be active as long as the client devices that are present behind the IOMMU are active.

This patch adds the dev_pm_ops callbacks to provide the system suspend/resume functionality through the appropriate runtime PM callbacks. The PM runtime_resume and runtime_suspend callbacks are already used to enable, configure and disable the IOMMUs during the attaching and detaching of the client devices to the IOMMUs, and the new PM callbacks reuse the same code by invoking the pm_runtime_force_suspend() and pm_runtime_force_resume() API. The functionality in dev_pm_ops .prepare() checks if the IOMMU device was already runtime suspended, and skips invoking the suspend/resume PM callbacks. The suspend/resume PM callbacks are plugged in through the 'late' pm ops to ensure that the IOMMU devices will be suspended only after its master devices (remoteproc devices) are suspended and restored before them.

NOTE:
There are two other existing API, omap_iommu_save_ctx() and omap_iommu_restore_ctx(). These are left as is to support suspend/resume of devices on legacy OMAP3 SoC.

---
 drivers/iommu/omap-iommu.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
drivers/iommu/omap-iommu.c +26 -0
index 51bf9d7..694cd38 100644
@@ -65,6 +65,9 @@ static struct omap_iommu_domain *to_omap_domain(struct iommu_domain *dom)
 /**
  * omap_iommu_save_ctx - Save registers for pm off-mode support
  * @dev:	client device
+ *
+ * This should be treated as an deprecated API. It is preserved only
+ * to maintain existing functionality for OMAP3 ISP driver.
  **/
 void omap_iommu_save_ctx(struct device *dev)
 {
@@ -92,6 +95,9 @@ void omap_iommu_save_ctx(struct device *dev)
 /**
  * omap_iommu_restore_ctx - Restore registers for pm off-mode support
  * @dev:	client device
+ *
+ * This should be treated as an deprecated API. It is preserved only
+ * to maintain existing functionality for OMAP3 ISP driver.
  **/
 void omap_iommu_restore_ctx(struct device *dev)
 {
@@ -1021,6 +1027,23 @@ static int omap_iommu_runtime_resume(struct device *dev)
 	return ret;
 }
 
+/**
+ * omap_iommu_suspend_prepare - prepare() dev_pm_ops implementation
+ * @dev:	iommu device
+ *
+ * This function performs the necessary checks to determine if the IOMMU
+ * device needs suspending or not. The function checks if the runtime_pm
+ * status of the device is suspended, and returns 1 in that case. This
+ * results in the PM core to skip invoking any of the Sleep PM callbacks
+ * (suspend, suspend_late, resume, resume_early etc).
+ */
+static int omap_iommu_prepare(struct device *dev)
+{
+	if (pm_runtime_status_suspended(dev))
+		return 1;
+	return 0;
+}
+
 static bool omap_iommu_can_register(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node;
@@ -1198,6 +1221,9 @@ static int omap_iommu_remove(struct platform_device *pdev)
 }
 
 static const struct dev_pm_ops omap_iommu_pm_ops = {
+	.prepare = omap_iommu_prepare,
+	SET_LATE_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+				     pm_runtime_force_resume)
 	SET_RUNTIME_PM_OPS(omap_iommu_runtime_suspend,
 			   omap_iommu_runtime_resume, NULL)
 };
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#6 lore raw

PATCH 6/8 iommu/omap: introduce new API for runtime suspend/resume control

In reply to Tero Kristo via iommu

From: Suman Anna <[email protected]>

This patch adds the support for the OMAP IOMMUs to be suspended during the auto suspend/resume of the OMAP remoteproc devices. The remote processors are auto suspended after a certain time of idle or inactivity period. This is done by introducing two new API, omap_iommu_domain_deactivate() and omap_iommu_domain_activate() to allow the client users/master devices of the IOMMU devices to deactivate & activate the IOMMU devices from their runtime suspend/resume operations. There is no API exposed by the IOMMU layer at present, and so these new API are added directly in the OMAP IOMMU driver to minimize framework changes.

The API simply decrements and increments the runtime usage count of the IOMMU devices and let the context be saved/restored using the existing runtime pm callbacks.

---
 drivers/iommu/omap-iommu.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/omap-iommu.h |  8 +++++++
 2 files changed, 66 insertions(+)
drivers/iommu/omap-iommu.c +58 -0
index 694cd38..5e0dfca 100644
@@ -939,6 +939,64 @@ static void omap_iommu_restore_tlb_entries(struct omap_iommu *obj)
 }
 
 /**
+ * omap_iommu_domain_deactivate - deactivate attached iommu devices
+ * @domain: iommu domain attached to the target iommu device
+ *
+ * This API allows the client devices of IOMMU devices to suspend
+ * the IOMMUs they control at runtime, after they are idled and
+ * suspended all activity. System Suspend will leverage the PM
+ * driver late callbacks.
+ **/
+int omap_iommu_domain_deactivate(struct iommu_domain *domain)
+{
+	struct omap_iommu_domain *omap_domain = to_omap_domain(domain);
+	struct omap_iommu_device *iommu;
+	struct omap_iommu *oiommu;
+	int i;
+
+	if (!omap_domain->dev)
+		return 0;
+
+	iommu = omap_domain->iommus;
+	iommu += (omap_domain->num_iommus - 1);
+	for (i = 0; i < omap_domain->num_iommus; i++, iommu--) {
+		oiommu = iommu->iommu_dev;
+		pm_runtime_put_sync(oiommu->dev);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(omap_iommu_domain_deactivate);
+
+/**
+ * omap_iommu_domain_activate - activate attached iommu devices
+ * @domain: iommu domain attached to the target iommu device
+ *
+ * This API allows the client devices of IOMMU devices to resume the
+ * IOMMUs they control at runtime, before they can resume operations.
+ * System Resume will leverage the PM driver late callbacks.
+ **/
+int omap_iommu_domain_activate(struct iommu_domain *domain)
+{
+	struct omap_iommu_domain *omap_domain = to_omap_domain(domain);
+	struct omap_iommu_device *iommu;
+	struct omap_iommu *oiommu;
+	int i;
+
+	if (!omap_domain->dev)
+		return 0;
+
+	iommu = omap_domain->iommus;
+	for (i = 0; i < omap_domain->num_iommus; i++, iommu++) {
+		oiommu = iommu->iommu_dev;
+		pm_runtime_get_sync(oiommu->dev);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(omap_iommu_domain_activate);
+
+/**
  * omap_iommu_runtime_suspend - disable an iommu device
  * @dev:	iommu device
  *
include/linux/omap-iommu.h +8 -0
index 153bf25..36b6457 100644
@@ -10,12 +10,20 @@
 #ifndef _OMAP_IOMMU_H_
 #define _OMAP_IOMMU_H_
 
+struct iommu_domain;
+
 #ifdef CONFIG_OMAP_IOMMU
 extern void omap_iommu_save_ctx(struct device *dev);
 extern void omap_iommu_restore_ctx(struct device *dev);
+
+int omap_iommu_domain_deactivate(struct iommu_domain *domain);
+int omap_iommu_domain_activate(struct iommu_domain *domain);
 #else
 static inline void omap_iommu_save_ctx(struct device *dev) {}
 static inline void omap_iommu_restore_ctx(struct device *dev) {}
+
+static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain) {}
+static inline int omap_iommu_domain_activate(struct iommu_domain *domain) {}
 #endif
 
 #endif
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
maintainer
#7 lore raw

PATCH 1/1 iommu/omap: Fix compilation warnings suspend/resume control

Review

Quoted 2 lines · Tero Kristo
(quoted diff) +2 -0
+static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain) {}
+static inline int omap_iommu_domain_activate(struct iommu_domain *domain) {}

This caused compilation warnings, I queued the fix below.

From 70c8a42adf482fba35e9e39ccd441a5ff8ffb657 Mon Sep 17 00:00:00 2001
From: Joerg Roedel <[email protected]>
Date: Fri, 9 Aug 2019 18:09:13 +0200
Subject: [PATCH] iommu/omap: Fix compilation warnings

A recent patches introduced a new compiler warning because two functions with non-void return type have no return statement in omap-iommu.h for CONFIG_OMAP_IOMMU=n.

Fix this by adding return statements to these functions.

Fixes d9c4d8a6cc0f8 ('iommu/omap: introduce new API for runtime suspend/resume control')
---
 include/linux/omap-iommu.h | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
include/linux/omap-iommu.h +9 -2
index 36b645726813..2c32ca09df02 100644
@@ -22,8 +22,15 @@ int omap_iommu_domain_activate(struct iommu_domain *domain);
 static inline void omap_iommu_save_ctx(struct device *dev) {}
 static inline void omap_iommu_restore_ctx(struct device *dev) {}
 
-static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain) {}
-static inline int omap_iommu_domain_activate(struct iommu_domain *domain) {}
+static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain)
+{
+	return -ENODEV;
+}
+
+static inline int omap_iommu_domain_activate(struct iommu_domain *domain)
+{
+	return -ENODEV;
+}
 #endif
 
 #endif
Signature · 6 lines
2.16.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#8 lore raw

Re: [PATCH] iommu/omap: Fix compilation warnings suspend/resume control

Quoted 11 lines · Joerg Roedel

Review

Quoted 2 lines · Tero Kristo
(quoted diff) +2 -0
+static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain) {}
+static inline int omap_iommu_domain_activate(struct iommu_domain *domain) {}

This caused compilation warnings, I queued the fix below.

From 70c8a42adf482fba35e9e39ccd441a5ff8ffb657 Mon Sep 17 00:00:00 2001
From: Joerg Roedel <[email protected]>
Date: Fri, 9 Aug 2019 18:09:13 +0200
Subject: [PATCH] iommu/omap: Fix compilation warnings

Quoted 5 lines

A recent patches introduced a new compiler warning because two functions with non-void return type have no return statement in omap-iommu.h for CONFIG_OMAP_IOMMU=n.

Fix this by adding return statements to these functions.

Thanks Joerg, looks like Tero missed them when he forward-ported the patches.

regards
Suman

Quoted 20 lines
Fixes d9c4d8a6cc0f8 ('iommu/omap: introduce new API for runtime suspend/resume control')
---
 include/linux/omap-iommu.h | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
include/linux/omap-iommu.h +2 -2
index 36b645726813..2c32ca09df02 100644
@@ -22,8 +22,15 @@ int omap_iommu_domain_activate(struct iommu_domain *domain);
 static inline void omap_iommu_save_ctx(struct device *dev) {}
 static inline void omap_iommu_restore_ctx(struct device *dev) {}
 
-static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain) {}
-static inline int omap_iommu_domain_activate(struct iommu_domain *domain) {}
+static inline int omap_iommu_domain_deactivate(struct iommu_domain *domain)
+{

include/linux/omap-iommu.h int omap_iommu_domain_activate(struct iommu_domain *domain); · line 36

Quoted 10 lines
include/linux/omap-iommu.h +7 -0
+	return -ENODEV;
+}
+
+static inline int omap_iommu_domain_activate(struct iommu_domain *domain)
+{
+	return -ENODEV;
+}
 #endif
 
 #endif

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#9 lore raw

PATCH 7/8 iommu/omap: add support for late attachment of iommu devices

In reply to Tero Kristo via iommu

Current implementation of OMAP IOMMU enforces strict ordering of device probe, initiated by iommu and followed by remoteproc later. This doesn't work too well with the new setup done with ti-sysc changes which may have the devices probed at pretty much any order. To overcome this limitation, if iommu has not been probed yet when a consumer tries to attach to it, add the device to orphan device list which will be parsed during iommu probe to see if any orphan devices should be attached.

---
 drivers/iommu/omap-iommu.c | 56 +++++++++++++++++++++++++++++++++++++++++++---
 drivers/iommu/omap-iommu.h |  4 +++-
 2 files changed, 56 insertions(+), 4 deletions(-)
drivers/iommu/omap-iommu.c +53 -3
index 5e0dfca..1beb318 100644
@@ -35,6 +35,15 @@
 
 static const struct iommu_ops omap_iommu_ops;
 
+struct orphan_dev {
+	struct device *dev;
+	struct list_head node;
+};
+
+static LIST_HEAD(orphan_dev_list);
+
+static DEFINE_SPINLOCK(orphan_lock);
+
 #define to_iommu(dev)	((struct omap_iommu *)dev_get_drvdata(dev))
 
 /* bitmap of the page sizes currently supported */
@@ -53,6 +62,8 @@
 static struct platform_driver omap_iommu_driver;
 static struct kmem_cache *iopte_cachep;
 
+static int _omap_iommu_add_device(struct device *dev);
+
 /**
  * to_omap_domain - Get struct omap_iommu_domain from generic iommu_domain
  * @dom:	generic iommu domain handle
@@ -1166,6 +1177,7 @@ static int omap_iommu_probe(struct platform_device *pdev)
 	struct omap_iommu *obj;
 	struct resource *res;
 	struct device_node *of = pdev->dev.of_node;
+	struct orphan_dev *orphan_dev, *tmp;
 
 	if (!of) {
 		pr_err("%s: only DT-based devices are supported\n", __func__);
@@ -1249,6 +1261,14 @@ static int omap_iommu_probe(struct platform_device *pdev)
 
 	dev_info(&pdev->dev, "%s registered\n", obj->name);
 
+	list_for_each_entry_safe(orphan_dev, tmp, &orphan_dev_list, node) {
+		err = _omap_iommu_add_device(orphan_dev->dev);
+		if (!err) {
+			list_del(&orphan_dev->node);
+			kfree(orphan_dev);
+		}
+	}
+
 	return 0;
 
 out_sysfs:
@@ -1638,7 +1658,7 @@ static phys_addr_t omap_iommu_iova_to_phys(struct iommu_domain *domain,
 	return ret;
 }
 
-static int omap_iommu_add_device(struct device *dev)
+static int _omap_iommu_add_device(struct device *dev)
 {
 	struct omap_iommu_arch_data *arch_data, *tmp;
 	struct omap_iommu *oiommu;
@@ -1647,6 +1667,8 @@ static int omap_iommu_add_device(struct device *dev)
 	struct platform_device *pdev;
 	int num_iommus, i;
 	int ret;
+	struct orphan_dev *orphan_dev;
+	unsigned long flags;
 
 	/*
 	 * Allocate the archdata iommu structure for DT-based devices.
@@ -1678,10 +1700,26 @@ static int omap_iommu_add_device(struct device *dev)
 		}
 
 		pdev = of_find_device_by_node(np);
-		if (WARN_ON(!pdev)) {
+		if (!pdev) {
 			of_node_put(np);
 			kfree(arch_data);
-			return -EINVAL;
+			spin_lock_irqsave(&orphan_lock, flags);
+			list_for_each_entry(orphan_dev, &orphan_dev_list,
+					    node) {
+				if (orphan_dev->dev == dev)
+					break;
+			}
+			spin_unlock_irqrestore(&orphan_lock, flags);
+
+			if (orphan_dev && orphan_dev->dev == dev)
+				return -EPROBE_DEFER;
+
+			orphan_dev = kzalloc(sizeof(*orphan_dev), GFP_KERNEL);
+			orphan_dev->dev = dev;
+			spin_lock_irqsave(&orphan_lock, flags);
+			list_add(&orphan_dev->node, &orphan_dev_list);
+			spin_unlock_irqrestore(&orphan_lock, flags);
+			return -EPROBE_DEFER;
 		}
 
 		oiommu = platform_get_drvdata(pdev);
@@ -1692,6 +1730,7 @@ static int omap_iommu_add_device(struct device *dev)
 		}
 
 		tmp->iommu_dev = oiommu;
+		tmp->dev = &pdev->dev;
 
 		of_node_put(np);
 	}
@@ -1726,6 +1765,17 @@ static int omap_iommu_add_device(struct device *dev)
 	return 0;
 }
 
+static int omap_iommu_add_device(struct device *dev)
+{
+	int ret;
+
+	ret = _omap_iommu_add_device(dev);
+	if (ret == -EPROBE_DEFER)
+		return 0;
+
+	return ret;
+}
+
 static void omap_iommu_remove_device(struct device *dev)
 {
 	struct omap_iommu_arch_data *arch_data = dev->archdata.iommu;
drivers/iommu/omap-iommu.h +3 -1
index 1d15aa8..18ee713 100644
@@ -87,7 +87,8 @@ struct omap_iommu {
 
 /**
  * struct omap_iommu_arch_data - omap iommu private data
- * @iommu_dev: handle of the iommu device
+ * @iommu_dev: handle of the OMAP iommu device
+ * @dev: handle of the iommu device
  *
  * This is an omap iommu private data object, which binds an iommu user
  * to its iommu device. This object should be placed at the iommu user's
@@ -96,6 +97,7 @@ struct omap_iommu {
  */
 struct omap_iommu_arch_data {
 	struct omap_iommu *iommu_dev;
+	struct device *dev;
 };
 
 struct cr_regs {
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#10 lore raw

PATCH 8/8 iommu/omap: remove pm_runtime_irq_safe flag for OMAP IOMMUs

In reply to Tero Kristo via iommu

This is not needed for anything, and prevents proper PM transitions for parent devices which is bad in case of ti-sysc; this effectively kills PM completely. Thus, remove the flag.

---
 drivers/iommu/omap-iommu.c | 1 -
 1 file changed, 1 deletion(-)
drivers/iommu/omap-iommu.c +0 -1
index 1beb318..c8a2142 100644
@@ -1254,7 +1254,6 @@ static int omap_iommu_probe(struct platform_device *pdev)
 			goto out_sysfs;
 	}
 
-	pm_runtime_irq_safe(obj->dev);
 	pm_runtime_enable(obj->dev);
 
 	omap_iommu_debugfs_add(obj);
Signature · 8 lines
1.9.1

--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu