Skip to contentSkip to conversation

drm/i915: Fix false-positive assert_rpm_wakelock_held in i915_pmic_bus_access_notifier

PATCH Superseded Hans de Goede 1 participant 5 messages May 19, 2017 – May 28, 2017

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/4 drm/i915: Fix false-positive assert_rpm_wakelock_held in i915_pmic_bus_access_notifier

assert_rpm_wakelock_held is triggered from i915_pmic_bus_access_notifier even though it gets unregistered on (runtime) suspend, this is caused by a race happening under the following circumstances:

intel_runtime_pm_put does:

   atomic_dec(&dev_priv->pm.wakeref_count);

   pm_runtime_mark_last_busy(kdev);
   pm_runtime_put_autosuspend(kdev);

And pm_runtime_put_autosuspend calls intel_runtime_suspend from a workqueue, so there is ample of time between the atomic_dec() and intel_runtime_suspend() unregistering the notifier. If the notifier gets called in this windowd assert_rpm_wakelock_held falsely triggers (at this point we're not runtime-suspended yet).

This commit adds disable_rpm_wakeref_asserts and enable_rpm_wakeref_asserts calls around the intel_uncore_forcewake_get(FORCEWAKE_ALL) call in i915_pmic_bus_access_notifier fixing the false-positive WARN_ON.

---
 drivers/gpu/drm/i915/intel_uncore.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/gpu/drm/i915/intel_uncore.c +7 -0
index 6d1ea26b2493..dc5e4f3e5a43 100644
@@ -1286,8 +1286,15 @@ static int i915_pmic_bus_access_notifier(struct notifier_block *nb,
 		 * bus, which will be busy after this notification, leading to:
 		 * "render: timed out waiting for forcewake ack request."
 		 * errors.
+		 *
+		 * This notifier may get called between intel_runtime_pm_put()
+		 * doing atomic_dec(wakeref_count) and intel_runtime_resume()
+		 * unregistering this notifier, which leads to false-positive
+		 * assert_rpm_wakelock_held() triggering.
 		 */
+		disable_rpm_wakeref_asserts(dev_priv);
 		intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+		enable_rpm_wakeref_asserts(dev_priv);
 		break;
 	case MBI_PMIC_BUS_ACCESS_END:
 		intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
Signature · 6 lines
2.12.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 2/4 drm/i915: Re-register PMIC bus access notifier on runtime resume

intel_uncore_suspend() unregisters the uncore code's PMIC bus access notifier and gets called on both normal and runtime suspend.

intel_uncore_resume_early() re-registers the notifier, but only on normal resume. Add a new intel_uncore_runtime_resume() function which only re-registers the notifier and call that on runtime resume.

---
 drivers/gpu/drm/i915/i915_drv.c     | 2 ++
 drivers/gpu/drm/i915/i915_drv.h     | 1 +
 drivers/gpu/drm/i915/intel_uncore.c | 6 ++++++
 3 files changed, 9 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +2 -0
index 3036d4835b0f..f037bc9db73d 100644
@@ -2464,6 +2464,8 @@ static int intel_runtime_resume(struct device *kdev)
 		ret = vlv_resume_prepare(dev_priv, true);
 	}
 
+	intel_uncore_runtime_resume(dev_priv);
+
 	/*
 	 * No point of rolling back things in case of an error, as the best
 	 * we can do is to hope that things will still work (and disable RPM).
drivers/gpu/drm/i915/i915_drv.h +1 -0
index c9b0949f6c1a..f7db152bb21f 100644
@@ -3073,6 +3073,7 @@ extern bool intel_uncore_arm_unclaimed_mmio_detection(struct drm_i915_private *d
 extern void intel_uncore_fini(struct drm_i915_private *dev_priv);
 extern void intel_uncore_suspend(struct drm_i915_private *dev_priv);
 extern void intel_uncore_resume_early(struct drm_i915_private *dev_priv);
+extern void intel_uncore_runtime_resume(struct drm_i915_private *dev_priv);
 const char *intel_uncore_forcewake_domain_to_str(const enum forcewake_domain_id id);
 void intel_uncore_forcewake_get(struct drm_i915_private *dev_priv,
 				enum forcewake_domains domains);
drivers/gpu/drm/i915/intel_uncore.c +6 -0
index dc5e4f3e5a43..882cf7187a57 100644
@@ -438,6 +438,12 @@ void intel_uncore_resume_early(struct drm_i915_private *dev_priv)
 	i915_check_and_clear_faults(dev_priv);
 }
 
+void intel_uncore_runtime_resume(struct drm_i915_private *dev_priv)
+{
+	iosf_mbi_register_pmic_bus_access_notifier(
+		&dev_priv->uncore.pmic_bus_access_nb);
+}
+
 void intel_uncore_sanitize(struct drm_i915_private *dev_priv)
 {
 	i915.enable_rc6 = sanitize_rc6_option(dev_priv, i915.enable_rc6);
Signature · 6 lines
2.12.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

PATCH 3/4 drm/i915: Call uncore_suspend before platform suspend handlers

In reply to Hans de Goede

Quoting Ville: "the forcewake timer might still be active until the uncore suspend, and having active forcewakes while we've already told the GT wake stuff to stop acting normally doesn't seem quite right to me."

---
 drivers/gpu/drm/i915/i915_drv.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +4 -2
index f037bc9db73d..b7a02c3b5314 100644
@@ -2366,6 +2366,8 @@ static int intel_runtime_suspend(struct device *kdev)
 
 	intel_runtime_pm_disable_interrupts(dev_priv);
 
+	intel_uncore_suspend(dev_priv);
+
 	ret = 0;
 	if (IS_GEN9_LP(dev_priv)) {
 		bxt_display_core_uninit(dev_priv);
@@ -2378,6 +2380,8 @@ static int intel_runtime_suspend(struct device *kdev)
 
 	if (ret) {
 		DRM_ERROR("Runtime suspend failed, disabling it (%d)\n", ret);
+		intel_uncore_runtime_resume(dev_priv);
+
 		intel_runtime_pm_enable_interrupts(dev_priv);
 
 		enable_rpm_wakeref_asserts(dev_priv);
@@ -2385,8 +2389,6 @@ static int intel_runtime_suspend(struct device *kdev)
 		return ret;
 	}
 
-	intel_uncore_suspend(dev_priv);
-
 	enable_rpm_wakeref_asserts(dev_priv);
 	WARN_ON_ONCE(atomic_read(&dev_priv->pm.wakeref_count));
 
Signature · 6 lines
2.12.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH 4/4 drm/i915: Acquire PUNIT->PMIC bus for intel_uncore_forcewake_reset()

In reply to Hans de Goede

intel_uncore_forcewake_reset() does forcewake puts and gets as such we need to make sure that no-one tries to access the PUNIT->PMIC bus (on systems where this bus is shared) while it runs, otherwise bad things happen.

Normally this is taken care of by the i915_pmic_bus_access_notifier() which does an intel_uncore_forcewake_get(FORCEWAKE_ALL) when some other driver tries to access the PMIC bus, so that later forcewake gets are no-ops (for the duration of the bus access).

But intel_uncore_forcewake_reset gets called in 3 cases:
1) Before registering the pmic_bus_access_notifier
2) After unregistering the pmic_bus_access_notifier
3) To reset forcewake state on a GPU reset

In all 3 cases the i915_pmic_bus_access_notifier() protection is insufficient. This commit fixes the pmic bus access race this causes by making intel_uncore_forcewake_reset() call iosf_mbi_punit_acquire() (and iosf_mbi_punit_release() when done).

Note that iosf_mbi_punit_acquire() locks a mutex and thus intel_uncore_forcewake_reset() may sleep after this commit. I've checked all callers and they all already take other mutexes, so this is not a problem.

---
 drivers/gpu/drm/i915/intel_uncore.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/intel_uncore.c +4 -0
index 882cf7187a57..0fb829cc1cbd 100644
@@ -252,6 +252,9 @@ static void intel_uncore_forcewake_reset(struct drm_i915_private *dev_priv,
 	int retry_count = 100;
 	enum forcewake_domains fw, active_domains;
 
+	/* Acquire the PUNIT->PMIC bus before modifying forcewake settings */
+	iosf_mbi_punit_acquire();
+
 	/* Hold uncore.lock across reset to prevent any register access
 	 * with forcewake not set correctly. Wait until all pending
 	 * timers are run before holding.
@@ -308,6 +311,7 @@ static void intel_uncore_forcewake_reset(struct drm_i915_private *dev_priv,
 		assert_forcewakes_inactive(dev_priv);
 
 	spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+	iosf_mbi_punit_release();
 }
 
 static u64 gen9_edram_size(struct drm_i915_private *dev_priv)
Signature · 6 lines
2.12.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw
In reply to Hans de Goede

Hi,

Quoted 17 lines · Hans de Goede

assert_rpm_wakelock_held is triggered from i915_pmic_bus_access_notifier even though it gets unregistered on (runtime) suspend, this is caused by a race happening under the following circumstances:

intel_runtime_pm_put does:

    atomic_dec(&dev_priv->pm.wakeref_count);

    pm_runtime_mark_last_busy(kdev);
    pm_runtime_put_autosuspend(kdev);

And pm_runtime_put_autosuspend calls intel_runtime_suspend from a workqueue, so there is ample of time between the atomic_dec() and intel_runtime_suspend() unregistering the notifier. If the notifier gets called in this windowd assert_rpm_wakelock_held falsely triggers (at this point we're not runtime-suspended yet).

Quoted 7 lines

This commit adds disable_rpm_wakeref_asserts and enable_rpm_wakeref_asserts calls around the intel_uncore_forcewake_get(FORCEWAKE_ALL) call in i915_pmic_bus_access_notifier fixing the false-positive WARN_ON.

Ping, what is the status of this series?

I've been running this series on several Bay and Cherry Trail devices since I posted this without issues.

As this fixes a couple of corner case wrt the pmic bus access sharing merged for 4.12, it would be nice to get these queued up and of possible added to a 4.12-rc#.

The only patch which changes code / functionality which was present in 4.11 is the 3th patch, which I added because Ville requested the change. The other 3 can be queued up for 4.12 without the 3th patch.

Regards,

Hans

Quoted 15 lines
---
  drivers/gpu/drm/i915/intel_uncore.c | 7 +++++++
  1 file changed, 7 insertions(+)
drivers/gpu/drm/i915/intel_uncore.c +2 -0
index 6d1ea26b2493..dc5e4f3e5a43 100644
@@ -1286,8 +1286,15 @@ static int i915_pmic_bus_access_notifier(struct notifier_block *nb,
  		 * bus, which will be busy after this notification, leading to:
  		 * "render: timed out waiting for forcewake ack request."
  		 * errors.
+		 *
+		 * This notifier may get called between intel_runtime_pm_put()

drivers/gpu/drm/i915/intel_uncore.c static int i915_pmic_bus_access_notifier(struct notifier_block *nb, · line 1300

Quoted 10 lines
drivers/gpu/drm/i915/intel_uncore.c +5 -0
+		 * doing atomic_dec(wakeref_count) and intel_runtime_resume()
+		 * unregistering this notifier, which leads to false-positive
+		 * assert_rpm_wakelock_held() triggering.
  		 */
+		disable_rpm_wakeref_asserts(dev_priv);
  		intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+		enable_rpm_wakeref_asserts(dev_priv);
  		break;
  	case MBI_PMIC_BUS_ACCESS_END:
  		intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx