Skip to contentSkip to conversation

arm: omap: clk/clkdm/pwrdm/voltdm usecounting changes

PATCH Superseded Tero Kristo 4 participants 30 messages Feb 15, 2012 – Feb 29, 2012

Subsystems (5)

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v2 cover arm: omap: clk/clkdm/pwrdm/voltdm usecounting changes

Changes compared to previous version:

- moved voltdm_pwrdm_enable / disable hooks from idle code to pwrdm_pre_transition / pwrdm_post_transition functions (patch 4)
- added patches 7 and 8 to fix one issue that was visible with off-mode on omap3: attempting to disable wkdeps for per_clkdm causes crash dumps to be generated by omap_l3_smx (this was actually part of the original larger set but I dropped it accidentally from v1), this issue surfaces with this patch set as the kernel actually tries to manually idle per_clkdm (probably first time in history)
- added some parameter sanity checks (pwrdm/voltdm = null checks)

Similar tests done as with previous version, additionally off-mode tests with omap3.

-Tero

#1 lore raw

PATCH v2 1/8 arm: omap: clk: add support for omap_clk_for_each

This works similarly to e.g. pwrdm_for_each(). Needed by enhanced usecounting debug functionality that will be added to pm-debug.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/plat-omap/clock.c              |   30 ++++++++++++++++++++++++++++++
 arch/arm/plat-omap/include/plat/clock.h |    2 ++
 2 files changed, 32 insertions(+), 0 deletions(-)
arch/arm/plat-omap/clock.c +30 -0
index 567e4b5..e17ee5e2 100644
@@ -356,6 +356,36 @@ int omap_clk_enable_autoidle_all(void)
 	return 0;
 }
 
+/**
+ * omap_clk_for_each - call a function for each registered clock
+ * @fn: pointer to callback function
+ * @data: void * data to pass to callback function
+ *
+ * Call @fn for each registered clock, passing @data to each function.
+ * @fn must return 0 for success or any other value for failure. If
+ * @fn returns non-zero, the iteration across clocks will stop and
+ * the non-zero return value will be passed to the caller of
+ * omap_clk_for_each(). @fn is called with clockfw_lock held.
+ */
+int omap_clk_for_each(int (*fn)(struct clk *clk, void *user), void *user)
+{
+	struct clk *c;
+	unsigned long flags;
+	int ret = 0;
+
+	spin_lock_irqsave(&clockfw_lock, flags);
+
+	list_for_each_entry(c, &clocks, node) {
+		ret = fn(c, user);
+		if (ret)
+			break;
+	}
+
+	spin_unlock_irqrestore(&clockfw_lock, flags);
+
+	return ret;
+}
+
 int omap_clk_disable_autoidle_all(void)
 {
 	struct clk *c;
arch/arm/plat-omap/include/plat/clock.h +2 -0
index 240a7b9..c84f2a4 100644
@@ -300,6 +300,8 @@ extern void propagate_rate(struct clk *clk);
 extern void recalculate_root_clocks(void);
 extern unsigned long followparent_recalc(struct clk *clk);
 extern void clk_enable_init_clocks(void);
+extern int omap_clk_for_each(int (*fn)(struct clk *clk, void *user),
+				void *user);
 unsigned long omap_fixed_divisor_recalc(struct clk *clk);
 #ifdef CONFIG_CPU_FREQ
 extern void clk_init_cpufreq_table(struct cpufreq_frequency_table **table);
Signature · 1 line
1.7.4.1
#2 lore raw

PATCH v2 2/8 arm: omap3+: voltage/pwrdm/clkdm/clock add recursive usecount tracking

In reply to Tero Kristo

This patch fixes the usecount tracking for omap3+, previously the usecount numbers were rather bogus and were not really useful for any purpose. Now usecount numbers track the number of really active clients on each domain. This patch also adds support for usecount tracking on powerdomain level and autoidle flag for clocks that are hardware controlled and should be skipped in usecount calculations.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/clkt_iclk.c         |   21 +++++++++++++++++++
 arch/arm/mach-omap2/clockdomain.c       |   34 +++++++++++++++++++++++++++++-
 arch/arm/mach-omap2/clockdomain.h       |    2 +
 arch/arm/mach-omap2/powerdomain.c       |   16 ++++++++++++++
 arch/arm/mach-omap2/powerdomain.h       |    5 ++++
 arch/arm/plat-omap/clock.c              |    6 +++++
 arch/arm/plat-omap/include/plat/clock.h |    2 +
 7 files changed, 84 insertions(+), 2 deletions(-)
arch/arm/mach-omap2/clkt_iclk.c +21 -0
index 3d43fba..f8c2e77 100644
@@ -21,6 +21,7 @@
 #include "clock2xxx.h"
 #include "cm2xxx_3xxx.h"
 #include "cm-regbits-24xx.h"
+#include "clockdomain.h"
 
 /* Private functions */
 
@@ -34,6 +35,16 @@ void omap2_clkt_iclk_allow_idle(struct clk *clk)
 	v = __raw_readl((__force void __iomem *)r);
 	v |= (1 << clk->enable_bit);
 	__raw_writel(v, (__force void __iomem *)r);
+
+	/* Remove this clock from parent clockdomain usecounts */
+	if (clk->usecount && clk->clkdm)
+		clkdm_usecount_dec(clk->clkdm);
+
+	/*
+	 * Mark as autoidle, so we continue to ignore this clock in
+	 * parent clkdm usecount calculations
+	 */
+	clk->autoidle = true;
 }
 
 /* XXX */
@@ -46,6 +57,16 @@ void omap2_clkt_iclk_deny_idle(struct clk *clk)
 	v = __raw_readl((__force void __iomem *)r);
 	v &= ~(1 << clk->enable_bit);
 	__raw_writel(v, (__force void __iomem *)r);
+
+	/* Add clock back to parent clockdomain usecount */
+	if (clk->usecount && clk->clkdm)
+		clkdm_usecount_inc(clk->clkdm);
+
+	/*
+	 * Disable autoidle flag so further clkdm usecounts take this
+	 * clock into account
+	 */
+	clk->autoidle = false;
 }
 
 /* Public data */
arch/arm/mach-omap2/clockdomain.c +32 -2
index ad07689..c0b6187 100644
@@ -907,6 +907,28 @@ bool clkdm_in_hwsup(struct clockdomain *clkdm)
 
 /* Clockdomain-to-clock/hwmod framework interface code */
 
+int clkdm_usecount_inc(struct clockdomain *clkdm)
+{
+	int usecount;
+
+	usecount = atomic_inc_return(&clkdm->usecount);
+
+	if (usecount == 1)
+		pwrdm_clkdm_enable(clkdm->pwrdm.ptr);
+	return usecount;
+}
+
+int clkdm_usecount_dec(struct clockdomain *clkdm)
+{
+	int usecount;
+
+	usecount = atomic_dec_return(&clkdm->usecount);
+
+	if (usecount == 0)
+		pwrdm_clkdm_disable(clkdm->pwrdm.ptr);
+	return usecount;
+}
+
 static int _clkdm_clk_hwmod_enable(struct clockdomain *clkdm)
 {
 	unsigned long flags;
@@ -919,7 +941,7 @@ static int _clkdm_clk_hwmod_enable(struct clockdomain *clkdm)
 	 * should be called for every clock instance or hwmod that is
 	 * enabled, so the clkdm can be force woken up.
 	 */
-	if ((atomic_inc_return(&clkdm->usecount) > 1) && autodeps)
+	if ((clkdm_usecount_inc(clkdm) > 1) && autodeps)
 		return 0;
 
 	spin_lock_irqsave(&clkdm->lock, flags);
@@ -945,7 +967,7 @@ static int _clkdm_clk_hwmod_disable(struct clockdomain *clkdm)
 		return -ERANGE;
 	}
 
-	if (atomic_dec_return(&clkdm->usecount) > 0)
+	if (clkdm_usecount_dec(clkdm) > 0)
 		return 0;
 
 	spin_lock_irqsave(&clkdm->lock, flags);
@@ -982,6 +1004,10 @@ int clkdm_clk_enable(struct clockdomain *clkdm, struct clk *clk)
 	if (!clk)
 		return -EINVAL;
 
+	/* If autoidle clock, do not update clkdm usecounts */
+	if (clk->autoidle)
+		return 0;
+
 	return _clkdm_clk_hwmod_enable(clkdm);
 }
 
@@ -1008,6 +1034,10 @@ int clkdm_clk_disable(struct clockdomain *clkdm, struct clk *clk)
 	if (!clk)
 		return -EINVAL;
 
+	/* If autoidle clock, do not update clkdm usecounts */
+	if (clk->autoidle)
+		return 0;
+
 	return _clkdm_clk_hwmod_disable(clkdm);
 }
 
arch/arm/mach-omap2/clockdomain.h +2 -0
index f7b5860..373399a 100644
@@ -191,6 +191,8 @@ int clkdm_clk_enable(struct clockdomain *clkdm, struct clk *clk);
 int clkdm_clk_disable(struct clockdomain *clkdm, struct clk *clk);
 int clkdm_hwmod_enable(struct clockdomain *clkdm, struct omap_hwmod *oh);
 int clkdm_hwmod_disable(struct clockdomain *clkdm, struct omap_hwmod *oh);
+int clkdm_usecount_inc(struct clockdomain *clkdm);
+int clkdm_usecount_dec(struct clockdomain *clkdm);
 
 extern void __init omap242x_clockdomains_init(void);
 extern void __init omap243x_clockdomains_init(void);
arch/arm/mach-omap2/powerdomain.c +16 -0
index 8a18d1b..3b70f69 100644
@@ -985,6 +985,22 @@ int pwrdm_clkdm_state_switch(struct clockdomain *clkdm)
 	return -EINVAL;
 }
 
+void pwrdm_clkdm_enable(struct powerdomain *pwrdm)
+{
+	if (!pwrdm)
+		return;
+
+	atomic_inc(&pwrdm->usecount);
+}
+
+void pwrdm_clkdm_disable(struct powerdomain *pwrdm)
+{
+	if (!pwrdm)
+		return;
+
+	atomic_dec(&pwrdm->usecount);
+}
+
 int pwrdm_pre_transition(void)
 {
 	pwrdm_for_each(_pwrdm_pre_transition_cb, NULL);
arch/arm/mach-omap2/powerdomain.h +5 -0
index 0d72a8a..32f44c8 100644
@@ -120,6 +120,7 @@ struct powerdomain {
 	unsigned state_counter[PWRDM_MAX_PWRSTS];
 	unsigned ret_logic_off_counter;
 	unsigned ret_mem_off_counter[PWRDM_MAX_MEM_BANKS];
+	atomic_t usecount;
 
 #ifdef CONFIG_PM_DEBUG
 	s64 timer;
@@ -216,6 +217,10 @@ int pwrdm_state_switch(struct powerdomain *pwrdm);
 int pwrdm_clkdm_state_switch(struct clockdomain *clkdm);
 int pwrdm_pre_transition(void);
 int pwrdm_post_transition(void);
+
+void pwrdm_clkdm_enable(struct powerdomain *pwrdm);
+void pwrdm_clkdm_disable(struct powerdomain *pwrdm);
+
 int pwrdm_set_lowpwrstchange(struct powerdomain *pwrdm);
 int pwrdm_get_context_loss_count(struct powerdomain *pwrdm);
 bool pwrdm_can_ever_lose_context(struct powerdomain *pwrdm);
arch/arm/plat-omap/clock.c +6 -0
index e17ee5e2..1f80df8 100644
@@ -283,6 +283,12 @@ int clk_register(struct clk *clk)
 		list_add(&clk->sibling, &root_clks);
 
 	list_add(&clk->node, &clocks);
+	/*
+	 * If clock has no ops, it is handled by hardware and thus will
+	 * idle automatically
+	 */
+	if (clk->ops == &clkops_null)
+		clk->autoidle = true;
 	if (clk->init)
 		clk->init(clk);
 	mutex_unlock(&clocks_mutex);
arch/arm/plat-omap/include/plat/clock.h +2 -0
index c84f2a4..14b44c0 100644
@@ -208,6 +208,7 @@ struct dpll_data {
  * @init: fn ptr to do clock-specific initialization
  * @enable_bit: bitshift to write to enable/disable the clock (see @enable_reg)
  * @usecount: number of users that have requested this clock to be enabled
+ * @autoidle: indicates hardware controlled clock (not used in domain usecounts)
  * @fixed_div: when > 0, this clock's rate is its parent's rate / @fixed_div
  * @flags: see "struct clk.flags possibilities" above
  * @clksel_reg: for clksel clks, register va containing src/divisor select
@@ -254,6 +255,7 @@ struct clk {
 	void			(*init)(struct clk *);
 	u8			enable_bit;
 	s8			usecount;
+	bool			autoidle;
 	u8			fixed_div;
 	u8			flags;
 #ifdef CONFIG_ARCH_OMAP2PLUS
Signature · 1 line
1.7.4.1
#3 lore raw

PATCH v2 3/8 arm: omap3+: voltage: add support for voltagedomain usecounts

In reply to Tero Kristo

These are updated based on powerdomain usecounts. Also added support for voltdm->sleep and voltdm->wakeup calls that will be invoked once voltagedomain enters sleep or wakes up based on usecount numbers. These will be used for controlling voltage scaling functionality.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/powerdomain.c |    6 +++-
 arch/arm/mach-omap2/voltage.c     |   54 +++++++++++++++++++++++++++++++++++++
 arch/arm/mach-omap2/voltage.h     |   11 +++++++
 3 files changed, 69 insertions(+), 2 deletions(-)
arch/arm/mach-omap2/powerdomain.c +4 -2
index 3b70f69..076eb42 100644
@@ -990,7 +990,8 @@ void pwrdm_clkdm_enable(struct powerdomain *pwrdm)
 	if (!pwrdm)
 		return;
 
-	atomic_inc(&pwrdm->usecount);
+	if (atomic_inc_return(&pwrdm->usecount) == 1)
+		voltdm_pwrdm_enable(pwrdm->voltdm.ptr);
 }
 
 void pwrdm_clkdm_disable(struct powerdomain *pwrdm)
@@ -998,7 +999,8 @@ void pwrdm_clkdm_disable(struct powerdomain *pwrdm)
 	if (!pwrdm)
 		return;
 
-	atomic_dec(&pwrdm->usecount);
+	if (!atomic_dec_return(&pwrdm->usecount))
+		voltdm_pwrdm_disable(pwrdm->voltdm.ptr);
 }
 
 int pwrdm_pre_transition(void)
arch/arm/mach-omap2/voltage.c +54 -0

66 lines not on this page · raw message

arch/arm/mach-omap2/voltage.h +11 -0
index 16a1b09..8829201 100644
@@ -54,10 +54,14 @@ struct omap_vfsm_instance {
  * @pwrdm_list: list_head linking all powerdomains in this voltagedomain
  * @vc: pointer to VC channel associated with this voltagedomain
  * @vp: pointer to VP associated with this voltagedomain
+ * @usecount: number of users for this voltagedomain
+ * @target_state: calculated target state for the children of this domain
  * @read: read a VC/VP register
  * @write: write a VC/VP register
  * @read: read-modify-write a VC/VP register
  * @sys_clk: system clock name/frequency, used for various timing calculations
+ * @sleep: function to call once the domain enters idle
+ * @wakeup: function to call once the domain wakes up from idle
  * @scale: function used to scale the voltage of the voltagedomain
  * @nominal_volt: current nominal voltage for this voltage domain
  * @volt_data: voltage table having the distinct voltages supported
@@ -73,6 +77,9 @@ struct voltagedomain {
 	struct omap_vp_instance *vp;
 	struct omap_voltdm_pmic *pmic;
 
+	atomic_t usecount;
+	int target_state;
+
 	/* VC/VP register access functions: SoC specific */
 	u32 (*read) (u8 offset);
 	void (*write) (u32 val, u8 offset);
@@ -83,6 +90,8 @@ struct voltagedomain {
 		u32 rate;
 	} sys_clk;
 
+	void (*sleep) (struct voltagedomain *voltdm);
+	void (*wakeup) (struct voltagedomain *voltdm);
 	int (*scale) (struct voltagedomain *voltdm,
 		      unsigned long target_volt);
 
@@ -161,6 +170,8 @@ extern void omap44xx_voltagedomains_init(void);
 struct voltagedomain *voltdm_lookup(const char *name);
 void voltdm_init(struct voltagedomain **voltdm_list);
 int voltdm_add_pwrdm(struct voltagedomain *voltdm, struct powerdomain *pwrdm);
+void voltdm_pwrdm_enable(struct voltagedomain *voltdm);
+void voltdm_pwrdm_disable(struct voltagedomain *voltdm);
 int voltdm_for_each(int (*fn)(struct voltagedomain *voltdm, void *user),
 		    void *user);
 int voltdm_for_each_pwrdm(struct voltagedomain *voltdm,
Signature · 1 line
1.7.4.1
#4 lore raw

PATCH v2 4/8 arm: omap3: add manual control for mpu / core pwrdm usecounting

In reply to Tero Kristo

mpu / core powerdomain usecounts are now statically increased by 1 during MPU activity. This allows the domains to reflect actual usage, and will allow the usecount to reach 0 just before entering wfi. Proper powerdomain usecounts are propageted to voltagedomain level also, and will allow vc callbacks to be triggered at right point of time.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/powerdomain.c |   40 +++++++++++++++++++++++++++++++++++++
 1 files changed, 40 insertions(+), 0 deletions(-)
arch/arm/mach-omap2/powerdomain.c +40 -0
index 076eb42..b0f9b29 100644
@@ -47,6 +47,7 @@ enum {
 static LIST_HEAD(pwrdm_list);
 
 static struct pwrdm_ops *arch_pwrdm;
+static struct powerdomain *mpu_pwrdm, *core_pwrdm;
 
 /* Private functions */
 
@@ -1006,11 +1007,17 @@ void pwrdm_clkdm_disable(struct powerdomain *pwrdm)
 int pwrdm_pre_transition(void)
 {
 	pwrdm_for_each(_pwrdm_pre_transition_cb, NULL);
+	/* Decrease mpu / core usecounts to indicate we are entering idle */
+	pwrdm_clkdm_disable(mpu_pwrdm);
+	pwrdm_clkdm_disable(core_pwrdm);
 	return 0;
 }
 
 int pwrdm_post_transition(void)
 {
+	/* Increase mpu / core usecounts to indicate we are leaving idle */
+	pwrdm_clkdm_enable(mpu_pwrdm);
+	pwrdm_clkdm_enable(core_pwrdm);
 	pwrdm_for_each(_pwrdm_post_transition_cb, NULL);
 	return 0;
 }
@@ -1090,3 +1097,36 @@ bool pwrdm_can_ever_lose_context(struct powerdomain *pwrdm)
 
 	return 0;
 }
+
+/**
+ * pwrdm_usecount_init - initialize special powerdomain usecounts
+ *
+ * Initializes usecounts for the powerdomains that have static
+ * dependencies with MPU idle cycle, namely mpu_pwrdm and core_pwrdm.
+ * These powerdomains will get their usecounts increased / decreased
+ * each sleep cycle so that they reach 0 just before entering wfi,
+ * and are increased to 1 just after it. This allows the dependent
+ * voltage domains to follow idle cycle properly and trigger their
+ * callbacks for sleep / wakeup.
+ */
+static int __init pwrdm_usecount_init(void)
+{
+	mpu_pwrdm = pwrdm_lookup("mpu_pwrdm");
+	if (!mpu_pwrdm) {
+		pr_err("%s: failed to get mpu_pwrdm\n", __func__);
+		return -EINVAL;
+	}
+
+	pwrdm_clkdm_enable(mpu_pwrdm);
+
+	core_pwrdm = pwrdm_lookup("core_pwrdm");
+	if (!core_pwrdm) {
+		pr_err("%s: failed to get core_pwrdm\n", __func__);
+		return -EINVAL;
+	}
+
+	pwrdm_clkdm_enable(core_pwrdm);
+
+	return 0;
+}
+late_initcall(pwrdm_usecount_init);
Signature · 1 line
1.7.4.1
#5 lore raw

PATCH v2 5/8 arm: omap3: set autoidle flags for a few clocks

In reply to Tero Kristo

dpll3, dpll4 and sdrc_ick are controlled automatically by hardware. Thus, reflect this with the autoidle flags, the clocks will no longer show as active in usecount dumps and will allow the voltdm->sleep / wakeup calls to work properly.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/clock3xxx_data.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
arch/arm/mach-omap2/clock3xxx_data.c +3 -0
index d75e5f6..f3ee983 100644
@@ -432,6 +432,7 @@ static struct clk dpll3_ck = {
 	.round_rate	= &omap2_dpll_round_rate,
 	.clkdm_name	= "dpll3_clkdm",
 	.recalc		= &omap3_dpll_recalc,
+	.autoidle	= true,
 };
 
 /*
@@ -615,6 +616,7 @@ static struct clk dpll4_ck = {
 	.set_rate	= &omap3_dpll4_set_rate,
 	.clkdm_name	= "dpll4_clkdm",
 	.recalc		= &omap3_dpll_recalc,
+	.autoidle	= true,
 };
 
 /*
@@ -1744,6 +1746,7 @@ static struct clk sdrc_ick = {
 	.flags		= ENABLE_ON_INIT,
 	.clkdm_name	= "core_l3_clkdm",
 	.recalc		= &followparent_recalc,
+	.autoidle	= true,
 };
 
 static struct clk gpmc_fck = {
Signature · 1 line
1.7.4.1
#6 lore raw

PATCH v2 6/8 arm: omap: pm-debug: enhanced usecount debug support

In reply to Tero Kristo

Voltdm, pwrdm, clkdm, hwmod and clk usecounts are now separeted to their own file, 'usecount'. This file shows the usecounts for every active domain and their children recursively. 'count' file now only shows power state counts for powerdomains.

This patch also provices a way to do printk dumps from kernel code, by calling the pm_dbg_dump_X functions. The plan is to call these functions once an error condition is detected, e.g. failed suspend.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/pm-debug.c |  128 ++++++++++++++++++++++++++++++++++------
 arch/arm/mach-omap2/pm.h       |    6 ++
 2 files changed, 115 insertions(+), 19 deletions(-)
arch/arm/mach-omap2/pm-debug.c +109 -19
index 4411163..444588e 100644
@@ -51,6 +51,7 @@ static int pm_dbg_init(void);
 enum {
 	DEBUG_FILE_COUNTERS = 0,
 	DEBUG_FILE_TIMERS,
+	DEBUG_FILE_USECOUNT,
 };
 
 static const char pwrdm_state_names[][PWRDM_MAX_PWRSTS] = {
@@ -75,23 +76,6 @@ void pm_dbg_update_time(struct powerdomain *pwrdm, int prev)
 	pwrdm->timer = t;
 }
 
-static int clkdm_dbg_show_counter(struct clockdomain *clkdm, void *user)
-{
-	struct seq_file *s = (struct seq_file *)user;
-
-	if (strcmp(clkdm->name, "emu_clkdm") == 0 ||
-		strcmp(clkdm->name, "wkup_clkdm") == 0 ||
-		strncmp(clkdm->name, "dpll", 4) == 0)
-		return 0;
-
-	seq_printf(s, "%s->%s (%d)", clkdm->name,
-			clkdm->pwrdm.ptr->name,
-			atomic_read(&clkdm->usecount));
-	seq_printf(s, "\n");
-
-	return 0;
-}
-
 static int pwrdm_dbg_show_counter(struct powerdomain *pwrdm, void *user)
 {
 	struct seq_file *s = (struct seq_file *)user;
@@ -145,11 +129,112 @@ static int pwrdm_dbg_show_timer(struct powerdomain *pwrdm, void *user)
 	return 0;
 }
 
+static struct voltagedomain *parent_voltdm;
+static struct powerdomain *parent_pwrdm;
+static struct clockdomain *parent_clkdm;
+
+#define PM_DBG_PRINT(s, fmt, args...)			\
+	{						\
+		if (s)					\
+			seq_printf(s, fmt, ## args);	\
+		else					\
+			pr_info(fmt, ## args);		\
+	}
+
+static int _pm_dbg_dump_clk(struct clk *clk, void *user)
+{
+	struct seq_file *s = user;
+
+	if (clk->clkdm == parent_clkdm && clk->usecount && !clk->autoidle)
+		PM_DBG_PRINT(s, "      ck:%s: %d\n", clk->name, clk->usecount);
+
+	return 0;
+}
+
+static int _pm_dbg_dump_hwmod(struct omap_hwmod *oh, void *user)
+{
+	struct seq_file *s = user;
+
+	if (oh->clkdm != parent_clkdm)
+		return 0;
+
+	if (oh->_state != _HWMOD_STATE_ENABLED)
+		return 0;
+
+	PM_DBG_PRINT(s, "      oh:%s: enabled\n", oh->name);
+
+	return 0;
+}
+
+static int _pm_dbg_dump_clkdm(struct clockdomain *clkdm, void *user)
+{
+	struct seq_file *s = user;
+	u32 usecount;
+
+	if (clkdm->pwrdm.ptr == parent_pwrdm) {
+		usecount = atomic_read(&clkdm->usecount);
+		if (usecount) {
+			PM_DBG_PRINT(s, "    cd:%s: %d\n", clkdm->name,
+				usecount);
+			parent_clkdm = clkdm;
+			omap_hwmod_for_each(_pm_dbg_dump_hwmod, s);
+			omap_clk_for_each(_pm_dbg_dump_clk, s);
+		}
+	}
+	return 0;
+}
+
+static int _pm_dbg_dump_pwrdm(struct powerdomain *pwrdm, void *user)
+{
+	struct seq_file *s = user;
+	u32 usecount;
+
+	if (pwrdm->voltdm.ptr == parent_voltdm) {
+		usecount = atomic_read(&pwrdm->usecount);
+		if (usecount) {
+			PM_DBG_PRINT(s, "  pd:%s: %d\n", pwrdm->name, usecount);
+			parent_pwrdm = pwrdm;
+			clkdm_for_each(_pm_dbg_dump_clkdm, s);
+		}
+	}
+	return 0;
+}
+
+void pm_dbg_dump_pwrdm(struct powerdomain *pwrdm)
+{
+	pr_info("pd:%s: %d\n", pwrdm->name, atomic_read(&pwrdm->usecount));
+	parent_pwrdm = pwrdm;
+	clkdm_for_each(_pm_dbg_dump_clkdm, NULL);
+}
+
+void pm_dbg_dump_voltdm(struct voltagedomain *voltdm)
+{
+	pr_info("vd:%s: %d\n", voltdm->name, atomic_read(&voltdm->usecount));
+	parent_voltdm = voltdm;
+	pwrdm_for_each(_pm_dbg_dump_pwrdm, NULL);
+}
+
+static int _voltdm_dbg_show_counters(struct voltagedomain *voltdm, void *user)
+{
+	struct seq_file *s = user;
+
+	seq_printf(s, "vd:%s: %d\n", voltdm->name,
+		atomic_read(&voltdm->usecount));
+
+	parent_voltdm = voltdm;
+	pwrdm_for_each(_pm_dbg_dump_pwrdm, s);
+	return 0;
+}
+
+static int pm_dbg_show_usecount(struct seq_file *s, void *unused)
+{
+	voltdm_for_each(_voltdm_dbg_show_counters, s);
+	return 0;
+}
+
 static int pm_dbg_show_counters(struct seq_file *s, void *unused)
 {
 	pwrdm_for_each(pwrdm_dbg_show_counter, s);
-	clkdm_for_each(clkdm_dbg_show_counter, s);
-
 	return 0;
 }
 
@@ -162,6 +247,9 @@ static int pm_dbg_show_timers(struct seq_file *s, void *unused)
 static int pm_dbg_open(struct inode *inode, struct file *file)
 {
 	switch ((int)inode->i_private) {
+	case DEBUG_FILE_USECOUNT:
+		return single_open(file, pm_dbg_show_usecount,
+			&inode->i_private);
 	case DEBUG_FILE_COUNTERS:
 		return single_open(file, pm_dbg_show_counters,
 			&inode->i_private);
@@ -271,6 +359,8 @@ static int __init pm_dbg_init(void)
 		d, (void *)DEBUG_FILE_COUNTERS, &debug_fops);
 	(void) debugfs_create_file("time", S_IRUGO,
 		d, (void *)DEBUG_FILE_TIMERS, &debug_fops);
+	(void) debugfs_create_file("usecount", S_IRUGO,
+		d, (void *)DEBUG_FILE_USECOUNT, &debug_fops);
 
 	pwrdm_for_each(pwrdms_setup, (void *)d);
 
arch/arm/mach-omap2/pm.h +6 -0
index b737b11..ec8f874 100644
@@ -61,10 +61,16 @@ inline void omap3_pm_init_cpuidle(struct cpuidle_params *cpuidle_board_params)
 extern int omap3_pm_get_suspend_state(struct powerdomain *pwrdm);
 extern int omap3_pm_set_suspend_state(struct powerdomain *pwrdm, int state);
 
+struct clk;
+
 #ifdef CONFIG_PM_DEBUG
 extern u32 enable_off_mode;
+extern void pm_dbg_dump_pwrdm(struct powerdomain *pwrdm);
+extern void pm_dbg_dump_voltdm(struct voltagedomain *voltdm);
 #else
 #define enable_off_mode 0
+static inline void pm_dbg_dump_pwrdm(struct powerdomain *pwrdm) { }
+static inline void pm_dbg_dump_voltdm(struct voltagedomain *voltdm) { }
 #endif
 
 #if defined(CONFIG_PM_DEBUG) && defined(CONFIG_DEBUG_FS)
Signature · 1 line
1.7.4.1
#7 lore raw

PATCH v2 7/8 arm: omap: clockdomain: add support for preventing domain transitions

In reply to Tero Kristo

Some clockdomains can't support manual domain transitions triggered by clock framework, and must be prevented from doing so. Added clkdm flag CLKDM_NO_MANUAL_TRANS for doing this.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/clockdomain.c |    6 ++++++
 arch/arm/mach-omap2/clockdomain.h |    3 +++
 2 files changed, 9 insertions(+), 0 deletions(-)
arch/arm/mach-omap2/clockdomain.c +6 -0
index c0b6187..850665d 100644
@@ -944,6 +944,9 @@ static int _clkdm_clk_hwmod_enable(struct clockdomain *clkdm)
 	if ((clkdm_usecount_inc(clkdm) > 1) && autodeps)
 		return 0;
 
+	if (clkdm->flags & CLKDM_NO_MANUAL_TRANS)
+		return 0;
+
 	spin_lock_irqsave(&clkdm->lock, flags);
 	arch_clkdm->clkdm_clk_enable(clkdm);
 	pwrdm_wait_transition(clkdm->pwrdm.ptr);
@@ -970,6 +973,9 @@ static int _clkdm_clk_hwmod_disable(struct clockdomain *clkdm)
 	if (clkdm_usecount_dec(clkdm) > 0)
 		return 0;
 
+	if (clkdm->flags & CLKDM_NO_MANUAL_TRANS)
+		return 0;
+
 	spin_lock_irqsave(&clkdm->lock, flags);
 	arch_clkdm->clkdm_clk_disable(clkdm);
 	pwrdm_clkdm_state_switch(clkdm);
arch/arm/mach-omap2/clockdomain.h +3 -0
index 373399a..8693bec 100644
@@ -31,12 +31,15 @@
  *
  * CLKDM_NO_AUTODEPS: Prevent "autodeps" from being added/removed from this
  *     clockdomain.  (Currently, this applies to OMAP3 clockdomains only.)
+ * CLKDM_NO_MANUAL_TRANS: Prevent clockdomain code from attempting to change
+ *     clockdomain state manually. Needed for PER domain on omap3.
  */
 #define CLKDM_CAN_FORCE_SLEEP			(1 << 0)
 #define CLKDM_CAN_FORCE_WAKEUP			(1 << 1)
 #define CLKDM_CAN_ENABLE_AUTO			(1 << 2)
 #define CLKDM_CAN_DISABLE_AUTO			(1 << 3)
 #define CLKDM_NO_AUTODEPS			(1 << 4)
+#define CLKDM_NO_MANUAL_TRANS			(1 << 5)
 
 #define CLKDM_CAN_HWSUP		(CLKDM_CAN_ENABLE_AUTO | CLKDM_CAN_DISABLE_AUTO)
 #define CLKDM_CAN_SWSUP		(CLKDM_CAN_FORCE_SLEEP | CLKDM_CAN_FORCE_WAKEUP)
Signature · 1 line
1.7.4.1
#8 lore raw

PATCH v2 7/8 arm: omap: clockdomain: add support for preventing domain transitions

Quoted 7 lines · Tero Kristo

Some clockdomains can't support manual domain transitions triggered by clock framework, and must be prevented from doing so. Added clkdm flag CLKDM_NO_MANUAL_TRANS for doing this.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>

Dumb Q: what's the difference between this new flag and CLKDM_CAN_SWSUP?

Kevin

#9 lore raw

PATCH v2 7/8 arm: omap: clockdomain: add support for preventing domain transitions

Quoted 11 lines · Kevin Hilman
Quoted 7 lines · Tero Kristo

Some clockdomains can't support manual domain transitions triggered by clock framework, and must be prevented from doing so. Added clkdm flag CLKDM_NO_MANUAL_TRANS for doing this.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>

Dumb Q: what's the difference between this new flag and CLKDM_CAN_SWSUP?

CAN_SWSUP controls software wakeup / sleep (clktrctrl values 1 and 2), but it still allows hwsup based transitions during runtime (basically using values 0 & 3 for the same register.) NO_MANUAL_TRANS prevents both, but still allows hwauto mode being enabled in the first place.

I was thinking about adding a flag for preventing the autodep disabling as this was the root cause for the problem I saw, but I wanted to optimize the _clkdm_clk_hwmod_enable/disable for this case, as the domain is not going through any transitions.

-Tero

#10 lore raw

PATCH v2 7/8 arm: omap: clockdomain: add support for preventing domain transitions

Tero,

Quoted 18 lines · Tero Kristo
Quoted 11 lines · Kevin Hilman
Quoted 7 lines · Tero Kristo

Some clockdomains can't support manual domain transitions triggered by clock framework, and must be prevented from doing so. Added clkdm flag CLKDM_NO_MANUAL_TRANS for doing this.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>

Dumb Q: what's the difference between this new flag and CLKDM_CAN_SWSUP?

CAN_SWSUP controls software wakeup / sleep (clktrctrl values 1 and 2), but it still allows hwsup based transitions during runtime (basically using values 0 & 3 for the same register.) NO_MANUAL_TRANS prevents both, but still allows hwauto mode being enabled in the first place.

Quoted 5 lines

I was thinking about adding a flag for preventing the autodep disabling as this was the root cause for the problem I saw, but I wanted to optimize the _clkdm_clk_hwmod_enable/disable for this case, as the domain is not going through any transitions.

Which clock-domain you are refering here which needs this flag ?

Regards
santosh

#11 lore raw

PATCH v2 7/8 arm: omap: clockdomain: add support for preventing domain transitions

Quoted 20 lines · Santosh

Tero,

Quoted 18 lines · Tero Kristo
Quoted 11 lines · Kevin Hilman
Quoted 7 lines · Tero Kristo

Some clockdomains can't support manual domain transitions triggered by clock framework, and must be prevented from doing so. Added clkdm flag CLKDM_NO_MANUAL_TRANS for doing this.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>

Dumb Q: what's the difference between this new flag and CLKDM_CAN_SWSUP?

CAN_SWSUP controls software wakeup / sleep (clktrctrl values 1 and 2), but it still allows hwsup based transitions during runtime (basically using values 0 & 3 for the same register.) NO_MANUAL_TRANS prevents both, but still allows hwauto mode being enabled in the first place.

Quoted 7 lines
Quoted 5 lines

I was thinking about adding a flag for preventing the autodep disabling as this was the root cause for the problem I saw, but I wanted to optimize the _clkdm_clk_hwmod_enable/disable for this case, as the domain is not going through any transitions.

Which clock-domain you are refering here which needs this flag ?

per_clkdm, see patch 8 in this same set for comments. This happens with omap3.

Quoted 3 lines

Regards
santosh

#12 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

In reply to Tero Kristo

Attempting this will cause problems especially with off-mode enabled. Previously this issue was hidden by the fact that per_clkdm never attempted manual idle by software, as the usecounts for the clockdomain were broken.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/clockdomains3xxx_data.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/arm/mach-omap2/clockdomains3xxx_data.c +1 -1
index b84e138..db31bbf 100644
@@ -282,7 +282,7 @@ static struct clockdomain usbhost_clkdm = {
 static struct clockdomain per_clkdm = {
 	.name		= "per_clkdm",
 	.pwrdm		= { .name = "per_pwrdm" },
-	.flags		= CLKDM_CAN_HWSUP_SWSUP,
+	.flags		= CLKDM_CAN_HWSUP_SWSUP | CLKDM_NO_MANUAL_TRANS,
 	.dep_bit	= OMAP3430_EN_PER_SHIFT,
 	.wkdep_srcs	= per_wkdeps,
 	.sleepdep_srcs	= per_sleepdeps,
Signature · 1 line
1.7.4.1
#13 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Kevin

Quoted 24 lines

Previously this issue was hidden by the fact that per_clkdm never attempted manual idle by software, as the usecounts for the clockdomain were broken.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/clockdomains3xxx_data.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/arm/mach-omap2/clockdomains3xxx_data.c +1 -1
index b84e138..db31bbf 100644
@@ -282,7 +282,7 @@ static struct clockdomain usbhost_clkdm = {
 static struct clockdomain per_clkdm = {
 	.name		= "per_clkdm",
 	.pwrdm		= { .name = "per_pwrdm" },
-	.flags		= CLKDM_CAN_HWSUP_SWSUP,
+	.flags		= CLKDM_CAN_HWSUP_SWSUP | CLKDM_NO_MANUAL_TRANS,
 	.dep_bit	= OMAP3430_EN_PER_SHIFT,
 	.wkdep_srcs	= per_wkdeps,
 	.sleepdep_srcs	= per_sleepdeps,
#14 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

[   79.010345] PM: Syncing filesystems ... done.
[   79.083801] Freezing user space processes ... (elapsed 0.01 seconds)
done.
[   79.110839] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   79.141845] Suspending console(s) (use no_console_suspend to debug)
[   79.266815] PM: suspend of devices complete after 115.551 msecs
[   79.269958] PM: late suspend of devices complete after 3.142 msecs
[   79.270050] Disabling non-boot CPUs ...
[   79.697235] Powerdomain (cam_pwrdm) didn't enter target state 0
[   79.697265] Could not enter target state in pm_suspend
[   79.699829] PM: early resume of devices complete after 2.257 msecs
[   79.699920] ------------[ cut here ]------------
[   79.699981] WARNING: at arch/arm/mach-omap2/omap_l3_smx.c:161
omap3_l3_app_ir
q+0xd8/0x130()
[   79.699981] In-band Error seen by MPU  at address 0
[   79.699981] Modules linked in:
[   79.700042] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c00428d0>] (warn_
slowpath_common+0x4c/0x64)
[   79.700073] [<c00428d0>] (warn_slowpath_common+0x4c/0x64) from
[<c004297c>] (
warn_slowpath_fmt+0x30/0x40)
[   79.700103] [<c004297c>] (warn_slowpath_fmt+0x30/0x40) from
[<c0034a2c>] (oma
p3_l3_app_irq+0xd8/0x130)
[   79.700103] [<c0034a2c>] (omap3_l3_app_irq+0xd8/0x130) from
[<c00a09e4>] (han
dle_irq_event_percpu+0x5c/0x22c)
[   79.700134] [<c00a09e4>] (handle_irq_event_percpu+0x5c/0x22c) from

[<c00a0bf0

Quoted 1 line

] (handle_irq_event+0x3c/0x5c)

[   79.700164] [<c00a0bf0>] (handle_irq_event+0x3c/0x5c) from
[<c00a3a70>] (hand
le_level_irq+0xac/0x118)
[   79.700195] [<c00a3a70>] (handle_level_irq+0xac/0x118) from
[<c00a0490>] (gen
eric_handle_irq+0x34/0x44)
[   79.700225] [<c00a0490>] (generic_handle_irq+0x34/0x44) from
[<c00151ec>] (ha
IdRQ+0x4c/0xac)
[   79.700256] [<c00151ec>] (handle_IRQ+0x4c/0xac) from [<c0008548>]
(omap3_intc
_handle_irq+0x44/0x4c)
[   79.700256] [<c0008548>] (omap3_intc_handle_irq+0x44/0x4c) from
[<c0476824>] 
(__irq_svc+0x44/0x60)
[   79.700286] Exception stack(0xcefe7e78 to 0xcefe7ec0)
[   79.700286] 7e60:
00007
a93 cedb25d0
[   79.700317] 7e80: 00000000 cedb2140 60000153 c0676994 00000000
c0676994 60000
153 00000000
[   79.700347] 7ea0: c0676940 000a3008 c0721d40 cefe7ec0 00007a94
c04765e4 20000
153 ffffffff
[   79.700347] [<c0476824>] (__irq_svc+0x44/0x60) from [<c04765e4>]
(_raw_spin_u
nlock_irqrestore+0x34/0x44)
[   79.700378] [<c04765e4>] (_raw_spin_unlock_irqrestore+0x34/0x44) from
[<c00a5
e18>] (resume_irqs+0x9c/0xbc)
[   79.700408] [<c00a5e18>] (resume_irqs+0x9c/0xbc) from [<c008061c>]
(suspend_d
evices_and_enter+0x114/0x2d8)
[   79.700439] [<c008061c>] (suspend_devices_and_enter+0x114/0x2d8) from
[<c0080
91c>] (enter_state+0x13c/0x184)
[   79.700469] [<c008091c>] (enter_state+0x13c/0x184) from [<c007f5fc>]
(state_s
tore+0xd0/0x170)
[   79.700500] [<c007f5fc>] (state_store+0xd0/0x170) from [<c02536a8>]
(kobj_att
r_store+0x18/0x1c)
[   79.700531] [<c02536a8>] (kobj_attr_store+0x18/0x1c) from
[<c0167a9c>] (sysfs
_write_file+0xfc/0x180)
[   79.700561] [<c0167a9c>] (sysfs_write_file+0xfc/0x180) from
[<c0107ed4>] (vfs
_write+0xb0/0x134)
[   79.700561] [<c0107ed4>] (vfs_write+0xb0/0x134) from [<c0108028>]
(sys_write+
0x40/0x70)
[   79.700592] [<c0108028>] (sys_write+0x40/0x70) from [<c0014160>]
(ret_fast_sy
scall+0x0/0x3c)
[   79.700622] ---[ end trace 338e34a6f123bc2b ]---
[   80.121765] PM: resume of devices complete after 420.012 msecs
[   80.414886] Restarting tasks ... done.
Quoted 26 lines

Kevin

Quoted 24 lines

Previously this issue was hidden by the fact that per_clkdm never attempted manual idle by software, as the usecounts for the clockdomain were broken.

Cc Paul Walmsley <[email protected]>
Cc Kevin Hilman <[email protected]>
---
 arch/arm/mach-omap2/clockdomains3xxx_data.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/arm/mach-omap2/clockdomains3xxx_data.c +1 -1
index b84e138..db31bbf 100644
@@ -282,7 +282,7 @@ static struct clockdomain usbhost_clkdm = {
 static struct clockdomain per_clkdm = {
 	.name		= "per_clkdm",
 	.pwrdm		= { .name = "per_pwrdm" },
-	.flags		= CLKDM_CAN_HWSUP_SWSUP,
+	.flags		= CLKDM_CAN_HWSUP_SWSUP | CLKDM_NO_MANUAL_TRANS,
 	.dep_bit	= OMAP3430_EN_PER_SHIFT,
 	.wkdep_srcs	= per_wkdeps,
 	.sleepdep_srcs	= per_sleepdeps,
#15 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

regards
Santosh

#16 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 17 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

Quoted 7 lines

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

-Tero

#17 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

        if (per_next_state < PWRDM_POWER_ON) {
                per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
                omap2_gpio_resume_after_idle();
                wake_per();
                if (per_prev_state == PWRDM_POWER_OFF)
                        omap3_per_restore_context();
        }

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

-Tero

#18 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

? ? ? ?if (per_next_state < PWRDM_POWER_ON) {
? ? ? ? ? ? ? ?per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm); ? ? ? ? ? ? ? ?omap2_gpio_resume_after_idle();
? ? ? ? ? ? ? ?wake_per();
? ? ? ? ? ? ? ?if (per_prev_state == PWRDM_POWER_OFF)
? ? ? ? ? ? ? ? ? ? ? ?omap3_per_restore_context();
? ? ? ?}

Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Regards
Santosh

#19 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

-Tero

#20 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

Kevin

#21 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

-Tero

#22 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

What platform are you testing on?

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Can you debug this a little further so we can explain what's going on here?

Thanks,

Kevin

#23 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 164 lines · Kevin Hilman
Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 5 lines
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

What platform are you testing on?

This is with omap3530 beagle.

Quoted 3 lines

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Did you try without Tarun's series and see what happens in that case?

Quoted 2 lines

Can you debug this a little further so we can explain what's going on here?

I don't have capability to test with any other omap3 device than beagle, so don't know what I can accomplish here. I just got omap4 cswr working yesterday with mainline, so I can try with that one and see how it behaves (omap4 requires some additional patching for the usecounting though.)

-Tero

#24 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

In reply to Kevin Hilman
Quoted 168 lines · Kevin Hilman
Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

Quoted 8 lines

What platform are you testing on?

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Can you debug this a little further so we can explain what's going on here?

Attached one trial patch you could try to use to see if it fixes the issues you are seeing with 3430/n900. I'll look at Tarun's set and see if I can figure out what is causing the warning above.

-Tero
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-arm-omap3-prevent-dpll4-manual-enable-disable-preven.patch
Type: text/x-patch
Size: 2116 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20120224/143dd71b/attachment.bin>

#25 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

In reply to Kevin Hilman
Quoted 168 lines · Kevin Hilman
Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman
Quoted 1 line · Tero Kristo

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

Quoted 9 lines

What platform are you testing on?

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Can you debug this a little further so we can explain what's going on here?

After some debugging, it looks like the root cause for the WARN dump I see with GPIO PM runtime + usecounting is the fact that PER domain enters SW assisted idle and remains OFF after wakeup from OFF mode, and it will remain there until some functional clock is enabled. GPIO6 module is only enabling its interface clock, but as this is autoidled, it will not wake up PER domain. This will cause the initial _wait_target_ready(oh) to fail with EBUSY, leaving the hwmod into idle state. Next cpuidle entry will attempt to disable the already idle hwmod causing this dump.

This can be avoided by preventing PER domain idle on omap3530. I believe on omap3430 you also need to prevent similar stuff for CORE domain.

Did you try the patch I posted last week for the core domain + dplls? Does it help with 3430? I guess n900 could hang in the ROM code if PER domain is idle during wakeup, as it is a HS device also and behaves differently during wakeup from OFF.

-Tero

#26 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

+Paul

Quoted 175 lines · Tero Kristo
Quoted 174 lines · Kevin Hilman
Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman

Tero Kristo <[email protected]> writes:
P

Quoted 1 line

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

What platform are you testing on?

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Quoted 8 lines
Quoted 3 lines

Can you debug this a little further so we can explain what's going on here?

After some debugging, it looks like the root cause for the WARN dump I see with GPIO PM runtime + usecounting is the fact that PER domain enters SW assisted idle and remains OFF after wakeup from OFF mode, and it will remain there until some functional clock is enabled.

OK, but before your usecounting changes, the runtime PM enable was successfully bringing GPIO out of idle.

In the case of GPIO, the iclk is also the functional clock (it's the main_clk in the hwmod.)

Quoted 5 lines

GPIO6 module is only enabling its interface clock, but as this is autoidled, it will not wake up PER domain. This will cause the initial _wait_target_ready(oh) to fail with EBUSY, leaving the hwmod into idle state. Next cpuidle entry will attempt to disable the already idle hwmod causing this dump.

I see. Thanks for tracking it down.

To me, this suggests that the autoidle changes in your usecounting series aren't quite right, especially for modules like GPIO where the iclk is also the functional clock.

I added Paul to this discussion to see what his thoughts are on this.

Quoted 1 line

This can be avoided by preventing PER domain idle on omap3530.

That doesn't sound quite right either.

Quoted 4 lines

I believe on omap3430 you also need to prevent similar stuff for CORE domain.

Did you try the patch I posted last week for the core domain + dplls?

I didn't try it until just now and it' doesn't compile because the new flag you're using doesn't seem to be defined anywhere.

Kevin

Quoted 11 lines

Does it help with 3430? I guess n900 could hang in the ROM code if PER domain is idle during wakeup, as it is a HS device also and behaves differently during wakeup from OFF.

-Tero

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

#27 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 183 lines · Kevin Hilman

+Paul

Quoted 179 lines · Tero Kristo
Quoted 177 lines · Kevin Hilman
Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman

Tero Kristo <[email protected]> writes:
P

Quoted 1 line

Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

What platform are you testing on?

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Can you debug this a little further so we can explain what's going on here?

Quoted 7 lines
Quoted 4 lines

After some debugging, it looks like the root cause for the WARN dump I see with GPIO PM runtime + usecounting is the fact that PER domain enters SW assisted idle and remains OFF after wakeup from OFF mode, and it will remain there until some functional clock is enabled.

OK, but before your usecounting changes, the runtime PM enable was successfully bringing GPIO out of idle.

Well, the thing is in this case, PER never idles by SW, it is only following MPU to idle and back. PER awakens immediately by wakeup from OFF, which is not the case with the usecounting set.

Quoted 9 lines

In the case of GPIO, the iclk is also the functional clock (it's the main_clk in the hwmod.)

Quoted 5 lines

GPIO6 module is only enabling its interface clock, but as this is autoidled, it will not wake up PER domain. This will cause the initial _wait_target_ready(oh) to fail with EBUSY, leaving the hwmod into idle state. Next cpuidle entry will attempt to disable the already idle hwmod causing this dump.

Quoted 5 lines

I see. Thanks for tracking it down.

To me, this suggests that the autoidle changes in your usecounting series aren't quite right, especially for modules like GPIO where the iclk is also the functional clock.

Hmm, it might be possible to change the behavior to mark a few of the iclks as functional clocks. Or alternatively take a hwmod based approach that will just track active hwmods instead of clocks + hwmods and leave the clocks as they are. Still the PER domain will require additional tweaks due to the wake dep stuff. I think I'll experiment with this a bit and see.

Quoted 2 lines

I added Paul to this discussion to see what his thoughts are on this.

Yea, would be nice to get Paul's comments how he thinks this should be handled.

Quoted 4 lines
Quoted 2 lines

This can be avoided by preventing PER domain idle on omap3530.

That doesn't sound quite right either.

I think I was somewhat confusing with this comment. PER domain idle is not prevented, we just prevent the manual SW triggered transitions for PER.

Quoted 8 lines
Quoted 5 lines

I believe on omap3430 you also need to prevent similar stuff for CORE domain.

Did you try the patch I posted last week for the core domain + dplls?

I didn't try it until just now and it' doesn't compile because the new flag you're using doesn't seem to be defined anywhere.

You need to use that patch in addition to the usecount set.

-Tero

Quoted 14 lines

Kevin

Quoted 11 lines

Does it help with 3430? I guess n900 could hang in the ROM code if PER domain is idle during wakeup, as it is a HS device also and behaves differently during wakeup from OFF.

-Tero

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

#28 lore raw

PATCH v2 8/8 arm: omap3: prevent per_clkdm from attempting manual domain transitions

Quoted 211 lines · Tero Kristo
Quoted 183 lines · Kevin Hilman

+Paul

Quoted 179 lines · Tero Kristo
Quoted 177 lines · Kevin Hilman
Quoted 162 lines · Tero Kristo
Quoted 77 lines · Kevin Hilman
Quoted 66 lines · Tero Kristo
Quoted 57 lines · Santosh
Quoted 48 lines · Tero Kristo
Quoted 22 lines · Tero Kristo
Quoted 21 lines · Santosh
Quoted 8 lines · Tero Kristo
Quoted 6 lines · Kevin Hilman

Tero Kristo <[email protected]> writes:
P
Attempting this will cause problems especially with off-mode enabled.

Please be more verbose about the problems seen, and the root cause(s).

Quoted 8 lines

I was actually looking forward for some help with this commit message, as I am still not quite sure what is going on in here. :) Here is the log for suspend (btw, cam_pwrdm does not go to off in mainline yet, but I think that is probably fixed by the patch from Paul, omap_set_pwrdm_state() does not work properly.) The warning comes out after wakeup from off-mode, and it is triggered by the disabling of autodeps before off-mode entry.

This mostly indicates that one of the per clock-domain module clock turning ON seems to be not working well with auto deps disabled. This leads to interconnect violation.

Quoted 7 lines
Quoted 3 lines

if not tried already, can you put the per_clockdomain in SW_WKUP in the low power code early resume path and see if this error goes away.

This seems to get rid of the dump also. It looks like some driver resume is not behaving nicely, I am trying to pinpoint the culprit currently and see whether it can provide more info.

Okay, I have some more info about this now.

What happens is that when entering off-mode, PER domain remains OFF even during the execution of the exit phase from omap_sram_idle. Adding a manual SW_WKUP it comes up and there are no issues. If autodeps are enabled on the domain, it comes back from off mode as active.

Looking further in the code, we have this at the end of omap_sram_idle:

       if (per_next_state < PWRDM_POWER_ON) {
               per_prev_state = pwrdm_read_prev_pwrst(per_pwrdm);
               omap2_gpio_resume_after_idle();
               wake_per();
               if (per_prev_state == PWRDM_POWER_OFF)
                       omap3_per_restore_context();
       }
Quoted 8 lines

... which seems to assume that per domain is on. Gpio code does not control any clocks currently, as it only requires the interface clock to be on, and as this is autoidled....

Any comments how we should handle this? Shall we just keep these two patches for handling this or add some different hackery for the gpio issue?

Quoted 7 lines

Good. I thought too that issue will disappear.
The issue is pretty clear. Technically every driver pm_runtime() code should be able to manage a clock->clockdomain->power domain power up/down sequence. That should work without auto deps.

Do you narrowed down which driver resume is creating the dump ? UART , GPIO ?

Quoted 8 lines

It is the gpio base stuff called from omap_sram_idle(), basically the restore context part. If I force enable per domain before the code snippet before, there is no dump, but if it is done after, I get the dump.

The thing is that gpio driver doesn't currently have this kind of mechanism for the restore context part, at least not on omap3 due to above clocking issue (only autoidled interface clock is used.)

Quoted 6 lines

I'm not sure if it fully addresses this, but Tarun's series converts GPIO to runtime PM.

Can you try with Tarun's series. See the for_3.4/gpio_cleanup_fixes_v9 branch here:
git://gitorious.org/~tarunkanti/omap-sw-develoment/tarunkantis-linux-omap-dev.git

It does something for the issue, but I still get this during suspend to off:

[   11.284973] PM: Syncing filesystems ... done.
[   11.379241] Freezing user space processes ... (elapsed 0.02 seconds)
done.
[   11.408233] Freezing remaining freezable tasks ... (elapsed 0.02
seconds) don
e.
[   11.439239] Suspending console(s) (use no_console_suspend to debug)
[   11.564178] PM: suspend of devices complete after 115.506 msecs
[   11.567382] PM: late suspend of devices complete after 3.204 msecs
[   11.567443] Disabling non-boot CPUs ...
[   12.004089] Powerdomain (cam_pwrdm) didn't enter target state 0
[   12.004119] Could not enter target state in pm_suspend
[   12.009368] PM: early resume of devices complete after 4.944 msecs
[   12.436645] PM: resume of devices complete after 426.086 msecs
[   12.480285] Restarting tasks ... done.
/sys/kernel/debu[   12.488433] ------------[ cut here ]------------
[   12.494415] WARNING: at arch/arm/mach-omap2/omap_hwmod.c:1604 _idle
+0x164/0x1
7c()
[   12.502258] omap_hwmod: gpio6: idle state can only be entered from
enabled st
ate
[   12.509979] Modules linked in:
[   12.513214] [<c001bcd0>] (unwind_backtrace+0x0/0xf4) from
[<c0042968>] (warn_
slowpath_common+0x4c/0x64)
[   12.523071] [<c0042968>] (warn_slowpath_common+0x4c/0x64) from
[<c0042a14>] (
warn_slowpath_fmt+0x30/0x40)
[   12.533081] [<c0042a14>] (warn_slowpath_fmt+0x30/0x40) from
[<c0028d68>] (_id
le+0x164/0x17c)
[   12.541931] [<c0028d68>] (_idle+0x164/0x17c) from [<c00298b0>]
(omap_hwmod_id
le+0x28/0x3c)
[   12.550567] [<c00298b0>] (omap_hwmod_idle+0x28/0x3c) from
[<c003c664>] (omap_
device_idle_hwmods+0x24/0x3c)
[   12.560699] [<c003c664>] (omap_device_idle_hwmods+0x24/0x3c) from
[<c003c898>
] (_omap_device_deactivate+0xa4/0x138)
[   12.571594] [<c003c898>] (_omap_device_deactivate+0xa4/0x138) from

[<c003c954

Quoted 1 line

] (omap_device_idle+0x28/0x54)

[   12.581909] [<c003c954>] (omap_device_idle+0x28/0x54) from
[<c003c99c>] (_od_
runtime_suspend+0x1c/0x24)
[   12.591735] [<c003c99c>] (_od_runtime_suspend+0x1c/0x24) from
[<c02c5010>] (_
_rpm_callback+0x2c/0x78)
[   12.601379] [<c02c5010>] (__rpm_callback+0x2c/0x78) from [<c02c54d0>]
(rpm_su
spend+0x264/0x6c4)
[   12.610504] [<c02c54d0>] (rpm_suspend+0x264/0x6c4) from [<c02c6998>]
(__pm_ru
ntime_suspend+0x5c/0x74)
[   12.620178] [<c02c6998>] (__pm_runtime_suspend+0x5c/0x74) from
[<c02731a4>] (
omap2_gpio_prepare_for_idle+0x50/0x64)
[   12.631103] [<c02731a4>] (omap2_gpio_prepare_for_idle+0x50/0x64) from
[<c002b
d30>] (omap_sram_idle+0xa0/0x3b0)
[   12.641571] [<c002bd30>] (omap_sram_idle+0xa0/0x3b0) from
[<c002c1d8>] (omap3
_pm_idle+0x60/0x178)
[   12.650848] [<c002c1d8>] (omap3_pm_idle+0x60/0x178) from [<c0015c10>]
(cpu_id
le+0xc4/0x108)
[   12.659606] [<c0015c10>] (cpu_idle+0xc4/0x108) from [<c0626828>]
(start_kerne
l+0x2b0/0x304)
[   12.668365] ---[ end trace 441b8fea2b56dcb1 ]---
Quoted 3 lines

Also, this goes away if I manually force wakeup for the per domain, so this might be caused by some additional latency.

What platform are you testing on?

When I test Tarun's series with off-mode on 3430/n900 suspend/resume works fine. But when I add your v2 series, it never comes out of suspend, and I don't get a kernel dump either.

Can you debug this a little further so we can explain what's going on here?

Quoted 7 lines
Quoted 4 lines

After some debugging, it looks like the root cause for the WARN dump I see with GPIO PM runtime + usecounting is the fact that PER domain enters SW assisted idle and remains OFF after wakeup from OFF mode, and it will remain there until some functional clock is enabled.

OK, but before your usecounting changes, the runtime PM enable was successfully bringing GPIO out of idle.

Well, the thing is in this case, PER never idles by SW, it is only following MPU to idle and back. PER awakens immediately by wakeup from OFF, which is not the case with the usecounting set.

Quoted 9 lines

In the case of GPIO, the iclk is also the functional clock (it's the main_clk in the hwmod.)

Quoted 5 lines

GPIO6 module is only enabling its interface clock, but as this is autoidled, it will not wake up PER domain. This will cause the initial _wait_target_ready(oh) to fail with EBUSY, leaving the hwmod into idle state. Next cpuidle entry will attempt to disable the already idle hwmod causing this dump.

Quoted 5 lines

I see. Thanks for tracking it down.

To me, this suggests that the autoidle changes in your usecounting series aren't quite right, especially for modules like GPIO where the iclk is also the functional clock.

Quoted 6 lines

Hmm, it might be possible to change the behavior to mark a few of the iclks as functional clocks. Or alternatively take a hwmod based approach that will just track active hwmods instead of clocks + hwmods and leave the clocks as they are. Still the PER domain will require additional tweaks due to the wake dep stuff. I think I'll experiment with this a bit and see.

If I change the gpio*_ick clkops from clkops_omap2_iclk_dflt_wait to clkops_omap2_dflt_wait, the issue is gone, this basically prevents autoidle for these clocks. The hwmod approach does not work for omap3 currently, as most of the hwmods don't have clkdm defined for them (only a couple of USB hwmods have this.)

Anyway, I think we still need to prevent the manual transition tries for per / core domains to avoid any trouble. There are even a couple of omap3 erratas related to per wkdeps and how they should not be disabled when going to idle.

-Tero

Quoted 44 lines
Quoted 3 lines

I added Paul to this discussion to see what his thoughts are on this.

Yea, would be nice to get Paul's comments how he thinks this should be handled.

Quoted 4 lines
Quoted 2 lines

This can be avoided by preventing PER domain idle on omap3530.

That doesn't sound quite right either.

I think I was somewhat confusing with this comment. PER domain idle is not prevented, we just prevent the manual SW triggered transitions for PER.

Quoted 8 lines
Quoted 5 lines

I believe on omap3430 you also need to prevent similar stuff for CORE domain.

Did you try the patch I posted last week for the core domain + dplls?

I didn't try it until just now and it' doesn't compile because the new flag you're using doesn't seem to be defined anywhere.

You need to use that patch in addition to the usecount set.

-Tero

Quoted 14 lines

Kevin

Quoted 11 lines

Does it help with 3430? I guess n900 could hang in the ROM code if PER domain is idle during wakeup, as it is a HS device also and behaves differently during wakeup from OFF.

-Tero

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

#29 lore raw

v2 cover arm: omap: clk/clkdm/pwrdm/voltdm usecounting changes

In reply to Tero Kristo

Hi Tero,

Great to see those changes merged in.
FWIW:

Jean

Quoted 21 lines · Tero Kristo

Changes compared to previous version:

- moved voltdm_pwrdm_enable / disable hooks from idle code to ?pwrdm_pre_transition / pwrdm_post_transition functions (patch 4)
- added patches 7 and 8 to fix one issue that was visible with off-mode ?on omap3: attempting to disable wkdeps for per_clkdm causes crash ?dumps to be generated by omap_l3_smx (this was actually part of the ?original larger set but I dropped it accidentally from v1), this issue ?surfaces with this patch set as the kernel actually tries to manually ?idle per_clkdm (probably first time in history)
- added some parameter sanity checks (pwrdm/voltdm = null checks)

Similar tests done as with previous version, additionally off-mode tests with omap3.

-Tero

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majordomo at vger.kernel.org
More majordomo info at ?http://vger.kernel.org/majordomo-info.html