ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Subsystems (3)
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with CPU_IDLE enabled. The root cause of the issue is missing interrupts while in idle state. Commit cb7094e8 {cpuidle / omap4 :
use CPUIDLE_FLAG_TIMER_STOP flag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commit ff999b8a {ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.
Patch reverts commit cb7094 {cpuidle / omap4 : use CPUIDLE_FLAG_TIMER_STOP flag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.
--- arch/arm/mach-omap2/cpuidle44xx.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/arm/mach-omap2/cpuidle44xx.c +8 -4
@@ -14,6 +14,7 @@#include <linux/cpuidle.h> #include <linux/cpu_pm.h> #include <linux/export.h> +#include <linux/clockchips.h> #include <asm/cpuidle.h> #include <asm/proc-fns.h>
@@ -83,6 +84,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, {struct idle_statedata *cx = state_ptr + index;u32 mpuss_can_lose_context =0 ; +int cpu_id =smp_processor_id ();/* * CPU0 has to wait and stay ON until CPU1 is OFF state.
@@ -110,6 +112,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, mpuss_can_lose_context = (cx->mpu_state ==PWRDM_POWER_RET ) && (cx->mpu_logic_state ==PWRDM_POWER_OFF ); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved.
@@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id); + fail:cpuidle_coupled_parallel_barrier (dev, &abort_barrier); cpu_done[dev->cpu] = false;
@@ -189,8 +195,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR",
@@ -199,8 +204,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR",
Signature · 1 line
1.7.9.5
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 8 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.
Quoted 4 lines
Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.
Thanks for figuring this regression out. Applying into fixes with Cc stable.
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
In reply to Santosh ShilimkarQuoted 74 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-omap2/cpuidle44xx.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)arch/arm/mach-omap2/cpuidle44xx.c +7
-2@@ -14,6 +14,7 @@#include <linux/cpuidle.h> #include <linux/cpu_pm.h> #include <linux/export.h> +#include <linux/clockchips.h> #include <asm/cpuidle.h> #include <asm/proc-fns.h> @@ -83,6 +84,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, {struct idle_statedata *cx = state_ptr + index;u32 mpuss_can_lose_context =0 ; +int cpu_id =smp_processor_id ();/* * CPU0 has to wait and stay ON until CPU1 is OFF state. @@ -110,6 +112,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, mpuss_can_lose_context = (cx->mpu_state ==PWRDM_POWER_RET ) && (cx->mpu_logic_state ==PWRDM_POWER_OFF ); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id); + fail:cpuidle_coupled_parallel_barrier (dev, &abort_barrier); cpu_done[dev->cpu] = false;@@ -189,8 +195,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR",
arch/arm/mach-omap2/cpuidle44xx.c static struct cpuidle_driver omap4_idle_driver = { · line 210
Quoted 9 lines
arch/arm/mach-omap2/cpuidle44xx.c +1
-2@@ -199,8 +204,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR",
Shouldn't the broadcast timer to be setup with CLOCK_EVT_NOTIFY_BROADCAST_ON also ?
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 78 lines · Daniel Lezcano
Quoted 77 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-omap2/cpuidle44xx.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)arch/arm/mach-omap2/cpuidle44xx.c +7
-2@@ -14,6 +14,7 @@#include <linux/cpuidle.h> #include <linux/cpu_pm.h> #include <linux/export.h> +#include <linux/clockchips.h> #include <asm/cpuidle.h> #include <asm/proc-fns.h> @@ -83,6 +84,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, {struct idle_statedata *cx = state_ptr + index;u32 mpuss_can_lose_context =0 ; +int cpu_id =smp_processor_id ();/* * CPU0 has to wait and stay ON until CPU1 is OFF state. @@ -110,6 +112,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, mpuss_can_lose_context = (cx->mpu_state ==PWRDM_POWER_RET ) && (cx->mpu_logic_state ==PWRDM_POWER_OFF ); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id); + fail:cpuidle_coupled_parallel_barrier (dev, &abort_barrier); cpu_done[dev->cpu] = false;@@ -189,8 +195,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR",@@ -199,8 +204,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100,
Quoted 9 lines
arch/arm/mach-omap2/cpuidle44xx.c static struct cpuidle_driver omap4_idle_driver = { · line 210
Quoted 6 lines
arch/arm/mach-omap2/cpuidle44xx.c +1
-2- .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR",Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?
Which is already taken care by __cpuidle_register_driver(). Note that setup code is still used from generic code...
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 24 lines · Santosh Shilimkar
Quoted 23 lines · Daniel Lezcano
Quoted 22 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>---
[ ... ]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
arch/arm/mach-omap2/cpuidle44xx.c +3
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id);
[ ... ]
Quoted 5 lines
Quoted 3 lines
Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?Which is already taken care by
__cpuidle_register_driver(). Note that setup code is still used from generic code...
Nope, if the flag CPUIDLE_FLAG_TIMER_STOP is not set, the cpuidle framework won't setup the timer.
static void __cpuidle_driver_init(struct cpuidle_driver *drv) {
...
for (i = drv->state_count - 1; i >= 0 ; i--) {
if (drv->states[i].flags & CPUIDLE_FLAG_TIMER_STOP) {
drv->bctimer = 1;
break;
}
}
...}
static int __cpuidle_register_driver(struct cpuidle_driver *drv)
{
...
if (drv->bctimer)
on_each_cpu_mask(drv->cpumask,
cpuidle_setup_broadcast_timer,
(void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1);
...
}So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 79 lines · Daniel Lezcano
Quoted 24 lines · Santosh Shilimkar
Quoted 23 lines · Daniel Lezcano
Quoted 22 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>---
[ ... ]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
arch/arm/mach-omap2/cpuidle44xx.c +3
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id);[ ... ]
Quoted 5 lines
Quoted 3 lines
Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?Which is already taken care by
__cpuidle_register_driver(). Note that setup code is still used from generic code...Nope, if the flag
CPUIDLE_FLAG_TIMER_STOPis not set, the cpuidle framework won't setup the timer.static void
__cpuidle_driver_init(struct cpuidle_driver *drv){... for (i = drv->state_count - 1; i >= 0 ; i--) { if (drv->states[i].flags & CPUIDLE_FLAG_TIMER_STOP) { drv->bctimer = 1; break; } } ...}
static int
__cpuidle_register_driver(struct cpuidle_driver *drv){ ... if (drv->bctimer) on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, (void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1); ... }
Quoted 2 lines
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
OK I'll hold on sending anything until there's some conclusion.
Are you able to reproduce the problem with current Linux next?
Regards,
Tony
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 8 lines · Tony Lindgren
Quoted 3 lines · Daniel Lezcano
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
OK I'll hold on sending anything until there's some conclusion.
Are you able to reproduce the problem with current Linux next?
BTW, I'm also noticing now hangs on omap3 with my PM patches on v3.15-rc4 that seem similar to the panda cpuidle hang.
The hangs on omap3 do not happen on my v3.14 based branch, so I'm wondering if there are some recent cpuidle regressions too in play?
Regards,
Tony
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 8 lines · Tony Lindgren
Quoted 7 lines · Tony Lindgren
Quoted 3 lines · Daniel Lezcano
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
OK I'll hold on sending anything until there's some conclusion.
Quoted 8 lines
Quoted 1 line
Are you able to reproduce the problem with current Linux next?
BTW, I'm also noticing now hangs on omap3 with my PM patches on v3.15-rc4 that seem similar to the panda cpuidle hang.
The hangs on omap3 do not happen on my v3.14 based branch, so I'm wondering if there are some recent cpuidle regressions too in play?
Looks like the omap3 idle hang is caused by commit 6ddeb6d84459 (dmaengine: omap-dma: move IRQ handling to omap-dma). This may not be related to omap4 cpu_idle hang, but it might.
I'm trying to figure out if it's related to something like omap_dma_global_context_save and omap_dma_global_context_restore.
Meanwhile, to test the DMA changes have an effect on the omap4 cpu_idle issue, you can try reverting the following two patches:
aa4c5b962a7a dmaengine: omap-dma: more consolidation of CCR register setup 6ddeb6d84459 dmaengine: omap-dma: move IRQ handling to omap-dma
Regards,
Tony
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 22 lines · Tony Lindgren
Quoted 8 lines · Tony Lindgren
Quoted 7 lines · Tony Lindgren
Quoted 3 lines · Daniel Lezcano
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
OK I'll hold on sending anything until there's some conclusion.
Quoted 8 lines
Quoted 1 line
Are you able to reproduce the problem with current Linux next?
BTW, I'm also noticing now hangs on omap3 with my PM patches on v3.15-rc4 that seem similar to the panda cpuidle hang.
The hangs on omap3 do not happen on my v3.14 based branch, so I'm wondering if there are some recent cpuidle regressions too in play?
Looks like the omap3 idle hang is caused by commit
6ddeb6d84459(dmaengine: omap-dma: move IRQ handling to omap-dma). This may not be related to omap4cpu_idlehang, but it might.
Quoted 8 lines
I'm trying to figure out if it's related to something like
omap_dma_global_context_saveandomap_dma_global_context_restore.Meanwhile, to test the DMA changes have an effect on the omap4
cpu_idleissue, you can try reverting the following two patches:
aa4c5b962a7admaengine: omap-dma: more consolidation of CCR register setup6ddeb6d84459dmaengine: omap-dma: move IRQ handling to omap-dma
OK with the above reverted still seeing hangs on my panda es. So it seems the cpu_idle issue is different.
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
In reply to Daniel LezcanoQuoted 43 lines · Daniel Lezcano
Quoted 24 lines · Santosh Shilimkar
Quoted 23 lines · Daniel Lezcano
Quoted 22 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>---
[ ... ]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
arch/arm/mach-omap2/cpuidle44xx.c +3
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id);[ ... ]
Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?Which is already taken care by
__cpuidle_register_driver(). Note that setup code is still used from generic code...Nope, if the flag
CPUIDLE_FLAG_TIMER_STOPis not set, the cpuidle framework won't setup the timer.
I see. I assumed it was taken care. So we might have setup the timer too.
Quoted 7 lines
static void
__cpuidle_driver_init(struct cpuidle_driver *drv){... for (i = drv->state_count - 1; i >= 0 ; i--) { if (drv->states[i].flags & CPUIDLE_FLAG_TIMER_STOP) {
May be you should start the bc timer in case 'CPUIDLE_FLAG_TIMER_STOP' or 'CPUIDLE_FLAG_COUPLED'
Quoted 21 lines
drv->bctimer = 1; break; } } ...}
static int
__cpuidle_register_driver(struct cpuidle_driver *drv){ ... if (drv->bctimer) on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, (void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1); ... }
Quoted 2 lines
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Daniel,
Quoted 83 lines · Santosh Shilimkar
Quoted 43 lines · Daniel Lezcano
Quoted 24 lines · Santosh Shilimkar
Quoted 23 lines · Daniel Lezcano
Quoted 22 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>---
[ ... ]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
arch/arm/mach-omap2/cpuidle44xx.c +3
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id);[ ... ]
Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?Which is already taken care by
__cpuidle_register_driver(). Note that setup code is still used from generic code...Nope, if the flag
CPUIDLE_FLAG_TIMER_STOPis not set, the cpuidle framework won't setup the timer.I see. I assumed it was taken care. So we might have setup the timer too.
Quoted 7 lines
static void
__cpuidle_driver_init(struct cpuidle_driver *drv){... for (i = drv->state_count - 1; i >= 0 ; i--) { if (drv->states[i].flags & CPUIDLE_FLAG_TIMER_STOP) {May be you should start the bc timer in case '
CPUIDLE_FLAG_TIMER_STOP' or 'CPUIDLE_FLAG_COUPLED'Quoted 21 lines
drv->bctimer = 1; break; } } ...}
static int
__cpuidle_register_driver(struct cpuidle_driver *drv){ ... if (drv->bctimer) on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, (void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1); ... }
Quoted 8 lines
Quoted 2 lines
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under CPUIDLE_FLAG_COUPLED seems a good way forward.
Do let me know.
Regards,
Santosh
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 92 lines · Santosh Shilimkar
Daniel,
Quoted 89 lines · Santosh Shilimkar
Quoted 43 lines · Daniel Lezcano
Quoted 24 lines · Santosh Shilimkar
Quoted 23 lines · Daniel Lezcano
Quoted 22 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>---
[ ... ]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
arch/arm/mach-omap2/cpuidle44xx.c +3
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id);[ ... ]
Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?Which is already taken care by
__cpuidle_register_driver(). Note that setup code is still used from generic code...Nope, if the flag
CPUIDLE_FLAG_TIMER_STOPis not set, the cpuidle framework won't setup the timer.I see. I assumed it was taken care. So we might have setup the timer too.
Quoted 7 lines
static void
__cpuidle_driver_init(struct cpuidle_driver *drv){... for (i = drv->state_count - 1; i >= 0 ; i--) { if (drv->states[i].flags & CPUIDLE_FLAG_TIMER_STOP) {May be you should start the bc timer in case '
CPUIDLE_FLAG_TIMER_STOP' or 'CPUIDLE_FLAG_COUPLED'Quoted 21 lines
drv->bctimer = 1; break; } } ...}
static int
__cpuidle_register_driver(struct cpuidle_driver *drv){ ... if (drv->bctimer) on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, (void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1); ... }Quoted 2 lines
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Quoted 7 lines
Quoted 2 lines
Ofcourse if you have better fix, then great.
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under
CPUIDLE_FLAG_COUPLEDseems a good way forward.Do let me know.
Did you see Alex Shi's email [cc'ed] ? Reverting this change makes the panda ES to hang.
I am not convinced the culprit is this code you are trying to revert.
I will try to reproduce the bug on my board.
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 95 lines · Daniel Lezcano
Quoted 94 lines · Santosh Shilimkar
Daniel,
Quoted 91 lines · Santosh Shilimkar
Quoted 43 lines · Daniel Lezcano
Quoted 24 lines · Santosh Shilimkar
Quoted 23 lines · Daniel Lezcano
Quoted 22 lines · Santosh Shilimkar
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>---
[ ... ]
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
(quoted diff) +3
-0+ clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id); + /* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved.@@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, if (dev->cpu == 0 && mpuss_can_lose_context) cpu_cluster_pm_exit(); + clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);[ ... ]
Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Shouldn't the broadcast timer to be setup with
CLOCK_EVT_NOTIFY_BROADCAST_ONalso ?Which is already taken care by
__cpuidle_register_driver(). Note that setup code is still used from generic code...Nope, if the flag
CPUIDLE_FLAG_TIMER_STOPis not set, the cpuidle framework won't setup the timer.I see. I assumed it was taken care. So we might have setup the timer too.
Quoted 7 lines
static void
__cpuidle_driver_init(struct cpuidle_driver *drv){... for (i = drv->state_count - 1; i >= 0 ; i--) { if (drv->states[i].flags & CPUIDLE_FLAG_TIMER_STOP) {May be you should start the bc timer in case '
CPUIDLE_FLAG_TIMER_STOP' or 'CPUIDLE_FLAG_COUPLED'Quoted 21 lines
drv->bctimer = 1; break; } } ...}
static int
__cpuidle_register_driver(struct cpuidle_driver *drv){ ... if (drv->bctimer) on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, (void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1); ... }Quoted 2 lines
So the broadcast timer does not operate with this revert. Moreover, I am not sure reverting this patch is the right solution.
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
Quoted 8 lines
Quoted 5 lines
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under
CPUIDLE_FLAG_COUPLEDseems a good way forward.Do let me know.
Did you see Alex Shi's email [cc'ed] ? Reverting this change makes the panda ES to hang.
The hang is definitely due to the bctimer not started. As I said, I assumed it was and then you corrected saying it is under the flag.
Quoted 2 lines
I am not convinced the culprit is this code you are trying to revert.
fair enough. Thats why I said if you have an alternative fix thats great.
Quoted 2 lines
I will try to reproduce the bug on my board.
Sure...
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 2 lines · Santosh Shilimkar
Quoted 1 line · Daniel Lezcano
On 05/15/2014 07:03 PM, Santosh Shilimkar wrote:
[..]
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under
CPUIDLE_FLAG_COUPLEDseems a good way forward.Do let me know.
Quoted 9 lines
Quoted 2 lines
Did you see Alex Shi's email [cc'ed] ? Reverting this change makes the panda ES to hang.
The hang is definitely due to the bctimer not started. As I said, I assumed it was and then you corrected saying it is under the flag.
Quoted 2 lines
I am not convinced the culprit is this code you are trying to revert.
fair enough. Thats why I said if you have an alternative fix thats great.
For record, below is updated patch with bctimer started which was missed in earlier version. I haven't tested it though.
Alex,
Please give a try with your test-case and see if you still see the hang. Am just curious about your issue and hence the request..
Regards,
Santosh
Quoted 1 line
From
bb3b82cc5645b83bedf1343d03cc956f27f6fc83Mon Sep 17 00:00:00 2001
From: Santosh Shilimkar <[email protected]>
Date: Mon, 12 May 2014 17:37:59 -0400
Subject: [PATCH] ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
On OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with CPU_IDLE enabled. The root cause of the issue is missing interrupts while in idle state. Commit cb7094e8 {cpuidle / omap4 :
use CPUIDLE_FLAG_TIMER_STOP flag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commit ff999b8a {ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.
Patch reverts commit cb7094 {cpuidle / omap4 : use CPUIDLE_FLAG_TIMER_STOP flag} and 54769d6 {cpuidle: OMAP4: remove timer broadcast initialization} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.
--- arch/arm/mach-omap2/cpuidle44xx.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
arch/arm/mach-omap2/cpuidle44xx.c +21 -4
@@ -14,6 +14,7 @@#include <linux/cpuidle.h> #include <linux/cpu_pm.h> #include <linux/export.h> +#include <linux/clockchips.h> #include <asm/cpuidle.h> #include <asm/proc-fns.h>
@@ -83,6 +84,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, {struct idle_statedata *cx = state_ptr + index;u32 mpuss_can_lose_context =0 ; +int cpu_id =smp_processor_id ();/* * CPU0 has to wait and stay ON until CPU1 is OFF state.
@@ -110,6 +112,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, mpuss_can_lose_context = (cx->mpu_state ==PWRDM_POWER_RET ) && (cx->mpu_logic_state ==PWRDM_POWER_OFF ); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved.
@@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id); + fail:cpuidle_coupled_parallel_barrier (dev, &abort_barrier); cpu_done[dev->cpu] = false;
@@ -172,6 +178,16 @@ fail:return index; } +/* +* For each cpu, setup the broadcast timer because local timers +* stops for the states above C1. +*/ +static void omap_setup_broadcast_timer (void *arg) +{ +int cpu =smp_processor_id (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ON , &cpu); +} +static struct cpuidle_driver omap4_idle_driver = { .name ="omap4_idle" , .owner =THIS_MODULE ,
@@ -189,8 +205,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR",
@@ -199,8 +214,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR",
@@ -231,5 +245,8 @@ int __init omap4_idle_init(void)if (!cpu_clkdm[0 ] || !cpu_clkdm[1 ])return -ENODEV ; +/* Configure the broadcast timer on each cpu */ +on_each_cpu (omap_setup_broadcast_timer,NULL ,1 ); +return cpuidle_register (&omap4_idle_driver, cpu_online_mask); }
Signature · 1 line
1.7.9.5
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 3 lines · Santosh Shilimkar
Alex,
Please give a try with your test-case and see if you still see the hang. Am just curious about your issue and hence the request..
there is no test case. I just patched your patches, then boot new kernel, system quickly to hang without any serial port oops, I am using pandaES/ubuntu.
Signature · 2 lines
Thanks
AlexPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
In reply to Santosh ShilimkarTony,
Quoted 25 lines · Santosh Shilimkar
Quoted 2 lines · Santosh Shilimkar
Quoted 1 line · Daniel Lezcano
On 05/15/2014 07:03 PM, Santosh Shilimkar wrote:
[..]
Quoted 19 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
Quoted 8 lines
Quoted 5 lines
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under
CPUIDLE_FLAG_COUPLEDseems a good way forward.Do let me know.
Did you see Alex Shi's email [cc'ed] ? Reverting this change makes the panda ES to hang.
The hang is definitely due to the bctimer not started. As I said, I assumed it was and then you corrected saying it is under the flag.
Quoted 2 lines
I am not convinced the culprit is this code you are trying to revert.
Quoted 9 lines
Quoted 2 lines
fair enough. Thats why I said if you have an alternative fix thats great.
For record, below is updated patch with bctimer started which was missed in earlier version. I haven't tested it though.
Alex,
Please give a try with your test-case and see if you still see the hang. Am just curious about your issue and hence the request..
Alex tested below patch and he don't see the hang so the patch is addressing the issue.
If Daniel works out an alternate fix to avoid reverts, that will be great but if not, we should merge the below patch. I let you take call on it.
Quoted 118 lines
From
bb3b82cc5645b83bedf1343d03cc956f27f6fc83Mon Sep 17 00:00:00 2001
From: Santosh Shilimkar <[email protected]>
Date: Mon, 12 May 2014 17:37:59 -0400
Subject: [PATCH] ARM: OMAP4: Fix the boot regression withCPU_IDLEenabledOn OMAP4 panda board, there have been several bug reports about boot hang and lock-ups with
CPU_IDLEenabled. The root cause of the issue is missing interrupts while in idle state. Commitcb7094e8{cpuidle / omap4 :
useCPUIDLE_FLAG_TIMER_STOPflag} moved the broadcast notifiers to common code for right reasons but on OMAP4 which suffers from a nasty ROM code bug with GIC, commitff999b8a{ARM: OMAP4460: Workaround for ROM bug ..}, we loose interrupts which leads to issues like lock-up, hangs etc.Patch reverts commit cb7094 {cpuidle / omap4 : use
CPUIDLE_FLAG_TIMER_STOPflag} and 54769d6 {cpuidle: OMAP4: remove timer broadcast initialization} to avoid the issue. With this change, OMAP4 panda boards, the mentioned issues are getting fixed. We no longer loose interrupts which was the cause of the regression.Cc Roger Quadros <[email protected]>Cc Kevin Hilman <[email protected]>Cc Tony Lindgren <[email protected]>Cc Daniel Lezcano <[email protected]>Reported-tested-by Roger Quadros <[email protected]>Reported-tested-by Kevin Hilman <[email protected]>Tested-by Tony Lindgren <[email protected]>Signed-off-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-omap2/cpuidle44xx.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)arch/arm/mach-omap2/cpuidle44xx.c +21
-4@@ -14,6 +14,7 @@#include <linux/cpuidle.h> #include <linux/cpu_pm.h> #include <linux/export.h> +#include <linux/clockchips.h> #include <asm/cpuidle.h> #include <asm/proc-fns.h> @@ -83,6 +84,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, {struct idle_statedata *cx = state_ptr + index;u32 mpuss_can_lose_context =0 ; +int cpu_id =smp_processor_id ();/* * CPU0 has to wait and stay ON until CPU1 is OFF state. @@ -110,6 +112,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, mpuss_can_lose_context = (cx->mpu_state ==PWRDM_POWER_RET ) && (cx->mpu_logic_state ==PWRDM_POWER_OFF ); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu_id); +/* * Call idle CPU PM enter notifier chain so that * VFP and per CPU interrupt context is saved. @@ -165,6 +169,8 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,if (dev->cpu ==0 && mpuss_can_lose_context)cpu_cluster_pm_exit (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu_id); + fail:cpuidle_coupled_parallel_barrier (dev, &abort_barrier); cpu_done[dev->cpu] = false;@@ -172,6 +178,16 @@ fail:return index; } +/* +* For each cpu, setup the broadcast timer because local timers +* stops for the states above C1. +*/ +static void omap_setup_broadcast_timer (void *arg) +{ +int cpu =smp_processor_id (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ON , &cpu); +} +static struct cpuidle_driver omap4_idle_driver = { .name ="omap4_idle" , .owner =THIS_MODULE ,@@ -189,8 +205,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR",@@ -199,8 +214,7 @@ static struct cpuidle_driver omap4_idle_driver = {/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100, - .flags =CPUIDLE_FLAG_TIME_VALID |CPUIDLE_FLAG_COUPLED | - CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR",@@ -231,5 +245,8 @@ int __init omap4_idle_init(void)if (!cpu_clkdm[0 ] || !cpu_clkdm[1 ])return -ENODEV ; +/* Configure the broadcast timer on each cpu */ +on_each_cpu (omap_setup_broadcast_timer,NULL ,1 ); +return cpuidle_register (&omap4_idle_driver, cpu_online_mask); }
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 37 lines · Santosh Shilimkar
Tony,
Quoted 25 lines · Santosh Shilimkar
Quoted 2 lines · Santosh Shilimkar
Quoted 1 line · Daniel Lezcano
On 05/15/2014 07:03 PM, Santosh Shilimkar wrote:
[..]
Quoted 19 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
Quoted 8 lines
Quoted 5 lines
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under
CPUIDLE_FLAG_COUPLEDseems a good way forward.Do let me know.
Did you see Alex Shi's email [cc'ed] ? Reverting this change makes the panda ES to hang.
The hang is definitely due to the bctimer not started. As I said, I assumed it was and then you corrected saying it is under the flag.
Quoted 2 lines
I am not convinced the culprit is this code you are trying to revert.
Quoted 9 lines
Quoted 2 lines
fair enough. Thats why I said if you have an alternative fix thats great.
For record, below is updated patch with bctimer started which was missed in earlier version. I haven't tested it though.
Alex,
Please give a try with your test-case and see if you still see the hang. Am just curious about your issue and hence the request..
Quoted 5 lines
Alex tested below patch and he don't see the hang so the patch is addressing the issue.
If Daniel works out an alternate fix to avoid reverts, that will be great but if not, we should merge the below patch. I let you take call on it.
Daniel any news on this?
And just to recap, this problem can be reproduced with current Linux next with omap2plus_defconfig with CONFIG_CPU_IDLE enabled. The system should hang during the boot at some point.
And for the record, the omap3 hang fix is now posted to the lists as "[PATCH] ARM: OMAP2+: Fix DMA hang after off-idle". This should not have anything to do with the omap4 cpu_idle hang as omap4 does not currently lose context during idle.
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 41 lines · Tony Lindgren
Quoted 40 lines · Santosh Shilimkar
Tony,
Quoted 25 lines · Santosh Shilimkar
Quoted 2 lines · Santosh Shilimkar
Quoted 1 line · Daniel Lezcano
On 05/15/2014 07:03 PM, Santosh Shilimkar wrote:
[..]
Quoted 19 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
With above mentioned change, it should work. Other alternatives is OMAP4 driver does its won registration where it can start the timer. The way it was before the consolidation.
Ofcourse if you have better fix, then great.
Quoted 8 lines
Quoted 5 lines
What is your suggestion. We *must* fix the regression asap. I think $subject patch with an update to bctimer start under
CPUIDLE_FLAG_COUPLEDseems a good way forward.Do let me know.
Did you see Alex Shi's email [cc'ed] ? Reverting this change makes the panda ES to hang.
The hang is definitely due to the bctimer not started. As I said, I assumed it was and then you corrected saying it is under the flag.
Quoted 2 lines
I am not convinced the culprit is this code you are trying to revert.
Quoted 9 lines
Quoted 2 lines
fair enough. Thats why I said if you have an alternative fix thats great.
For record, below is updated patch with bctimer started which was missed in earlier version. I haven't tested it though.
Alex,
Please give a try with your test-case and see if you still see the hang. Am just curious about your issue and hence the request..Alex tested below patch and he don't see the hang so the patch is addressing the issue.
Quoted 8 lines
Quoted 2 lines
If Daniel works out an alternate fix to avoid reverts, that will be great but if not, we should merge the below patch. I let you take call on it.
Daniel any news on this?
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.
I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Quoted 4 lines
And for the record, the omap3 hang fix is now posted to the lists as "[PATCH] ARM: OMAP2+: Fix DMA hang after off-idle". This should not have anything to do with the omap4
cpu_idlehang as omap4 does not currently lose context during idle.
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Well we do have several automated build and boot systems failing because of this with multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:
https://bugzilla.kernel.org/show_bug.cgi?id=75421
It seems that doing the revert is not enough based on the page above.
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 10 lines · Tony Lindgren
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Quoted 9 lines
Well we do have several automated build and boot systems failing because of this with
multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:https://bugzilla.kernel.org/show_bug.cgi?id=75421
It seems that doing the revert is not enough based on the page above.
Thats not true. The above link used the half patch and not the updated patch. Updated patch worked for Alex also. As you can see they saw RCU stalls and they go away after the updated patch.
Can you please point them to try out the updated patch ?
Quoted 5 lines
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
I am certain that the updated patch fixed the regression for sure. The issue is really not generic enough since its related an OMAP ROM errata which needs that special handling of interrupt re-trigger etc. You don't need that for other platforms so they are not likely get affected.
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 17 lines · Santosh Shilimkar
Quoted 16 lines · Tony Lindgren
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Well we do have several automated build and boot systems failing because of this with
multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:
Quoted 8 lines
Quoted 3 lines
It seems that doing the revert is not enough based on the page above.
Thats not true. The above link used the half patch and not the updated patch. Updated patch worked for Alex also. As you can see they saw RCU stalls and they go away after the updated patch.
Can you please point them to try out the updated patch ?
OK good point. I added a link to the updated patch in bugzilla.
Quoted 10 lines
Quoted 5 lines
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
I am certain that the updated patch fixed the regression for sure. The issue is really not generic enough since its related an OMAP ROM errata which needs that special handling of interrupt re-trigger etc. You don't need that for other platforms so they are not likely get affected.
OK makes sense to me considering the ROM code. Daniel, are you OK with that or do you still want to investigate further?
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 35 lines · Tony Lindgren
Quoted 17 lines · Santosh Shilimkar
Quoted 16 lines · Tony Lindgren
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Well we do have several automated build and boot systems failing because of this with
multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:Quoted 8 lines
Quoted 3 lines
It seems that doing the revert is not enough based on the page above.
Thats not true. The above link used the half patch and not the updated patch. Updated patch worked for Alex also. As you can see they saw RCU stalls and they go away after the updated patch.
Can you please point them to try out the updated patch ?
OK good point. I added a link to the updated patch in bugzilla.
Quoted 5 lines
Quoted 5 lines
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
Quoted 8 lines
Quoted 5 lines
I am certain that the updated patch fixed the regression for sure. The issue is really not generic enough since its related an OMAP ROM errata which needs that special handling of interrupt re-trigger etc. You don't need that for other platforms so they are not likely get affected.
OK makes sense to me considering the ROM code. Daniel, are you OK with that or do you still want to investigate further?
For the moment I am a bit short in time for some other tasks. So feel free to apply the revert and I will look for a proper fix when I will have time.
Thanks
-- Daniel
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 42 lines · Daniel Lezcano
Quoted 41 lines · Tony Lindgren
Quoted 17 lines · Santosh Shilimkar
Quoted 16 lines · Tony Lindgren
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Well we do have several automated build and boot systems failing because of this with
multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:Quoted 8 lines
Quoted 3 lines
It seems that doing the revert is not enough based on the page above.
Thats not true. The above link used the half patch and not the updated patch. Updated patch worked for Alex also. As you can see they saw RCU stalls and they go away after the updated patch.
Can you please point them to try out the updated patch ?
OK good point. I added a link to the updated patch in bugzilla.
Quoted 10 lines
Quoted 5 lines
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
I am certain that the updated patch fixed the regression for sure. The issue is really not generic enough since its related an OMAP ROM errata which needs that special handling of interrupt re-trigger etc. You don't need that for other platforms so they are not likely get affected.
Quoted 5 lines
Quoted 2 lines
OK makes sense to me considering the ROM code. Daniel, are you OK with that or do you still want to investigate further?
For the moment I am a bit short in time for some other tasks. So feel free to apply the revert and I will look for a proper fix when I will have time.
Added Tobias to Cc. At the bugzilla link Tobias is saying he used the right patch from Santosh to test and it still fails.
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 46 lines · Tony Lindgren
Quoted 45 lines · Daniel Lezcano
Quoted 43 lines · Tony Lindgren
Quoted 17 lines · Santosh Shilimkar
Quoted 16 lines · Tony Lindgren
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Well we do have several automated build and boot systems failing because of this with
multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:Quoted 8 lines
Quoted 3 lines
It seems that doing the revert is not enough based on the page above.
Thats not true. The above link used the half patch and not the updated patch. Updated patch worked for Alex also. As you can see they saw RCU stalls and they go away after the updated patch.
Can you please point them to try out the updated patch ?
OK good point. I added a link to the updated patch in bugzilla.
Quoted 10 lines
Quoted 5 lines
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
I am certain that the updated patch fixed the regression for sure. The issue is really not generic enough since its related an OMAP ROM errata which needs that special handling of interrupt re-trigger etc. You don't need that for other platforms so they are not likely get affected.
OK makes sense to me considering the ROM code. Daniel, are you OK with that or do you still want to investigate further?
Quoted 6 lines
Quoted 2 lines
For the moment I am a bit short in time for some other tasks. So feel free to apply the revert and I will look for a proper fix when I will have time.
Added Tobias to Cc. At the bugzilla link Tobias is saying he used the right patch from Santosh to test and it still fails.
So apparently Santosh, Kevin and Alex say they are not facing the issue anymore with the revert. But Tobias is still facing this issue.
There is this simple program [1] which usually makes cpuidle to hang the system more quickly when there is a problem somewhere. May be worth to check with it.
Hope that helps.
-- Daniel
[1] https://git.linaro.org/power/pm-qa.git/blob/HEAD:/cpuidle/cpuidle_killer.c
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 8 lines · Daniel Lezcano
So apparently Santosh, Kevin and Alex say they are not facing the issue anymore with the revert. But Tobias is still facing this issue.
There is this simple program [1] which usually makes cpuidle to hang the system more quickly when there is a problem somewhere. May be worth to check with it.
Quoted 7 lines
Hope that helps.
-- Daniel
[1]
https://git.linaro.org/power/pm-qa.git/blob/HEAD:/cpuidle/cpuidle_killer.c
Hello,
just to clarify. I never encountered the boot failure problem on my Panda ES in the first place. The issue with CPU_IDLE for me, like I also say in the bugreport, showed itself through instability of the USB. I saw the boot issue on the ml, read that it was caused by CPU_IDLE, and thought I should make Tony aware that I have a similar problem.
Now, after applying the latest version of the patch that Santosh proposed, the situation with CPU_IDLE improves. But it is still not stable for me, since after some while, working with an SSH session of WiFi (which IIRC is not connected via USB on the board), this 'BUG' storm hits. This does _not_ happen when I disable CPU_IDLE completly.
But even if I don't connect via WiFi at all, just boot and let me system run with serial console connected, after some time I get a kernel 'WARNING':
http://www.math.uni-bielefeld.de/~tjakobi/archive/dmesg.1.log
So something with CPU_IDLE is not right here. However I can't say if it's exactly this boot issue which this thread was initially about. Maybe seperate issues are at work here, who knows.
Anyway, I'm going to have a look at that cpuidle killer thingy later.
-- Tobias
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 24 lines · Tobias Jakobi
Quoted 8 lines · Daniel Lezcano
So apparently Santosh, Kevin and Alex say they are not facing the issue anymore with the revert. But Tobias is still facing this issue.
There is this simple program [1] which usually makes cpuidle to hang the system more quickly when there is a problem somewhere. May be worth to check with it.
Quoted 7 lines
Hope that helps.
-- Daniel
[1]
https://git.linaro.org/power/pm-qa.git/blob/HEAD:/cpuidle/cpuidle_killer.cHello,
just to clarify. I never encountered the boot failure problem on my Panda ES in the first place. The issue with
CPU_IDLEfor me, like I also say in the bugreport, showed itself through instability of the USB. I saw the boot issue on the ml, read that it was caused byCPU_IDLE, and thought I should make Tony aware that I have a similar problem.
Quoted 5 lines
Now, after applying the latest version of the patch that Santosh proposed, the situation with
CPU_IDLEimproves. But it is still not stable for me, since after some while, working with an SSH session of WiFi (which IIRC is not connected via USB on the board), this 'BUG' storm hits. This does _not_ happen when I disableCPU_IDLEcompletly.
OK that seems like a different problem.
Quoted 9 lines
But even if I don't connect via WiFi at all, just boot and let me system run with serial console connected, after some time I get a kernel 'WARNING':
http://www.math.uni-bielefeld.de/~tjakobi/archive/dmesg.1.logSo something with
CPU_IDLEis not right here. However I can't say if it's exactly this boot issue which this thread was initially about. Maybe seperate issues are at work here, who knows.Anyway, I'm going to have a look at that cpuidle killer thingy later.
OK. I gave it a try, but did not get any errors at least after about 30 minutes of testing on my panda es.
I'll apply the patch from Santosh as that gets things booting again with linux next.
Regards,
Tony
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
In reply to Tobias JakobiQuoted 4 lines · Tobias Jakobi
But even if I don't connect via WiFi at all, just boot and let me system run with serial console connected, after some time I get a kernel 'WARNING':
http://www.math.uni-bielefeld.de/~tjakobi/archive/dmesg.1.log
BTW, care to update the bugzilla page with the second warning in this log?
That's the WARNING: CPU: 1 PID: 0 at kernel/timer.c:1147 that's at 238 seconds.
Regards,
Tony
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 11 lines · Tony Lindgren
Quoted 4 lines · Tobias Jakobi
But even if I don't connect via WiFi at all, just boot and let me system run with serial console connected, after some time I get a kernel 'WARNING':
http://www.math.uni-bielefeld.de/~tjakobi/archive/dmesg.1.logBTW, care to update the bugzilla page with the second warning in this log?
That's the WARNING: CPU: 1 PID: 0 at
kernel/timer.c:1147that's at 238 seconds.
Also, with Santosh's fix applied, can you also try disabling one or more of the idle states for cpuidle and see if that helps?
Something like this patch below. If that helps with the WARNING above you're getting it narrows down the problem down quite a bit.
Regards,
Tony
arch/arm/mach-omap2/cpuidle44xx.c +2 -0
@@ -44,11 +44,13 @@ static struct idle_statedata omap4_idle_data[] = { .mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_RET, }, +#if 0 { .cpu_state = PWRDM_POWER_OFF, .mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_OFF, }, +#endif };static struct powerdomain *mpu_pd, *cpu_pd[MAX_CPUS ];
A message that is not in the archive (sent off-list or not yet received): CAKnoXLxj42veWCbRjX98iT3FNAb92J4yR+UAL716JFSOb=pcrQ@mail.gmail.com
Tony Lindgren
maintainerPATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 36 lines · Daniel Lezcano
Quoted 34 lines · Tony Lindgren
Quoted 3 lines · Tony Lindgren
Quoted 2 lines · Tobias Jakobi
But even if I don't connect via WiFi at all, just boot and let me
system
Quoted 1 line
Quoted 1 line
run with serial console connected, after some time I get a kernel
'WARNING':
Quoted 7 lines
BTW, care to update the bugzilla page with the second warning in this log?
That's the WARNING: CPU: 1 PID: 0 at
kernel/timer.c:1147that's at 238 seconds.Also, with Santosh's fix applied, can you also try disabling one or more of the idle states for cpuidle and see if that helps?
Something like this patch below. If that helps with the WARNING above you're getting it narrows down the problem down quite a bit.
Regards,
Tony
arch/arm/mach-omap2/cpuidle44xx.c +1
-0@@ -44,11 +44,13 @@ static struct idle_statedata omap4_idle_data[] = { .mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_RET, }, +#if 0 { .cpu_state = PWRDM_POWER_OFF,
Quoted 8 lines
arch/arm/mach-omap2/cpuidle44xx.c static struct idle_statedata omap4_idle_data[] = { · line 55
Quoted 5 lines
arch/arm/mach-omap2/cpuidle44xx.c +1
-0.mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_OFF, }, +#endif Hmm, I am afraid that will lead to a fault. Safer to set the
state_count= 2 instead.
Hmm don't we have state_count = ARRAY_SIZE(omap4_idle_data) or am I missing something?
Regards,
Tony
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 46 lines · Tony Lindgren
Quoted 36 lines · Daniel Lezcano
Quoted 34 lines · Tony Lindgren
Quoted 3 lines · Tony Lindgren
Quoted 2 lines · Tobias Jakobi
But even if I don't connect via WiFi at all, just boot and let me
system
Quoted 1 line
Quoted 1 line
run with serial console connected, after some time I get a kernel
'WARNING':
Quoted 7 lines
BTW, care to update the bugzilla page with the second warning in this log?
That's the WARNING: CPU: 1 PID: 0 at
kernel/timer.c:1147that's at 238 seconds.Also, with Santosh's fix applied, can you also try disabling one or more of the idle states for cpuidle and see if that helps?
Something like this patch below. If that helps with the WARNING above you're getting it narrows down the problem down quite a bit.
Regards,
Tony
arch/arm/mach-omap2/cpuidle44xx.c +1
-0@@ -44,11 +44,13 @@ static struct idle_statedata omap4_idle_data[] = { .mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_RET, }, +#if 0 { .cpu_state = PWRDM_POWER_OFF,Quoted 8 lines
arch/arm/mach-omap2/cpuidle44xx.c static struct idle_statedata omap4_idle_data[] = { · line 55
Quoted 5 lines
arch/arm/mach-omap2/cpuidle44xx.c +1
-0.mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_OFF, }, +#endif Hmm, I am afraid that will lead to a fault. Safer to set the
state_count= 2 instead.
Quoted 3 lines
Hmm don't we have
state_count=ARRAY_SIZE(omap4_idle_data)or am I missing something?
I don't think you are missing anything. The change should work.
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
Quoted 47 lines · Santosh Shilimkar
Quoted 46 lines · Tony Lindgren
Quoted 44 lines · Daniel Lezcano
Quoted 29 lines · Tony Lindgren
Quoted 3 lines · Tony Lindgren
Quoted 2 lines · Tobias Jakobi
But even if I don't connect via WiFi at all, just boot and let me
system
Quoted 1 line
Quoted 1 line
run with serial console connected, after some time I get a kernel
'WARNING':
Quoted 7 lines
BTW, care to update the bugzilla page with the second warning in this log?
That's the WARNING: CPU: 1 PID: 0 at
kernel/timer.c:1147that's at 238 seconds.Also, with Santosh's fix applied, can you also try disabling one or more of the idle states for cpuidle and see if that helps?
Something like this patch below. If that helps with the WARNING above you're getting it narrows down the problem down quite a bit.
Regards,
Tony
arch/arm/mach-omap2/cpuidle44xx.c +0
-0@@ -44,11 +44,13 @@ static struct idle_statedata omap4_idle_data[] = { .mpu_state = PWRDM_POWER_RET,arch/arm/mach-omap2/cpuidle44xx.c static struct idle_statedata omap4_idle_data[] = { · line 54
Quoted 10 lines
arch/arm/mach-omap2/cpuidle44xx.c +2
-0.mpu_logic_state = PWRDM_POWER_RET, }, +#if 0 { .cpu_state = PWRDM_POWER_OFF, .mpu_state = PWRDM_POWER_RET, .mpu_logic_state = PWRDM_POWER_OFF, }, +#endif Hmm, I am afraid that will lead to a fault. Safer to set the
state_count= 2 instead.
Quoted 4 lines
Quoted 3 lines
Hmm don't we have
state_count=ARRAY_SIZE(omap4_idle_data)or am I missing something?I don't think you are missing anything. The change should work.
Oh, yes. Sorry, over-sighted that. I am too used with the other drivers assigning the state_count with a number.
Signature · 10 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/1 ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled
In reply to Tony LindgrenQuoted 46 lines · Tony Lindgren
Quoted 45 lines · Daniel Lezcano
Quoted 43 lines · Tony Lindgren
Quoted 17 lines · Santosh Shilimkar
Quoted 16 lines · Tony Lindgren
Quoted 8 lines · Daniel Lezcano
Quoted 4 lines · Tony Lindgren
And just to recap, this problem can be reproduced with current Linux next with
omap2plus_defconfigwithCONFIG_CPU_IDLEenabled. The system should hang during the boot at some point.I can take the time to investigate a bit more but not right now. What is your deadline before committing the reverts ?
Well we do have several automated build and boot systems failing because of this with
multi_v7_defconfig. And users are complaining, see this report from Tobias Jakobi:Quoted 8 lines
Quoted 3 lines
It seems that doing the revert is not enough based on the page above.
Thats not true. The above link used the half patch and not the updated patch. Updated patch worked for Alex also. As you can see they saw RCU stalls and they go away after the updated patch.
Can you please point them to try out the updated patch ?
OK good point. I added a link to the updated patch in bugzilla.
Quoted 10 lines
Quoted 5 lines
I'd prefer we'd fix this issue properly for sure, it seems that we're not quite understanding what's going on. And this might hit other platforms too when they start implementing deeper PM idle states in the mainline kernel.
I am certain that the updated patch fixed the regression for sure. The issue is really not generic enough since its related an OMAP ROM errata which needs that special handling of interrupt re-trigger etc. You don't need that for other platforms so they are not likely get affected.
OK makes sense to me considering the ROM code. Daniel, are you OK with that or do you still want to investigate further?
Quoted 7 lines
Quoted 2 lines
For the moment I am a bit short in time for some other tasks. So feel free to apply the revert and I will look for a proper fix when I will have time.
Added Tobias to Cc. At the bugzilla link Tobias is saying he used the right patch from Santosh to test and it still fails.
The link of the patch in bugzilla shows that it was not the updated patch and that was my reference point. The other bugzilla issue seems to be different and mostly not related to cpuidle. I could be wrong.
----------------------
dmesg | grep idle:
cpuidle: using governor ladder
cpuidle: using governor menu
grep . /sys/devices/system/cpu/cpu*/cpuidle/*/*:
(now attached as 'cpuidle states')
The 'BUG' looks like this:
BUG: scheduling while atomic: swapper/1/0/0xffff0000 Modules linked in: ctr ccm btrfs xor xor_neon zlib_inflate zlib_deflate omapfb cfbfillrect cfbimgblt cfbcopyarea raid6_pq arc4 wl12xx wlcore mac80211 cfg80211 rfkill usb_storage snd_soc_omap_hdmi snd_soc_omap_hdmi_card wlcore_sdio CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 3.15.0-rc5+ #1 Backtrace: [<c001152c>] (dump_backtrace) from [<c00116c8>] (show_stack+0x18/0x1c) r6:eb894000 r5:00000001 r4:00000000 r3:00000000 [<c00116b0>] (show_stack) from [<c0430a94>] (dump_stack+0x7c/0x98) [<c0430a18>] (dump_stack) from [<c042ec10>] (__schedule_bug+0x48/0x60) r4:00000000 r3:00000153 [<c042ebc8>] (__schedule_bug) from [<c0432248>] (__schedule+0x41c/0x4b4) r4:2b9fe000 r3:00000000 [<c0431e2c>] (__schedule) from [<c04323d4>] (schedule+0x38/0x88) r10:eb894000 r9:eb894000 r8:00000000 r7:eb894000 r6:c05b0450 r5:c05b04a0 r4:eb894000 [<c043239c>] (schedule) from [<c04326fc>] (schedule_preempt_disabled+0x10/0x14) [<c04326ec>] (schedule_preempt_disabled) from [<c006d8f0>] (cpu_startup_entry+0xec/0x22c) [<c006d804>] (cpu_startup_entry) from [<c00131a4>] (secondary_start_kernel+0xe8/0x100) r7:c05de240
[<c00130bc>] (secondary_start_kernel) from [<80008664>] (0x80008664)
r4:ab87c06a r3:c000864c
----------------------