Skip to contentSkip to conversation

next: suspicious RCU usage message since commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'

DISCUSSION Guenter Roeck 2 participants 13 messages Apr 24, 2016 – May 16, 2016

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

Thanks,
Guenter

---
# bad: [73442f7becf1b305c966ad37f5991d030f8bc025] Add linux-next specific files for 20160422
# good: [c3b46c73264b03000d1e18b22f5caf63332547c9] Linux 4.6-rc4
git bisect start 'HEAD' 'v4.6-rc4'
# good: [f59d58f4c4c5d63b2f1768ab94465f9e98cd771d] Merge remote-tracking branch 'l2-mtd/master'
git bisect good f59d58f4c4c5d63b2f1768ab94465f9e98cd771d
# good: [ee634dd861741fa3112263c5fcd562e7953a8f3e] Merge remote-tracking branch 'tip/auto-latest'
git bisect good ee634dd861741fa3112263c5fcd562e7953a8f3e
# bad: [56ce28694cb53fb5609feca2735736583a35d5d1] Merge remote-tracking branch 'staging/staging-next'
git bisect bad 56ce28694cb53fb5609feca2735736583a35d5d1
# good: [9acc4500b44723d12bec63519038ef820479eaad] staging/lustre/llite: rename struct ccc_thread_info to vvp_thread_info
git bisect good 9acc4500b44723d12bec63519038ef820479eaad
# bad: [95585a7b8c5f8eac40402378bc7482376f17a3ff] Merge remote-tracking branch 'driver-core/driver-core-next'
git bisect bad 95585a7b8c5f8eac40402378bc7482376f17a3ff
# bad: [3052fd5d2b38b26deaa195aec34f18bacccdf40c] torture: Add starvation events to error summary
git bisect bad 3052fd5d2b38b26deaa195aec34f18bacccdf40c
# good: [a54062c0d95921d4fb0edc8d268021bf387e6c75] rcutorture: Add boot-time adjustment of leaf fanout
git bisect good a54062c0d95921d4fb0edc8d268021bf387e6c75
# bad: [dcd36d01fb3f99d1d5df01714f6ccbe3fbbaf81f] Merge branches 'doc.2016.04.19a', 'exp.2016.03.31d', 'fixes.2016.03.31d' and 'torture.2016.04.21a' into HEAD git bisect bad dcd36d01fb3f99d1d5df01714f6ccbe3fbbaf81f
# good: [3b5f668e715bc19610ad967ef97a7e8c55a186ec] rcu: Overlap wakeups with next expedited grace period
git bisect good 3b5f668e715bc19610ad967ef97a7e8c55a186ec
# good: [5413e24c943da33306047fc091fa34fa4f261b3b] documentation: Sharpen up the no-readers quick quiz
git bisect good 5413e24c943da33306047fc091fa34fa4f261b3b
# bad: [293e2421fe25839500207eda123cc4475f8d17b8] rcu: Remove superfluous versions of rcu_read_lock_sched_held()
git bisect bad 293e2421fe25839500207eda123cc4475f8d17b8
# good: [fcfd0a237bfcf0c314005007e9d76e55a25e2bad] rcu: Make FQS schedule advance only if FQS happened
git bisect good fcfd0a237bfcf0c314005007e9d76e55a25e2bad
# good: [8c7c4829a81c1838f18c12ce5a3a5c29a08bf0a8] rcu: Awaken grace-period kthread if too long since FQS
git bisect good 8c7c4829a81c1838f18c12ce5a3a5c29a08bf0a8

# first bad commit: [293e2421fe25839500207eda123cc4475f8d17b8] rcu: Remove superfluous versions of rcu_read_lock_sched_held()
#1 lore raw
Quoted 29 lines · Guenter Roeck

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 2 lines

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

I believe that the real fix is not a revert of that commit, but rather that some of the tracing statements need an "_rcuidle" suffix.

Something like the following (untested, probably does not build) patch.

Thanx, Paul

------------------------------------------------------------------------

commit ca91304178e1cf53ee391236a0ac3969cc814e5f

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700

arm: Use _rcuidle tracepoint to allow use from idle

arch/arm/mach-omap2/powerdomain.c +2 -2
index 78af6d8cf2e2..12b66b5bcc55 100644
@@ -523,8 +523,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}
#2 lore raw

Hi Paul,

Quoted 56 lines · Paul E. McKenney
Quoted 29 lines · Guenter Roeck

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 2 lines

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

I believe that the real fix is not a revert of that commit, but rather that some of the tracing statements need an "_rcuidle" suffix.

Something like the following (untested, probably does not build) patch.

Thanx, Paul

------------------------------------------------------------------------

commit ca91304178e1cf53ee391236a0ac3969cc814e5f

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700

arm: Use _rcuidle tracepoint to allow use from idle

arch/arm/mach-omap2/powerdomain.c +0 -0
index 78af6d8cf2e2..12b66b5bcc55 100644

arch/arm/mach-omap2/powerdomain.c int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst) · line 530

Quoted 10 lines
arch/arm/mach-omap2/powerdomain.c +2 -2
@@ -523,8 +523,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
  	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
  		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
  		/* Program the pwrdm desired target state */
  		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
  	}

It does build. After applying it, I get a different traceback.

[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14) [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0) [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c) [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14) [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c) [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30) [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240) [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8) [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408) [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0) [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Guenter

#3 lore raw
Quoted 84 lines · Guenter Roeck

Hi Paul,

Quoted 56 lines · Paul E. McKenney
Quoted 29 lines · Guenter Roeck

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 2 lines

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

I believe that the real fix is not a revert of that commit, but rather that some of the tracing statements need an "_rcuidle" suffix.

Something like the following (untested, probably does not build) patch.

Thanx, Paul

------------------------------------------------------------------------

commit ca91304178e1cf53ee391236a0ac3969cc814e5f

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700

arm: Use _rcuidle tracepoint to allow use from idle

arch/arm/mach-omap2/powerdomain.c +0 -0
index 78af6d8cf2e2..12b66b5bcc55 100644

arch/arm/mach-omap2/powerdomain.c int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst) · line 530

Quoted 10 lines
arch/arm/mach-omap2/powerdomain.c +2 -2
@@ -523,8 +523,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}

It does build. After applying it, I get a different traceback.

[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14) [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0) [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c) [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14) [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c) [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30) [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240) [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8) [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408) [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0) [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)

Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Thanx, Paul

#4 lore raw
Quoted 85 lines · Paul E. McKenney
Quoted 84 lines · Guenter Roeck

Hi Paul,

Quoted 56 lines · Paul E. McKenney
Quoted 29 lines · Guenter Roeck

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 2 lines

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

I believe that the real fix is not a revert of that commit, but rather that some of the tracing statements need an "_rcuidle" suffix.

Something like the following (untested, probably does not build) patch.

Thanx, Paul

------------------------------------------------------------------------

commit ca91304178e1cf53ee391236a0ac3969cc814e5f

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700

arm: Use _rcuidle tracepoint to allow use from idle

arch/arm/mach-omap2/powerdomain.c +0 -0
index 78af6d8cf2e2..12b66b5bcc55 100644

arch/arm/mach-omap2/powerdomain.c int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst) · line 530

Quoted 10 lines
arch/arm/mach-omap2/powerdomain.c +2 -2
@@ -523,8 +523,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
  	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
  		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
  		/* Program the pwrdm desired target state */
  		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
  	}

It does build. After applying it, I get a different traceback.

[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14) [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0) [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c) [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14) [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c) [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30) [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240) [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8) [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408) [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0) [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)

Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

Thanks,
Guenter

#5 lore raw
Quoted 86 lines · Guenter Roeck
Quoted 85 lines · Paul E. McKenney
Quoted 84 lines · Guenter Roeck

Hi Paul,

Quoted 56 lines · Paul E. McKenney
Quoted 29 lines · Guenter Roeck

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 2 lines

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

I believe that the real fix is not a revert of that commit, but rather that some of the tracing statements need an "_rcuidle" suffix.

Something like the following (untested, probably does not build) patch.

Thanx, Paul

------------------------------------------------------------------------

commit ca91304178e1cf53ee391236a0ac3969cc814e5f

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700

arm: Use _rcuidle tracepoint to allow use from idle

arch/arm/mach-omap2/powerdomain.c +0 -0
index 78af6d8cf2e2..12b66b5bcc55 100644

arch/arm/mach-omap2/powerdomain.c int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst) · line 530

Quoted 10 lines
arch/arm/mach-omap2/powerdomain.c +2 -2
@@ -523,8 +523,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}

It does build. After applying it, I get a different traceback.

[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14) [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0) [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c) [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14) [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c) [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30) [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240) [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8) [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408) [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0) [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)

Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

Either way, please let me know.

Thanx, Paul

#6 lore raw
Quoted 90 lines · Paul E. McKenney
Quoted 89 lines · Guenter Roeck
Quoted 88 lines · Paul E. McKenney
Quoted 86 lines · Guenter Roeck

Hi Paul,

Quoted 56 lines · Paul E. McKenney
Quoted 29 lines · Guenter Roeck

Hi,

I see the following log message when running a qemu test for 'beagle' with omap2plus_defconfig.

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc4-next-20160422 #1 Not tainted
-------------------------------
include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
Hardware name: Generic OMAP3-GP (Flattened Device Tree)
[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
[<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
[<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
[<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
[<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
[<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
[<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 2 lines

bisect points to commit 'rcu: Remove superfluous versions of rcu_read_lock_sched_held()'. Bisect log is attached.

I believe that the real fix is not a revert of that commit, but rather that some of the tracing statements need an "_rcuidle" suffix.

Something like the following (untested, probably does not build) patch.

Thanx, Paul

------------------------------------------------------------------------

commit ca91304178e1cf53ee391236a0ac3969cc814e5f

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700

arm: Use _rcuidle tracepoint to allow use from idle

arch/arm/mach-omap2/powerdomain.c +0 -0
index 78af6d8cf2e2..12b66b5bcc55 100644

arch/arm/mach-omap2/powerdomain.c int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst) · line 530

Quoted 10 lines
arch/arm/mach-omap2/powerdomain.c +2 -2
@@ -523,8 +523,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
  	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
  		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
  		/* Program the pwrdm desired target state */
  		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
  	}

It does build. After applying it, I get a different traceback.

[<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14) [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0) [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c) [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14) [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c) [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30) [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240) [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8) [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408) [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0) [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Quoted 8 lines
Quoted 5 lines
Quoted 4 lines

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

I think it would be best if you send a single patch which fixes both calls.

Guenter

#7 lore raw
Quoted 4 lines · Guenter Roeck
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Paul E. McKenney

On Sun, Apr 24, 2016 at 04:56:38PM -0700, Guenter Roeck wrote:

[ . . . ]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

I think it would be best if you send a single patch which fixes both calls.

Like this one?

If so, could you please run it to make sure that it actually fixes the problem? And if it does, would you be willing to give me a Tested-by?

Thanx, Paul

------------------------------------------------------------------------

commit e3c9e1a46984b8c3e4204f138aced7630c4b85c2

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700
    arm: Use _rcuidle tracepoint to allow use from idle
    
    Testing on ARM encountered the following pair of lockdep-RCU splats:
    
    ------------------------------------------------------------------------
    
    ===============================
    [ INFO: suspicious RCU usage. ]
    4.6.0-rc4-next-20160422 #1 Not tainted
    -------------------------------
    include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!
    
    other info that might help us debug this:
    
    RCU used illegally from idle CPU!
    rcu_scheduler_active = 1, debug_locks = 0
    RCU used illegally from extended quiescent state!
    no locks held by swapper/0/0.
    
    stack backtrace:
    CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
    Hardware name: Generic OMAP3-GP (Flattened Device Tree)
    [<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
    [<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
    [<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
    [<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
    [<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
    [<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
    [<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
    
    ------------------------------------------------------------------------
    
    [<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
    [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0)
    [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c)
    [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14)
    [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c)
    [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30)
    [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240)
    [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8)
    [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408)
    [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0)
    [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
    
    ------------------------------------------------------------------------
    
    These are caused by event tracing from the idle loop.  This commit
    therefore adds the _rcuidle suffix to make RCU aware of this implicit
    use of RCU by event tracing, thus preventing both splats.
    
    Reported-by: Guenter Roeck <[email protected]>
    Signed-off-by: Paul E. McKenney <[email protected]>
    Cc: Tony Lindgren <[email protected]>
    Cc: Russell King <[email protected]>
    Cc: Steven Rostedt <[email protected]>
    Cc: <[email protected]>
    Cc: <[email protected]>
arch/arm/mach-omap2/powerdomain.c +5 -4
index 78af6d8cf2e2..daf2753de7aa 100644
@@ -186,8 +186,9 @@ static int _pwrdm_state_switch(struct powerdomain *pwrdm, int flag)
 			trace_state = (PWRDM_TRACE_STATES_FLAG |
 				       ((next & OMAP_POWERSTATE_MASK) << 8) |
 				       ((prev & OMAP_POWERSTATE_MASK) << 0));
-			trace_power_domain_target(pwrdm->name, trace_state,
-						  smp_processor_id());
+			trace_power_domain_target_rcuidle(pwrdm->name,
+							  trace_state,
+							  smp_processor_id());
 		}
 		break;
 	default:
@@ -523,8 +524,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}
author Tested-by
#8 lore raw
Quoted 17 lines · Paul E. McKenney
Quoted 4 lines · Guenter Roeck
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Paul E. McKenney

On Sun, Apr 24, 2016 at 04:56:38PM -0700, Guenter Roeck wrote:

[ . . . ]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

Quoted 9 lines
Quoted 3 lines
Quoted 2 lines

I can, but please feel free to send mine along with yours, if you wish.

I think it would be best if you send a single patch which fixes both calls.

Like this one?

If so, could you please run it to make sure that it actually fixes the problem? And if it does, would you be willing to give me a Tested-by?

It does. Tested-by: inline below.

Thanks,
Guenter

Quoted 52 lines

Thanx, Paul

------------------------------------------------------------------------

commit e3c9e1a46984b8c3e4204f138aced7630c4b85c2

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700
    arm: Use _rcuidle tracepoint to allow use from idle
    
    Testing on ARM encountered the following pair of lockdep-RCU splats:
    
    ------------------------------------------------------------------------
    
    ===============================
    [ INFO: suspicious RCU usage. ]
    4.6.0-rc4-next-20160422 #1 Not tainted
    -------------------------------
    include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!
    
    other info that might help us debug this:
    
    RCU used illegally from idle CPU!
    rcu_scheduler_active = 1, debug_locks = 0
    RCU used illegally from extended quiescent state!
    no locks held by swapper/0/0.
    
    stack backtrace:
    CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
    Hardware name: Generic OMAP3-GP (Flattened Device Tree)
    [<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
    [<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
    [<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
    [<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
    [<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
    [<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
    [<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
    
    ------------------------------------------------------------------------
    
    [<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
    [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0)
    [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c)
    [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14)
    [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c)
    [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30)
    [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240)
    [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8)
    [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408)
    [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0)
    [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 7 lines
    ------------------------------------------------------------------------
    
    These are caused by event tracing from the idle loop.  This commit
    therefore adds the _rcuidle suffix to make RCU aware of this implicit
    use of RCU by event tracing, thus preventing both splats.
    
    Reported-by: Guenter Roeck <[email protected]>
Quoted 35 lines
arch/arm/mach-omap2/powerdomain.c +5 -4
index 78af6d8cf2e2..daf2753de7aa 100644
@@ -186,8 +186,9 @@ static int _pwrdm_state_switch(struct powerdomain *pwrdm, int flag)
 			trace_state = (PWRDM_TRACE_STATES_FLAG |
 				       ((next & OMAP_POWERSTATE_MASK) << 8) |
 				       ((prev & OMAP_POWERSTATE_MASK) << 0));
-			trace_power_domain_target(pwrdm->name, trace_state,
-						  smp_processor_id());
+			trace_power_domain_target_rcuidle(pwrdm->name,
+							  trace_state,
+							  smp_processor_id());
 		}
 		break;
 	default:
@@ -523,8 +524,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}
#9 lore raw
Quoted 20 lines · Guenter Roeck
Quoted 19 lines · Paul E. McKenney
Quoted 4 lines · Guenter Roeck
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Paul E. McKenney

On Sun, Apr 24, 2016 at 04:56:38PM -0700, Guenter Roeck wrote:

[ . . . ]

Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

I think it would be best if you send a single patch which fixes both calls.

Like this one?

If so, could you please run it to make sure that it actually fixes the problem? And if it does, would you be willing to give me a Tested-by?

It does. Tested-by: inline below.

Got it, thank you!

If the ARM guys are willing to take this, it might hit the next merge window, or perhaps they will take it as an exception. If I push it up my usual route, it will be a bit later.

I just now sent it out, so hopefully they will grab it. ;-)

Thanx, Paul

Quoted 101 lines

Thanks,
Guenter

Quoted 52 lines

Thanx, Paul

------------------------------------------------------------------------

commit e3c9e1a46984b8c3e4204f138aced7630c4b85c2

Author: Paul E. McKenney <[email protected]>
Date:   Sun Apr 24 14:30:16 2016 -0700
    arm: Use _rcuidle tracepoint to allow use from idle
    
    Testing on ARM encountered the following pair of lockdep-RCU splats:
    
    ------------------------------------------------------------------------
    
    ===============================
    [ INFO: suspicious RCU usage. ]
    4.6.0-rc4-next-20160422 #1 Not tainted
    -------------------------------
    include/trace/events/power.h:328 suspicious rcu_dereference_check() usage!
    
    other info that might help us debug this:
    
    RCU used illegally from idle CPU!
    rcu_scheduler_active = 1, debug_locks = 0
    RCU used illegally from extended quiescent state!
    no locks held by swapper/0/0.
    
    stack backtrace:
    CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc4-next-20160422 #1
    Hardware name: Generic OMAP3-GP (Flattened Device Tree)
    [<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
    [<c010b64c>] (show_stack) from [<c047acbc>] (dump_stack+0xa8/0xe0)
    [<c047acbc>] (dump_stack) from [<c012bc10>] (pwrdm_set_next_pwrst+0xf8/0x1cc)
    [<c012bc10>] (pwrdm_set_next_pwrst) from [<c01269fc>] (omap3_enter_idle_bm+0x1b8/0x1e8)
    [<c01269fc>] (omap3_enter_idle_bm) from [<c05fa0b8>] (cpuidle_enter_state+0x84/0x408)
    [<c05fa0b8>] (cpuidle_enter_state) from [<c0182c1c>] (cpu_startup_entry+0x1c8/0x3f0)
    [<c0182c1c>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
    
    ------------------------------------------------------------------------
    
    [<c010f55c>] (unwind_backtrace) from [<c010b64c>] (show_stack+0x10/0x14)
    [<c010b64c>] (show_stack) from [<c047ac3c>] (dump_stack+0xa8/0xe0)
    [<c047ac3c>] (dump_stack) from [<c012c340>] (_pwrdm_state_switch+0x188/0x32c)
    [<c012c340>] (_pwrdm_state_switch) from [<c012c4f0>] (_pwrdm_post_transition_cb+0xc/0x14)
    [<c012c4f0>] (_pwrdm_post_transition_cb) from [<c012ba74>] (pwrdm_for_each+0x30/0x5c)
    [<c012ba74>] (pwrdm_for_each) from [<c012c72c>] (pwrdm_post_transition+0x24/0x30)
    [<c012c72c>] (pwrdm_post_transition) from [<c012548c>] (omap_sram_idle+0xfc/0x240)
    [<c012548c>] (omap_sram_idle) from [<c0126934>] (omap3_enter_idle_bm+0xf0/0x1e8)
    [<c0126934>] (omap3_enter_idle_bm) from [<c05fa038>] (cpuidle_enter_state+0x84/0x408)
    [<c05fa038>] (cpuidle_enter_state) from [<c0182b90>] (cpu_startup_entry+0x1c8/0x3f0)
    [<c0182b90>] (cpu_startup_entry) from [<c0b00c20>] (start_kernel+0x354/0x3cc)
Quoted 7 lines
    ------------------------------------------------------------------------
    
    These are caused by event tracing from the idle loop.  This commit
    therefore adds the _rcuidle suffix to make RCU aware of this implicit
    use of RCU by event tracing, thus preventing both splats.
    
    Reported-by: Guenter Roeck <[email protected]>
Quoted 35 lines
arch/arm/mach-omap2/powerdomain.c +5 -4
index 78af6d8cf2e2..daf2753de7aa 100644
@@ -186,8 +186,9 @@ static int _pwrdm_state_switch(struct powerdomain *pwrdm, int flag)
 			trace_state = (PWRDM_TRACE_STATES_FLAG |
 				       ((next & OMAP_POWERSTATE_MASK) << 8) |
 				       ((prev & OMAP_POWERSTATE_MASK) << 0));
-			trace_power_domain_target(pwrdm->name, trace_state,
-						  smp_processor_id());
+			trace_power_domain_target_rcuidle(pwrdm->name,
+							  trace_state,
+							  smp_processor_id());
 		}
 		break;
 	default:
@@ -523,8 +524,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target(pwrdm->name, pwrst,
-					  smp_processor_id());
+		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
+						  smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}
#10 lore raw
Quoted 30 lines · Paul E. McKenney
Quoted 20 lines · Guenter Roeck
Quoted 19 lines · Paul E. McKenney
Quoted 4 lines · Guenter Roeck
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Paul E. McKenney

On Sun, Apr 24, 2016 at 04:56:38PM -0700, Guenter Roeck wrote:

[ . . . ]

Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

I think it would be best if you send a single patch which fixes both calls.

Like this one?

If so, could you please run it to make sure that it actually fixes the problem? And if it does, would you be willing to give me a Tested-by?

It does. Tested-by: inline below.

Quoted 8 lines

Got it, thank you!

If the ARM guys are willing to take this, it might hit the next merge window, or perhaps they will take it as an exception. If I push it up my usual route, it will be a bit later.

I just now sent it out, so hopefully they will grab it. ;-)

Hi Paul,

Either it fixes a zero-day bug which was exposed by 'rcu: Remove superfluous versions ...", or it fixes a problem with that commit, so I would hope that _someone_ will pick it up.

Anyway, thanks a lot for looking into it, and for the patch!

Guenter

#11 lore raw
In reply to Paul E. McKenney
Quoted 30 lines · Paul E. McKenney
Quoted 20 lines · Guenter Roeck
Quoted 19 lines · Paul E. McKenney
Quoted 4 lines · Guenter Roeck
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Paul E. McKenney

On Sun, Apr 24, 2016 at 04:56:38PM -0700, Guenter Roeck wrote:

[ . . . ]

Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

I think it would be best if you send a single patch which fixes both calls.

Like this one?

If so, could you please run it to make sure that it actually fixes the problem? And if it does, would you be willing to give me a Tested-by?

It does. Tested-by: inline below.

Quoted 8 lines

Got it, thank you!

If the ARM guys are willing to take this, it might hit the next merge window, or perhaps they will take it as an exception. If I push it up my usual route, it will be a bit later.

I just now sent it out, so hopefully they will grab it. ;-)

The problem is still seen in next-20160513, so it looks like the patch was not accepted.

I recently learned that arm has a special way of submitting patches. See
http://www.arm.linux.org.uk/developer/patches/ for details. If I understand correctly, you'll have to send the patch to [email protected], and it has to be formatted correctly (eg no "[PATCH]" in the subject line, and some other information added). I never tried it myself, so I don't really know how exactly it works.

Guenter

#12 lore raw
Quoted 39 lines · Guenter Roeck
Quoted 30 lines · Paul E. McKenney
Quoted 20 lines · Guenter Roeck
Quoted 19 lines · Paul E. McKenney
Quoted 4 lines · Guenter Roeck
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Paul E. McKenney

On Sun, Apr 24, 2016 at 04:56:38PM -0700, Guenter Roeck wrote:

[ . . . ]

Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

After making the same change in _pwrdm_state_switch(), the traceback is gone from my tests (beagle, beagle-xm, and overo-tobi).

Very good!

(And yes, you normally find these one at a time...)

Are you going to submit a formal patch ?

I can, but please feel free to send mine along with yours, if you wish.

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

I think it would be best if you send a single patch which fixes both calls.

Like this one?

If so, could you please run it to make sure that it actually fixes the problem? And if it does, would you be willing to give me a Tested-by?

It does. Tested-by: inline below.

Quoted 8 lines

Got it, thank you!

If the ARM guys are willing to take this, it might hit the next merge window, or perhaps they will take it as an exception. If I push it up my usual route, it will be a bit later.

I just now sent it out, so hopefully they will grab it. ;-)

Quoted 7 lines

The problem is still seen in next-20160513, so it looks like the patch was not accepted.

I recently learned that arm has a special way of submitting patches. See
http://www.arm.linux.org.uk/developer/patches/ for details. If I understand correctly, you'll have to send the patch to [email protected], and it has to be formatted correctly (eg no "[PATCH]" in the subject line, and some other information added). I never tried it myself, so I don't really know how exactly it works.

Nor have I. But Tony Lindgren asked me to send them via -tip. which I can do. But I do need to straighten out the commit logs a bit beforehand. So I will send an updated series to LKML later today and if there are no objections, send a pull request to Ingo.

Thanx, Paul