s390/idle: Fix suspicious RCU usage
Subsystems (18)
[email protected]
maintainerPATCH 1/1 s390/idle: Fix suspicious RCU usage
After commit eb1f00237aca ("lockdep,trace: Expose tracepoints") the lock tracepoints are visible to lockdep and RCU-lockdep is finding a bunch more RCU violations that were previously hidden.
Switch the idle->seqcount over to using raw_write_*() to avoid the lockdep annotation and thus the lock tracepoints.
--- arch/s390/kernel/idle.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
arch/s390/kernel/idle.c +2 -3
@@ -39,14 +39,13 @@ void enabled_wait(void)local_irq_restore (flags);/* Account time spent with enabled wait psw loaded as idle time. */ -/* XXX seqcount has tracepoints that require RCU */ -write_seqcount_begin (&idle->seqcount); +raw_write_seqcount_begin (&idle->seqcount); idle_time = idle->clock_idle_exit - idle->clock_idle_enter; idle->clock_idle_enter = idle->clock_idle_exit =0ULL ; idle->idle_time += idle_time; idle->idle_count++;account_idle_time (cputime_to_nsecs (idle_time)); -write_seqcount_end (&idle->seqcount); +raw_write_seqcount_end (&idle->seqcount); }NOKPROBE_SYMBOL (enabled_wait);
Quoted 5 lines · On Tue, Sep 08, 2020 at 03:30:31PM +0200, [email protected] wrote:
After commit
eb1f00237aca("lockdep,trace: Expose tracepoints") the lock tracepoints are visible to lockdep and RCU-lockdep is finding a bunch more RCU violations that were previously hidden.
Quoted 8 lines
Switch the
idle->seqcountover to using raw_write_*() to avoid the lockdep annotation and thus the lock tracepoints.Reported-by Guenter Roeck <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- arch/s390/kernel/idle.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
Applied, thank you!
Sven Schnelle
reviewerHi Peter,
Quoted 10 lines · [email protected] writes:
After commit
eb1f00237aca("lockdep,trace: Expose tracepoints") the lock tracepoints are visible to lockdep and RCU-lockdep is finding a bunch more RCU violations that were previously hidden.Switch the
idle->seqcountover to using raw_write_*() to avoid the lockdep annotation and thus the lock tracepoints.Reported-by Guenter Roeck <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>[..]
I'm still seeing the splat below on s390 when irq tracing is enabled:
[ 1273.747948] =============================
[ 1273.747953] WARNING: suspicious RCU usage
[ 1273.747960] 5.9.0-20201006.rc8.git0.162c12a918a1.300.fc32.s390x+debug #1 Not tainted
[ 1273.747965] -----------------------------
[ 1273.747971] include/trace/events/lock.h:74 suspicious rcu_dereference_check() usage!
[ 1273.747976] other info that might help us debug this:
[ 1273.747982] rcu_scheduler_active = 2, debug_locks = 1
[ 1273.747987] RCU used illegally from extended quiescent state!
[ 1273.747993] 1 lock held by swapper/8/0:
[ 1273.747998] #0: 000000010f7281b8 (max_trace_lock){..-.}-{2:2}, at: check_critical_timing+0x7c/0x1c8
[ 1273.748019] stack backtrace:
[ 1273.748034] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.9.0-20201006.rc8.git0.162c12a918a1.300.fc32.s390x+debug #1
[ 1273.748040] Hardware name: IBM 2964 NC9 702 (z/VM 6.4.0)
[ 1273.748045] Call Trace:
[ 1273.748053] [<000000010e656de0>] show_stack+0x90/0xf8
[ 1273.748060] [<000000010eea94da>] dump_stack+0xa2/0xd8
[ 1273.748068] [<000000010e711cc8>] trace_lock_acquired+0x178/0x180
[ 1273.748075] [<000000010e718ed4>] lock_contended+0x24/0xd8
[ 1273.748083] [<000000010f26d148>] _raw_spin_lock_irqsave+0xb0/0xd8
[ 1273.748089] [<000000010e7ec404>] check_critical_timing+0x7c/0x1c8
[ 1273.748096] [<000000010e7ecaa8>] tracer_hardirqs_on+0x128/0x148
[ 1273.748103] [<000000010e7eae0c>] trace_hardirqs_on+0x6c/0x1b0
[ 1273.748110] [<000000010e644ba8>] arch_cpu_idle+0x28/0x38
[ 1273.748116] [<000000010f26ccd6>] default_idle_call+0x56/0x98
[ 1273.748124] [<000000010e6da81a>] do_idle+0xf2/0x1b0
[ 1273.748130] [<000000010e6dab4e>] cpu_startup_entry+0x36/0x40
[ 1273.748137] [<000000010e6590fa>] smp_start_secondary+0x82/0x88
[ 1273.748142] 1 lock held by swapper/8/0:
[ 1273.748147] #0: 000000010f7281b8 (max_trace_lock){..-.}-{2:2}, at: check_critical_timing+0x7c/0x1c8I think this happens because trace_lock_acquired gets called from idle context?
Regards
Sven
Peter Zijlstra
maintainerQuoted 8 lines · Sven Schnelle
Hi Peter,
Quoted 4 lines · [email protected] writes:
After commit
eb1f00237aca("lockdep,trace: Expose tracepoints") the lock tracepoints are visible to lockdep and RCU-lockdep is finding a bunch more RCU violations that were previously hidden.
Quoted 8 lines
Quoted 6 lines
Switch the
idle->seqcountover to using raw_write_*() to avoid the lockdep annotation and thus the lock tracepoints.Reported-by Guenter Roeck <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>[..]I'm still seeing the splat below on s390 when irq tracing is enabled:
Damn... :/
This one is tricky, trouble seems to be that arch_cpu_idle() is defined to enable interrupts (no doubt because ot x86 :/), but we call it before rcu_exit_idle().
What a mess... let me rummage around the various archs to see what makes most sense here.
Peter Zijlstra
maintainerQuoted 18 lines · Peter Zijlstra
Quoted 8 lines · Sven Schnelle
Hi Peter,
Quoted 4 lines · [email protected] writes:
After commit
eb1f00237aca("lockdep,trace: Expose tracepoints") the lock tracepoints are visible to lockdep and RCU-lockdep is finding a bunch more RCU violations that were previously hidden.Quoted 8 lines
Quoted 6 lines
Switch the
idle->seqcountover to using raw_write_*() to avoid the lockdep annotation and thus the lock tracepoints.Reported-by Guenter Roeck <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>[..]I'm still seeing the splat below on s390 when irq tracing is enabled:
Quoted 8 lines
Damn... :/
This one is tricky, trouble seems to be that
arch_cpu_idle()is defined to enable interrupts (no doubt because ot x86 :/), but we call it beforercu_exit_idle().What a mess... let me rummage around the various archs to see what makes most sense here.
Maybe something like so, I've not yet tested it. I need to figure out how to force x86 into this path.
--- arch/alpha/kernel/process.c | 2 +- arch/arm/kernel/process.c | 2 +- arch/arm64/kernel/process.c | 2 +- arch/csky/kernel/process.c | 2 +- arch/h8300/kernel/process.c | 2 +- arch/hexagon/kernel/process.c | 2 +- arch/ia64/kernel/process.c | 2 +- arch/microblaze/kernel/process.c | 2 +- arch/mips/kernel/idle.c | 12 ++++++------ arch/nios2/kernel/process.c | 2 +- arch/openrisc/kernel/process.c | 2 +- arch/parisc/kernel/process.c | 2 +- arch/powerpc/kernel/idle.c | 4 ++-- arch/riscv/kernel/process.c | 2 +- arch/s390/kernel/idle.c | 2 +- arch/sh/kernel/idle.c | 2 +- arch/sparc/kernel/process_32.c | 2 +- arch/sparc/kernel/process_64.c | 4 ++-- arch/um/kernel/process.c | 2 +- arch/x86/include/asm/mwait.h | 2 -- arch/x86/kernel/process.c | 8 +++++--- kernel/sched/idle.c | 28 +++++++++++++++++++++++++++- 22 files changed, 58 insertions(+), 32 deletions(-)
arch/alpha/kernel/process.c +1 -1
@@ -57,7 +57,7 @@ EXPORT_SYMBOL(pm_power_off);void arch_cpu_idle (void ) {wtint (0 ); -local_irq_enable (); +raw_local_irq_enable (); }void arch_cpu_idle_dead (void )
arch/arm/kernel/process.c +1 -1
@@ -71,7 +71,7 @@ void arch_cpu_idle(void)arm_pm_idle (); elsecpu_do_idle (); -local_irq_enable (); +raw_local_irq_enable (); }void arch_cpu_idle_prepare (void )
arch/arm64/kernel/process.c +1 -1
@@ -124,7 +124,7 @@ void arch_cpu_idle(void)* tricks */ cpu_do_idle (); -local_irq_enable (); +raw_local_irq_enable (); }#ifdef CONFIG_HOTPLUG_CPU
arch/csky/kernel/process.c +1 -1
@@ -102,6 +102,6 @@ void arch_cpu_idle(void)#ifdef CONFIG_CPU_PM_STOP asmvolatile ("stop\n" );#endif - local_irq_enable(); + raw_local_irq_enable(); }#endif
arch/h8300/kernel/process.c +1 -1
@@ -57,7 +57,7 @@ asmlinkage void ret_from_kernel_thread(void);*/ void arch_cpu_idle (void ) { -local_irq_enable (); +raw_local_irq_enable (); __asm__("sleep" ); }
arch/hexagon/kernel/process.c +1 -1
@@ -44,7 +44,7 @@ void arch_cpu_idle(void) {__vmwait ();/* interrupts wake us up, but irqs are still disabled */ -local_irq_enable (); +raw_local_irq_enable (); }/*
arch/ia64/kernel/process.c +1 -1
@@ -252,7 +252,7 @@ void arch_cpu_idle(void)if (mark_idle) (*mark_idle)(1 ); -safe_halt (); +raw_safe_halt ();if (mark_idle) (*mark_idle)(0 );
arch/microblaze/kernel/process.c +1 -1
@@ -149,5 +149,5 @@ int dump_fpu(struct pt_regs *regs, elf_fpregset_t *fpregs)void arch_cpu_idle (void ) { -local_irq_enable (); +raw_local_irq_enable (); }
arch/mips/kernel/idle.c +6 -6
@@ -33,19 +33,19 @@ static void __cpuidle r3081_wait(void) {unsigned long cfg =read_c0_conf ();write_c0_conf (cfg |R30XX_CONF_HALT ); -local_irq_enable (); +raw_local_irq_enable (); } static void __cpuidle r39xx_wait(void) {if (!need_resched ())write_c0_conf (read_c0_conf () |TX39_CONF_HALT ); -local_irq_enable (); +raw_local_irq_enable (); } void __cpuidle r4k_wait(void) { -local_irq_enable (); +raw_local_irq_enable ();__r4k_wait (); }
@@ -64,7 +64,7 @@ void __cpuidle r4k_wait_irqoff(void)" .set arch=r4000 \n" " wait \n" " .set pop \n"); -local_irq_enable (); +raw_local_irq_enable (); }/*
@@ -84,7 +84,7 @@ static void __cpuidle rm7k_wait_irqoff(void)" wait \n" " mtc0 $1, $12 # stalls until W stage \n" " .set pop \n"); -local_irq_enable (); +raw_local_irq_enable (); }/*
@@ -257,7 +257,7 @@ void arch_cpu_idle(void)if (cpu_wait)cpu_wait ();else -local_irq_enable (); +raw_local_irq_enable (); }#ifdef CONFIG_CPU_IDLE
arch/nios2/kernel/process.c +1 -1
@@ -33,7 +33,7 @@ EXPORT_SYMBOL(pm_power_off);void arch_cpu_idle (void ) { -local_irq_enable (); +raw_local_irq_enable (); }/*
arch/openrisc/kernel/process.c +1 -1
@@ -79,7 +79,7 @@ void machine_power_off(void)*/ void arch_cpu_idle (void ) { -local_irq_enable (); +raw_local_irq_enable ();if (mfspr (SPR_UPR ) &SPR_UPR_PMP )mtspr (SPR_PMR ,mfspr (SPR_PMR ) |SPR_PMR_DME ); }
arch/parisc/kernel/process.c +1 -1
@@ -169,7 +169,7 @@ void __cpuidle arch_cpu_idle_dead(void) void __cpuidle arch_cpu_idle(void) { -local_irq_enable (); +raw_local_irq_enable ();/* nop on real hardware, qemu will idle sleep. */ asmvolatile ("or %%r10,%%r10,%%r10\n" :::);
arch/powerpc/kernel/idle.c +2 -2
@@ -60,9 +60,9 @@ void arch_cpu_idle(void)* interrupts enabled, some don't. */ if (irqs_disabled ()) -local_irq_enable (); +raw_local_irq_enable (); }else { -local_irq_enable (); +raw_local_irq_enable ();/* * Go into low thread priority and possibly * low power mode.
arch/riscv/kernel/process.c +1 -1
@@ -36,7 +36,7 @@ extern asmlinkage void ret_from_kernel_thread(void);void arch_cpu_idle (void ) {wait_for_interrupt (); -local_irq_enable (); +raw_local_irq_enable (); }void show_regs (struct pt_regs *regs)
arch/s390/kernel/idle.c +1 -1
@@ -123,7 +123,7 @@ void arch_cpu_idle_enter(void)void arch_cpu_idle (void ) {enabled_wait (); -local_irq_enable (); +raw_local_irq_enable (); }void arch_cpu_idle_exit (void )
arch/sh/kernel/idle.c +1 -1
@@ -22,7 +22,7 @@ static void (*sh_idle)(void);void default_idle (void ) {set_bl_bit (); -local_irq_enable (); +raw_local_irq_enable ();/* Isn't this racy ? */ cpu_sleep ();clear_bl_bit ();
arch/sparc/kernel/process_32.c +1 -1
@@ -74,7 +74,7 @@ void arch_cpu_idle(void) {if (sparc_idle) (*sparc_idle)(); -local_irq_enable (); +raw_local_irq_enable (); }/* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */
arch/sparc/kernel/process_64.c +2 -2
@@ -62,11 +62,11 @@ void arch_cpu_idle(void) {if (tlb_type != hypervisor) {touch_nmi_watchdog (); -local_irq_enable (); +raw_local_irq_enable (); }else {unsigned long pstate; -local_irq_enable (); +raw_local_irq_enable ();/* The sun4v sleeping code requires that we have PSTATE.IE cleared over * the cpu sleep hypervisor call.
arch/um/kernel/process.c +1 -1
@@ -217,7 +217,7 @@ void arch_cpu_idle(void) { cpu_tasks[current_thread_info ()->cpu].pid =os_getpid ();um_idle_sleep (); -local_irq_enable (); +raw_local_irq_enable (); }int __cant_sleep (void ) {
arch/x86/include/asm/mwait.h +0 -2
@@ -88,8 +88,6 @@ static inline void __mwaitx(unsigned long eax, unsigned long ebx,static inline void __sti_mwait (unsigned long eax,unsigned long ecx) { -trace_hardirqs_on (); -mds_idle_clear_cpu_buffers ();/* "mwait %eax, %ecx;" */ asm volatile("sti; .byte 0x0f, 0x01, 0xc9;"
arch/x86/kernel/process.c +5 -3
@@ -685,7 +685,7 @@ void arch_cpu_idle(void)*/ void __cpuidle default_idle(void) { -safe_halt (); +raw_safe_halt (); }#if defined(CONFIG_APM_MODULE ) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE )EXPORT_SYMBOL (default_idle);
@@ -736,6 +736,8 @@ void stop_this_cpu(void *dummy)/* * AMD Erratum 400 aware idle routine. We handle it the same way as C3 power * states (local apic timer and TSC stop). +* +* XXX this function is completely buggered vs RCU and tracing. */ static void amd_e400_idle (void ) {
@@ -801,9 +803,9 @@ static __cpuidle void mwait_idle(void)if (!need_resched ())__sti_mwait (0 ,0 );else -local_irq_enable (); +raw_local_irq_enable (); }else { -local_irq_enable (); +raw_local_irq_enable (); }__current_clr_polling (); }
kernel/sched/idle.c +27 -1
@@ -78,7 +78,7 @@ void __weak arch_cpu_idle_dead(void) { }void __weak arch_cpu_idle (void ) { cpu_idle_force_poll =1 ; -local_irq_enable (); +raw_local_irq_enable (); }/**
@@ -94,9 +94,35 @@ void __cpuidle default_idle_call(void)trace_cpu_idle (1 ,smp_processor_id ());stop_critical_timings (); + +/* +* arch_cpu_idle() is supposed to enable IRQs, however +* we can't do that because of RCU and tracing. +* +* Trace IRQs enable here, then switch off RCU, and have +* arch_cpu_idle() use raw_local_irq_enable(). Note that +* rcu_idle_enter() relies on lockdep IRQ state, so switch that +* last -- this is very similar to the entry code. +*/ +trace_hardirqs_on_prepare (); +lockdep_hardirqs_on_prepare (_THIS_IP_);rcu_idle_enter (); +lockdep_hardirqs_on (_THIS_IP_); +arch_cpu_idle (); + +/* +* OK, so IRQs are enabled here, but RCU needs them disabled to +* turn itself back on.. funny thing is that disabling IRQs +* will cause tracing, which needs RCU. Jump through hoops to +* make it 'work'. +*/ +raw_local_irq_disable (); +lockdep_hardirqs_off (_THIS_IP_);rcu_idle_exit (); +lockdep_hardirqs_on (_THIS_IP_); +raw_local_irq_enable (); +start_critical_timings ();trace_cpu_idle (PWR_EVENT_EXIT ,smp_processor_id ()); }
Sven Schnelle
reviewerHi Peter,
Quoted 25 lines · Peter Zijlstra
Quoted 24 lines · Peter Zijlstra
Quoted 8 lines · Sven Schnelle
Hi Peter,
Quoted 4 lines · [email protected] writes:
After commit
eb1f00237aca("lockdep,trace: Expose tracepoints") the lock tracepoints are visible to lockdep and RCU-lockdep is finding a bunch more RCU violations that were previously hidden.Quoted 8 lines
Quoted 6 lines
Switch the
idle->seqcountover to using raw_write_*() to avoid the lockdep annotation and thus the lock tracepoints.Reported-by Guenter Roeck <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>[..]I'm still seeing the splat below on s390 when irq tracing is enabled:
Damn... :/
This one is tricky, trouble seems to be that
arch_cpu_idle()is defined to enable interrupts (no doubt because ot x86 :/), but we call it beforercu_exit_idle().
Quoted 5 lines
Quoted 2 lines
What a mess... let me rummage around the various archs to see what makes most sense here.
Maybe something like so, I've not yet tested it. I need to figure out how to force x86 into this path.
I've gave this patch a quick test on linux-next from today and haven't seen the splat again. However it wasn't happening all the time, so will test it a bit longer. I haven't looked into the tracing code in detail, but i guess it was only happening when the lock was contented.
The only thing with this patch is that rcu complains that it gets called with interrupts enabled on s390 when rcu_irq_enter() is called. But a few trace_hardirqs_{on,off} at the beginning and end of the IRQ handlers are fixing this. Will check why this worked in the past.
Sven