Skip to contentSkip to conversation

per_cpu_counter_sum lockdep warning

DISCUSSION Balbir Singh 2 participants 4 messages Jun 4, 2008 VFS

lore
Share & tools
lore
#0 lore raw

Saw this warning on an x86_64 box, while booting up 2.6.26-rc4. Has anybody else seen it? Working on it?

=================================
[ INFO: inconsistent lock state ]
2.6.26-rc4 #2
---------------------------------
inconsistent {in-hardirq-W} -> {hardirq-on-W} usage.
init/1 [HC0[0]:SC0[0]:HE1:SE1] takes:
(&fbc->lock){+...}, at: [<ffffffff80386382>] __percpu_counter_sum+0xf/0x5a
{in-hardirq-W} state was registered at:
[<ffffffffffffffff>] 0xffffffffffffffff

irq event stamp: 694563
hardirqs last  enabled at (694563): [<ffffffff80297533>] kmem_cache_free+0x1a9/0x1bb
hardirqs last disabled at (694562): [<ffffffff802973aa>] kmem_cache_free+0x20/0x1bb
softirqs last  enabled at (692960): [<ffffffff8023a27b>] __do_softirq+0xda/0xe3
softirqs last disabled at (692945): [<ffffffff8020c4ac>] call_softirq+0x1c/0x28

other info that might help us debug this:
no locks held by init/1.

stack backtrace:

Pid: 1, comm: init Not tainted 2.6.26-rc4 #2

Call Trace:
 [<ffffffff802518e6>] print_usage_bug+0x15e/0x16f
 [<ffffffff8025281f>] mark_lock+0x22f/0x416
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff80253576>] __lock_acquire+0x4e7/0xc8a
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff80253da7>] lock_acquire+0x8e/0xb2
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff805990d7>] _spin_lock+0x26/0x53
 [<ffffffff80386382>] __percpu_counter_sum+0xf/0x5a
 [<ffffffff803139e2>] ext3_statfs+0xd6/0x160
 [<ffffffff8029c6be>] vfs_statfs+0x4e/0x6d
 [<ffffffff8029c85f>] vfs_statfs_native+0x22/0x48
 [<ffffffff8029c945>] sys_statfs+0x4d/0x89
 [<ffffffff80252bfe>] ? trace_hardirqs_on+0xf9/0x124
 [<ffffffff8037da3d>] ? __up_read+0x8f/0x97
 [<ffffffff8024ad5a>] ? up_read+0x26/0x2a
 [<ffffffff80598cf0>] ? trace_hardirqs_on_thunk+0x35/0x3a
 [<ffffffff80252bfe>] ? trace_hardirqs_on+0xf9/0x124
 [<ffffffff80598cf0>] ? trace_hardirqs_on_thunk+0x35/0x3a
 [<ffffffff8020b1fb>] system_call_after_swapgs+0x7b/0x80
Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#1 lore raw
Quoted 2 lines · Balbir Singh

Saw this warning on an x86_64 box, while booting up 2.6.26-rc4. Has anybody else seen it? Working on it?

I've neither seen it, nor am I working on it, but I can decode it.

Quoted 1 line

inconsistent {in-hardirq-W} -> {hardirq-on-W} usage.

Translation: "This lock was previously grabbed in hardirq context. Now someone's taking it in process context without interrupts disabled. That could lead to a deadlock."

Quoted 2 lines

init/1 [HC0[0]:SC0[0]:HE1:SE1] takes:
(&fbc->lock){+...}, at: [<ffffffff80386382>] __percpu_counter_sum+0xf/0x5a

That's the name of the lock -- &fbc->lock and the function where it happens.

Quoted 2 lines

{in-hardirq-W} state was registered at:
[<ffffffffffffffff>] 0xffffffffffffffff

Drat, no backtrace for the guy who took the lock in hardirq context.

Quoted 11 lines
Call Trace:
 [<ffffffff802518e6>] print_usage_bug+0x15e/0x16f
 [<ffffffff8025281f>] mark_lock+0x22f/0x416
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff80253576>] __lock_acquire+0x4e7/0xc8a
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff80253da7>] lock_acquire+0x8e/0xb2
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff805990d7>] _spin_lock+0x26/0x53
 [<ffffffff80386382>] __percpu_counter_sum+0xf/0x5a
 [<ffffffff803139e2>] ext3_statfs+0xd6/0x160

ext3_statfs was the one who asked for the lock to be taken without disabling interrupts.

Some percpu counters are supposed to be used from interrupt context. These are created with percpu_counter_init_irq. Others are not and should be created with percpu_counter_init. It seems like someone's made a mess of that rule. This is likely to be a driver, IMO. Perhaps you could work on tracking this down?

Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."
#2 lore raw
Quoted 4 lines · Matthew Wilcox
Quoted 2 lines · Balbir Singh

Saw this warning on an x86_64 box, while booting up 2.6.26-rc4. Has anybody else seen it? Working on it?

Quoted 8 lines

I've neither seen it, nor am I working on it, but I can decode it.

Quoted 1 line

inconsistent {in-hardirq-W} -> {hardirq-on-W} usage.

Translation: "This lock was previously grabbed in hardirq context. Now someone's taking it in process context without interrupts disabled. That could lead to a deadlock."

I understand this part. I did not want to interpret the data, but I think that's a better way of reporting problems.

Quoted 27 lines
Quoted 2 lines

init/1 [HC0[0]:SC0[0]:HE1:SE1] takes:
(&fbc->lock){+...}, at: [<ffffffff80386382>] __percpu_counter_sum+0xf/0x5a

That's the name of the lock -- &fbc->lock and the function where it happens.

Quoted 2 lines

{in-hardirq-W} state was registered at:
[<ffffffffffffffff>] 0xffffffffffffffff

Drat, no backtrace for the guy who took the lock in hardirq context.

Quoted 11 lines
Call Trace:
 [<ffffffff802518e6>] print_usage_bug+0x15e/0x16f
 [<ffffffff8025281f>] mark_lock+0x22f/0x416
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff80253576>] __lock_acquire+0x4e7/0xc8a
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff80253da7>] lock_acquire+0x8e/0xb2
 [<ffffffff80386382>] ? __percpu_counter_sum+0xf/0x5a
 [<ffffffff805990d7>] _spin_lock+0x26/0x53
 [<ffffffff80386382>] __percpu_counter_sum+0xf/0x5a
 [<ffffffff803139e2>] ext3_statfs+0xd6/0x160

ext3_statfs was the one who asked for the lock to be taken without disabling interrupts.

Quoted 6 lines

Some percpu counters are supposed to be used from interrupt context. These are created with percpu_counter_init_irq. Others are not and should be created with percpu_counter_init. It seems like someone's made a mess of that rule. This is likely to be a driver, IMO. Perhaps you could work on tracking this down?

Sure, I will. Let me poke harder, I'll recheck all patches I have applied (if any) on my current tree.

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#3 lore raw
In reply to Matthew Wilcox
Quoted 7 lines

Some percpu counters are supposed to be used from interrupt context. These are created with percpu_counter_init_irq. Others are not and should be created with percpu_counter_init. It seems like someone's made a mess of that rule. This is likely to be a driver, IMO. Perhaps you could work on tracking this down?

Sorry for the noise. It was patch I had applied on my tree, not yet in mainline. The explanation though did help me find and fix the problem.

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL