Skip to contentSkip to conversation

How to avoid spurious lockdep warnings?

DISCUSSION Jeremy Fitzhardinge 3 participants 5 messages Mar 20, 2008 – Mar 22, 2008

lore
Share & tools
lore
#0 lore raw

In a Xen system, when a new pagetable is about to be put in use it is "pinned", meaning that each page in the pagetable is registered with the hypervisor. This is done in arch/x86/xen/mmu.c:pin_page().

In order to make this efficient, the hypercalls for pinning are batched, so that multiple pages are submitted at once in a single multicall. While a page is batched pending the hypercall, its corresponding pte_lock is held.

This means that the code can end up holding multiple pte locks at once, though it is guaranteed to never try to hold the same lock at once. However, because these locks are in the same lock class, I get a spurious warning from lockdep. Is there some way I can get rid of this warning?

=============================================
[ INFO: possible recursive locking detected ] 2.6.25-rc6-x86-latest.git-dirty #297
---------------------------------------------
init/1 is trying to acquire lock:
(__pte_lockptr(page)){--..}, at: [<c0105038>] pin_page+0x6a/0x167

but task is already holding lock:
(__pte_lockptr(page)){--..}, at: [<c0105038>] pin_page+0x6a/0x167

other info that might help us debug this:
4 locks held by init/1:

#0:  (&mm->mmap_sem){----}, at: [<c012645c>] copy_process+0x97e/0x122f
#1:  (&mm->mmap_sem/1){--..}, at: [<c012646c>] copy_process+0x98e/0x122f
#2:  (&mm->page_table_lock){--..}, at: [<c0104bf5>] xen_dup_mmap+0x11/0x24
#3:  (__pte_lockptr(page)){--..}, at: [<c0105038>] pin_page+0x6a/0x167
stack backtrace:
Pid: 1, comm: init Not tainted 2.6.25-rc6-x86-latest.git-dirty #297
[<c0144049>] __lock_acquire+0x821/0xb50
[<c01443ee>] lock_acquire+0x76/0x9d
[<c0105038>] ? pin_page+0x6a/0x167
[<c0454537>] _spin_lock+0x23/0x32
[<c0105038>] ? pin_page+0x6a/0x167
[<c0105038>] pin_page+0x6a/0x167
[<c0104520>] pgd_walk+0x18f/0x1e1
[<c0104fce>] ? pin_page+0x0/0x167
[<c0104b20>] xen_pgd_pin+0x46/0x10a
[<c0104bfd>] xen_dup_mmap+0x19/0x24
[<c0126619>] copy_process+0xb3b/0x122f
[<c0126e6d>] do_fork+0xab/0x1dd
[<c017341b>] ? vfs_write+0xf1/0x108
[<c010877a>] ? sysenter_past_esp+0xba/0xc8
[<c0106cd5>] sys_clone+0x1f/0x21
[<c0108731>] sysenter_past_esp+0x71/0xc8
=======================

Thanks,
J

#1 lore raw
Quoted 9 lines · Jeremy Fitzhardinge

In a Xen system, when a new pagetable is about to be put in use it is "pinned", meaning that each page in the pagetable is registered with the hypervisor. This is done in arch/x86/xen/mmu.c:pin_page().

In order to make this efficient, the hypercalls for pinning are batched, so that multiple pages are submitted at once in a single multicall. While a page is batched pending the hypercall, its corresponding pte_lock is held.

Quoted 5 lines

This means that the code can end up holding multiple pte locks at once, though it is guaranteed to never try to hold the same lock at once. However, because these locks are in the same lock class, I get a spurious warning from lockdep. Is there some way I can get rid of this warning?

How many locks at once?

#2 lore raw
Quoted 10 lines · Peter Zijlstra
Quoted 6 lines · Jeremy Fitzhardinge

In a Xen system, when a new pagetable is about to be put in use it is "pinned", meaning that each page in the pagetable is registered with the hypervisor. This is done in arch/x86/xen/mmu.c:pin_page().

In order to make this efficient, the hypercalls for pinning are batched, so that multiple pages are submitted at once in a single multicall. While a page is batched pending the hypercall, its corresponding pte_lock is held.

This means that the code can end up holding multiple pte locks at once, though it is guaranteed to never try to hold the same lock at once. However, because these locks are in the same lock class, I get a spurious warning from lockdep. Is there some way I can get rid of this warning?

How many locks at once?

(We discussed this, but for the record...)

The main limit is the batch size, which is currently 32. There's nothing magic about this number, so it may change (I can't imagine it getting much larger however, since a 32x mitigation of hypercall overhead is already pretty good).

J

#3 lore raw
In reply to Jeremy Fitzhardinge
Quoted 9 lines · On Thu, 20 Mar 2008 16:02:11 -0700 Jeremy Fitzhardinge <[email protected]> wrote:

In a Xen system, when a new pagetable is about to be put in use it is "pinned", meaning that each page in the pagetable is registered with the hypervisor. This is done in arch/x86/xen/mmu.c:pin_page().

In order to make this efficient, the hypercalls for pinning are batched, so that multiple pages are submitted at once in a single multicall. While a page is batched pending the hypercall, its corresponding pte_lock is held.

Quoted 5 lines

This means that the code can end up holding multiple pte locks at once, though it is guaranteed to never try to hold the same lock at once. However, because these locks are in the same lock class, I get a spurious warning from lockdep. Is there some way I can get rid of this warning?

what's the ordering guarantee between these locks ?

Signature · 3 lines
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#4 lore raw
Quoted 20 lines · Arjan van de Ven
Quoted 8 lines · On Thu, 20 Mar 2008 16:02:11 -0700 Jeremy Fitzhardinge <[email protected]> wrote:

In a Xen system, when a new pagetable is about to be put in use it is "pinned", meaning that each page in the pagetable is registered with the hypervisor. This is done in arch/x86/xen/mmu.c:pin_page().

In order to make this efficient, the hypercalls for pinning are batched, so that multiple pages are submitted at once in a single multicall. While a page is batched pending the hypercall, its corresponding pte_lock is held.

Quoted 6 lines

This means that the code can end up holding multiple pte locks at once, though it is guaranteed to never try to hold the same lock at once. However, because these locks are in the same lock class, I get a spurious warning from lockdep. Is there some way I can get rid of this warning?

Quoted 2 lines

what's the ordering guarantee between these locks ?

Pagetable virtual address order. move_ptes can also lock two ptes within one pagetable, without regard to order, but they're protected by mmap_sem, which is also held during pinning.

J