kvm: use the correct RCU API

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/mmu.c:3020 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/8550:
#0: (&kvm->slots_lock){+.+.+.}, at: [<ffffffffa011a6ac>] kvm_set_memory_region+0x29/0x50 [kvm] #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa012262d>] kvm_arch_commit_memory_region+0xa6/0xe2 [kvm]

stack backtrace:
Pid: 8550, comm: qemu-system-x86 Not tainted 2.6.34-rc4-tip-01028-g939eab1 #27
Call Trace:
  [<ffffffff8106c59e>] lockdep_rcu_dereference+0xaa/0xb3
  [<ffffffffa012f6c1>] kvm_mmu_calculate_mmu_pages+0x44/0x7d [kvm]
  [<ffffffffa012263e>] kvm_arch_commit_memory_region+0xb7/0xe2 [kvm]
  [<ffffffffa011a5d7>] __kvm_set_memory_region+0x636/0x6e2 [kvm]
  [<ffffffffa011a6ba>] kvm_set_memory_region+0x37/0x50 [kvm]
  [<ffffffffa015e956>] vmx_set_tss_addr+0x46/0x5a [kvm_intel]
  [<ffffffffa0126592>] kvm_arch_vm_ioctl+0x17a/0xcf8 [kvm]
  [<ffffffff810a8692>] ? unlock_page+0x27/0x2c
  [<ffffffff810bf879>] ? __do_fault+0x3a9/0x3e1
  [<ffffffffa011b12f>] kvm_vm_ioctl+0x364/0x38d [kvm]
  [<ffffffff81060cfa>] ? up_read+0x23/0x3d
  [<ffffffff810f3587>] vfs_ioctl+0x32/0xa6
  [<ffffffff810f3b19>] do_vfs_ioctl+0x495/0x4db
  [<ffffffff810e6b2f>] ? fget_light+0xc2/0x241
  [<ffffffff810e416c>] ? do_sys_open+0x104/0x116
  [<ffffffff81382d6d>] ? retint_swapgs+0xe/0x13
  [<ffffffff810f3ba6>] sys_ioctl+0x47/0x6a
  [<ffffffff810021db>] system_call_fastpath+0x16/0x1b

Quoted 42 lines · Lai Jiangshan

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/mmu.c:3020 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/8550:
#0: (&kvm->slots_lock){+.+.+.}, at: [<ffffffffa011a6ac>] kvm_set_memory_region+0x29/0x50 [kvm] #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa012262d>] kvm_arch_commit_memory_region+0xa6/0xe2 [kvm]

stack backtrace:
Pid: 8550, comm: qemu-system-x86 Not tainted 2.6.34-rc4-tip-01028-g939eab1 #27
Call Trace:
 [<ffffffff8106c59e>] lockdep_rcu_dereference+0xaa/0xb3
 [<ffffffffa012f6c1>] kvm_mmu_calculate_mmu_pages+0x44/0x7d [kvm]
 [<ffffffffa012263e>] kvm_arch_commit_memory_region+0xb7/0xe2 [kvm]
 [<ffffffffa011a5d7>] __kvm_set_memory_region+0x636/0x6e2 [kvm]
 [<ffffffffa011a6ba>] kvm_set_memory_region+0x37/0x50 [kvm]
 [<ffffffffa015e956>] vmx_set_tss_addr+0x46/0x5a [kvm_intel]
 [<ffffffffa0126592>] kvm_arch_vm_ioctl+0x17a/0xcf8 [kvm]
 [<ffffffff810a8692>] ? unlock_page+0x27/0x2c
 [<ffffffff810bf879>] ? __do_fault+0x3a9/0x3e1
 [<ffffffffa011b12f>] kvm_vm_ioctl+0x364/0x38d [kvm]
 [<ffffffff81060cfa>] ? up_read+0x23/0x3d
 [<ffffffff810f3587>] vfs_ioctl+0x32/0xa6
 [<ffffffff810f3b19>] do_vfs_ioctl+0x495/0x4db
 [<ffffffff810e6b2f>] ? fget_light+0xc2/0x241
 [<ffffffff810e416c>] ? do_sys_open+0x104/0x116
 [<ffffffff81382d6d>] ? retint_swapgs+0xe/0x13
 [<ffffffff810f3ba6>] sys_ioctl+0x47/0x6a
 [<ffffffff810021db>] system_call_fastpath+0x16/0x1b

include/linux/kvm_host.h void kvm_exit(void); · line 248

Quoted 7 lines

include/linux/kvm_host.h +7 -0

+static inline struct kvm_memslots *kvm_memslots(struct kvm *kvm)
+{
+	return rcu_dereference_check(kvm->memslots,
+			srcu_read_lock_held(&kvm->srcu)
+			|| lockdep_is_held(&kvm->slots_lock));
+}
+

This open-codes srcu_dereference(). I guess we need an srcu_dereference_check(). Paul?

btw, perhaps it is possible not to call rcu_dereference from the write paths.

Quoted 43 lines · Avi Kivity

Quoted 42 lines · Lai Jiangshan

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/mmu.c:3020 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/8550:
#0: (&kvm->slots_lock){+.+.+.}, at: [<ffffffffa011a6ac>] kvm_set_memory_region+0x29/0x50 [kvm] #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa012262d>] kvm_arch_commit_memory_region+0xa6/0xe2 [kvm]

stack backtrace:
Pid: 8550, comm: qemu-system-x86 Not tainted 2.6.34-rc4-tip-01028-g939eab1 #27
Call Trace:
 [<ffffffff8106c59e>] lockdep_rcu_dereference+0xaa/0xb3
 [<ffffffffa012f6c1>] kvm_mmu_calculate_mmu_pages+0x44/0x7d [kvm]
 [<ffffffffa012263e>] kvm_arch_commit_memory_region+0xb7/0xe2 [kvm]
 [<ffffffffa011a5d7>] __kvm_set_memory_region+0x636/0x6e2 [kvm]
 [<ffffffffa011a6ba>] kvm_set_memory_region+0x37/0x50 [kvm]
 [<ffffffffa015e956>] vmx_set_tss_addr+0x46/0x5a [kvm_intel]
 [<ffffffffa0126592>] kvm_arch_vm_ioctl+0x17a/0xcf8 [kvm]
 [<ffffffff810a8692>] ? unlock_page+0x27/0x2c
 [<ffffffff810bf879>] ? __do_fault+0x3a9/0x3e1
 [<ffffffffa011b12f>] kvm_vm_ioctl+0x364/0x38d [kvm]
 [<ffffffff81060cfa>] ? up_read+0x23/0x3d
 [<ffffffff810f3587>] vfs_ioctl+0x32/0xa6
 [<ffffffff810f3b19>] do_vfs_ioctl+0x495/0x4db
 [<ffffffff810e6b2f>] ? fget_light+0xc2/0x241
 [<ffffffff810e416c>] ? do_sys_open+0x104/0x116
 [<ffffffff81382d6d>] ? retint_swapgs+0xe/0x13
 [<ffffffff810f3ba6>] sys_ioctl+0x47/0x6a
 [<ffffffff810021db>] system_call_fastpath+0x16/0x1b

Quoted 10 lines

include/linux/kvm_host.h void kvm_exit(void); · line 248

Quoted 7 lines

include/linux/kvm_host.h +7 -0

+static inline struct kvm_memslots *kvm_memslots(struct kvm *kvm)
+{
+	return rcu_dereference_check(kvm->memslots,
+			srcu_read_lock_held(&kvm->srcu)
+			|| lockdep_is_held(&kvm->slots_lock));
+}
+

This open-codes srcu_dereference(). I guess we need an srcu_dereference_check(). Paul?

One is coming in Arnd's sparse-based patchset. It is probably best to open-code this in the meantime and clean up later, but I will double-check with Arnd.

Quoted 2 lines

btw, perhaps it is possible not to call rcu_dereference from the write paths.

There is an rcu_dereference_protected() on its way to mainline to handle the case where the reference is always protected by a lock. Why not just access it directly? Because if you do that, the sparse-based checks will yell at you.

There is also an rcu_access_pointer() on its way to mainline for cases where you only want to test the pointer itself, not dereference it.

Thanx, Paul

Quoted 44 lines · Paul E. McKenney

Quoted 43 lines · Avi Kivity

Quoted 42 lines · Lai Jiangshan

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/mmu.c:3020 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/8550:
#0: (&kvm->slots_lock){+.+.+.}, at: [<ffffffffa011a6ac>] kvm_set_memory_region+0x29/0x50 [kvm] #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa012262d>] kvm_arch_commit_memory_region+0xa6/0xe2 [kvm]

stack backtrace:
Pid: 8550, comm: qemu-system-x86 Not tainted 2.6.34-rc4-tip-01028-g939eab1 #27
Call Trace:
 [<ffffffff8106c59e>] lockdep_rcu_dereference+0xaa/0xb3
 [<ffffffffa012f6c1>] kvm_mmu_calculate_mmu_pages+0x44/0x7d [kvm]
 [<ffffffffa012263e>] kvm_arch_commit_memory_region+0xb7/0xe2 [kvm]
 [<ffffffffa011a5d7>] __kvm_set_memory_region+0x636/0x6e2 [kvm]
 [<ffffffffa011a6ba>] kvm_set_memory_region+0x37/0x50 [kvm]
 [<ffffffffa015e956>] vmx_set_tss_addr+0x46/0x5a [kvm_intel]
 [<ffffffffa0126592>] kvm_arch_vm_ioctl+0x17a/0xcf8 [kvm]
 [<ffffffff810a8692>] ? unlock_page+0x27/0x2c
 [<ffffffff810bf879>] ? __do_fault+0x3a9/0x3e1
 [<ffffffffa011b12f>] kvm_vm_ioctl+0x364/0x38d [kvm]
 [<ffffffff81060cfa>] ? up_read+0x23/0x3d
 [<ffffffff810f3587>] vfs_ioctl+0x32/0xa6
 [<ffffffff810f3b19>] do_vfs_ioctl+0x495/0x4db
 [<ffffffff810e6b2f>] ? fget_light+0xc2/0x241
 [<ffffffff810e416c>] ? do_sys_open+0x104/0x116
 [<ffffffff81382d6d>] ? retint_swapgs+0xe/0x13
 [<ffffffff810f3ba6>] sys_ioctl+0x47/0x6a
 [<ffffffff810021db>] system_call_fastpath+0x16/0x1b

Quoted 11 lines

Quoted 10 lines

include/linux/kvm_host.h void kvm_exit(void); · line 248

Quoted 7 lines

include/linux/kvm_host.h +7 -0

+static inline struct kvm_memslots *kvm_memslots(struct kvm *kvm)
+{
+	return rcu_dereference_check(kvm->memslots,
+			srcu_read_lock_held(&kvm->srcu)
+			|| lockdep_is_held(&kvm->slots_lock));
+}
+

This open-codes srcu_dereference(). I guess we need an srcu_dereference_check(). Paul?

rcu_dereference_check() is useful when rcu_dereference(), rcu_dereference_bh(), rcu_dereference_sched() and srcu_dereference() are not appropriate.

I think we don't need srcu_dereference_check() nor rcu_dereference_bh_check() nor rcu_dereference_sched_check().

Quoted 15 lines

One is coming in Arnd's sparse-based patchset. It is probably best to open-code this in the meantime and clean up later, but I will double-check with Arnd.

Quoted 2 lines

btw, perhaps it is possible not to call rcu_dereference from the write paths.

There is an rcu_dereference_protected() on its way to mainline to handle the case where the reference is always protected by a lock. Why not just access it directly? Because if you do that, the sparse-based checks will yell at you.

There is also an rcu_access_pointer() on its way to mainline for cases where you only want to test the pointer itself, not dereference it.

Quoted 1 line

Thanx, Paul

I reviewed the code, the functions can be called from the srcu-read-site or update-site, rcu_dereference_check() can simplify the code.

If we use rcu_dereference_protected(), we may need duplicate the functions.

I think there is very small overhead of using rcu_dereference(), so we can call it from write paths.

Applied the patch I just sent and let CONFIG_PROVE_RCU=y, we can got the following dmesg. And we found that it is because some codes in KVM dereferences srcu-protected pointer without srcu_read_lock() held or update-side lock held.

It is not hard to fix, the problem is that:
Where is the most proper place to put a srcu_read_lock()?

I can not determine the answer, so I report this bug instead of fixing it.

Quoted 11 lines · Lai Jiangshan

Applied the patch I just sent and let CONFIG_PROVE_RCU=y, we can got the following dmesg. And we found that it is because some codes in KVM dereferences srcu-protected pointer without srcu_read_lock() held or update-side lock held.

It is not hard to fix, the problem is that:
Where is the most proper place to put a srcu_read_lock()?

I can not determine the answer, so I report this bug instead of fixing it.

I think the else branch in complete_pio() should work. Marcelo?

Longer term I'd like to see the lock taken at the high levels (ioctls, in virt/kvm) and dropped only for guest entry and when we explicitly sleep (hlt emulation).

Note: complete_pio() is gone in the current code.

Quoted 14 lines · Avi Kivity

Quoted 11 lines · Lai Jiangshan

Applied the patch I just sent and let CONFIG_PROVE_RCU=y, we can got the following dmesg. And we found that it is because some codes in KVM dereferences srcu-protected pointer without srcu_read_lock() held or update-side lock held.

It is not hard to fix, the problem is that:
Where is the most proper place to put a srcu_read_lock()?

I can not determine the answer, so I report this bug instead of fixing it.

I think the else branch in complete_pio() should work. Marcelo?

Quoted 5 lines

Longer term I'd like to see the lock taken at the high levels (ioctls, in virt/kvm) and dropped only for guest entry and when we explicitly sleep (hlt emulation).

Note: complete_pio() is gone in the current code.

Yes, this was fixed by 7fb2ea1e6.

Quoted 15 lines · Marcelo Tosatti

Quoted 14 lines · Avi Kivity

Quoted 11 lines · Lai Jiangshan

Applied the patch I just sent and let CONFIG_PROVE_RCU=y, we can got the following dmesg. And we found that it is because some codes in KVM dereferences srcu-protected pointer without srcu_read_lock() held or update-side lock held.

It is not hard to fix, the problem is that:
Where is the most proper place to put a srcu_read_lock()?

I can not determine the answer, so I report this bug instead of fixing it.

I think the else branch in complete_pio() should work. Marcelo?

Quoted 9 lines

Quoted 5 lines

Longer term I'd like to see the lock taken at the high levels (ioctls, in virt/kvm) and dropped only for guest entry and when we explicitly sleep (hlt emulation).

Note: complete_pio() is gone in the current code.

Yes, this was fixed by 7fb2ea1e6.

Applied the patch I sent yesterday and let CONFIG_PROVE_RCU=y I can get the following dmesg.

Under very simple test, these is no complaint from PROVE_RCU after this patch applied.

More test or reviewing of code are need in future.

----------
Subject: [PATCH] kvm: add missing srcu_read_lock()

I got this dmesg due to srcu_read_lock() is missing in kvm_mmu_notifier_release().

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/x86.h:72 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/3100:
#0: (rcu_read_lock){.+.+..}, at: [<ffffffff810d73dc>] __mmu_notifier_release+0x38/0xdf #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa0130a6a>] kvm_mmu_zap_all+0x21/0x5e [kvm]

stack backtrace:
Pid: 3100, comm: qemu-system-x86 Not tainted 2.6.34-rc3-22949-gbc8a97a-dirty #2
Call Trace:
 [<ffffffff8106afd9>] lockdep_rcu_dereference+0xaa/0xb3
 [<ffffffffa0123a89>] unalias_gfn+0x56/0xab [kvm]
 [<ffffffffa0119600>] gfn_to_memslot+0x16/0x25 [kvm]
 [<ffffffffa012ffca>] gfn_to_rmap+0x17/0x6e [kvm]
 [<ffffffffa01300c1>] rmap_remove+0xa0/0x19d [kvm]
 [<ffffffffa0130649>] kvm_mmu_zap_page+0x109/0x34d [kvm]
 [<ffffffffa0130a7e>] kvm_mmu_zap_all+0x35/0x5e [kvm]
 [<ffffffffa0122870>] kvm_arch_flush_shadow+0x16/0x22 [kvm]
 [<ffffffffa01189e0>] kvm_mmu_notifier_release+0x15/0x17 [kvm]
 [<ffffffff810d742c>] __mmu_notifier_release+0x88/0xdf
 [<ffffffff810d73dc>] ? __mmu_notifier_release+0x38/0xdf
 [<ffffffff81040848>] ? exit_mm+0xe0/0x115
 [<ffffffff810c2cb0>] exit_mmap+0x2c/0x17e
 [<ffffffff8103c472>] mmput+0x2d/0xd4
 [<ffffffff81040870>] exit_mm+0x108/0x115

[...]

Signed-off-by Lai Jiangshan <[email protected]>

---

virt/kvm/kvm_main.c +4 -0

index a5dfea1..a6d639d 100644
@@ -341,7 +341,11 @@ static void kvm_mmu_notifier_release(struct mmu_notifier *mn,
 				     struct mm_struct *mm)
 {
 	struct kvm *kvm = mmu_notifier_to_kvm(mn);
+	int idx;
+
+	idx = srcu_read_lock(&kvm->srcu);
 	kvm_arch_flush_shadow(kvm);
+	srcu_read_unlock(&kvm->srcu, idx);
 }
 
 static const struct mmu_notifier_ops kvm_mmu_notifier_ops = {

Quoted 37 lines

Subject: [PATCH] kvm: add missing srcu_read_lock()

I got this dmesg due to srcu_read_lock() is missing in kvm_mmu_notifier_release().

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/x86.h:72 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/3100:
#0: (rcu_read_lock){.+.+..}, at: [<ffffffff810d73dc>] __mmu_notifier_release+0x38/0xdf #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa0130a6a>] kvm_mmu_zap_all+0x21/0x5e [kvm]

stack backtrace:
Pid: 3100, comm: qemu-system-x86 Not tainted 2.6.34-rc3-22949-gbc8a97a-dirty #2
Call Trace:
  [<ffffffff8106afd9>] lockdep_rcu_dereference+0xaa/0xb3
  [<ffffffffa0123a89>] unalias_gfn+0x56/0xab [kvm]
  [<ffffffffa0119600>] gfn_to_memslot+0x16/0x25 [kvm]
  [<ffffffffa012ffca>] gfn_to_rmap+0x17/0x6e [kvm]
  [<ffffffffa01300c1>] rmap_remove+0xa0/0x19d [kvm]
  [<ffffffffa0130649>] kvm_mmu_zap_page+0x109/0x34d [kvm]
  [<ffffffffa0130a7e>] kvm_mmu_zap_all+0x35/0x5e [kvm]
  [<ffffffffa0122870>] kvm_arch_flush_shadow+0x16/0x22 [kvm]
  [<ffffffffa01189e0>] kvm_mmu_notifier_release+0x15/0x17 [kvm]
  [<ffffffff810d742c>] __mmu_notifier_release+0x88/0xdf
  [<ffffffff810d73dc>] ? __mmu_notifier_release+0x38/0xdf
  [<ffffffff81040848>] ? exit_mm+0xe0/0x115
  [<ffffffff810c2cb0>] exit_mmap+0x2c/0x17e
  [<ffffffff8103c472>] mmput+0x2d/0xd4
  [<ffffffff81040870>] exit_mm+0x108/0x115

Quoted 2 lines

[...]

Queued, thank you, Lai!

Quoted 38 lines · Paul E. McKenney

Quoted 38 lines

Subject: [PATCH] kvm: add missing srcu_read_lock()

I got this dmesg due to srcu_read_lock() is missing in kvm_mmu_notifier_release().

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/x86.h:72 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/3100:
#0: (rcu_read_lock){.+.+..}, at: [<ffffffff810d73dc>] __mmu_notifier_release+0x38/0xdf #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa0130a6a>] kvm_mmu_zap_all+0x21/0x5e [kvm]

stack backtrace:
Pid: 3100, comm: qemu-system-x86 Not tainted 2.6.34-rc3-22949-gbc8a97a-dirty #2
Call Trace:
 [<ffffffff8106afd9>] lockdep_rcu_dereference+0xaa/0xb3
 [<ffffffffa0123a89>] unalias_gfn+0x56/0xab [kvm]
 [<ffffffffa0119600>] gfn_to_memslot+0x16/0x25 [kvm]
 [<ffffffffa012ffca>] gfn_to_rmap+0x17/0x6e [kvm]
 [<ffffffffa01300c1>] rmap_remove+0xa0/0x19d [kvm]
 [<ffffffffa0130649>] kvm_mmu_zap_page+0x109/0x34d [kvm]
 [<ffffffffa0130a7e>] kvm_mmu_zap_all+0x35/0x5e [kvm]
 [<ffffffffa0122870>] kvm_arch_flush_shadow+0x16/0x22 [kvm]
 [<ffffffffa01189e0>] kvm_mmu_notifier_release+0x15/0x17 [kvm]
 [<ffffffff810d742c>] __mmu_notifier_release+0x88/0xdf
 [<ffffffff810d73dc>] ? __mmu_notifier_release+0x38/0xdf
 [<ffffffff81040848>] ? exit_mm+0xe0/0x115
 [<ffffffff810c2cb0>] exit_mmap+0x2c/0x17e
 [<ffffffff8103c472>] mmput+0x2d/0xd4
 [<ffffffff81040870>] exit_mm+0x108/0x115

[...]

Quoted 1 line

Queued, thank you, Lai!

Paul, I'd like to merge this via the kvm tree.

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/mmu.c:3020 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/8550:
#0: (&kvm->slots_lock){+.+.+.}, at: [<ffffffffa011a6ac>] kvm_set_memory_region+0x29/0x50 [kvm] #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa012262d>] kvm_arch_commit_memory_region+0xa6/0xe2 [kvm]

stack backtrace:
Pid: 8550, comm: qemu-system-x86 Not tainted 2.6.34-rc4-tip-01028-g939eab1 #27
Call Trace:
 [<ffffffff8106c59e>] lockdep_rcu_dereference+0xaa/0xb3
 [<ffffffffa012f6c1>] kvm_mmu_calculate_mmu_pages+0x44/0x7d [kvm]
 [<ffffffffa012263e>] kvm_arch_commit_memory_region+0xb7/0xe2 [kvm]
 [<ffffffffa011a5d7>] __kvm_set_memory_region+0x636/0x6e2 [kvm]
 [<ffffffffa011a6ba>] kvm_set_memory_region+0x37/0x50 [kvm]
 [<ffffffffa015e956>] vmx_set_tss_addr+0x46/0x5a [kvm_intel]
 [<ffffffffa0126592>] kvm_arch_vm_ioctl+0x17a/0xcf8 [kvm]
 [<ffffffff810a8692>] ? unlock_page+0x27/0x2c
 [<ffffffff810bf879>] ? __do_fault+0x3a9/0x3e1
 [<ffffffffa011b12f>] kvm_vm_ioctl+0x364/0x38d [kvm]

 [<ffffffff81060cfa>] ? up_read+0x23/0x3d
 [<ffffffff810f3587>] vfs_ioctl+0x32/0xa6
 [<ffffffff810f3b19>] do_vfs_ioctl+0x495/0x4db
 [<ffffffff810e6b2f>] ? fget_light+0xc2/0x241
 [<ffffffff810e416c>] ? do_sys_open+0x104/0x116
 [<ffffffff81382d6d>] ? retint_swapgs+0xe/0x13
 [<ffffffff810f3ba6>] sys_ioctl+0x47/0x6a
 [<ffffffff810021db>] system_call_fastpath+0x16/0x1b

Signed-off-by Lai Jiangshan <[email protected]>

---

arch/ia64/kvm/kvm-ia64.c +1 -1

index 73c5c2b..52b8ece 100644
@@ -1379,7 +1379,7 @@ static void kvm_release_vm_pages(struct kvm *kvm)
 	int i, j;
 	unsigned long base_gfn;
 
-	slots = rcu_dereference(kvm->memslots);
+	slots = kvm_memslots(kvm);
 	for (i = 0; i < slots->nmemslots; i++) {
 		memslot = &slots->memslots[i];
 		base_gfn = memslot->base_gfn;

arch/s390/kvm/kvm-s390.h +1 -1

index 60f09ab..cfa9d17 100644
@@ -72,7 +72,7 @@ static inline void kvm_s390_vcpu_set_mem(struct kvm_vcpu *vcpu)
 	struct kvm_memslots *memslots;
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
-	memslots = rcu_dereference(vcpu->kvm->memslots);
+	memslots = kvm_memslots(vcpu->kvm);
 
 	mem = &memslots->memslots[0];
 

arch/x86/kvm/mmu.c +4 -3

index 71faa04..bb61881 100644
@@ -792,7 +792,7 @@ static int kvm_handle_hva(struct kvm *kvm, unsigned long hva,
 	int retval = 0;
 	struct kvm_memslots *slots;
 
-	slots = rcu_dereference(kvm->memslots);
+	slots = kvm_memslots(kvm);
 
 	for (i = 0; i < slots->nmemslots; i++) {
 		struct kvm_memory_slot *memslot = &slots->memslots[i];

@@ -3017,7 +3017,8 @@ unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm)
 	unsigned int  nr_pages = 0;
 	struct kvm_memslots *slots;
 
-	slots = rcu_dereference(kvm->memslots);
+	slots = kvm_memslots(kvm);
+
 	for (i = 0; i < slots->nmemslots; i++)
 		nr_pages += slots->memslots[i].npages;
 

@@ -3292,7 +3293,7 @@ static int count_rmaps(struct kvm_vcpu *vcpu)
 	int i, j, k, idx;
 
 	idx = srcu_read_lock(&kvm->srcu);
-	slots = rcu_dereference(kvm->memslots);
+	slots = kvm_memslots(kvm);
 	for (i = 0; i < KVM_MEMORY_SLOTS; ++i) {
 		struct kvm_memory_slot *m = &slots->memslots[i];
 		struct kvm_rmap_desc *d;

arch/x86/kvm/vmx.c +1 -1

index 077cac5..725e7b6 100644
@@ -1514,7 +1514,7 @@ static gva_t rmode_tss_base(struct kvm *kvm)
 		struct kvm_memslots *slots;
 		gfn_t base_gfn;
 
-		slots = rcu_dereference(kvm->memslots);
+		slots = kvm_memslots(kvm);
 		base_gfn = kvm->memslots->memslots[0].base_gfn +
 				 kvm->memslots->memslots[0].npages - 3;
 		return base_gfn << PAGE_SHIFT;

arch/x86/kvm/x86.c +2 -2

index 6120e33..4dcd62c 100644
@@ -2408,7 +2408,7 @@ gfn_t unalias_gfn_instantiation(struct kvm *kvm, gfn_t gfn)
 	struct kvm_mem_alias *alias;
 	struct kvm_mem_aliases *aliases;
 
-	aliases = rcu_dereference(kvm->arch.aliases);
+	aliases = kvm_aliases(kvm);
 
 	for (i = 0; i < aliases->naliases; ++i) {
 		alias = &aliases->aliases[i];

@@ -2427,7 +2427,7 @@ gfn_t unalias_gfn(struct kvm *kvm, gfn_t gfn)
 	struct kvm_mem_alias *alias;
 	struct kvm_mem_aliases *aliases;
 
-	aliases = rcu_dereference(kvm->arch.aliases);
+	aliases = kvm_aliases(kvm);
 
 	for (i = 0; i < aliases->naliases; ++i) {
 		alias = &aliases->aliases[i];

arch/x86/kvm/x86.h +7 -0

index 2d10163..e218fed 100644
@@ -65,4 +65,11 @@ static inline int is_paging(struct kvm_vcpu *vcpu)
 	return kvm_read_cr0_bits(vcpu, X86_CR0_PG);
 }
 
+static inline struct kvm_mem_aliases *kvm_aliases(struct kvm *kvm)
+{
+	return rcu_dereference_check(kvm->arch.aliases,
+			srcu_read_lock_held(&kvm->srcu)
+			|| lockdep_is_held(&kvm->slots_lock));
+}
+
 #endif

include/linux/kvm_host.h +7 -0

index a3fd0f9..f735752 100644
@@ -239,6 +239,13 @@ void kvm_exit(void);
 void kvm_get_kvm(struct kvm *kvm);
 void kvm_put_kvm(struct kvm *kvm);
 
+static inline struct kvm_memslots *kvm_memslots(struct kvm *kvm)
+{
+	return rcu_dereference_check(kvm->memslots,
+			srcu_read_lock_held(&kvm->srcu)
+			|| lockdep_is_held(&kvm->slots_lock));
+}
+
 #define HPA_MSB ((sizeof(hpa_t) * 8) - 1)
 #define HPA_ERR_MASK ((hpa_t)1 << HPA_MSB)
 static inline int is_error_hpa(hpa_t hpa) { return hpa >> HPA_MSB; }

virt/kvm/iommu.c +2 -2

index 80fd3ad..37ca71e 100644
@@ -78,7 +78,7 @@ static int kvm_iommu_map_memslots(struct kvm *kvm)
 	int i, r = 0;
 	struct kvm_memslots *slots;
 
-	slots = rcu_dereference(kvm->memslots);
+	slots = kvm_memslots(kvm);
 
 	for (i = 0; i < slots->nmemslots; i++) {
 		r = kvm_iommu_map_pages(kvm, &slots->memslots[i]);

@@ -217,7 +217,7 @@ static int kvm_iommu_unmap_memslots(struct kvm *kvm)
 	int i;
 	struct kvm_memslots *slots;
 
-	slots = rcu_dereference(kvm->memslots);
+	slots = kvm_memslots(kvm);
 
 	for (i = 0; i < slots->nmemslots; i++) {
 		kvm_iommu_put_pages(kvm, slots->memslots[i].base_gfn,

virt/kvm/kvm_main.c +8 -5

index b03f863..58b2147 100644
@@ -829,7 +829,7 @@ EXPORT_SYMBOL_GPL(kvm_is_error_hva);
 struct kvm_memory_slot *gfn_to_memslot_unaliased(struct kvm *kvm, gfn_t gfn)
 {
 	int i;
-	struct kvm_memslots *slots = rcu_dereference(kvm->memslots);
+	struct kvm_memslots *slots = kvm_memslots(kvm);
 
 	for (i = 0; i < slots->nmemslots; ++i) {
 		struct kvm_memory_slot *memslot = &slots->memslots[i];

@@ -851,7 +851,7 @@ struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn)
 int kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn)
 {
 	int i;
-	struct kvm_memslots *slots = rcu_dereference(kvm->memslots);
+	struct kvm_memslots *slots = kvm_memslots(kvm);
 
 	gfn = unalias_gfn_instantiation(kvm, gfn);
 	for (i = 0; i < KVM_MEMORY_SLOTS; ++i) {

@@ -895,7 +895,7 @@ out:
 int memslot_id(struct kvm *kvm, gfn_t gfn)
 {
 	int i;
-	struct kvm_memslots *slots = rcu_dereference(kvm->memslots);
+	struct kvm_memslots *slots = kvm_memslots(kvm);
 	struct kvm_memory_slot *memslot = NULL;
 
 	gfn = unalias_gfn(kvm, gfn);

@@ -1979,7 +1979,9 @@ int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		     int len, const void *val)
 {
 	int i;
-	struct kvm_io_bus *bus = rcu_dereference(kvm->buses[bus_idx]);
+	struct kvm_io_bus *bus;
+
+	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
 	for (i = 0; i < bus->dev_count; i++)
 		if (!kvm_iodevice_write(bus->devs[i], addr, len, val))
 			return 0;

@@ -1991,8 +1993,9 @@ int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		    int len, void *val)
 {
 	int i;
-	struct kvm_io_bus *bus = rcu_dereference(kvm->buses[bus_idx]);
+	struct kvm_io_bus *bus;
 
+	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
 	for (i = 0; i < bus->dev_count; i++)
 		if (!kvm_iodevice_read(bus->devs[i], addr, len, val))
 			return 0;

Quoted 31 lines · Lai Jiangshan

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.

===================================================
[ INFO: suspicious rcu_dereference_check() usage. ]
---------------------------------------------------
arch/x86/kvm/mmu.c:3020 invoked rcu_dereference_check() without protection!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 0
2 locks held by qemu-system-x86/8550:
#0: (&kvm->slots_lock){+.+.+.}, at: [<ffffffffa011a6ac>] kvm_set_memory_region+0x29/0x50 [kvm] #1: (&(&kvm->mmu_lock)->rlock){+.+...}, at: [<ffffffffa012262d>] kvm_arch_commit_memory_region+0xa6/0xe2 [kvm]

stack backtrace:
Pid: 8550, comm: qemu-system-x86 Not tainted 2.6.34-rc4-tip-01028-g939eab1 #27
Call Trace:
  [<ffffffff8106c59e>] lockdep_rcu_dereference+0xaa/0xb3
  [<ffffffffa012f6c1>] kvm_mmu_calculate_mmu_pages+0x44/0x7d [kvm]
  [<ffffffffa012263e>] kvm_arch_commit_memory_region+0xb7/0xe2 [kvm]
  [<ffffffffa011a5d7>] __kvm_set_memory_region+0x636/0x6e2 [kvm]
  [<ffffffffa011a6ba>] kvm_set_memory_region+0x37/0x50 [kvm]
  [<ffffffffa015e956>] vmx_set_tss_addr+0x46/0x5a [kvm_intel]
  [<ffffffffa0126592>] kvm_arch_vm_ioctl+0x17a/0xcf8 [kvm]
  [<ffffffff810a8692>] ? unlock_page+0x27/0x2c
  [<ffffffff810bf879>] ? __do_fault+0x3a9/0x3e1
  [<ffffffffa011b12f>] kvm_vm_ioctl+0x364/0x38d [kvm]

Quoted 9 lines

  [<ffffffff81060cfa>] ? up_read+0x23/0x3d
  [<ffffffff810f3587>] vfs_ioctl+0x32/0xa6
  [<ffffffff810f3b19>] do_vfs_ioctl+0x495/0x4db
  [<ffffffff810e6b2f>] ? fget_light+0xc2/0x241
  [<ffffffff810e416c>] ? do_sys_open+0x104/0x116
  [<ffffffff81382d6d>] ? retint_swapgs+0xe/0x13
  [<ffffffff810f3ba6>] sys_ioctl+0x47/0x6a
  [<ffffffff810021db>] system_call_fastpath+0x16/0x1b

I have queued this one up, thank you, Lai!

The RCU/SRCU API have already changed for proving RCU usage.

I got the following dmesg when PROVE_RCU=y because we used incorrect API. This patch coverts rcu_deference() to srcu_dereference() or family API.