Skip to contentSkip to conversation

KVM: PPC: Take the kvm->srcu lock in kvmppc_h_logical_ci_load/store()

PATCH In review Thomas Huth 4 participants 6 messages Sep 18, 2015 – Sep 21, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: PPC: Take the kvm->srcu lock in kvmppc_h_logical_ci_load/store()

Access to the kvm->buses (like with the kvm_io_bus_read() and -write() functions) has to be protected via the kvm->srcu lock.
The kvmppc_h_logical_ci_load() and -store() functions are missing this lock so far, so let's add it there, too.
This fixes the problem that the kernel reports "suspicious RCU usage" when lock debugging is enabled.

Fixes 99342cf8044420eebdf9297ca03a14cb6a7085a1
---
 arch/powerpc/kvm/book3s.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/powerpc/kvm/book3s.c +6 -0
index d75bf32..096e5eb 100644
@@ -828,12 +828,15 @@ int kvmppc_h_logical_ci_load(struct kvm_vcpu *vcpu)
 	unsigned long size = kvmppc_get_gpr(vcpu, 4);
 	unsigned long addr = kvmppc_get_gpr(vcpu, 5);
 	u64 buf;
+	int srcu_idx;
 	int ret;
 
 	if (!is_power_of_2(size) || (size > sizeof(buf)))
 		return H_TOO_HARD;
 
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
 	ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, addr, size, &buf);
+	srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx);
 	if (ret != 0)
 		return H_TOO_HARD;
 
@@ -868,6 +871,7 @@ int kvmppc_h_logical_ci_store(struct kvm_vcpu *vcpu)
 	unsigned long addr = kvmppc_get_gpr(vcpu, 5);
 	unsigned long val = kvmppc_get_gpr(vcpu, 6);
 	u64 buf;
+	int srcu_idx;
 	int ret;
 
 	switch (size) {
@@ -891,7 +895,9 @@ int kvmppc_h_logical_ci_store(struct kvm_vcpu *vcpu)
 		return H_TOO_HARD;
 	}
 
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
 	ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, size, &buf);
+	srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx);
 	if (ret != 0)
 		return H_TOO_HARD;
 
Signature · 1 line
1.8.3.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 9 lines · Thomas Huth

Access to the kvm->buses (like with the kvm_io_bus_read() and -write() functions) has to be protected via the kvm->srcu lock.
The kvmppc_h_logical_ci_load() and -store() functions are missing this lock so far, so let's add it there, too.
This fixes the problem that the kernel reports "suspicious RCU usage" when lock debugging is enabled.

Fixes 99342cf8044420eebdf9297ca03a14cb6a7085a1

Nice catch. Looks like I missed this because the places kvm_io_bus_{read,write}() are called on x86 are buried about 5 layers below where the srcu lock is taken :/.

Quoted 42 lines
---
 arch/powerpc/kvm/book3s.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/powerpc/kvm/book3s.c +6 -0
index d75bf32..096e5eb 100644
@@ -828,12 +828,15 @@ int kvmppc_h_logical_ci_load(struct kvm_vcpu *vcpu)
 	unsigned long size = kvmppc_get_gpr(vcpu, 4);
 	unsigned long addr = kvmppc_get_gpr(vcpu, 5);
 	u64 buf;
+	int srcu_idx;
 	int ret;
 
 	if (!is_power_of_2(size) || (size > sizeof(buf)))
 		return H_TOO_HARD;
 
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
 	ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, addr, size, &buf);
+	srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx);
 	if (ret != 0)
 		return H_TOO_HARD;
 
@@ -868,6 +871,7 @@ int kvmppc_h_logical_ci_store(struct kvm_vcpu *vcpu)
 	unsigned long addr = kvmppc_get_gpr(vcpu, 5);
 	unsigned long val = kvmppc_get_gpr(vcpu, 6);
 	u64 buf;
+	int srcu_idx;
 	int ret;
 
 	switch (size) {
@@ -891,7 +895,9 @@ int kvmppc_h_logical_ci_store(struct kvm_vcpu *vcpu)
 		return H_TOO_HARD;
 	}
 
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
 	ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, size, &buf);
+	srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx);
 	if (ret != 0)
 		return H_TOO_HARD;
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
maintainer
#2 lore raw
Quoted 45 lines · David Gibson
Quoted 9 lines · Thomas Huth

Access to the kvm->buses (like with the kvm_io_bus_read() and -write() functions) has to be protected via the kvm->srcu lock. The kvmppc_h_logical_ci_load() and -store() functions are missing this lock so far, so let's add it there, too. This fixes the problem that the kernel reports "suspicious RCU usage" when lock debugging is enabled.

Fixes 99342cf8044420eebdf9297ca03a14cb6a7085a1 Signed-off-by:

Thomas Huth <[email protected]>

Nice catch. Looks like I missed this because the places kvm_io_bus_{read,write}() are called on x86 are buried about 5 layers below where the srcu lock is taken :/.

Quoted 28 lines

--- arch/powerpc/kvm/book3s.c | 6 ++++++ 1 file changed, 6 insertions(+)

(quoted diff) +0 -0

b/arch/powerpc/kvm/book3s.c index d75bf32..096e5eb 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -828,12 +828,15 @@ int kvmppc_h_logical_ci_load(struct kvm_vcpu *vcpu) unsigned long size = kvmppc_get_gpr(vcpu, 4); unsigned
long addr = kvmppc_get_gpr(vcpu, 5); u64 buf; + int srcu_idx; int
ret;

if (!is_power_of_2(size) || (size > sizeof(buf))) return H_TOO_HARD;

(quoted diff) +1 -0
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu); ret =

kvm_io_bus_read(vcpu, KVM_MMIO_BUS, addr, size, &buf); + srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx); if (ret != 0) return H_TOO_HARD;

(quoted diff) +0 -0
@@ -868,6 +871,7 @@ int kvmppc_h_logical_ci_store(struct kvm_vcpu
 *vcpu) unsigned long addr = kvmppc_get_gpr(vcpu, 5); unsigned

long val = kvmppc_get_gpr(vcpu, 6); u64 buf; + int srcu_idx; int
ret;

switch (size) { @@ -891,7 +895,9 @@ int kvmppc_h_logical_ci_store(struct kvm_vcpu *vcpu) return H_TOO_HARD; }

Quoted 6 lines
Quoted 5 lines
(quoted diff) +1 -0
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu); ret =

kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, size, &buf); + srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx); if (ret != 0) return H_TOO_HARD;

Paul,

shall I take this directly into my tree for -rc3?

Paolo

#3 lore raw
Quoted 19 lines · Paolo Bonzini
Quoted 16 lines · David Gibson
Quoted 9 lines · Thomas Huth

Access to the kvm->buses (like with the kvm_io_bus_read() and -write() functions) has to be protected via the kvm->srcu lock. The kvmppc_h_logical_ci_load() and -store() functions are missing this lock so far, so let's add it there, too. This fixes the problem that the kernel reports "suspicious RCU usage" when lock debugging is enabled.

Fixes 99342cf8044420eebdf9297ca03a14cb6a7085a1 Signed-off-by:

Thomas Huth <[email protected]>

Nice catch. Looks like I missed this because the places kvm_io_bus_{read,write}() are called on x86 are buried about 5 layers below where the srcu lock is taken :/.

...

Quoted 5 lines

Paul,

shall I take this directly into my tree for -rc3?

Paolo

I have that and two other fixes in my kvm-ppc-fixes branch on kernel.org. They were in linux-next today. I was going to send you a pull request tomorrow, but if you are about to send stuff off to Linus you could pull now from:

git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc.git kvm-ppc-fixes

The three patches in there are:

Gautham R. Shenoy (1):
KVM: PPC: Book3S HV: Pass the correct trap argument to kvmhv_commence_exit

Paul Mackerras (1):
KVM: PPC: Book3S HV: Fix handling of interrupted VCPUs

Thomas Huth (1):

      KVM: PPC: Book3S: Take the kvm->srcu lock in kvmppc_h_logical_ci_load/store()

The one from Gautham is a 1-liner that has been around for months and got missed, and is obviously correct. The one from me fixes a regression that was introduced in 4.3-rc1 by one of my patches, which causes oopses and soft lockups due to a use-after-free bug.

Thanks,
Paul.

maintainer
#4 lore raw
Quoted 5 lines · Paul Mackerras

I was going to send you a
pull request tomorrow, but if you are about to send stuff off to Linus you could pull now from:

git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc.git kvm-ppc-fixes

Ok, I'll pull from here tomorrow. The pull request will go to Linus around Thursday.

Paolo

Quoted 15 lines

The three patches in there are:

Gautham R. Shenoy (1):
KVM: PPC: Book3S HV: Pass the correct trap argument to kvmhv_commence_exit

Paul Mackerras (1):
KVM: PPC: Book3S HV: Fix handling of interrupted VCPUs

Thomas Huth (1):

      KVM: PPC: Book3S: Take the kvm->srcu lock in kvmppc_h_logical_ci_load/store()

The one from Gautham is a 1-liner that has been around for months and got missed, and is obviously correct. The one from me fixes a regression that was introduced in 4.3-rc1 by one of my patches, which causes oopses and soft lockups due to a use-after-free bug.

#5 lore raw
In reply to David Gibson
Quoted 8 lines · David Gibson
Quoted 7 lines · Thomas Huth

Access to the kvm->buses (like with the kvm_io_bus_read() and -write() functions) has to be protected via the kvm->srcu lock.
The kvmppc_h_logical_ci_load() and -store() functions are missing this lock so far, so let's add it there, too.
This fixes the problem that the kernel reports "suspicious RCU usage" when lock debugging is enabled.

Quoted 6 lines
Quoted 2 lines
Fixes 99342cf8044420eebdf9297ca03a14cb6a7085a1

Nice catch. Looks like I missed this because the places kvm_io_bus_{read,write}() are called on x86 are buried about 5 layers below where the srcu lock is taken :/.

AFAIK the philosophy for taking the srcu lock is completely different between powerpc and x86. On powerpc it is only taken when needed (and released immediately afterwards), while the x86 code tries to hold it the whole time while not being in the guest and not being in userspace. See vcpu_enter_guest() in the x86 code for example, the lock is dropped before entering the guest, and taken again before leaving this function.

Thomas