KVM: arm64: Fix bugs related to mp_state updates
cover KVM: arm64: Fix bugs related to mp_state updates
This series adds fixes that were missing in the patch [1].
The patch [1] added the mp_state_lock to serialize writes to kvm_vcpu_arch::{mp_state, reset_state}, and promoted all accessors of mp_state to {READ,WRITE}_ONCE() as readers do not acquire the mp_state_lock.
Since the patch [1] didn't fix all the relevant code, fix the code that weren't addressed yet.
This series is based on v6.3-rc7 with the series [2] applied.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
Reiji Watanabe (2):
KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
KVM: arm64: Have kvm_psci_vcpu_on() use WRITE_ONCE() to update
mp_state
arch/arm64/kvm/arm.c | 5 ++++- arch/arm64/kvm/psci.c | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)
Signature · 1 line
2.40.0.396.gfff15efe05-goog
PATCH 1/2 KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
kvm_arch_vcpu_ioctl_vcpu_init() doesn't acquire mp_state_lock when setting the mp_state to KVM_MP_STATE_RUNNABLE. Fix the code to acquire the lock.
--- arch/arm64/kvm/arm.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/arm64/kvm/arm.c +4 -1
@@ -1244,8 +1244,11 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,*/ if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features))kvm_arm_vcpu_power_off (vcpu); -else +else { +spin_lock (&vcpu->arch.mp_state_lock);WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); + }return 0 ; }
Signature · 1 line
2.40.0.396.gfff15efe05-goog
Marc Zyngier
maintainerRe: [PATCH v1 1/2] KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
Quoted 17 lines · Reiji Watanabe
kvm_arch_vcpu_ioctl_vcpu_init()doesn't acquiremp_state_lockwhen setting themp_statetoKVM_MP_STATE_RUNNABLE. Fix the code to acquire the lock.Signed-off-by Reiji Watanabe <[email protected]>--- arch/arm64/kvm/arm.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)arch/arm64/kvm/arm.c +0
-0@@ -1244,8 +1244,11 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,*/ if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features))
arch/arm64/kvm/arm.c static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, · line 1254
Quoted 10 lines
arch/arm64/kvm/arm.c +4
-1kvm_arm_vcpu_power_off (vcpu); - else + else { +spin_lock (&vcpu->arch.mp_state_lock);WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); + }return 0 ; }
I'm not entirely convinced that this fixes anything. What does the lock hazard against given that the write is atomic? But maybe a slightly more readable of this would be to expand the critical section this way:
arch/arm64/kvm/arm.c +5 -1
@@ -1246,11 +1246,15 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,/* * Handle the "start in power-off" case. */ +spin_lock (&vcpu->arch.mp_state_lock); +if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features)) -kvm_arm_vcpu_power_off (vcpu); +__kvm_arm_vcpu_power_off (vcpu);else WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); +return 0 ; }
Thoughts?
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v1 1/2] KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
Hi Marc,
Quoted 26 lines · Marc Zyngier
Quoted 24 lines · Reiji Watanabe
kvm_arch_vcpu_ioctl_vcpu_init()doesn't acquiremp_state_lockwhen setting themp_statetoKVM_MP_STATE_RUNNABLE. Fix the code to acquire the lock.Signed-off-by Reiji Watanabe <[email protected]>--- arch/arm64/kvm/arm.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)arch/arm64/kvm/arm.c +4
-1@@ -1244,8 +1244,11 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,*/ if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features))kvm_arm_vcpu_power_off (vcpu); -else +else { +spin_lock (&vcpu->arch.mp_state_lock);WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); + }
Quoted 5 lines
arch/arm64/kvm/arm.c static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, · line 1259
Quoted 2 lines
arch/arm64/kvm/arm.c +0
-0return 0 ; }I'm not entirely convinced that this fixes anything. What does the lock hazard against given that the write is atomic? But maybe a
It appears that kvm_psci_vcpu_on() expects the vCPU's mp_state to not be changed by holding the lock. Although I don't think this code practically causes any real issues now, I am a little concerned about leaving one instance that updates mpstate without acquiring the lock, in terms of future maintenance, as holding the lock won't prevent mp_state from being updated.
What do you think ?
Quoted 24 lines
slightly more readable of this would be to expand the critical section this way:
arch/arm64/kvm/arm.c +5
-1@@ -1246,11 +1246,15 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,/* * Handle the "start in power-off" case. */ +spin_lock (&vcpu->arch.mp_state_lock); +if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features)) -kvm_arm_vcpu_power_off (vcpu); +__kvm_arm_vcpu_power_off (vcpu);else WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); +return 0 ; }
Quoted 1 line
Thoughts?
Yes, it looks better!
Thank you,
Reiji
Marc Zyngier
maintainerRe: [PATCH v1 1/2] KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
Quoted 36 lines · Reiji Watanabe
Hi Marc,
Quoted 26 lines · Marc Zyngier
Quoted 24 lines · Reiji Watanabe
kvm_arch_vcpu_ioctl_vcpu_init()doesn't acquiremp_state_lockwhen setting themp_statetoKVM_MP_STATE_RUNNABLE. Fix the code to acquire the lock.Signed-off-by Reiji Watanabe <[email protected]>--- arch/arm64/kvm/arm.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)arch/arm64/kvm/arm.c +4
-1@@ -1244,8 +1244,11 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,*/ if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features))kvm_arm_vcpu_power_off (vcpu); -else +else { +spin_lock (&vcpu->arch.mp_state_lock);WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); + }Quoted 5 lines
arch/arm64/kvm/arm.c static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, · line 1259
Quoted 2 lines
arch/arm64/kvm/arm.c +0
-0return 0 ; }I'm not entirely convinced that this fixes anything. What does the lock hazard against given that the write is atomic? But maybe a
Quoted 8 lines
It appears that
kvm_psci_vcpu_on()expects the vCPU'smp_stateto not be changed by holding the lock. Although I don't think this code practically causes any real issues now, I am a little concerned about leaving one instance that updates mpstate without acquiring the lock, in terms of future maintenance, as holding the lock won't preventmp_statefrom being updated.What do you think ?
Right, fair enough. It is probably better to take the lock and not have to think of this sort of things... I'm becoming more lazy by the minute!
Quoted 20 lines
Quoted 20 lines
slightly more readable of this would be to expand the critical section this way:
arch/arm64/kvm/arm.c +3
-1@@ -1246,11 +1246,15 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,/* * Handle the "start in power-off" case. */ +spin_lock (&vcpu->arch.mp_state_lock); +if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features)) -kvm_arm_vcpu_power_off (vcpu); +__kvm_arm_vcpu_power_off (vcpu);else WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE );
Quoted 8 lines
Quoted 6 lines
arch/arm64/kvm/arm.c +2
-0+spin_unlock (&vcpu->arch.mp_state_lock); +return 0 ; }Thoughts?
Yes, it looks better!
Cool. I've applied this change to your patch, applied the series to the lock inversion branch, and remerged the branch in -next.
We're getting there! ;-)
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v1 1/2] KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
Quoted 77 lines · Marc Zyngier
Quoted 36 lines · Reiji Watanabe
Hi Marc,
Quoted 26 lines · Marc Zyngier
Quoted 24 lines · Reiji Watanabe
kvm_arch_vcpu_ioctl_vcpu_init()doesn't acquiremp_state_lockwhen setting themp_statetoKVM_MP_STATE_RUNNABLE. Fix the code to acquire the lock.Signed-off-by Reiji Watanabe <[email protected]>--- arch/arm64/kvm/arm.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)arch/arm64/kvm/arm.c +4
-1@@ -1244,8 +1244,11 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,*/ if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features))kvm_arm_vcpu_power_off (vcpu); -else +else { +spin_lock (&vcpu->arch.mp_state_lock);WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); + }Quoted 5 lines
arch/arm64/kvm/arm.c static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, · line 1259
Quoted 2 lines
arch/arm64/kvm/arm.c +0
-0return 0 ; }I'm not entirely convinced that this fixes anything. What does the lock hazard against given that the write is atomic? But maybe a
Quoted 8 lines
It appears that
kvm_psci_vcpu_on()expects the vCPU'smp_stateto not be changed by holding the lock. Although I don't think this code practically causes any real issues now, I am a little concerned about leaving one instance that updates mpstate without acquiring the lock, in terms of future maintenance, as holding the lock won't preventmp_statefrom being updated.What do you think ?
Right, fair enough. It is probably better to take the lock and not have to think of this sort of things... I'm becoming more lazy by the minute!
Quoted 25 lines
Quoted 25 lines
slightly more readable of this would be to expand the critical section this way:
arch/arm64/kvm/arm.c +5
-1@@ -1246,11 +1246,15 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,/* * Handle the "start in power-off" case. */ +spin_lock (&vcpu->arch.mp_state_lock); +if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features)) -kvm_arm_vcpu_power_off (vcpu); +__kvm_arm_vcpu_power_off (vcpu);else WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE ); +spin_unlock (&vcpu->arch.mp_state_lock); +return 0 ; }
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Thoughts?
Yes, it looks better!
Cool. I've applied this change to your patch, applied the series to the lock inversion branch, and remerged the branch in -next.
We're getting there! ;-)
Thank you, Marc!
Reiji
PATCH 2/2 KVM: arm64: Have kvm_psci_vcpu_on() use WRITE_ONCE() to update mp_state
In reply to Reiji WatanabeAll accessors of kvm_vcpu_arch::mp_state should be {READ,WRITE}_ONCE(), since readers of the mp_state don't acquire the mp_state_lock. Nonetheless, kvm_psci_vcpu_on() updates the mp_state without using WRITE_ONCE(). So, fix the code to update the mp_state using WRITE_ONCE.
--- arch/arm64/kvm/psci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/psci.c +1 -1
@@ -110,7 +110,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)*/ smp_wmb (); - vcpu->arch.mp_state.mp_state =KVM_MP_STATE_RUNNABLE ; +WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE );kvm_vcpu_wake_up (vcpu); out_unlock:
Signature · 1 line
2.40.0.396.gfff15efe05-goog
Quoted 8 lines · Reiji Watanabe
This series adds fixes that were missing in the patch [1].
The patch [1] added the
mp_state_lockto serialize writes to kvm_vcpu_arch::{mp_state,reset_state}, and promoted all accessors ofmp_stateto {READ,WRITE}_ONCE() as readers do not acquire themp_state_lock.[...]
Applied to next, thanks!
[1/2] KVM: arm64: Acquire mp_state_lock in kvm_arch_vcpu_ioctl_vcpu_init()
commit: 4ff910be01c0ca28c2ea8b354dd47a3a17524489
[2/2] KVM: arm64: Have kvm_psci_vcpu_on() use WRITE_ONCE() to update mp_state
commit: a189884bdc9238aeba941c50f02e25eb584fafedCheers,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.