KVM: SEV: Mark nested locking of vcpu->lock
PATCH 1/1 KVM: SEV: Mark nested locking of vcpu->lock
svm_vm_migrate_from() uses sev_lock_vcpus_for_migration() to lock all source and target vcpu->locks. Mark the nested subclasses to avoid false positives from lockdep.
---
Tested by running sev_migrate_tests with lockdep enabled. Before we see a warning from sev_lock_vcpus_for_migration(). After we get no warnings.
--- arch/x86/kvm/svm/sev.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
arch/x86/kvm/svm/sev.c +7 -5
@@ -1591,15 +1591,16 @@ static void sev_unlock_two_vms(struct kvm *dst_kvm, struct kvm *src_kvm)atomic_set_release (&src_sev->migration_in_progress,0 ); } - -static int sev_lock_vcpus_for_migration (struct kvm *kvm) +static int sev_lock_vcpus_for_migration (struct kvm *kvm,unsigned int *subclass) {struct kvm_vcpu *vcpu;unsigned long i, j;kvm_for_each_vcpu (i, vcpu, kvm) { -if (mutex_lock_killable (&vcpu->mutex)) +if (mutex_lock_killable_nested (&vcpu->mutex, *subclass)) goto out_unlock; + + ++(*subclass); }return 0 ;
@@ -1717,6 +1718,7 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)struct kvm *source_kvm;bool charged = false;int ret; +unsigned int vcpu_mutex_subclass =0 ; source_kvm_file =fget (source_fd);if (!file_is_kvm (source_kvm_file)) {
@@ -1745,10 +1747,10 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) charged = true; } - ret =sev_lock_vcpus_for_migration (kvm); + ret =sev_lock_vcpus_for_migration (kvm, &vcpu_mutex_subclass);if (ret) goto out_dst_cgroup; - ret =sev_lock_vcpus_for_migration (source_kvm); + ret =sev_lock_vcpus_for_migration (source_kvm, &vcpu_mutex_subclass);if (ret) goto out_dst_vcpu;
Signature · 1 line
2.35.1.1094.g7c7d902a7c-goog
Sean Christopherson
maintainerQuoted 32 lines · Peter Gonda
svm_vm_migrate_from()usessev_lock_vcpus_for_migration()to lock all source and targetvcpu->locks. Mark the nested subclasses to avoid false positives from lockdep.Fixes b56639318bb2b ("KVM: SEV: Add support for SEV intra host migration")Reported-by John Sperbeck<[email protected]>Suggested-by David Rientjes <[email protected]>Signed-off-by Peter Gonda <[email protected]>Cc Paolo Bonzini <[email protected]>Cc Sean Christopherson <[email protected]>---
Tested by running
sev_migrate_testswith lockdep enabled. Before we see a warning fromsev_lock_vcpus_for_migration(). After we get no warnings.--- arch/x86/kvm/svm/sev.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)arch/x86/kvm/svm/sev.c +1
-2@@ -1591,15 +1591,16 @@ static void sev_unlock_two_vms(struct kvm *dst_kvm, struct kvm *src_kvm)atomic_set_release (&src_sev->migration_in_progress,0 ); } - -static int sev_lock_vcpus_for_migration (struct kvm *kvm) +static int sev_lock_vcpus_for_migration (struct kvm *kvm,unsigned int *subclass)
arch/x86/kvm/svm/sev.c static void sev_unlock_two_vms(struct kvm *dst_kvm, struct kvm *src_kvm) · line 1603
Quoted 10 lines
arch/x86/kvm/svm/sev.c +3
-1{struct kvm_vcpu *vcpu;unsigned long i, j;kvm_for_each_vcpu (i, vcpu, kvm) { -if (mutex_lock_killable (&vcpu->mutex)) +if (mutex_lock_killable_nested (&vcpu->mutex, *subclass)) goto out_unlock; + + ++(*subclass);
This is rather gross, and I'm guessing it adds extra work for the non-lockdep case, assuming the compiler isn't so clever that it can figure out that the result is never used. Not that this is a hot path...
Does each lock actually need a separate subclass? If so, why don't the other paths that lock all vCPUs complain?
If differentiating the two VMs is sufficient, then we can pass in SINGLE_DEPTH_NESTING for the second round of locks. If a per-vCPU subclass is required, we can use the vCPU index and assign evens to one and odds to the other, e.g. this should work and compiles to a nop when LOCKDEP is disabled (compile tested only). It's still gross, but we could pretty it up, e.g. add defines for the 0/1 param.
arch/x86/kvm/svm/sev.c +4 -5
@@ -1591,14 +1591,13 @@ static void sev_unlock_two_vms(struct kvm *dst_kvm, struct kvm *src_kvm)atomic_set_release (&src_sev->migration_in_progress,0 ); } - -static int sev_lock_vcpus_for_migration (struct kvm *kvm) +static int sev_lock_vcpus_for_migration (struct kvm *kvm,int mod) {struct kvm_vcpu *vcpu;unsigned long i, j;kvm_for_each_vcpu (i, vcpu, kvm) { -if (mutex_lock_killable (&vcpu->mutex)) +if (mutex_lock_killable_nested (&vcpu->mutex, i *2 + mod)) goto out_unlock; }
@@ -1745,10 +1744,10 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) charged = true; } - ret =sev_lock_vcpus_for_migration (kvm); + ret =sev_lock_vcpus_for_migration (kvm,0 );if (ret) goto out_dst_cgroup; - ret =sev_lock_vcpus_for_migration (source_kvm); + ret =sev_lock_vcpus_for_migration (source_kvm,1 );if (ret) goto out_dst_vcpu;
Quoted 5 lines
This is rather gross, and I'm guessing it adds extra work for the non-lockdep case, assuming the compiler isn't so clever that it can figure out that the result is never used. Not that this is a hot path...
Quoted 8 lines
Does each lock actually need a separate subclass? If so, why don't the other paths that lock all vCPUs complain?
If differentiating the two VMs is sufficient, then we can pass in
SINGLE_DEPTH_NESTINGfor the second round of locks. If a per-vCPU subclass is required, we can use the vCPU index and assign evens to one and odds to the other, e.g. this should work and compiles to a nop when LOCKDEP is disabled (compile tested only). It's still gross, but we could pretty it up, e.g. add defines for the 0/1 param.
I checked and the perf vCPU subclassing is required. If I just only use a SINGLE_DEPTH_NESTING on the second VM's vCPUs I still see the warning.
This odds and evens approach seems much better. I'll update to use that in the V2 unless there is a better idea.