Skip to contentSkip to conversation

KVM: Protect vCPU's PID with a rwlock

PATCH Applied Sean Christopherson 3 participants 12 messages Aug 2, 2024 – Oct 31, 2024

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

cover KVM: Protect vCPU's PID with a rwlock

Protect vcpu->pid with a rwlock instead of RCU, so that running a vCPU with a different task doesn't require a full RCU synchronization, which can introduce a non-trivial amount of jitter, especially on large systems.

I've had this mini-series sitting around for ~2 years, pretty much as-is. I could have sworn past me thought there was a flaw in using a rwlock, and so I never posted it, but for the life of me I can't think of any issues.

Extra eyeballs would be much appreciated.

Sean Christopherson (2):
KVM: Return '0' directly when there's no task to yield to
KVM: Protect vCPU's "last run PID" with rwlock, not RCU

 arch/arm64/include/asm/kvm_host.h |  2 +-
 include/linux/kvm_host.h          |  3 ++-
 virt/kvm/kvm_main.c               | 36 +++++++++++++++++--------------
 3 files changed, 23 insertions(+), 18 deletions(-)

base-commit: 332d2c1d713e232e163386c35a3ba0c1b90df83f

Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
author reviewer
#1 lore raw

PATCH 1/2 KVM: Return '0' directly when there's no task to yield to

Do "return 0" instead of initializing and returning a local variable in kvm_vcpu_yield_to(), e.g. so that it's more obvious what the function returns if there is no task.

No functional change intended.

---
 virt/kvm/kvm_main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
virt/kvm/kvm_main.c +2 -2
index d0788d0a72cc..91048a7ad3be 100644
@@ -3932,7 +3932,7 @@ int kvm_vcpu_yield_to(struct kvm_vcpu *target)
 {
 	struct pid *pid;
 	struct task_struct *task = NULL;
-	int ret = 0;
+	int ret;
 
 	rcu_read_lock();
 	pid = rcu_dereference(target->pid);
@@ -3940,7 +3940,7 @@ int kvm_vcpu_yield_to(struct kvm_vcpu *target)
 		task = get_pid_task(pid, PIDTYPE_PID);
 	rcu_read_unlock();
 	if (!task)
-		return ret;
+		return 0;
 	ret = yield_to(task, 1);
 	put_task_struct(task);
 
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
author reviewer
#2 lore raw

PATCH 2/2 KVM: Protect vCPU's "last run PID" with rwlock, not RCU

In reply to Sean Christopherson

To avoid jitter on KVM_RUN due to synchronize_rcu(), use a rwlock instead of RCU to protect vcpu->pid, a.k.a. the pid of the task last used to a vCPU. When userspace is doing M:N scheduling of tasks to vCPUs, e.g. to run SEV migration helper vCPUs during post-copy, the synchronize_rcu() needed to change the PID associated with the vCPU can stall for hundreds of milliseconds, which is problematic for latency sensitive post-copy operations.

In the directed yield path, do not acquire the lock if it's contended, i.e. if the associated PID is changing, as that means the vCPU's task is already running.

---
 arch/arm64/include/asm/kvm_host.h |  2 +-
 include/linux/kvm_host.h          |  3 ++-
 virt/kvm/kvm_main.c               | 32 +++++++++++++++++--------------
 3 files changed, 21 insertions(+), 16 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
index a33f5996ca9f..7199cb014806 100644
@@ -1115,7 +1115,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 void kvm_arm_halt_guest(struct kvm *kvm);
 void kvm_arm_resume_guest(struct kvm *kvm);
 
-#define vcpu_has_run_once(vcpu)	!!rcu_access_pointer((vcpu)->pid)
+#define vcpu_has_run_once(vcpu)	(!!READ_ONCE((vcpu)->pid))
 
 #ifndef __KVM_NVHE_HYPERVISOR__
 #define kvm_call_hyp_nvhe(f, ...)						\
include/linux/kvm_host.h +2 -1
index 689e8be873a7..d6f4e8b2b44c 100644
@@ -342,7 +342,8 @@ struct kvm_vcpu {
 #ifndef __KVM_HAVE_ARCH_WQP
 	struct rcuwait wait;
 #endif
-	struct pid __rcu *pid;
+	struct pid *pid;
+	rwlock_t pid_lock;
 	int sigset_active;
 	sigset_t sigset;
 	unsigned int halt_poll_ns;
virt/kvm/kvm_main.c +18 -14
index 91048a7ad3be..fabffd85fa34 100644
@@ -486,6 +486,7 @@ static void kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
 	vcpu->kvm = kvm;
 	vcpu->vcpu_id = id;
 	vcpu->pid = NULL;
+	rwlock_init(&vcpu->pid_lock);
 #ifndef __KVM_HAVE_ARCH_WQP
 	rcuwait_init(&vcpu->wait);
 #endif
@@ -513,7 +514,7 @@ static void kvm_vcpu_destroy(struct kvm_vcpu *vcpu)
 	 * the vcpu->pid pointer, and at destruction time all file descriptors
 	 * are already gone.
 	 */
-	put_pid(rcu_dereference_protected(vcpu->pid, 1));
+	put_pid(vcpu->pid);
 
 	free_page((unsigned long)vcpu->run);
 	kmem_cache_free(kvm_vcpu_cache, vcpu);
@@ -3930,15 +3931,17 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_kick);
 
 int kvm_vcpu_yield_to(struct kvm_vcpu *target)
 {
-	struct pid *pid;
 	struct task_struct *task = NULL;
 	int ret;
 
-	rcu_read_lock();
-	pid = rcu_dereference(target->pid);
-	if (pid)
-		task = get_pid_task(pid, PIDTYPE_PID);
-	rcu_read_unlock();
+	if (!read_trylock(&target->pid_lock))
+		return 0;
+
+	if (target->pid)
+		task = get_pid_task(target->pid, PIDTYPE_PID);
+
+	read_unlock(&target->pid_lock);
+
 	if (!task)
 		return 0;
 	ret = yield_to(task, 1);
@@ -4178,9 +4181,9 @@ static int vcpu_get_pid(void *data, u64 *val)
 {
 	struct kvm_vcpu *vcpu = data;
 
-	rcu_read_lock();
-	*val = pid_nr(rcu_dereference(vcpu->pid));
-	rcu_read_unlock();
+	read_lock(&vcpu->pid_lock);
+	*val = pid_nr(vcpu->pid);
+	read_unlock(&vcpu->pid_lock);
 	return 0;
 }
 
@@ -4466,7 +4469,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
 		r = -EINVAL;
 		if (arg)
 			goto out;
-		oldpid = rcu_access_pointer(vcpu->pid);
+		oldpid = vcpu->pid;
 		if (unlikely(oldpid != task_pid(current))) {
 			/* The thread running this VCPU changed. */
 			struct pid *newpid;
@@ -4476,9 +4479,10 @@ static long kvm_vcpu_ioctl(struct file *filp,
 				break;
 
 			newpid = get_task_pid(current, PIDTYPE_PID);
-			rcu_assign_pointer(vcpu->pid, newpid);
-			if (oldpid)
-				synchronize_rcu();
+			write_lock(&vcpu->pid_lock);
+			vcpu->pid = newpid;
+			write_unlock(&vcpu->pid_lock);
+
 			put_pid(oldpid);
 		}
 		vcpu->wants_to_run = !READ_ONCE(vcpu->run->immediate_exit__unsafe);
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
#3 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

Quoted 107 lines · Sean Christopherson

To avoid jitter on KVM_RUN due to synchronize_rcu(), use a rwlock instead of RCU to protect vcpu->pid, a.k.a. the pid of the task last used to a vCPU. When userspace is doing M:N scheduling of tasks to vCPUs, e.g. to run SEV migration helper vCPUs during post-copy, the synchronize_rcu() needed to change the PID associated with the vCPU can stall for hundreds of milliseconds, which is problematic for latency sensitive post-copy operations.

In the directed yield path, do not acquire the lock if it's contended, i.e. if the associated PID is changing, as that means the vCPU's task is already running.

---
 arch/arm64/include/asm/kvm_host.h |  2 +-
 include/linux/kvm_host.h          |  3 ++-
 virt/kvm/kvm_main.c               | 32 +++++++++++++++++--------------
 3 files changed, 21 insertions(+), 16 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
index a33f5996ca9f..7199cb014806 100644
@@ -1115,7 +1115,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 void kvm_arm_halt_guest(struct kvm *kvm);
 void kvm_arm_resume_guest(struct kvm *kvm);
 
-#define vcpu_has_run_once(vcpu)        !!rcu_access_pointer((vcpu)->pid)
+#define vcpu_has_run_once(vcpu)        (!!READ_ONCE((vcpu)->pid))
 
 #ifndef __KVM_NVHE_HYPERVISOR__
 #define kvm_call_hyp_nvhe(f, ...)                                              \
include/linux/kvm_host.h +2 -1
index 689e8be873a7..d6f4e8b2b44c 100644
@@ -342,7 +342,8 @@ struct kvm_vcpu {
 #ifndef __KVM_HAVE_ARCH_WQP
        struct rcuwait wait;
 #endif
-       struct pid __rcu *pid;
+       struct pid *pid;
+       rwlock_t pid_lock;
        int sigset_active;
        sigset_t sigset;
        unsigned int halt_poll_ns;
virt/kvm/kvm_main.c +13 -10
index 91048a7ad3be..fabffd85fa34 100644
@@ -486,6 +486,7 @@ static void kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
        vcpu->kvm = kvm;
        vcpu->vcpu_id = id;
        vcpu->pid = NULL;
+       rwlock_init(&vcpu->pid_lock);
 #ifndef __KVM_HAVE_ARCH_WQP
        rcuwait_init(&vcpu->wait);
 #endif
@@ -513,7 +514,7 @@ static void kvm_vcpu_destroy(struct kvm_vcpu *vcpu)
         * the vcpu->pid pointer, and at destruction time all file descriptors
         * are already gone.
         */
-       put_pid(rcu_dereference_protected(vcpu->pid, 1));
+       put_pid(vcpu->pid);
 
        free_page((unsigned long)vcpu->run);
        kmem_cache_free(kvm_vcpu_cache, vcpu);
@@ -3930,15 +3931,17 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_kick);
 
 int kvm_vcpu_yield_to(struct kvm_vcpu *target)
 {
-       struct pid *pid;
        struct task_struct *task = NULL;
        int ret;
 
-       rcu_read_lock();
-       pid = rcu_dereference(target->pid);
-       if (pid)
-               task = get_pid_task(pid, PIDTYPE_PID);
-       rcu_read_unlock();
+       if (!read_trylock(&target->pid_lock))
+               return 0;
+
+       if (target->pid)
+               task = get_pid_task(target->pid, PIDTYPE_PID);
+
+       read_unlock(&target->pid_lock);
+
        if (!task)
                return 0;
        ret = yield_to(task, 1);
@@ -4178,9 +4181,9 @@ static int vcpu_get_pid(void *data, u64 *val)
 {
        struct kvm_vcpu *vcpu = data;
 
-       rcu_read_lock();
-       *val = pid_nr(rcu_dereference(vcpu->pid));
-       rcu_read_unlock();
+       read_lock(&vcpu->pid_lock);
+       *val = pid_nr(vcpu->pid);
+       read_unlock(&vcpu->pid_lock);
        return 0;
 }
 

virt/kvm/kvm_main.c static long kvm_vcpu_ioctl(struct file *filp, · line 4472

Quoted 5 lines
virt/kvm/kvm_main.c +1 -1
@@ -4466,7 +4469,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
                r = -EINVAL;
                if (arg)
                        goto out;
-               oldpid = rcu_access_pointer(vcpu->pid);
+               oldpid = vcpu->pid;

Overall this patch looks correct, but this spot took me a moment, and I want to confirm. This skips the reader lock since writing only happens just below, under the vcpu lock, and we've already taken that lock?

Quoted 20 lines
virt/kvm/kvm_main.c +4 -4
                if (unlikely(oldpid != task_pid(current))) {
                        /* The thread running this VCPU changed. */
                        struct pid *newpid;
@@ -4476,9 +4479,10 @@ static long kvm_vcpu_ioctl(struct file *filp,
                                break;
 
                        newpid = get_task_pid(current, PIDTYPE_PID);
-                       rcu_assign_pointer(vcpu->pid, newpid);
-                       if (oldpid)
-                               synchronize_rcu();
+                       write_lock(&vcpu->pid_lock);
+                       vcpu->pid = newpid;
+                       write_unlock(&vcpu->pid_lock);
+
                        put_pid(oldpid);
                }
                vcpu->wants_to_run = !READ_ONCE(vcpu->run->immediate_exit__unsafe);
--

2.46.0.rc2.264.g509ed76dc8-goog

author reviewer
#4 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

Quoted 14 lines · Steve Rutherford
Quoted 13 lines · Sean Christopherson
virt/kvm/kvm_main.c +3 -3
@@ -4178,9 +4181,9 @@ static int vcpu_get_pid(void *data, u64 *val)
 {
        struct kvm_vcpu *vcpu = data;
 
-       rcu_read_lock();
-       *val = pid_nr(rcu_dereference(vcpu->pid));
-       rcu_read_unlock();
+       read_lock(&vcpu->pid_lock);
+       *val = pid_nr(vcpu->pid);
+       read_unlock(&vcpu->pid_lock);
        return 0;
 }
 
Quoted 11 lines

virt/kvm/kvm_main.c static long kvm_vcpu_ioctl(struct file *filp, · line 4472

Quoted 6 lines
virt/kvm/kvm_main.c +1 -1
@@ -4466,7 +4469,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
                r = -EINVAL;
                if (arg)
                        goto out;
-               oldpid = rcu_access_pointer(vcpu->pid);
+               oldpid = vcpu->pid;

Overall this patch looks correct, but this spot took me a moment, and I want to confirm. This skips the reader lock since writing only happens just below, under the vcpu lock, and we've already taken that lock?

Yep, exactly.

Reviewed-by on v1 2/2
#5 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

In reply to Sean Christopherson
Quoted 130 lines · Sean Christopherson

To avoid jitter on KVM_RUN due to synchronize_rcu(), use a rwlock instead of RCU to protect vcpu->pid, a.k.a. the pid of the task last used to a vCPU. When userspace is doing M:N scheduling of tasks to vCPUs, e.g. to run SEV migration helper vCPUs during post-copy, the synchronize_rcu() needed to change the PID associated with the vCPU can stall for hundreds of milliseconds, which is problematic for latency sensitive post-copy operations.

In the directed yield path, do not acquire the lock if it's contended, i.e. if the associated PID is changing, as that means the vCPU's task is already running.

---
 arch/arm64/include/asm/kvm_host.h |  2 +-
 include/linux/kvm_host.h          |  3 ++-
 virt/kvm/kvm_main.c               | 32 +++++++++++++++++--------------
 3 files changed, 21 insertions(+), 16 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
index a33f5996ca9f..7199cb014806 100644
@@ -1115,7 +1115,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 void kvm_arm_halt_guest(struct kvm *kvm);
 void kvm_arm_resume_guest(struct kvm *kvm);
 
-#define vcpu_has_run_once(vcpu)        !!rcu_access_pointer((vcpu)->pid)
+#define vcpu_has_run_once(vcpu)        (!!READ_ONCE((vcpu)->pid))
 
 #ifndef __KVM_NVHE_HYPERVISOR__
 #define kvm_call_hyp_nvhe(f, ...)                                              \
include/linux/kvm_host.h +2 -1
index 689e8be873a7..d6f4e8b2b44c 100644
@@ -342,7 +342,8 @@ struct kvm_vcpu {
 #ifndef __KVM_HAVE_ARCH_WQP
        struct rcuwait wait;
 #endif
-       struct pid __rcu *pid;
+       struct pid *pid;
+       rwlock_t pid_lock;
        int sigset_active;
        sigset_t sigset;
        unsigned int halt_poll_ns;
virt/kvm/kvm_main.c +18 -15
index 91048a7ad3be..fabffd85fa34 100644
@@ -486,6 +486,7 @@ static void kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
        vcpu->kvm = kvm;
        vcpu->vcpu_id = id;
        vcpu->pid = NULL;
+       rwlock_init(&vcpu->pid_lock);
 #ifndef __KVM_HAVE_ARCH_WQP
        rcuwait_init(&vcpu->wait);
 #endif
@@ -513,7 +514,7 @@ static void kvm_vcpu_destroy(struct kvm_vcpu *vcpu)
         * the vcpu->pid pointer, and at destruction time all file descriptors
         * are already gone.
         */
-       put_pid(rcu_dereference_protected(vcpu->pid, 1));
+       put_pid(vcpu->pid);
 
        free_page((unsigned long)vcpu->run);
        kmem_cache_free(kvm_vcpu_cache, vcpu);
@@ -3930,15 +3931,17 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_kick);
 
 int kvm_vcpu_yield_to(struct kvm_vcpu *target)
 {
-       struct pid *pid;
        struct task_struct *task = NULL;
        int ret;
 
-       rcu_read_lock();
-       pid = rcu_dereference(target->pid);
-       if (pid)
-               task = get_pid_task(pid, PIDTYPE_PID);
-       rcu_read_unlock();
+       if (!read_trylock(&target->pid_lock))
+               return 0;
+
+       if (target->pid)
+               task = get_pid_task(target->pid, PIDTYPE_PID);
+
+       read_unlock(&target->pid_lock);
+
        if (!task)
                return 0;
        ret = yield_to(task, 1);
@@ -4178,9 +4181,9 @@ static int vcpu_get_pid(void *data, u64 *val)
 {
        struct kvm_vcpu *vcpu = data;
 
-       rcu_read_lock();
-       *val = pid_nr(rcu_dereference(vcpu->pid));
-       rcu_read_unlock();
+       read_lock(&vcpu->pid_lock);
+       *val = pid_nr(vcpu->pid);
+       read_unlock(&vcpu->pid_lock);
        return 0;
 }
 
@@ -4466,7 +4469,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
                r = -EINVAL;
                if (arg)
                        goto out;
-               oldpid = rcu_access_pointer(vcpu->pid);
+               oldpid = vcpu->pid;
                if (unlikely(oldpid != task_pid(current))) {
                        /* The thread running this VCPU changed. */
                        struct pid *newpid;
@@ -4476,9 +4479,10 @@ static long kvm_vcpu_ioctl(struct file *filp,
                                break;
 
                        newpid = get_task_pid(current, PIDTYPE_PID);
-                       rcu_assign_pointer(vcpu->pid, newpid);
-                       if (oldpid)
-                               synchronize_rcu();
+                       write_lock(&vcpu->pid_lock);
+                       vcpu->pid = newpid;
+                       write_unlock(&vcpu->pid_lock);
+
                        put_pid(oldpid);
                }
                vcpu->wants_to_run = !READ_ONCE(vcpu->run->immediate_exit__unsafe);
--
Quoted 2 lines

2.46.0.rc2.264.g509ed76dc8-goog

#6 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

In reply to Sean Christopherson
Quoted 38 lines · Sean Christopherson

To avoid jitter on KVM_RUN due to synchronize_rcu(), use a rwlock instead of RCU to protect vcpu->pid, a.k.a. the pid of the task last used to a vCPU. When userspace is doing M:N scheduling of tasks to vCPUs, e.g. to run SEV migration helper vCPUs during post-copy, the synchronize_rcu() needed to change the PID associated with the vCPU can stall for hundreds of milliseconds, which is problematic for latency sensitive post-copy operations.

In the directed yield path, do not acquire the lock if it's contended, i.e. if the associated PID is changing, as that means the vCPU's task is already running.

---
 arch/arm64/include/asm/kvm_host.h |  2 +-
 include/linux/kvm_host.h          |  3 ++-
 virt/kvm/kvm_main.c               | 32 +++++++++++++++++--------------
 3 files changed, 21 insertions(+), 16 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
index a33f5996ca9f..7199cb014806 100644
@@ -1115,7 +1115,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 void kvm_arm_halt_guest(struct kvm *kvm);
 void kvm_arm_resume_guest(struct kvm *kvm);
 
-#define vcpu_has_run_once(vcpu)	!!rcu_access_pointer((vcpu)->pid)
+#define vcpu_has_run_once(vcpu)	(!!READ_ONCE((vcpu)->pid))
 
 #ifndef __KVM_NVHE_HYPERVISOR__
 #define kvm_call_hyp_nvhe(f, ...)						\
include/linux/kvm_host.h +0 -0
index 689e8be873a7..d6f4e8b2b44c 100644

include/linux/kvm_host.h struct kvm_vcpu { · line 349

Quoted 9 lines
include/linux/kvm_host.h +2 -1
@@ -342,7 +342,8 @@ struct kvm_vcpu {
 #ifndef __KVM_HAVE_ARCH_WQP
 	struct rcuwait wait;
 #endif
-	struct pid __rcu *pid;
+	struct pid *pid;
+	rwlock_t pid_lock;
 	int sigset_active;
 	sigset_t sigset;
 	unsigned int halt_poll_ns;

Adding yet another lock is never exciting, but this looks fine. Can you nest this lock inside of the vcpu->mutex acquisition in kvm_vm_ioctl_create_vcpu() so lockdep gets the picture?

include/linux/kvm_host.h static long kvm_vcpu_ioctl(struct file *filp, · line 4472

Quoted 6 lines
include/linux/kvm_host.h +1 -1
@@ -4466,7 +4469,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
 		r = -EINVAL;
 		if (arg)
 			goto out;
-		oldpid = rcu_access_pointer(vcpu->pid);
+		oldpid = vcpu->pid;

It'd be good to add a comment here about how this is guarded by the vcpu->mutex, as Steve points out.

Signature · 2 lines
Thanks,
Oliver
author reviewer
#7 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

Quoted 22 lines · Oliver Upton
Quoted 21 lines · Sean Christopherson
arch/arm64/include/asm/kvm_host.h +1 -1
index a33f5996ca9f..7199cb014806 100644
@@ -1115,7 +1115,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 void kvm_arm_halt_guest(struct kvm *kvm);
 void kvm_arm_resume_guest(struct kvm *kvm);
 
-#define vcpu_has_run_once(vcpu)	!!rcu_access_pointer((vcpu)->pid)
+#define vcpu_has_run_once(vcpu)	(!!READ_ONCE((vcpu)->pid))
 
 #ifndef __KVM_NVHE_HYPERVISOR__
 #define kvm_call_hyp_nvhe(f, ...)						\
include/linux/kvm_host.h +0 -0
index 689e8be873a7..d6f4e8b2b44c 100644
@@ -342,7 +342,8 @@ struct kvm_vcpu {
 #ifndef __KVM_HAVE_ARCH_WQP
 	struct rcuwait wait;
 #endif
Quoted 8 lines

include/linux/kvm_host.h struct kvm_vcpu { · line 349

Quoted 6 lines
include/linux/kvm_host.h +2 -1
-	struct pid __rcu *pid;
+	struct pid *pid;
+	rwlock_t pid_lock;
 	int sigset_active;
 	sigset_t sigset;
 	unsigned int halt_poll_ns;

Adding yet another lock is never exciting, but this looks fine.

Heh, my feelings too. Maybe that's why I didn't post this for two years.

Quoted 2 lines

Can you nest this lock inside of the vcpu->mutex acquisition in kvm_vm_ioctl_create_vcpu() so lockdep gets the picture?

I don't think that's necessary. Commit 42a90008f890 ("KVM: Ensure lockdep knows about kvm->lock vs. vcpu->mutex ordering rule") added the lock+unlock in kvm_vm_ioctl_create_vcpu() purely because actually taking vcpu->mutex inside kvm->lock is rare, i.e. lockdep would be unable to detect issues except for very specific VM types hitting very specific flows.

But for this lock, every arch is guaranteed to take the lock on the first KVM_RUN, as "oldpid" is '0' and guaranteed to mismatch task_pid(current). So I don't think we go out of our way to alert lockdep.

Quoted 9 lines

include/linux/kvm_host.h static long kvm_vcpu_ioctl(struct file *filp, · line 4472

Quoted 6 lines
include/linux/kvm_host.h +1 -1
@@ -4466,7 +4469,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
 		r = -EINVAL;
 		if (arg)
 			goto out;
-		oldpid = rcu_access_pointer(vcpu->pid);
+		oldpid = vcpu->pid;

It'd be good to add a comment here about how this is guarded by the vcpu->mutex, as Steve points out.

Roger that.

#8 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

Quoted 8 lines · Sean Christopherson
Quoted 2 lines

Can you nest this lock inside of the vcpu->mutex acquisition in kvm_vm_ioctl_create_vcpu() so lockdep gets the picture?

I don't think that's necessary. Commit 42a90008f890 ("KVM: Ensure lockdep knows about kvm->lock vs. vcpu->mutex ordering rule") added the lock+unlock in kvm_vm_ioctl_create_vcpu() purely because actually taking vcpu->mutex inside kvm->lock is rare, i.e. lockdep would be unable to detect issues except for very specific VM types hitting very specific flows.

I don't think the perceived rarity matters at all w/ this. Beyond the lockdep benefits, it is a self-documenting way to describe lock ordering. Dunno about you, but I haven't kept up with locking.rst at all :)

Having said that, an inversion would still be *very* obvious, as it would be trying to grab a mutex while holding a spinlock...

Signature · 2 lines
Thanks,
Oliver
author reviewer
#9 lore raw

Re: [PATCH 2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU

Quoted 11 lines · Oliver Upton
Quoted 8 lines · Sean Christopherson
Quoted 2 lines

Can you nest this lock inside of the vcpu->mutex acquisition in kvm_vm_ioctl_create_vcpu() so lockdep gets the picture?

I don't think that's necessary. Commit 42a90008f890 ("KVM: Ensure lockdep knows about kvm->lock vs. vcpu->mutex ordering rule") added the lock+unlock in kvm_vm_ioctl_create_vcpu() purely because actually taking vcpu->mutex inside kvm->lock is rare, i.e. lockdep would be unable to detect issues except for very specific VM types hitting very specific flows.

I don't think the perceived rarity matters at all w/ this.

Rarity definitely matters. If KVM was splattered with code that takes vcpu->mutex inside kvm->lock, then the mess that led to above commit likely would never had happened.

Quoted 2 lines

Beyond the lockdep benefits, it is a self-documenting way to describe lock ordering.

Lock acquisition alone won't suffice, many of the more unique locks in KVM need comments/documentation, e.g. to explain additional rules, assumptions that make things work, etc. We could obviously add comments for everything, but I don't see how that's clearly better than actual documentation. E.g. pid_lock is taken for read across vCPUs. Acquiring vcpu->pid_lock inside vcpu->mutex doesn't capture that at all.

It's also simply not realistic to enumerate every possible combination. Many of the combinations will likely never happen in practice, especially for spinlocks since their usage is quite targeted. Trying to document the "preferred" ordering between the various spinlocks would be an exercise in futility as so many would be 100% arbitrary due to lack of a use case.

KVM's mutexes are more interesting because they tend to be coarser, and thus are more prone to nesting, so maybe we could have lockdep-enforced documentation for those? But if we want to do that, I think we should have a dedicated helper (and possible arch hooks), not an ad hoc pile of locks in vCPU creation.

And we should have that discussion over here[*], because I was planning on posting a patch to revert the lockdep-only lock "documentation".

[*] https://lore.kernel.org/all/[email protected]

Quoted 1 line

Dunno about you, but I haven't kept up with locking.rst at all :)

Heh, x86 has done a decent job of documenting its lock usage. I would much rather add an entry in locking.rst for this new lock than add a lock+unlock in vCPU creation. Especially since the usage is rather uncommon (guaranteed single writer, readers are best-effort and cross-vCPU).

Quoted 6 lines

Having said that, an inversion would still be *very* obvious, as it would be trying to grab a mutex while holding a spinlock...

Signature · 2 lines
Thanks,
Oliver
Acked-by on the v1 series
#10 lore raw
In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson

Protect vcpu->pid with a rwlock instead of RCU, so that running a vCPU with a different task doesn't require a full RCU synchronization, which can introduce a non-trivial amount of jitter, especially on large systems.

I've had this mini-series sitting around for ~2 years, pretty much as-is. I could have sworn past me thought there was a flaw in using a rwlock, and so I never posted it, but for the life of me I can't think of any issues.

Extra eyeballs would be much appreciated.

Besides the nitpicks:

Signature · 2 lines
Thanks,
Oliver
author reviewer Applied
#11 lore raw
In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson

Protect vcpu->pid with a rwlock instead of RCU, so that running a vCPU with a different task doesn't require a full RCU synchronization, which can introduce a non-trivial amount of jitter, especially on large systems.

I've had this mini-series sitting around for ~2 years, pretty much as-is. I could have sworn past me thought there was a flaw in using a rwlock, and so I never posted it, but for the life of me I can't think of any issues.

[...]

Applied to kvm-x86 generic, with a comment explaining the vcpu->mutex is the true protector of vcpu->pid.

Oliver, I didn't add the requested lockdep notification, I really want to make that a separate discussion:

https://lore.kernel.org/all/[email protected]

[1/2] KVM: Return '0' directly when there's no task to yield to

      https://github.com/kvm-x86/linux/commit/6cf9ef23d942
[2/2] KVM: Protect vCPU's "last run PID" with rwlock, not RCU
      https://github.com/kvm-x86/linux/commit/3e7f43188ee2

--
https://github.com/kvm-x86/linux/tree/next