Skip to contentSkip to conversation

KVM: arm64: vgic-v4: Consistently request doorbell irq for blocking vCPU

PATCH Applied Oliver Upton 5 participants 13 messages Jul 10, 2023 – Jul 13, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: arm64: vgic-v4: Consistently request doorbell irq for blocking vCPU

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking. While this technically means we could drop the early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Fixes 8e01d9a396e6 ("KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put")
---
 arch/arm64/kvm/vgic/vgic-v3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
index c3b8e132d599..8c467e9f4f11 100644
@@ -749,7 +749,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
 {
 	struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
 
-	WARN_ON(vgic_v4_put(vcpu, false));
+	WARN_ON(vgic_v4_put(vcpu, kvm_vcpu_is_blocking(vcpu)));
 
 	vgic_v3_vmcr_sync(vcpu);
 
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 8 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Quoted 21 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking. While this technically means we could drop the early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Fixes 8e01d9a396e6 ("KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put")
---
 arch/arm64/kvm/vgic/vgic-v3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-v3.c +0 -0
index c3b8e132d599..8c467e9f4f11 100644

arch/arm64/kvm/vgic/vgic-v3.c void vgic_v3_put(struct kvm_vcpu *vcpu) · line 755

Quoted 8 lines
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
@@ -749,7 +749,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
 {
 	struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
 
-	WARN_ON(vgic_v4_put(vcpu, false));
+	WARN_ON(vgic_v4_put(vcpu, kvm_vcpu_is_blocking(vcpu)));
 
 	vgic_v3_vmcr_sync(vcpu);
 

Other than the above nitpicking, this looks good. Thanks both for the very detailed report and the fix.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#2 lore raw
Quoted 7 lines · Marc Zyngier
Quoted 5 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

Quoted 8 lines
Quoted 3 lines

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Quoted 27 lines
Quoted 27 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking. While this technically means we could drop the early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Fixes 8e01d9a396e6 ("KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put")
---
 arch/arm64/kvm/vgic/vgic-v3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
index c3b8e132d599..8c467e9f4f11 100644
@@ -749,7 +749,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
 {
 	struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
 
-	WARN_ON(vgic_v4_put(vcpu, false));
+	WARN_ON(vgic_v4_put(vcpu, kvm_vcpu_is_blocking(vcpu)));
 
Quoted 7 lines

arch/arm64/kvm/vgic/vgic-v3.c void vgic_v3_put(struct kvm_vcpu *vcpu) · line 755

Quoted 2 lines
arch/arm64/kvm/vgic/vgic-v3.c +0 -0
 	vgic_v3_vmcr_sync(vcpu);
 

Other than the above nitpicking, this looks good. Thanks both for the very detailed report and the fix.

Thanks!

--
Best,
Oliver

maintainer
#3 lore raw
Quoted 18 lines · Oliver Upton
Quoted 7 lines · Marc Zyngier
Quoted 5 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

Quoted 8 lines
Quoted 3 lines

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Quoted 4 lines

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Yup. And it is the transition via a new 'resident' state that blows it. No need to repost for that, just amend it locally.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
reviewer Reviewed-by
#4 lore raw
In reply to Oliver Upton
Quoted 17 lines · Oliver Upton
Quoted 7 lines · Marc Zyngier
Quoted 5 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

Quoted 8 lines
Quoted 3 lines

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Quoted 8 lines

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Quoted 3 lines
Quoted 3 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking.

Yup. Agreed!

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While this technically means we could drop the
early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Yeah, just like what we did in commit 07ab0f8d9a12 ("KVM: Call kvm_arch_vcpu_blocking early into the blocking sequence").

My only concern is that if the preemption happens before halt polling, we would enter the polling loop with VPE already resident on the RD and can't recognize any firing GICv4.x virtual interrupts (targeting this VPE) in polling. [1]

Given that making VPE resident on the vcpu block path (i.e., in kvm_vcpu_halt()) makes little sense (right?) and leads to this sort of problem, a crude idea is that we can probably keep track of the "nested" vgic_v4_{put,load} calls (instead of a single vpe->resident flag) and keep VPE *not resident* on the whole block path (like what we had before commit 8e01d9a396e6). And we then rely on kvm_vcpu_wfi/vgic_v4_load to actually schedule the VPE on...

The "need doorbell" rule would be simple as before: we do request DB only if there is a WFI trap (by kvm_vcpu_wfi/vgic_v4_load(vcpu, true)), and don't need it for any other cases.

Nevertheless [1] is just a matter of performance and shouldn't get in the way of we fixing the initial bug. ;-)

Thanks,
Zenghui

maintainer
#5 lore raw
Quoted 35 lines · Zenghui Yu
Quoted 17 lines · Oliver Upton
Quoted 7 lines · Marc Zyngier
Quoted 5 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

Quoted 8 lines
Quoted 3 lines

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Quoted 8 lines

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Quoted 3 lines
Quoted 3 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking.

Yup. Agreed!

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While this technically means we could drop the
early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Quoted 7 lines

Yeah, just like what we did in commit 07ab0f8d9a12 ("KVM: Call kvm_arch_vcpu_blocking early into the blocking sequence").

My only concern is that if the preemption happens before halt polling, we would enter the polling loop with VPE already resident on the RD and can't recognize any firing GICv4.x virtual interrupts (targeting this VPE) in polling. [1]

The status of the pending bit is recorded in pending_last, so we don't lose what was snapshot at the point of hitting WFI. But we indeed don't have any idea for something firing during the polling loop.

Quoted 7 lines

Given that making VPE resident on the vcpu block path (i.e., in kvm_vcpu_halt()) makes little sense (right?) and leads to this sort of problem, a crude idea is that we can probably keep track of the "nested" vgic_v4_{put,load} calls (instead of a single vpe->resident flag) and keep VPE *not resident* on the whole block path (like what we had before commit 8e01d9a396e6). And we then rely on kvm_vcpu_wfi/vgic_v4_load to actually schedule the VPE on...

I'm not sure about the nested tracking part, but it's easy enough to have a vcpu flag indicating that we're in WFI. So an *alternative* to the current fix would be something like this:

arch/arm64/include/asm/kvm_host.h +2 -0
index f54ba0a63669..417a0e85456b 100644
@@ -817,6 +817,8 @@ struct kvm_vcpu_arch {
 #define DBG_SS_ACTIVE_PENDING	__vcpu_single_flag(sflags, BIT(5))
 /* PMUSERENR for the guest EL0 is on physical CPU */
 #define PMUSERENR_ON_CPU	__vcpu_single_flag(sflags, BIT(6))
+/* WFI instruction trapped */
+#define IN_WFI			__vcpu_single_flag(sflags, BIT(7))
 
 /* vcpu entered with HCR_EL2.E2H set */
 #define VCPU_HCR_E2H		__vcpu_single_flag(oflags, BIT(0))
arch/arm64/kvm/arm.c +4 -2
index 236c5f1c9090..cf208d30a9ea 100644
@@ -725,13 +725,15 @@ void kvm_vcpu_wfi(struct kvm_vcpu *vcpu)
 	 */
 	preempt_disable();
 	kvm_vgic_vmcr_sync(vcpu);
-	vgic_v4_put(vcpu, true);
+	vcpu_set_flag(vcpu, IN_WFI);
+	vgic_v4_put(vcpu);
 	preempt_enable();
 
 	kvm_vcpu_halt(vcpu);
 	vcpu_clear_flag(vcpu, IN_WFIT);
 
 	preempt_disable();
+	vcpu_clear_flag(vcpu, IN_WFI);
 	vgic_v4_load(vcpu);
 	preempt_enable();
 }
@@ -799,7 +801,7 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
 		if (kvm_check_request(KVM_REQ_RELOAD_GICv4, vcpu)) {
 			/* The distributor enable bits were changed */
 			preempt_disable();
-			vgic_v4_put(vcpu, false);
+			vgic_v4_put(vcpu);
 			vgic_v4_load(vcpu);
 			preempt_enable();
 		}
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
index 49d35618d576..df61ead7c757 100644
@@ -780,7 +780,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
 	 * done a vgic_v4_put) and when running a nested guest (the
 	 * vPE was never resident in order to generate a doorbell).
 	 */
-	WARN_ON(vgic_v4_put(vcpu, false));
+	WARN_ON(vgic_v4_put(vcpu));
 
 	vgic_v3_vmcr_sync(vcpu);
 
arch/arm64/kvm/vgic/vgic-v4.c +5 -2
index c1c28fe680ba..339a55194b2c 100644
@@ -336,14 +336,14 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->vpes = NULL;
 }
 
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db)
+int vgic_v4_put(struct kvm_vcpu *vcpu)
 {
 	struct its_vpe *vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
 
 	if (!vgic_supports_direct_msis(vcpu->kvm) || !vpe->resident)
 		return 0;
 
-	return its_make_vpe_non_resident(vpe, need_db);
+	return its_make_vpe_non_resident(vpe, !!vcpu_get_flag(vcpu, IN_WFI));
 }
 
 int vgic_v4_load(struct kvm_vcpu *vcpu)
@@ -354,6 +354,9 @@ int vgic_v4_load(struct kvm_vcpu *vcpu)
 	if (!vgic_supports_direct_msis(vcpu->kvm) || vpe->resident)
 		return 0;
 
+	if (vcpu_get_flag(vcpu, IN_WFI))
+		return 0;
+
 	/*
 	 * Before making the VPE resident, make sure the redistributor
 	 * corresponding to our current CPU expects us here. See the
include/kvm/arm_vgic.h +1 -1
index 9b91a8135dac..765d801d1ddc 100644
@@ -446,7 +446,7 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 
 int vgic_v4_load(struct kvm_vcpu *vcpu);
 void vgic_v4_commit(struct kvm_vcpu *vcpu);
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+int vgic_v4_put(struct kvm_vcpu *vcpu);
 
 bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
 

Of course, it is totally untested... ;-) But I like that the doorbell request is solely driven by the WFI state, and we avoid leaking the knowledge outside of the vgic code.

Thoughts?

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#6 lore raw
Quoted 66 lines · Marc Zyngier
Quoted 35 lines · Zenghui Yu
Quoted 17 lines · Oliver Upton
Quoted 7 lines · Marc Zyngier
Quoted 5 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

Quoted 8 lines
Quoted 3 lines

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Quoted 8 lines

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Quoted 3 lines
Quoted 3 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking.

Yup. Agreed!

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While this technically means we could drop the
early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Quoted 7 lines

Yeah, just like what we did in commit 07ab0f8d9a12 ("KVM: Call kvm_arch_vcpu_blocking early into the blocking sequence").

My only concern is that if the preemption happens before halt polling, we would enter the polling loop with VPE already resident on the RD and can't recognize any firing GICv4.x virtual interrupts (targeting this VPE) in polling. [1]

The status of the pending bit is recorded in pending_last, so we don't lose what was snapshot at the point of hitting WFI. But we indeed don't have any idea for something firing during the polling loop.

Quoted 7 lines

Given that making VPE resident on the vcpu block path (i.e., in kvm_vcpu_halt()) makes little sense (right?) and leads to this sort of problem, a crude idea is that we can probably keep track of the "nested" vgic_v4_{put,load} calls (instead of a single vpe->resident flag) and keep VPE *not resident* on the whole block path (like what we had before commit 8e01d9a396e6). And we then rely on kvm_vcpu_wfi/vgic_v4_load to actually schedule the VPE on...

I'm not sure about the nested tracking part, but it's easy enough to have a vcpu flag indicating that we're in WFI. So an *alternative* to the current fix would be something like this:

arch/arm64/include/asm/kvm_host.h +0 -0
index f54ba0a63669..417a0e85456b 100644

arch/arm64/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 821

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -817,6 +817,8 @@ struct kvm_vcpu_arch {
 #define DBG_SS_ACTIVE_PENDING	__vcpu_single_flag(sflags, BIT(5))
 /* PMUSERENR for the guest EL0 is on physical CPU */
 #define PMUSERENR_ON_CPU	__vcpu_single_flag(sflags, BIT(6))
+/* WFI instruction trapped */
+#define IN_WFI			__vcpu_single_flag(sflags, BIT(7))

Ah, trust me that I was thinking about exactly the same vcpu flag when writing the last email. ;-) So here is my Ack for this alternative, thanks Marc for your quick reply!

Quoted 91 lines
arch/arm64/include/asm/kvm_host.h +0 -0
 /* vcpu entered with HCR_EL2.E2H set */
 #define VCPU_HCR_E2H		__vcpu_single_flag(oflags, BIT(0))
arch/arm64/kvm/arm.c +4 -2
index 236c5f1c9090..cf208d30a9ea 100644
@@ -725,13 +725,15 @@ void kvm_vcpu_wfi(struct kvm_vcpu *vcpu)
 	 */
 	preempt_disable();
 	kvm_vgic_vmcr_sync(vcpu);
-	vgic_v4_put(vcpu, true);
+	vcpu_set_flag(vcpu, IN_WFI);
+	vgic_v4_put(vcpu);
 	preempt_enable();
 
 	kvm_vcpu_halt(vcpu);
 	vcpu_clear_flag(vcpu, IN_WFIT);
 
 	preempt_disable();
+	vcpu_clear_flag(vcpu, IN_WFI);
 	vgic_v4_load(vcpu);
 	preempt_enable();
 }
@@ -799,7 +801,7 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
 		if (kvm_check_request(KVM_REQ_RELOAD_GICv4, vcpu)) {
 			/* The distributor enable bits were changed */
 			preempt_disable();
-			vgic_v4_put(vcpu, false);
+			vgic_v4_put(vcpu);
 			vgic_v4_load(vcpu);
 			preempt_enable();
 		}
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
index 49d35618d576..df61ead7c757 100644
@@ -780,7 +780,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
 	 * done a vgic_v4_put) and when running a nested guest (the
 	 * vPE was never resident in order to generate a doorbell).
 	 */
-	WARN_ON(vgic_v4_put(vcpu, false));
+	WARN_ON(vgic_v4_put(vcpu));
 
 	vgic_v3_vmcr_sync(vcpu);
 
arch/arm64/kvm/vgic/vgic-v4.c +5 -2
index c1c28fe680ba..339a55194b2c 100644
@@ -336,14 +336,14 @@ void vgic_v4_teardown(struct kvm *kvm)
 	its_vm->vpes = NULL;
 }
 
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db)
+int vgic_v4_put(struct kvm_vcpu *vcpu)
 {
 	struct its_vpe *vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
 
 	if (!vgic_supports_direct_msis(vcpu->kvm) || !vpe->resident)
 		return 0;
 
-	return its_make_vpe_non_resident(vpe, need_db);
+	return its_make_vpe_non_resident(vpe, !!vcpu_get_flag(vcpu, IN_WFI));
 }
 
 int vgic_v4_load(struct kvm_vcpu *vcpu)
@@ -354,6 +354,9 @@ int vgic_v4_load(struct kvm_vcpu *vcpu)
 	if (!vgic_supports_direct_msis(vcpu->kvm) || vpe->resident)
 		return 0;
 
+	if (vcpu_get_flag(vcpu, IN_WFI))
+		return 0;
+
 	/*
 	 * Before making the VPE resident, make sure the redistributor
 	 * corresponding to our current CPU expects us here. See the
include/kvm/arm_vgic.h +1 -1
index 9b91a8135dac..765d801d1ddc 100644
@@ -446,7 +446,7 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 
 int vgic_v4_load(struct kvm_vcpu *vcpu);
 void vgic_v4_commit(struct kvm_vcpu *vcpu);
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+int vgic_v4_put(struct kvm_vcpu *vcpu);
 
 bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
 
Quoted 3 lines

Of course, it is totally untested... ;-) But I like that the doorbell request is solely driven by the WFI state, and we avoid leaking the knowledge outside of the vgic code.

I'm happy with this approach and will have another look tomorrow. It'd also be great if Xiang can give this one a go on the appropriate HW.

Thanks,
Zenghui

#7 lore raw
Quoted 171 lines · Zenghui Yu
Quoted 66 lines · Marc Zyngier
Quoted 35 lines · Zenghui Yu
Quoted 17 lines · Oliver Upton
Quoted 7 lines · Marc Zyngier
Quoted 5 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

Quoted 8 lines
Quoted 3 lines

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Quoted 8 lines

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Quoted 3 lines
Quoted 3 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking.

Yup. Agreed!

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While this technically means we could drop the
early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Quoted 7 lines

Yeah, just like what we did in commit 07ab0f8d9a12 ("KVM: Call kvm_arch_vcpu_blocking early into the blocking sequence").

My only concern is that if the preemption happens before halt polling, we would enter the polling loop with VPE already resident on the RD and can't recognize any firing GICv4.x virtual interrupts (targeting this VPE) in polling. [1]

The status of the pending bit is recorded in pending_last, so we don't lose what was snapshot at the point of hitting WFI. But we indeed don't have any idea for something firing during the polling loop.

Quoted 7 lines

Given that making VPE resident on the vcpu block path (i.e., in kvm_vcpu_halt()) makes little sense (right?) and leads to this sort of problem, a crude idea is that we can probably keep track of the "nested" vgic_v4_{put,load} calls (instead of a single vpe->resident flag) and keep VPE *not resident* on the whole block path (like what we had before commit 8e01d9a396e6). And we then rely on kvm_vcpu_wfi/vgic_v4_load to actually schedule the VPE on...

I'm not sure about the nested tracking part, but it's easy enough to have a vcpu flag indicating that we're in WFI. So an *alternative* to the current fix would be something like this:

arch/arm64/include/asm/kvm_host.h +0 -0
index f54ba0a63669..417a0e85456b 100644

arch/arm64/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 821

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -817,6 +817,8 @@ struct kvm_vcpu_arch {
 #define DBG_SS_ACTIVE_PENDING    __vcpu_single_flag(sflags, BIT(5))
 /* PMUSERENR for the guest EL0 is on physical CPU */
 #define PMUSERENR_ON_CPU    __vcpu_single_flag(sflags, BIT(6))
+/* WFI instruction trapped */
+#define IN_WFI            __vcpu_single_flag(sflags, BIT(7))

Ah, trust me that I was thinking about exactly the same vcpu flag when writing the last email. ;-) So here is my Ack for this alternative, thanks Marc for your quick reply!

Quoted 94 lines
arch/arm64/include/asm/kvm_host.h +0 -0
 /* vcpu entered with HCR_EL2.E2H set */
 #define VCPU_HCR_E2H        __vcpu_single_flag(oflags, BIT(0))
arch/arm64/kvm/arm.c +4 -2
index 236c5f1c9090..cf208d30a9ea 100644
@@ -725,13 +725,15 @@ void kvm_vcpu_wfi(struct kvm_vcpu *vcpu)
      */
     preempt_disable();
     kvm_vgic_vmcr_sync(vcpu);
-    vgic_v4_put(vcpu, true);
+    vcpu_set_flag(vcpu, IN_WFI);
+    vgic_v4_put(vcpu);
     preempt_enable();
 
     kvm_vcpu_halt(vcpu);
     vcpu_clear_flag(vcpu, IN_WFIT);
 
     preempt_disable();
+    vcpu_clear_flag(vcpu, IN_WFI);
     vgic_v4_load(vcpu);
     preempt_enable();
 }
@@ -799,7 +801,7 @@ static int check_vcpu_requests(struct kvm_vcpu 
 *vcpu)
         if (kvm_check_request(KVM_REQ_RELOAD_GICv4, vcpu)) {
             /* The distributor enable bits were changed */
             preempt_disable();
-            vgic_v4_put(vcpu, false);
+            vgic_v4_put(vcpu);
             vgic_v4_load(vcpu);
             preempt_enable();
         }
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
index 49d35618d576..df61ead7c757 100644
@@ -780,7 +780,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
      * done a vgic_v4_put) and when running a nested guest (the
      * vPE was never resident in order to generate a doorbell).
      */
-    WARN_ON(vgic_v4_put(vcpu, false));
+    WARN_ON(vgic_v4_put(vcpu));
 
     vgic_v3_vmcr_sync(vcpu);
 
arch/arm64/kvm/vgic/vgic-v4.c +2 -2
index c1c28fe680ba..339a55194b2c 100644
@@ -336,14 +336,14 @@ void vgic_v4_teardown(struct kvm *kvm)
     its_vm->vpes = NULL;
 }
 
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db)
+int vgic_v4_put(struct kvm_vcpu *vcpu)
 {
     struct its_vpe *vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
 
     if (!vgic_supports_direct_msis(vcpu->kvm) || !vpe->resident)
         return 0;
 
-    return its_make_vpe_non_resident(vpe, need_db);
+    return its_make_vpe_non_resident(vpe, !!vcpu_get_flag(vcpu, 
 IN_WFI));
 }
 

int vgic_v4_load(struct kvm_vcpu *vcpu)

arch/arm64/kvm/vgic/vgic-v4.c +3 -0
@@ -354,6 +354,9 @@ int vgic_v4_load(struct kvm_vcpu *vcpu)
     if (!vgic_supports_direct_msis(vcpu->kvm) || vpe->resident)
         return 0;
 
+    if (vcpu_get_flag(vcpu, IN_WFI))
+        return 0;
+
     /*
      * Before making the VPE resident, make sure the redistributor
      * corresponding to our current CPU expects us here. See the
include/kvm/arm_vgic.h +1 -1
index 9b91a8135dac..765d801d1ddc 100644
@@ -446,7 +446,7 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, 
 int irq,
 
 int vgic_v4_load(struct kvm_vcpu *vcpu);
 void vgic_v4_commit(struct kvm_vcpu *vcpu);
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+int vgic_v4_put(struct kvm_vcpu *vcpu);
 
 bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
Quoted 6 lines
Quoted 3 lines

Of course, it is totally untested... ;-) But I like that the doorbell request is solely driven by the WFI state, and we avoid leaking the knowledge outside of the vgic code.

I'm happy with this approach and will have another look tomorrow. It'd also be great if Xiang can give this one a go on the appropriate HW.

Ok, I will test this approach and feedback the result when finished.

Quoted 5 lines

Thanks,
Zenghui
.

#8 lore raw
In reply to Marc Zyngier
Quoted 11 lines · Marc Zyngier
Quoted 7 lines

Given that making VPE resident on the vcpu block path (i.e., in kvm_vcpu_halt()) makes little sense (right?) and leads to this sort of problem, a crude idea is that we can probably keep track of the "nested" vgic_v4_{put,load} calls (instead of a single vpe->resident flag) and keep VPE *not resident* on the whole block path (like what we had before commit 8e01d9a396e6). And we then rely on kvm_vcpu_wfi/vgic_v4_load to actually schedule the VPE on...

I'm not sure about the nested tracking part, but it's easy enough to have a vcpu flag indicating that we're in WFI. So an *alternative* to the current fix would be something like this:

Yeah, I like your approach better. I've gone ahead and backed out my change and can take this instead once someone tests it out :)

Signature · 2 lines
Thanks,
Oliver
Tested-by
#9 lore raw
In reply to Marc Zyngier

Hi,

Quoted 153 lines · Marc Zyngier
Quoted 28 lines · Zenghui Yu
Quoted 10 lines · Oliver Upton
Quoted 9 lines · Marc Zyngier
Quoted 7 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

Quoted 11 lines
Quoted 4 lines

It'd be worth spelling out. You need to go via *three* schedule()
calls: one to be preempted (with DB set), one to be made resident again, and then the final one in kvm_vcpu_halt(), clearing the DB on vcpu_put() due to the bug.

Yeah, a bit lazy in the wording. What I had meant to imply was preemption happening after the doorbell is set up and before the thread has an opportunity to explicitly schedule out. Perhaps I should just say that.

Quoted 2 lines
Quoted 2 lines

Fix it by consistently requesting a doorbell irq in the vcpu put path if the vCPU is blocking.

Yup. Agreed!

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While this technically means we could drop the
early doorbell irq request in kvm_vcpu_wfi(), deliberately leave it intact such that vCPU halt polling can properly detect the wakeup condition before actually scheduling out a vCPU.

Quoted 7 lines

Yeah, just like what we did in commit 07ab0f8d9a12 ("KVM: Call kvm_arch_vcpu_blocking early into the blocking sequence").

My only concern is that if the preemption happens before halt polling, we would enter the polling loop with VPE already resident on the RD and can't recognize any firing GICv4.x virtual interrupts (targeting this VPE) in polling. [1]

The status of the pending bit is recorded in pending_last, so we don't lose what was snapshot at the point of hitting WFI. But we indeed don't have any idea for something firing during the polling loop.

Quoted 7 lines

Given that making VPE resident on the vcpu block path (i.e., in kvm_vcpu_halt()) makes little sense (right?) and leads to this sort of problem, a crude idea is that we can probably keep track of the "nested" vgic_v4_{put,load} calls (instead of a single vpe->resident flag) and keep VPE *not resident* on the whole block path (like what we had before commit 8e01d9a396e6). And we then rely on kvm_vcpu_wfi/vgic_v4_load to actually schedule the VPE on...

I'm not sure about the nested tracking part, but it's easy enough to have a vcpu flag indicating that we're in WFI. So an *alternative* to the current fix would be something like this:

arch/arm64/include/asm/kvm_host.h +2 -0
index f54ba0a63669..417a0e85456b 100644
@@ -817,6 +817,8 @@ struct kvm_vcpu_arch {
  #define DBG_SS_ACTIVE_PENDING	__vcpu_single_flag(sflags, BIT(5))
  /* PMUSERENR for the guest EL0 is on physical CPU */
  #define PMUSERENR_ON_CPU	__vcpu_single_flag(sflags, BIT(6))
+/* WFI instruction trapped */
+#define IN_WFI			__vcpu_single_flag(sflags, BIT(7))
 
  /* vcpu entered with HCR_EL2.E2H set */
  #define VCPU_HCR_E2H		__vcpu_single_flag(oflags, BIT(0))
arch/arm64/kvm/arm.c +4 -2
index 236c5f1c9090..cf208d30a9ea 100644
@@ -725,13 +725,15 @@ void kvm_vcpu_wfi(struct kvm_vcpu *vcpu)
  	 */
  	preempt_disable();
  	kvm_vgic_vmcr_sync(vcpu);
-	vgic_v4_put(vcpu, true);
+	vcpu_set_flag(vcpu, IN_WFI);
+	vgic_v4_put(vcpu);
  	preempt_enable();
 
  	kvm_vcpu_halt(vcpu);
  	vcpu_clear_flag(vcpu, IN_WFIT);
 
  	preempt_disable();
+	vcpu_clear_flag(vcpu, IN_WFI);
  	vgic_v4_load(vcpu);
  	preempt_enable();
  }
@@ -799,7 +801,7 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
  		if (kvm_check_request(KVM_REQ_RELOAD_GICv4, vcpu)) {
  			/* The distributor enable bits were changed */
  			preempt_disable();
-			vgic_v4_put(vcpu, false);
+			vgic_v4_put(vcpu);
  			vgic_v4_load(vcpu);
  			preempt_enable();
  		}
arch/arm64/kvm/vgic/vgic-v3.c +1 -1
index 49d35618d576..df61ead7c757 100644
@@ -780,7 +780,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
  	 * done a vgic_v4_put) and when running a nested guest (the
  	 * vPE was never resident in order to generate a doorbell).
  	 */
-	WARN_ON(vgic_v4_put(vcpu, false));
+	WARN_ON(vgic_v4_put(vcpu));
 
  	vgic_v3_vmcr_sync(vcpu);
 
arch/arm64/kvm/vgic/vgic-v4.c +5 -2
index c1c28fe680ba..339a55194b2c 100644
@@ -336,14 +336,14 @@ void vgic_v4_teardown(struct kvm *kvm)
  	its_vm->vpes = NULL;
  }
 
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db)
+int vgic_v4_put(struct kvm_vcpu *vcpu)
  {
  	struct its_vpe *vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
 
  	if (!vgic_supports_direct_msis(vcpu->kvm) || !vpe->resident)
  		return 0;
 
-	return its_make_vpe_non_resident(vpe, need_db);
+	return its_make_vpe_non_resident(vpe, !!vcpu_get_flag(vcpu, IN_WFI));
  }
 
  int vgic_v4_load(struct kvm_vcpu *vcpu)
@@ -354,6 +354,9 @@ int vgic_v4_load(struct kvm_vcpu *vcpu)
  	if (!vgic_supports_direct_msis(vcpu->kvm) || vpe->resident)
  		return 0;
 
+	if (vcpu_get_flag(vcpu, IN_WFI))
+		return 0;
+
  	/*
  	 * Before making the VPE resident, make sure the redistributor
  	 * corresponding to our current CPU expects us here. See the
include/kvm/arm_vgic.h +1 -1
index 9b91a8135dac..765d801d1ddc 100644
@@ -446,7 +446,7 @@ int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 
  int vgic_v4_load(struct kvm_vcpu *vcpu);
  void vgic_v4_commit(struct kvm_vcpu *vcpu);
-int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
+int vgic_v4_put(struct kvm_vcpu *vcpu);
 
  bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
 
Quoted 3 lines

Of course, it is totally untested... ;-) But I like that the doorbell request is solely driven by the WFI state, and we avoid leaking the knowledge outside of the vgic code.

I have tested this approach and it also solves the issue. Please feel free to add:

Quoted 5 lines

Thoughts?

	M.
#10 lore raw
Quoted 7 lines · chenxiang (M)
Quoted 3 lines

Of course, it is totally untested... ;-) But I like that the doorbell request is solely driven by the WFI state, and we avoid leaking the knowledge outside of the vgic code.

I have tested this approach and it also solves the issue. Please feel free to add:

Excellent, thanks for testing a couple of iterations for us here. Marc, do you want to add a changelog to this?

--
Thanks,
Oliver

maintainer
#11 lore raw
Quoted 6 lines · Oliver Upton
Quoted 4 lines · chenxiang (M)
Quoted 3 lines

Of course, it is totally untested... ;-) But I like that the doorbell request is solely driven by the WFI state, and we avoid leaking the knowledge outside of the vgic code.

Quoted 6 lines
Quoted 3 lines

I have tested this approach and it also solves the issue. Please feel free to add:

Excellent, thanks for testing a couple of iterations for us here. Marc, do you want to add a changelog to this?

Done [1]. I've nicked most of your changelog, so you get a CDB tag ;-)

Thanks,

	M.

[1] https://lore.kernel.org/all/[email protected]/

Signature · 1 line
Without deviation from the norm, progress is not possible.
author Applied
#12 lore raw
In reply to Oliver Upton
Quoted 9 lines · Oliver Upton

Xiang reports that VMs occasionally fail to boot on GICv4.1 systems when running a preemptible kernel, as it is possible that a vCPU is blocked without requesting a doorbell interrupt.

The issue is that any preemption that occurs between vgic_v4_put() and schedule() on the block path will mark the vPE as nonresident and *not* request a doorbell irq.

[...]

Applied to kvmarm/fixes, thanks!

[1/1] KVM: arm64: vgic-v4: Consistently request doorbell irq for blocking vCPU

      https://git.kernel.org/kvmarm/kvmarm/c/d30ea1f31ff5

--
Best,
Oliver