KVM: arm64: vgic: Decouple the check of the EnableLPIs bit from the ITS LPI translation
RFC PATCH 1/1 KVM: arm64: vgic: Decouple the check of the EnableLPIs bit from the ITS LPI translation
When the EnableLPIs bit is set to 0, any ITS LPI requests in the Redistributor would be ignored. And this check is independent from the ITS LPI translation. So it might be better to move the check of the EnableLPIs bit out of the LPI resolving, and also add it to the path that uses the translation cache. Besides it seems that by this the invalidating of the translation cache caused by the LPI disabling is unnecessary.
Not sure if I have missed something... Thanks.
--- arch/arm64/kvm/vgic/vgic-its.c | 9 +++++---- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 4 +--- 2 files changed, 6 insertions(+), 7 deletions(-)
arch/arm64/kvm/vgic/vgic-its.c +5 -4
@@ -683,9 +683,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its,if (!vcpu)return E_ITS_INT_UNMAPPED_INTERRUPT ; -if (!vcpu->arch.vgic_cpu.lpis_enabled) -return -EBUSY ; -vgic_its_cache_translation (kvm, its, devid, eventid, ite->irq); *irq = ite->irq;
@@ -738,6 +735,9 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,if (err)return err; +if (!irq->target_vcpu->arch.vgic_cpu.lpis_enabled) +return -EBUSY ; +if (irq->hw)return irq_set_irqchip_state (irq->host_irq,IRQCHIP_STATE_PENDING , true);
@@ -757,7 +757,8 @@ int vgic_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi) db = (u64 )msi->address_hi <<32 | msi->address_lo; irq =vgic_its_check_cache (kvm, db, msi->devid, msi->data); -if (!irq) + +if (!irq || !irq->target_vcpu->arch.vgic_cpu.lpis_enabled)return -EWOULDBLOCK ;raw_spin_lock_irqsave (&irq->irq_lock, flags);
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1 -3
@@ -242,10 +242,8 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu, vgic_cpu->lpis_enabled = val &GICR_CTLR_ENABLE_LPIS ; -if (was_enabled && !vgic_cpu->lpis_enabled) { +if (was_enabled && !vgic_cpu->lpis_enabled)vgic_flush_pending_lpis (vcpu); -vgic_its_invalidate_cache (vcpu->kvm); - }if (!was_enabled && vgic_cpu->lpis_enabled)vgic_enable_lpis (vcpu);
Signature · 6 lines
2.19.1 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerHi Shemming,
Quoted 5 lines · Shenming Lu
When the EnableLPIs bit is set to 0, any ITS LPI requests in the Redistributor would be ignored. And this check is independent from the ITS LPI translation. So it might be better to move the check of the EnableLPIs bit out of the LPI resolving, and also add it to the path that uses the translation cache.
But by doing that, you are moving the overhead of checking for EnableLPIs from the slow path (translation walk) to the fast path (cache hit), which seems counter-productive.
Quoted 5 lines
Besides it seems that
by this the invalidating of the translation cache caused by the LPI disabling is unnecessary.Not sure if I have missed something... Thanks.
I am certainly missing the purpose of this patch.
The effect of EnableLPIs being zero is to drop the result of any translation (a new pending bit) on the floor. Given that, it is immaterial whether this causes a new translation or hits in the cache, as the result is still to not pend a new interrupt.
I get the feeling that you are trying to optimise for the unusual case where EnableLPIs is 0 *and* you have a screaming device injecting tons of interrupt. If that is the case, I don't think this is worth it.
Thanks,
M.
Quoted 62 lines
Signed-off-by Shenming Lu <[email protected]>--- arch/arm64/kvm/vgic/vgic-its.c | 9 +++++---- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 4 +--- 2 files changed, 6 insertions(+), 7 deletions(-)arch/arm64/kvm/vgic/vgic-its.c +0
-3@@ -683,9 +683,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its, if (!vcpu)return E_ITS_INT_UNMAPPED_INTERRUPT ; -if (!vcpu->arch.vgic_cpu.lpis_enabled) -return -EBUSY ; -vgic_its_cache_translation (kvm, its, devid, eventid, ite->irq);arch/arm64/kvm/vgic/vgic-its.c +3
-0*irq = ite->irq;@@ -738,6 +735,9 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its, if (err) return err; +if (!irq->target_vcpu->arch.vgic_cpu.lpis_enabled) +return -EBUSY ; +if (irq->hw) return irq_set_irqchip_state(irq->host_irq,arch/arm64/kvm/vgic/vgic-its.c +2
-1IRQCHIP_STATE_PENDING, true);@@ -757,7 +757,8 @@ int vgic_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi) db = (u64 )msi->address_hi <<32 | msi->address_lo; irq =vgic_its_check_cache (kvm, db, msi->devid, msi->data); -if (!irq) + +if (!irq || !irq->target_vcpu->arch.vgic_cpu.lpis_enabled)return -EWOULDBLOCK ;arch/arm64/kvm/vgic/vgic-its.c +0
-0raw_spin_lock_irqsave (&irq->irq_lock, flags);arch/arm64/kvm/vgic/vgic-mmio-v3.c +1
-3@@ -242,10 +242,8 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu, vgic_cpu->lpis_enabled = val &GICR_CTLR_ENABLE_LPIS ; -if (was_enabled && !vgic_cpu->lpis_enabled) { +if (was_enabled && !vgic_cpu->lpis_enabled)vgic_flush_pending_lpis (vcpu); -vgic_its_invalidate_cache (vcpu->kvm); - }if (!was_enabled && vgic_cpu->lpis_enabled)
vgic_enable_lpis(vcpu);
Signature · 5 lines
Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 9 lines · Marc Zyngier
Hi Shemming,
Quoted 5 lines · Shenming Lu
When the EnableLPIs bit is set to 0, any ITS LPI requests in the Redistributor would be ignored. And this check is independent from the ITS LPI translation. So it might be better to move the check of the EnableLPIs bit out of the LPI resolving, and also add it to the path that uses the translation cache.
Quoted 3 lines
But by doing that, you are moving the overhead of checking for EnableLPIs from the slow path (translation walk) to the fast path (cache hit), which seems counter-productive.
Oh, I didn't notice the overhead of the checking, I thought it would be negligible...
Quoted 14 lines
Quoted 6 lines
Besides it seems that
by this the invalidating of the translation cache caused by the LPI disabling is unnecessary.Not sure if I have missed something... Thanks.
I am certainly missing the purpose of this patch.
The effect of EnableLPIs being zero is to drop the result of any translation (a new pending bit) on the floor. Given that, it is immaterial whether this causes a new translation or hits in the cache, as the result is still to not pend a new interrupt.
Quoted 4 lines
I get the feeling that you are trying to optimise for the unusual case where EnableLPIs is 0 *and* you have a screaming device injecting tons of interrupt. If that is the case, I don't think this is worth it.
In fact, I just found (imagining) that if the EnableLPIs bit is 0, the kvm_vgic_v4_set_forwarding() would fail when performing the LPI translation, but indeed we don't try to pend any interrupts there...
By the way, it seems that the LPI disabling would not affect the injection of VLPIs...
Thanks,
Shenming
Quoted 58 lines
Thanks,
M.
Quoted 53 lines
Signed-off-by Shenming Lu <[email protected]>---
arch/arm64/kvm/vgic/vgic-its.c | 9 +++++----
arch/arm64/kvm/vgic/vgic-mmio-v3.c | 4 +---2 files changed, 6 insertions(+), 7 deletions(-)arch/arm64/kvm/vgic/vgic-its.c +0
-3@@ -683,9 +683,6 @@ int vgic_its_resolve_lpi(struct kvm *kvm, struct vgic_its *its, if (!vcpu) return E_ITS_INT_UNMAPPED_INTERRUPT; - if (!vcpu->arch.vgic_cpu.lpis_enabled) - return -EBUSY; - vgic_its_cache_translation(kvm, its, devid, eventid, ite->irq);*irq =
ite->irq;arch/arm64/kvm/vgic/vgic-its.c +3
-0@@ -738,6 +735,9 @@ static int vgic_its_trigger_msi(struct kvm *kvm,struct vgic_its *its, if (err) return err; + if (!irq->target_vcpu->arch.vgic_cpu.lpis_enabled) + return -EBUSY; + if (irq->hw) return irq_set_irqchip_state(irq->host_irq,
IRQCHIP_STATE_PENDING, true);arch/arm64/kvm/vgic/vgic-its.c +2
-1@@ -757,7 +757,8 @@ int vgic_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi) db = (u64)msi->address_hi << 32 | msi->address_lo; irq = vgic_its_check_cache(kvm, db, msi->devid, msi->data); - if (!irq) + + if (!irq || !irq->target_vcpu->arch.vgic_cpu.lpis_enabled) return -EWOULDBLOCK;
raw_spin_lock_irqsave(&irq->irq_lock, flags);arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0@@ -242,10 +242,8 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu, vgic_cpu->lpis_enabled = val & GICR_CTLR_ENABLE_LPIS;
Quoted 9 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu, · line 249
Quoted 8 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1
-3- if (was_enabled && !vgic_cpu->lpis_enabled) { + if (was_enabled && !vgic_cpu->lpis_enabled) vgic_flush_pending_lpis(vcpu); - vgic_its_invalidate_cache(vcpu->kvm); - } if (!was_enabled && vgic_cpu->lpis_enabled) vgic_enable_lpis(vcpu);
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerQuoted 10 lines · Shenming Lu
Quoted 9 lines · Marc Zyngier
Hi Shemming,
Quoted 5 lines · Shenming Lu
When the EnableLPIs bit is set to 0, any ITS LPI requests in the Redistributor would be ignored. And this check is independent from the ITS LPI translation. So it might be better to move the check of the EnableLPIs bit out of the LPI resolving, and also add it to the path that uses the translation cache.
Quoted 6 lines
Quoted 3 lines
But by doing that, you are moving the overhead of checking for EnableLPIs from the slow path (translation walk) to the fast path (cache hit), which seems counter-productive.
Oh, I didn't notice the overhead of the checking, I thought it would be negligible...
It probably doesn't show on a modern box, but some of the slower systems might see it. Overall, this is a design decision to keep the translation cache as simple and straightforward as possible:
if anything affects the output of the cache, we invalidate it, and that's it.
Quoted 20 lines
Quoted 15 lines
Quoted 7 lines
Besides it seems that
by this the invalidating of the translation cache caused by the LPI disabling is unnecessary.Not sure if I have missed something... Thanks.
I am certainly missing the purpose of this patch.
The effect of EnableLPIs being zero is to drop the result of any translation (a new pending bit) on the floor. Given that, it is immaterial whether this causes a new translation or hits in the cache, as the result is still to not pend a new interrupt.
Quoted 4 lines
I get the feeling that you are trying to optimise for the unusual case where EnableLPIs is 0 *and* you have a screaming device injecting tons of interrupt. If that is the case, I don't think this is worth it.
Quoted 6 lines
In fact, I just found (imagining) that if the EnableLPIs bit is 0, the
kvm_vgic_v4_set_forwarding()would fail when performing the LPI translation, but indeed we don't try to pend any interrupts there...By the way, it seems that the LPI disabling would not affect the injection of VLPIs...
Yes, good point. We could unmap the VPE from all ITS, which would result in all translations to be discarded, but this has the really bad side effect of *also* preventing the delivery of vSGIs, which isn't what you'd expect.
Overall, I don't think there is a good way to support this, and maybe we should just prevent EnableLPIs to be turned off when using direct injection. After all, the architecture does allow that for GICv3 implementations, which is what we emulate.
Thanks,
M.
Signature · 5 lines
Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 15 lines · Marc Zyngier
Quoted 14 lines · Shenming Lu
Quoted 12 lines · Marc Zyngier
Hi Shemming,
Quoted 5 lines · Shenming Lu
When the EnableLPIs bit is set to 0, any ITS LPI requests in the Redistributor would be ignored. And this check is independent from the ITS LPI translation. So it might be better to move the check of the EnableLPIs bit out of the LPI resolving, and also add it to the path that uses the translation cache.
But by doing that, you are moving the overhead of checking for EnableLPIs from the slow path (translation walk) to the fast path (cache hit), which seems counter-productive.
Quoted 8 lines
Quoted 2 lines
Oh, I didn't notice the overhead of the checking, I thought it would be negligible...
It probably doesn't show on a modern box, but some of the slower systems might see it. Overall, this is a design decision to keep the translation cache as simple and straightforward as possible:
if anything affects the output of the cache, we invalidate it, and that's it.
Ok, get it.
Quoted 33 lines
Quoted 21 lines
Quoted 16 lines
Quoted 8 lines
Besides it seems that
by this the invalidating of the translation cache caused by the LPI disabling is unnecessary.Not sure if I have missed something... Thanks.
I am certainly missing the purpose of this patch.
The effect of EnableLPIs being zero is to drop the result of any translation (a new pending bit) on the floor. Given that, it is immaterial whether this causes a new translation or hits in the cache, as the result is still to not pend a new interrupt.
Quoted 4 lines
I get the feeling that you are trying to optimise for the unusual case where EnableLPIs is 0 *and* you have a screaming device injecting tons of interrupt. If that is the case, I don't think this is worth it.
Quoted 6 lines
In fact, I just found (imagining) that if the EnableLPIs bit is 0, the
kvm_vgic_v4_set_forwarding()would fail when performing the LPI translation, but indeed we don't try to pend any interrupts there...By the way, it seems that the LPI disabling would not affect the injection of VLPIs...
Yes, good point. We could unmap the VPE from all ITS, which would result in all translations to be discarded, but this has the really bad side effect of *also* preventing the delivery of vSGIs, which isn't what you'd expect.
Quoted 4 lines
Overall, I don't think there is a good way to support this, and maybe we should just prevent EnableLPIs to be turned off when using direct injection. After all, the architecture does allow that for GICv3 implementations, which is what we emulate.
Agreed, if there is no good way, we could just make the EnableLPIs clearing unsupported...
Thanks(Happy 2021),
Shenming
Quoted 4 lines
Thanks,
M.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm