Extensions for KVM MSI related ioctls
Hi Paolo,
we (Pavel, Eric and me) need your quick advice on how to proceed with a small API extension:
(tl;dr: skip to the numbered list at the end)
For using MSIs in a guest when running on an ARM(64) system using a GICv3 interrupt controller we need to have a device ID available. On real hardware this information is sampled from the bus by the ITS part of the interrupt controller.
To make this work for guests, we need to extend two ioctls which deal with MSIs: KVM_SIGNAL_MSI and KVM_SET_GSI_ROUTING.
The idea that we sketched so far in [1] and [2] is to use a new capability (KVM_CAP_MSI_DEVID) to advertise both a flag bit for KVM_SIGNAL_MSI and a new type for KVM_SET_GSI_ROUTING.
Since current kernels bail out on any flag value != 0 in KVM_SIGNAL_MSI, we need the new capability to tell userland about it in a reliable and portable way (to avoid hacks like #ifdef ARM && USES_IRQ_ROUTING in userland).
For KVM_SET_GSI_ROUTING there was the idea of using the very same flag value in it's own flag field, but I find it saner to use a new routing type instead (KVM_IRQ_ROUTING_EXTENDED_MSI). Both approaches would piggy-back on the existing struct kvm_irq_routing_msi and re-purpose the pad field in there.
Summarized:
1) Add a new KVM_CAP_MSI_DEVID to advertise the device ID extension.
2) Use a KVM_MSI_VALID_DEVID flag in KVM_SIGNAL_MSI to re-purpose part of struct kvm_msi.
3) Add a new routing type KVM_IRQ_ROUTING_EXTENDED_MSI for KVM_SET_GSI_ROUTING to add device IDs in struct kvm_irq_routing_msi.
Is that a sensible way to extend the KVM API?
Cheers,
Andre.
[1]:https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015622.html [2]:https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015689.html
Paolo Bonzini
maintainerQuoted 17 lines · Andre Przywara
For using MSIs in a guest when running on an ARM(64) system using a GICv3 interrupt controller we need to have a device ID available. On real hardware this information is sampled from the bus by the ITS part of the interrupt controller.
To make this work for guests, we need to extend two ioctls which deal with MSIs:KVM_SIGNAL_MSIandKVM_SET_GSI_ROUTING.The idea that we sketched so far in [1] and [2] is to use a new capability (
KVM_CAP_MSI_DEVID) to advertise both a flag bit forKVM_SIGNAL_MSIand a new type forKVM_SET_GSI_ROUTING.Since current kernels bail out on any flag value != 0 in
KVM_SIGNAL_MSI, we need the new capability to tell userland about it in a reliable and portable way (to avoid hacks like #ifdef ARM &&USES_IRQ_ROUTINGin userland).
Quoted 5 lines
For
KVM_SET_GSI_ROUTINGthere was the idea of using the very same flag value in it's own flag field, but I find it saner to use a new routing type instead (KVM_IRQ_ROUTING_EXTENDED_MSI). Both approaches would piggy-back on the existing structkvm_irq_routing_msiand re-purpose the pad field in there.
I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.
Quoted 4 lines
Summarized:
1) Add a newKVM_CAP_MSI_DEVIDto advertise the device ID extension.
2) Use aKVM_MSI_VALID_DEVIDflag inKVM_SIGNAL_MSIto re-purpose part of structkvm_msi.
These two sounds good.
Paolo
Quoted 4 lines
3) Add a new routing type
KVM_IRQ_ROUTING_EXTENDED_MSIforKVM_SET_GSI_ROUTINGto add device IDs in structkvm_irq_routing_msi.Is that a sensible way to extend the KVM API?
Hello!
Quoted 2 lines
I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to
KVM_IRQ_ROUTING_EXTENDED_MSI.
Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more consistent.
Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia
Quoted 8 lines · Pavel Fedin
Hello!
Quoted 2 lines
I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to
KVM_IRQ_ROUTING_EXTENDED_MSI.Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more consistent.
OK I will respin with user space flag.
Andre, what about the kernel routing entry struct. You wanted me to get rid of KVM_IRQ_ROUTING_EXTENDED_MSI there too. Will you be able to manage a usespace wrong setting if the type is not set?
Best Regards
Eric
Quoted 7 lines
Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia
Hi,
Quoted 10 lines · Eric Auger
Quoted 8 lines · Pavel Fedin
Hello!
Quoted 2 lines
I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to
KVM_IRQ_ROUTING_EXTENDED_MSI.Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more consistent.
Quoted 5 lines
OK I will respin with user space flag.
Andre, what about the kernel routing entry struct. You wanted me to get rid of
KVM_IRQ_ROUTING_EXTENDED_MSIthere too. Will you be able to manage a usespace wrong setting if the type is not set?
I am about to see how this all fits together, but I don't expect any serious problems, at least on the kvmtool side.
Instead of setting a different type I just set the flag and guard that by the capability: should be not an issue.
Cheers,
Andre.