Skip to contentSkip to conversation

Extensions for KVM MSI related ioctls

DISCUSSION Andre Przywara 4 participants 5 messages Jul 13, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi Paolo,

we (Pavel, Eric and me) need your quick advice on how to proceed with a small API extension:
(tl;dr: skip to the numbered list at the end)

For using MSIs in a guest when running on an ARM(64) system using a GICv3 interrupt controller we need to have a device ID available. On real hardware this information is sampled from the bus by the ITS part of the interrupt controller.
To make this work for guests, we need to extend two ioctls which deal with MSIs: KVM_SIGNAL_MSI and KVM_SET_GSI_ROUTING.

The idea that we sketched so far in [1] and [2] is to use a new capability (KVM_CAP_MSI_DEVID) to advertise both a flag bit for KVM_SIGNAL_MSI and a new type for KVM_SET_GSI_ROUTING.

Since current kernels bail out on any flag value != 0 in KVM_SIGNAL_MSI, we need the new capability to tell userland about it in a reliable and portable way (to avoid hacks like #ifdef ARM && USES_IRQ_ROUTING in userland).

For KVM_SET_GSI_ROUTING there was the idea of using the very same flag value in it's own flag field, but I find it saner to use a new routing type instead (KVM_IRQ_ROUTING_EXTENDED_MSI). Both approaches would piggy-back on the existing struct kvm_irq_routing_msi and re-purpose the pad field in there.

Summarized:
1) Add a new KVM_CAP_MSI_DEVID to advertise the device ID extension.
2) Use a KVM_MSI_VALID_DEVID flag in KVM_SIGNAL_MSI to re-purpose part of struct kvm_msi.
3) Add a new routing type KVM_IRQ_ROUTING_EXTENDED_MSI for KVM_SET_GSI_ROUTING to add device IDs in struct kvm_irq_routing_msi.

Is that a sensible way to extend the KVM API?

Cheers,
Andre.

[1]:https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015622.html [2]:https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015689.html

maintainer
#1 lore raw
Quoted 17 lines · Andre Przywara

For using MSIs in a guest when running on an ARM(64) system using a GICv3 interrupt controller we need to have a device ID available. On real hardware this information is sampled from the bus by the ITS part of the interrupt controller.
To make this work for guests, we need to extend two ioctls which deal with MSIs: KVM_SIGNAL_MSI and KVM_SET_GSI_ROUTING.

The idea that we sketched so far in [1] and [2] is to use a new capability (KVM_CAP_MSI_DEVID) to advertise both a flag bit for KVM_SIGNAL_MSI and a new type for KVM_SET_GSI_ROUTING.

Since current kernels bail out on any flag value != 0 in KVM_SIGNAL_MSI, we need the new capability to tell userland about it in a reliable and portable way (to avoid hacks like #ifdef ARM && USES_IRQ_ROUTING in userland).

Quoted 5 lines

For KVM_SET_GSI_ROUTING there was the idea of using the very same flag value in it's own flag field, but I find it saner to use a new routing type instead (KVM_IRQ_ROUTING_EXTENDED_MSI). Both approaches would piggy-back on the existing struct kvm_irq_routing_msi and re-purpose the pad field in there.

I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.

Quoted 4 lines

Summarized:
1) Add a new KVM_CAP_MSI_DEVID to advertise the device ID extension.
2) Use a KVM_MSI_VALID_DEVID flag in KVM_SIGNAL_MSI to re-purpose part of struct kvm_msi.

These two sounds good.

Paolo

Quoted 4 lines

3) Add a new routing type KVM_IRQ_ROUTING_EXTENDED_MSI for KVM_SET_GSI_ROUTING to add device IDs in struct kvm_irq_routing_msi.

Is that a sensible way to extend the KVM API?

#2 lore raw

Hello!

Quoted 2 lines

I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.

Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more consistent.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#3 lore raw
Quoted 8 lines · Pavel Fedin

Hello!

Quoted 2 lines

I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.

Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more consistent.

OK I will respin with user space flag.

Andre, what about the kernel routing entry struct. You wanted me to get rid of KVM_IRQ_ROUTING_EXTENDED_MSI there too. Will you be able to manage a usespace wrong setting if the type is not set?

Best Regards

Eric

Quoted 7 lines

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#4 lore raw

Hi,

Quoted 10 lines · Eric Auger
Quoted 8 lines · Pavel Fedin

Hello!

Quoted 2 lines

I think I prefer the flag. Offhand it sounds easier to add support for it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.

Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more consistent.

Quoted 5 lines

OK I will respin with user space flag.

Andre, what about the kernel routing entry struct. You wanted me to get rid of KVM_IRQ_ROUTING_EXTENDED_MSI there too. Will you be able to manage a usespace wrong setting if the type is not set?

I am about to see how this all fits together, but I don't expect any serious problems, at least on the kvmtool side.
Instead of setting a different type I just set the flag and guard that by the capability: should be not an issue.

Cheers,
Andre.