Skip to contentSkip to conversation

KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY should support MSI?

DISCUSSION Michael S. Tsirkin 5 participants 12 messages May 5, 2009 – May 20, 2009 KVM

lore
Share & tools
lore
#0 lore raw

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Signature · 1 line
MST
#1 lore raw
Quoted 9 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

If you can't, it would be better to change the ioctls before 2.6.30 is release IMO.

#2 lore raw
Quoted 6 lines · Marcelo Tosatti
Quoted 5 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Quoted 7 lines
Quoted 4 lines

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

Quoted 3 lines

If you can't, it would be better to change the ioctls before 2.6.30 is release IMO.

Signature · 1 line
MST
#3 lore raw
Quoted 14 lines · Michael S. Tsirkin
Quoted 12 lines · Marcelo Tosatti
Quoted 10 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Quoted 7 lines
Quoted 3 lines

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

MSIX_NR is the size of the table, while MSIX_ENTRY updates a single entry, if I read the code correctly.

Signature · 1 line
error compiling committee.c: too many arguments to function
#4 lore raw
Quoted 19 lines · Avi Kivity
Quoted 18 lines · Michael S. Tsirkin
Quoted 15 lines · Marcelo Tosatti
Quoted 10 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Quoted 6 lines
Quoted 3 lines

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

MSIX_NR is the size of the table, while MSIX_ENTRY updates a single entry, if I read the code correctly.

Right. So we'll need something like this for MSI as well. Actually maybe MSIX_NR MSIX_ENTRY should be renamed to MSI_NR / MSI_ENTRY and changed to do the right thing depending on the IRQ type?

Signature · 1 line
MST
#5 lore raw
Quoted 29 lines · Michael S. Tsirkin
Quoted 27 lines · Avi Kivity
Quoted 25 lines · Michael S. Tsirkin
Quoted 8 lines · Marcelo Tosatti
Quoted 5 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Quoted 10 lines
Quoted 6 lines

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

Quoted 7 lines
Quoted 2 lines

MSIX_NR is the size of the table, while MSIX_ENTRY updates a single entry, if I read the code correctly.

Right. So we'll need something like this for MSI as well. Actually maybe MSIX_NR MSIX_ENTRY should be renamed to MSI_NR / MSI_ENTRY and changed to do the right thing depending on the IRQ type?

Works for me. Sheng, is there a reason why it wasn't done like this?

btw, it could be further simplified by using irqfd. Instead of the host device tying directly into kvm, it could just trigger an eventfd; and we could terminate the eventfd either in kvm (irqfd) or in qemu.

Signature · 1 line
error compiling committee.c: too many arguments to function
#6 lore raw
Quoted 34 lines · Avi Kivity
Quoted 24 lines · Michael S. Tsirkin
Quoted 22 lines · Avi Kivity
Quoted 20 lines · Michael S. Tsirkin
Quoted 7 lines · Marcelo Tosatti
Quoted 5 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Quoted 8 lines
Quoted 5 lines

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

Quoted 8 lines
Quoted 3 lines

MSIX_NR is the size of the table, while MSIX_ENTRY updates a single entry, if I read the code correctly.

Right. So we'll need something like this for MSI as well. Actually maybe MSIX_NR MSIX_ENTRY should be renamed to MSI_NR / MSI_ENTRY and changed to do the right thing depending on the IRQ type?

Quoted 5 lines

Works for me. Sheng, is there a reason why it wasn't done like this?

btw, it could be further simplified by using irqfd. Instead of the host device tying directly into kvm, it could just trigger an eventfd; and we could terminate the eventfd either in kvm (irqfd) or in qemu.

This probably is outside the scope for 2.6.30 :)

Signature · 1 line
MST
#7 lore raw
In reply to Avi Kivity
Quoted 34 lines · Avi Kivity
Quoted 24 lines · Michael S. Tsirkin
Quoted 22 lines · Avi Kivity
Quoted 20 lines · Michael S. Tsirkin
Quoted 7 lines · Marcelo Tosatti
Quoted 5 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Quoted 8 lines
Quoted 5 lines

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

Quoted 8 lines
Quoted 3 lines

MSIX_NR is the size of the table, while MSIX_ENTRY updates a single entry, if I read the code correctly.

Right. So we'll need something like this for MSI as well. Actually maybe MSIX_NR MSIX_ENTRY should be renamed to MSI_NR / MSI_ENTRY and changed to do the right thing depending on the IRQ type?

Quoted 5 lines

Works for me. Sheng, is there a reason why it wasn't done like this?

btw, it could be further simplified by using irqfd. Instead of the host device tying directly into kvm, it could just trigger an eventfd; and we could terminate the eventfd either in kvm (irqfd) or in qemu.

If you are going wild, you could then split this code out from kvm into something like a UIO driver. E.g. qemu could then in theory support assigned devices even without VT-d hardware support in CPU.

Signature · 1 line
MST
#8 lore raw
Quoted 9 lines · Michael S. Tsirkin
Quoted 4 lines

Works for me. Sheng, is there a reason why it wasn't done like this?

btw, it could be further simplified by using irqfd. Instead of the host device tying directly into kvm, it could just trigger an eventfd; and we could terminate the eventfd either in kvm (irqfd) or in qemu.

If you are going wild, you could then split this code out from kvm into something like a UIO driver. E.g. qemu could then in theory support assigned devices even without VT-d hardware support in CPU.

That's my thinking. PCI interrupts don't work because we need to do some hacky stuff in there, but MSI should. Oh, and we could improve UIO support for interrupts when using MSI, since there's no need to acknowledge the interrupt.

Support we can tell the kernel to signal an eventfd whenever an MSI fires. We then ask kvm for an irqfd, and give that irqfd to the kernel for the MSI.

Voila, we assign an interrupt from userspace, without the device or kvm knowing anything about it. Like you say, we can assign the device to pure qemu, or to a userspace driver.

Beautiful, I finally found something to replace my old Lego set.

Signature · 1 line
error compiling committee.c: too many arguments to function
#9 lore raw
In reply to Avi Kivity
Quoted 9 lines · On Tuesday 05 May 2009 20:08:40 Avi Kivity wrote:
Quoted 8 lines · Michael S. Tsirkin
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Marcelo Tosatti
Quoted 4 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Well, one question: when did them merged? IIRC, MSI-X related things are still pending for 2.6.31... :)

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 8 lines
Quoted 5 lines

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

Can't you use more than one KVM_ASSIGN_SET_MSIX_ENTRY call per assigned device?

Sure, but only one KVM_ASSIGN_SET_MSIX_NR.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

MSIX_NR is the size of the table, while MSIX_ENTRY updates a single entry, if I read the code correctly.

Right. So we'll need something like this for MSI as well. Actually maybe MSIX_NR MSIX_ENTRY should be renamed to MSI_NR / MSI_ENTRY and changed to do the right thing depending on the IRQ type?

Works for me. Sheng, is there a reason why it wasn't done like this?

No, I think it's fine. Also some related structure should be modified. And one flag field should be add to kvm_assigned_msix_nr and kvm_assigned_msix_entry(using padding ones) to indicate the interrupt type, for we can't determined the irq type by device's status at that time.

Signature · 2 lines
regards
Yang, Sheng
#10 lore raw
In reply to Michael S. Tsirkin
Quoted 10 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

+1

#11 lore raw
Quoted 6 lines · On Tuesday 05 May 2009 19:29:14 Gregory Haskins wrote:
Quoted 5 lines · Michael S. Tsirkin

The new KVM_ASSIGN_SET_MSIX_NR and KVM_ASSIGN_SET_MSIX_ENTRY ioctls have been merged for 2.6.30. However, I note that PCI spec allows devices to support multiple vectors with MSI as well (support will be in linux 2.6.30).

Quoted 6 lines
Quoted 4 lines

Even though qemu for now only uses a single vector with MSI, it would seem that it's better to make the kernel/user interface generic straight away rather than add more ioctls later. What do you think? It might not be too late to fix this for 2.6.30.

+1

Hi

Sorry for I don't have time to work at it recently... And Avi's review patchset reminded me the merge for 2.6.31 would be soon. Anyone have interesting to get this done?

Thanks!

Signature · 2 lines
regards
Yang, Sheng