PCI Device Assignment status
Hi!
I wanted to know the status of PCI device assignment.
As far as I can see in the webpage and in the mailing list, it seems to be working ok if you have VT-d support on the motherboard and cpu. But if it isn't too much trouble, I wanted some confirmation about this, since I'm not sure and I don't want to buy hardware to test this when there is no way it's going to work :)
Also, I wanted to confirm that there is no limit in the maximun PCI cards that could be given to a guest (like VMWare which only lets you give at most 2 cards to a guest, if I'm not wrong) and which version of qemu-kvm/Linux kernel I need for propper support.
Thanks a lot,
Rodrigo
Quoted 9 lines · Rodrigo Campos
Hi!
I wanted to know the status of PCI device assignment.
As far as I can see in the webpage and in the mailing list, it seems to be working ok if you have VT-d support on the motherboard and cpu. But if it isn't too much trouble, I wanted some confirmation about this, since I'm not sure and I don't want to buy hardware to test this when there is no way it's going to work :)
Yes, it works if you have VT-d support (Intel) or AMD IOMMU (note this is different than the AMD GART that's often used as an IOMMU). The Intel boxes are a lot easier to find.
Quoted 4 lines
Also, I wanted to confirm that there is no limit in the maximun PCI cards that could be given to a guest (like VMWare which only lets you give at most 2 cards to a guest, if I'm not wrong) and which version of qemu-kvm/Linux kernel I need for propper support.
The only limit I'm aware of is the number of slots on a bus (32), a few of which are used by qemu emulated devices and can't be removed. Any of the 0.12 releases should work fine. Also note that there are also device and driver dependencies that can be far more subtle in whether the device can work with passthrough or not. Things like whether it has device dependent registers in pci config space, if the driver expects specific capabilities at certain offsets, if it's a multi-function device that shares nvram and uses the function as an index offset, drivers that assume a bridge topology for a given
vid/did, etc... SR-IOV devices are designed for this, so should
work, otherwise you might also post the cards you'd like to use and see if anyone has experience with them.
Alex
Quoted 8 lines · Alex Williamson
Quoted 4 lines · Rodrigo Campos
Also, I wanted to confirm that there is no limit in the maximun PCI cards that could be given to a guest (like VMWare which only lets you give at most 2 cards to a guest, if I'm not wrong) and which version of qemu-kvm/Linux kernel I need for propper support.
The only limit I'm aware of is the number of slots on a bus (32), a few of which are used by qemu emulated devices and can't be removed.
Great, so it shouldn't be a problem I guess =)
Quoted 1 line
Any of the 0.12 releases should work fine. Also note that there are
Cool, thanks!
Quoted 9 lines
also device and driver dependencies that can be far more subtle in whether the device can work with passthrough or not. Things like whether it has device dependent registers in pci config space, if the driver expects specific capabilities at certain offsets, if it's a multi-function device that shares nvram and uses the function as an index offset, drivers that assume a bridge topology for a given
vid/did, etc... SR-IOV devices are designed for this, so should
work, otherwise you might also post the cards you'd like to use and see if anyone has experience with them.
Ahh, I see. Thanks a lot for the information :)
I will need, for sure, at least this card:
http://www.dialogic.com/products/serial_protocol_adapters/Eiconcard_C_Series.htm?techspec=1®ID=4599
But I don't really know what the driver does. I will use it on QNX (yeah, I know it wouldn't be hard real-time :)), so it's not the driver/software it says there. It's this one, in case it matters/gives anyone some idea:
http://www.gandacar.ca/index.php?option=com_content&view=article&id=60&Itemid=89
I think that this card does almost everything in hardware and the driver only does some communication (although I don't know how)
Have anyone used this card with passthrough or have some intuition if it should work ? :)
Thanks a lot,
Rodrigo
Quoted 11 lines · Alex Williamson
Quoted 9 lines · Rodrigo Campos
Hi!
I wanted to know the status of PCI device assignment.
As far as I can see in the webpage and in the mailing list, it seems to be working ok if you have VT-d support on the motherboard and cpu. But if it isn't too much trouble, I wanted some confirmation about this, since I'm not sure and I don't want to buy hardware to test this when there is no way it's going to work :)
Quoted 3 lines
Yes, it works if you have VT-d support (Intel) or AMD IOMMU (note this is different than the AMD GART that's often used as an IOMMU). The Intel boxes are a lot easier to find.
For an AMD IOMMU you just need to buy a Mainboard with the AMD 890FX chipset. As far as I know all available boards support IOMMU with the latest BIOS.
On the server-side, look for an AMD SR56x0 chipset [where x=(5|7|9)].
Joerg
Quoted 12 lines · Joerg Roedel
Quoted 11 lines · Alex Williamson
Quoted 9 lines · Rodrigo Campos
Hi!
I wanted to know the status of PCI device assignment.
As far as I can see in the webpage and in the mailing list, it seems to be working ok if you have VT-d support on the motherboard and cpu. But if it isn't too much trouble, I wanted some confirmation about this, since I'm not sure and I don't want to buy hardware to test this when there is no way it's going to work :)
Quoted 8 lines
Quoted 3 lines
Yes, it works if you have VT-d support (Intel) or AMD IOMMU (note this is different than the AMD GART that's often used as an IOMMU). The Intel boxes are a lot easier to find.
For an AMD IOMMU you just need to buy a Mainboard with the AMD 890FX chipset. As far as I know all available boards support IOMMU with the latest BIOS.
On the server-side, look for an AMD SR56x0 chipset [where x=(5|7|9)].
Great, thanks a lot!
2010/9/3 Rodrigo Campos <[email protected]>:
Quoted 9 lines
Hi!
I wanted to know the status of PCI device assignment.
As far as I can see in the webpage and in the mailing list, it seems to be working ok if you have VT-d support on the motherboard and cpu. But if it isn't too much trouble, I wanted some confirmation about this, since I'm not sure and I don't want to buy hardware to test this when there is no way it's going to work :)
It highly depends on what you want to passthrough...if it's some well-known SR-IOV server NIC, then sure, it will probably work. If you want to passthrough various PCI devices in a regular desktop system or workstation, then forget it.
I've been playing that "various PCI devices in a regular desktop system" game for the last 8 months on my HTPC. I've been running with both stable versions of KVM as well as self-compiled versions from git during this time. The tests I have performed were done on two different VT-d capable boards (a Gigabyte EQ45M-S2 and a Intel DQ57TM). I've tried to passthrough 2 PCI TV tuners (Hauppauge 500 + Hauppauge 1300), 2 PCI USB 2.0 cards (can't remember the brands), 1 onboard USB controller (on a Gigabyte EQ45M-S2), and finally a PCI Express USB 3.0 card (Asrock). The only device which works correctly, is the Hauppauge 500 card. I bought the USB 3.0 card ONLY to see if it would help with a PCI Express card instead of the regular PCI cards I had tested...but no, it didn't change anything. In my tests the cards "almost works" in some cases - eg. the card gets correctly initialized in the guest and you can start to use it, but then you'll have timing issues, driver crashes, client program crashes, etc. etc.
Right now I'm awaiting RHEL 6 to see if passthrough has been improved here. If not, then as much as I would hate it, I'll switch back to Xen...people could passthrough all kinds of weird PCI cards 5 years ago with paravirtualization on Xen. Even though I see Xen as a dying platform, at least it works.
Best Regards
Kenni
Quoted 11 lines · Kenni Lund
2010/9/3 Rodrigo Campos <[email protected]>:
Quoted 9 lines
Hi!
I wanted to know the status of PCI device assignment.
As far as I can see in the webpage and in the mailing list, it seems to be working ok if you have VT-d support on the motherboard and cpu. But if it isn't too much trouble, I wanted some confirmation about this, since I'm not sure and I don't want to buy hardware to test this when there is no way it's going to work :)
Quoted 4 lines
It highly depends on what you want to passthrough...if it's some well-known SR-IOV server NIC, then sure, it will probably work. If you want to passthrough various PCI devices in a regular desktop system or workstation, then forget it.
It's none of this. It's an X.25 card (there are more details on the mail I've just sent to Alex)
Quoted 7 lines
I've been playing that "various PCI devices in a regular desktop system" game for the last 8 months on my HTPC. I've been running with both stable versions of KVM as well as self-compiled versions from git during this time. The tests I have performed were done on two different VT-d capable boards (a Gigabyte EQ45M-S2 and a Intel DQ57TM). I've tried to passthrough 2 PCI TV tuners (Hauppauge 500 + Hauppauge 1300), 2 PCI USB 2.0 cards (can't remember the brands), 1
Quoted 8 lines
onboard USB controller (on a Gigabyte EQ45M-S2), and finally a PCI Express USB 3.0 card (Asrock). The only device which works correctly, is the Hauppauge 500 card. I bought the USB 3.0 card ONLY to see if it would help with a PCI Express card instead of the regular PCI cards I had tested...but no, it didn't change anything. In my tests the cards "almost works" in some cases - eg. the card gets correctly initialized in the guest and you can start to use it, but then you'll have timing issues, driver crashes, client program crashes, etc. etc.
Ahhh, thanks for sharing your experience. I guess I will have to try then :)
Thanks,
Rodrigo
Quoted 1 line · On Fri, 3 Sep 2010 19:54:00 +0400 Rodrigo Campos <[email protected]> wrote:
I wanted to know the status of PCI device assignment.
I successfully use kvm for assignment second GPU in my notebook (Thinkpad T500) into guest OS, two months ago. But it require several hacks for correct vga-bios passthrough and it work only for linux guests -- all windows guests hung whole system at early boot. Currently I haven't enough free time to finish this investigation.
Quoted 10 lines · Konstantin Khlebnikov
Quoted 1 line · On Fri, 3 Sep 2010 19:54:00 +0400 Rodrigo Campos <[email protected]> wrote:
I wanted to know the status of PCI device assignment.
I successfully use kvm for assignment second GPU in my notebook (Thinkpad T500) into guest OS, two months ago. But it require several hacks for correct vga-bios passthrough and it work only for linux guests -- all windows guests hung whole system at early boot. Currently I haven't enough free time to finish this investigation.
Thanks, its good to know. Luckily I don't need to assign a second GPU, so I won't need those hacks :)
Thanks a lot,
Rodrigo