Device assignment, shared IRQs, uio_pci_generic
Hi all,
trying to pass some host device to a KVM guest (for driver hacking), I ran into massive IRQ sharing issues on the host. Classic device assignment refuses to work in this case. I remembered uio_pci_generic, which I thought was once designed to resolve this conflict. But it isn't used by KVM or qemu, is it?
So I crossed fingers and hacked IRQF_SHARED into assigned_device_enable_host_intx. Effect as expected: assignment work, host gets massive hick-ups (the graphic adapter is another user of this line...).
Are there already VFIO patches ready for testing that work with shared IRQs as well? And/or would it make sense to integrate IRQ sharing via PCI-level IRQ masking to the KVM core until VFIO becomes standard?
Jan
Quoted 7 lines · Jan Kiszka
Hi all,
trying to pass some host device to a KVM guest (for driver hacking), I ran into massive IRQ sharing issues on the host. Classic device assignment refuses to work in this case. I remembered
uio_pci_generic, which I thought was once designed to resolve this conflict. But it isn't used by KVM or qemu, is it?
Nope, VFIO started out being based on it, but has since branched off. KVM has never made use of it.
Quoted 8 lines
So I crossed fingers and hacked
IRQF_SHAREDintoassigned_device_enable_host_intx. Effect as expected: assignment work, host gets massive hick-ups (the graphic adapter is another user of this line...).Are there already VFIO patches ready for testing that work with shared IRQs as well? And/or would it make sense to integrate IRQ sharing via PCI-level IRQ masking to the KVM core until VFIO becomes standard?
There's a very old qemu VFIO patch floating around, but I plan to send out a new one this week, thus all the pre-patches I've been sending out. Adding PCI 2.3 interrupt disable support to KVM should be very possible, and I suspect fairly easy. Oddly, I've just been adding support for non-PCI 2.3 devices to VFIO so it can support all the old devices that KVM does already.
Alex
Quoted 21 lines · Alex Williamson
Quoted 7 lines · Jan Kiszka
Hi all,
trying to pass some host device to a KVM guest (for driver hacking), I ran into massive IRQ sharing issues on the host. Classic device assignment refuses to work in this case. I remembered
uio_pci_generic, which I thought was once designed to resolve this conflict. But it isn't used by KVM or qemu, is it?Nope, VFIO started out being based on it, but has since branched off. KVM has never made use of it.
Quoted 8 lines
So I crossed fingers and hacked
IRQF_SHAREDintoassigned_device_enable_host_intx. Effect as expected: assignment work, host gets massive hick-ups (the graphic adapter is another user of this line...).Are there already VFIO patches ready for testing that work with shared IRQs as well? And/or would it make sense to integrate IRQ sharing via PCI-level IRQ masking to the KVM core until VFIO becomes standard?
Quoted 6 lines
There's a very old qemu VFIO patch floating around, but I plan to send out a new one this week, thus all the pre-patches I've been sending out. Adding PCI 2.3 interrupt disable support to KVM should be very possible, and I suspect fairly easy. Oddly, I've just been adding support for non-PCI 2.3 devices to VFIO so it can support all the old devices that KVM does already.
It wasn't that easy, and I'm still not 100% sure I did it all correctly:
I just passed my SD card reader to a Linux guest which shares IRQ with five(!) other devices. That works well now (provided I disable SMIs - device pass-through appears to be another trigger for my SMI-related lockups under KVM, sigh).
Still not working here are ehci, ath9k and e1000e when passed through. They receive IRQs, but somehow the Linux guest drivers are unhappy about the device states (the e1000e detects "Hardware Unit Hang" e.g.). This is independent of my patches. Anyone any ideas? Improper configurations of the BIOS? I stumbled over this in the kernel log:
DMAR: BIOS has allocated no shadow GTT; disabling IOMMU for graphics
There were a few flaws in the current device assignment code /wrt host IRQ type selection, both kernel and qemu-kvm are affected. Will try to sort all my stuff next to send out patches.
Jan
Quoted 8 lines · Jan Kiszka
Still not working here are ehci, ath9k and e1000e when passed through. They receive IRQs, but somehow the Linux guest drivers are unhappy about the device states (the e1000e detects "Hardware Unit Hang" e.g.). This is independent of my patches. Anyone any ideas? Improper configurations of the BIOS? I stumbled over this in the kernel log:
DMAR: BIOS has allocated no shadow GTT; disabling IOMMU for graphics
Argh, no one told me that there is CONFIG_DMAR_DEFAULT_ON and that you have to pass "intel_iommu=on" if that config switch is off like here. Moreover, I somehow assumed "iommu=0" on the command line means "use IOMMU with index 0" (don't ask me why). Usability deserves some patches
- later.
So it's starting to work, already accessed some devices (ath9k did a successful scan), but now it's crashing in intel_iommu_attach_device. Looks like use after release. Sigh.
Jan