KVM PCI passthrough IRQ limitations

Hello,

I am attempting to make use of '-pcidevice' on my system, but have run into some problems. First, my setup:

Fedora 10 x86_64 host system
2.6.28.1-19.fc10 kernel from Koji
KVM-84
Debian Lenny guest VM

I am attempting to passthrough a Xilinx FPGA device to the Lenny guest.
No driver for this device exists on the host.

% /usr/local/kvm/bin/qemu-system-x86_64 ./vdisk.img -m 384 -pcidevice host=02:00.0
init_assigned_device: Registering real physical device 02:00.0 (bus=2 dev=0 func=0)
get_real_device: region 0 size 1024 start 0xe1200000 type 512 resource_fd 12 Failed to assign irq for "02:00.0": Input/output error
Perhaps you are assigning a device that shares an IRQ with another device?

When the Fedora host was first booted, the Xilinx was assigned IRQ 10, which appeared to be unshared:

02:00.0 Memory controller: Xilinx Corporation Device 0007
Subsystem: Xilinx Corporation Device 0007
Flags: bus master, fast devsel, latency 0, IRQ 10 Memory at e1200000 (64-bit, non-prefetchable) [size=1K]
Capabilities: <access denied>

However, after executing qemu-system-x86_64, lspci showed the Xilinx assigned to IRQ 16, which is shared with a SATA controller:

02:00.0 Memory controller: Xilinx Corporation Device 0007
Subsystem: Xilinx Corporation Device 0007
Flags: fast devsel, IRQ 16
Memory at e1200000 (64-bit, non-prefetchable) [size=1K]
Capabilities: [40] Power Management version 3
Capabilities: [48] Message Signalled Interrupts: Mask+ 64bit+
Count=1/1 Enable-
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number XX-XX-XX-XX-XX-XX-XX-XX

Moving the card to a different slot gave the same results.

My hope is to eventually have two Xilinx FPGA cards in the same system, and to passthrough each card to its own VM instance. Would such a scheme be possible? Is there a way around the IRQ assignment failure?

Thanks,

Jason

You mean both userspace and kernel module built externally?

any info in dmesg? what's lspci -t show?

Yes, I built both from the kvm-84.tar.gz package.

dmesg gives:

pci 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 IRQ handler type mismatch for IRQ 16
current handler: uhci_hcd:usb3
Pid: 3597, comm: qemu-system-x86 Not tainted 2.6.28.1-19.fc10.x86_64 #1

Call Trace:
 [<ffffffff810891f6>] __setup_irq+0x1f4/0x227
 [<ffffffffa01bc917>] ? kvm_assigned_dev_intr+0x0/0x2c [kvm]
 [<ffffffff810892fc>] request_irq+0xd3/0x15a
 [<ffffffffa01bc9d2>] assigned_device_update_intx+0x8f/0xa9 [kvm]
 [<ffffffffa01be711>] kvm_vm_ioctl+0x6af/0x79c [kvm]
 [<ffffffff8114d9bf>] ? inode_has_perm+0x5b/0x61
 [<ffffffff81051241>] ? __sigqueue_free+0x38/0x3c
 [<ffffffff81051f39>] ? __dequeue_signal+0x103/0x12f
 [<ffffffff810d82ec>] vfs_ioctl+0x2a/0x78
 [<ffffffff810d86cc>] do_vfs_ioctl+0x392/0x3c0
 [<ffffffff810d874f>] sys_ioctl+0x55/0x77
 [<ffffffff8101106a>] system_call_fastpath+0x16/0x1b

kvm: failed to enable INTx device!
pci 0000:02:00.0: PCI INT A disabled

lspci -t shows:

-[0000:00]-+-00.0

           +-01.0-[0000:01]--
           +-03.0-[0000:02]----00.0
           +-07.0-[0000:03]--
           +-10.0
           +-10.1
           +-14.0
           +-14.1
           +-14.2
           +-14.3
           +-19.0
           +-1a.0
           +-1a.1
           +-1a.2
           +-1a.7
           +-1b.0
           +-1c.0-[0000:04]--
           +-1c.1-[0000:05]--
           +-1c.4-[0000:06]----00.0
           +-1d.0
           +-1d.1
           +-1d.2
           +-1d.7
           +-1e.0-[0000:07]--+-02.0
           |                 \-03.0
           +-1f.0
           +-1f.2
           +-1f.3
           \-1f.5

Thanks,

Jason

This is a shared interrupt (and pci device assignment will request a non-shared interrupt). Are you using that usb host controller? Can you unbind it?

OK, only the one device behind this bridge

I attempted unbinding the usb host driver, but the IRQ was/is also shared by the SATA drive controller, so I'm stuck. I tried moving the card to a different slot but the IRQ assignments were the same.

Does it make any difference whether the device is MSI-capable or not?

Thanks,

Jason

Well, does it work with msi2intx=0 kvm module parameter?

If the device support MSI, msi2intx=1(default value) would resolve the sharing interrupt issue. So seems the device didn't support MSI...

regards
Yang, Sheng