PCI-passthrough - issues/questions/ideas

DISCUSSION Patrick Ringl 2 participants 2 messages Sep 10, 2011 – Sep 23, 2011 KVM

Share & tools

Patrick Ringl

author

2011-09-10 09:22 UTC #0 lore raw

Hi,

I just wanted to introduce a problem I currently face including some questions regarding my temporary fix. Anyway, I have a PCI-device that I want to passthrough to a hvm guest. Now there are several problems that add up:

a) the PCI-device is bound to a PCI-to-PCI bridge (which in turn is directly attached to the rootbus) (mainboard has a AMD 970/SB950 chipset). [since pciIsParent shows that the secondary bus equals the device's bus]:

bridge:

Quoted 2 lines

lspci -s00:14.4 -vvv | grep Bus:
Bus: primary=00, secondary=07, subordinate=07, sec-latency=64

PCI-device

Quoted 1 line

07:06.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)

b) neither the bridge nor the PCI device itself have the currently implemented reset functionality that you trigger in pciResetDevice

c) the PCI-device is mapped through the PCI bridge (IOMMU-wise):

ACPI IOMMU dump:

Quoted 3 lines

[    1.121239] AMD-Vi:   DEV_SELECT             devid: 00:14.4 flags: 00
[    1.121274] AMD-Vi:   DEV_ALIAS_RANGE         devid: 07:00.0 flags: 00 devid_to: 00:14.4
[    1.121311] AMD-Vi:   DEV_RANGE_END         devid: 07:1f.7

What I did to get (temporarily and in a rather hackish (maybe even wrong) manner) rid of the problem, is to ignore the error thrown in pciResetDevice when no reset had been possible at all.

    if (ret < 0) {
/*
    -- I know what you did last summer!

        virErrorPtr err = virGetLastError();
        pciReportError(VIR_ERR_INTERNAL_ERROR,
                       _("Unable to reset PCI device %s: %s"),
                       dev->name,
                       err ? err->message : _("no FLR, PM reset or bus 
reset available"));
*/
        ret = 0;
    }

Concludingly I'd ask the following questions:

a) Why is a secondary_bus_reset a bad idea if the "device in question"'s _primary_ bus is the root bus?
b) Why would it be a bad idea adding some sort of 'override attribute' to the guest's config, so libvirt may intentionally skip the reset? What are the possible consequences if no reset takes place at all?
c) What options do I have besides implementing c) or just going the dirty way and ignore the case when no reset is possible (like I described above)?

PS : Since I am not subscribed, please CC to me
PPS: Please cut me slack ;)

regards,
Patrick

Chris Wright

2011-09-23 19:11 UTC #1 lore raw

Quoted 50 lines · * Patrick Ringl ([email protected]) wrote:

Hi,
I just wanted to introduce a problem I currently face including some questions regarding my temporary fix.
Anyway, I have a PCI-device that I want to passthrough to a hvm guest. Now there are several problems that add up:
a) the PCI-device is bound to a PCI-to-PCI bridge (which in turn is directly attached to the rootbus) (mainboard has a AMD 970/SB950 chipset).
[since pciIsParent shows that the secondary bus equals the device's bus]:
bridge:
Quoted 2 lines
lspci -s00:14.4 -vvv | grep Bus:
Bus: primary=00, secondary=07, subordinate=07, sec-latency=64
PCI-device
Quoted 1 line
07:06.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
b) neither the bridge nor the PCI device itself have the currently implemented reset functionality that you trigger in pciResetDevice
c) the PCI-device is mapped through the PCI bridge (IOMMU-wise):
ACPI IOMMU dump:
Quoted 2 lines
[    1.121239] AMD-Vi:   DEV_SELECT             devid: 00:14.4 flags: 00
[    1.121274] AMD-Vi:   DEV_ALIAS_RANGE         devid: 07:00.0
flags: 00 devid_to: 00:14.4
Quoted 1 line
[    1.121311] AMD-Vi:   DEV_RANGE_END         devid: 07:1f.7
What I did to get (temporarily and in a rather hackish (maybe even wrong) manner) rid of the problem, is to ignore the error thrown in pciResetDevice when no reset had been possible at all.
    if (ret < 0) {
/*
    -- I know what you did last summer!

        virErrorPtr err = virGetLastError();
        pciReportError(VIR_ERR_INTERNAL_ERROR,
                       _("Unable to reset PCI device %s: %s"),
                       dev->name,
                       err ? err->message : _("no FLR, PM reset or
bus reset available"));
*/
        ret = 0;
    }

Quoted 4 lines

Concludingly I'd ask the following questions:
a) Why is a secondary_bus_reset a bad idea if the "device in question"'s _primary_ bus is the root bus?

That's not the issue the code is guarding against. It is guarding against issuing a secondary bus reset on the root bus. Your dev->bus should be 7, not 0. However, the part that should be failing is pciTrySecondaryBusReset(). And this will fail if there are other devices on that bus (07) that are not assigned to your guest, because a secondary bus resest will reset _all_ devices on the secondary bus.

Quoted 4 lines

b) Why would it be a bad idea adding some sort of 'override attribute' to the guest's config, so libvirt may intentionally skip the reset? What are the possible consequences if no reset takes place at all?

The problem with skipping the reset is primarily a security concern. Device state will leak between users of the device. An override is possible, you can discuss that with libvirt developers to see if they'd support an insecure flag like that.

Quoted 3 lines

c) What options do I have besides implementing c) or just going the dirty way and ignore the case when no reset is possible (like I described above)?

You can try assiging all devices on bus 7 to the guest. This should allow a sbus reset to be issued.

thanks,
-chris