hw/virtio: Add PCIe capability to virtio devices
PATCH v5 1/1 hw/virtio: Add PCIe capability to virtio devices
The virtio devices are converted to PCI-Express if they are plugged into a PCI-Express bus and the 'modern' protocol is enabled.
Devices plugged directly into the Root Complex as Integrated Endpoints remain PCI.
---
v4 -> v5:
- Addressed Michael S. Tsirkin's comments:
- Renamed disable-pcie => x-disable-pcie
- Rebased on master
v3 -> v4:
- Addressed Eduardo Habkost's comments:
- used a single virtio-pci.disable-pcie=on entry for HW_COMPAT, instead of one entry for each subclass
v2 -> v3:
- Addressed Michael S. Tsirkin's comments:
- enable pcie only for 2.5+ machines.
v1 -> v2:
- Addressed Michael S. Tsirkin's comments:
- Added the minimum required capabilities for PCIe devices
- Integrated Endpoints remain PCI
- Use pcie_endpoint_cap_init instead of manually creating the pcie capability.
- Regarding Gerd Hoffman's comments:
- Creating virtio-pcie devices:
For the moment I prefer to not duplicate the virtio definitions, at least until we don't have a consensus (Personally I don't like it)
- Removing the IO bar:
This would be my next patch on the "virtio to express" series, I plan to remove it only for "modern" devices.
Thanks,
Marcel
hw/virtio/virtio-pci.c | 22 ++++++++++++++++++++++ hw/virtio/virtio-pci.h | 2 ++ include/hw/compat.h | 4 ++++ 3 files changed, 28 insertions(+)
hw/virtio/virtio-pci.c +22 -0
@@ -1592,6 +1592,26 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp)address_space_init (&proxy->modern_as, &proxy->modern_cfg,"virtio-pci-cfg-as" ); +if (!(proxy->flags &VIRTIO_PCI_FLAG_DISABLE_PCIE ) + && !(proxy->flags &VIRTIO_PCI_FLAG_DISABLE_MODERN ) + &&pci_bus_is_express (pci_dev->bus) + && !pci_bus_is_root (pci_dev->bus)) { +int pos; + + pci_dev->cap_present |=QEMU_PCI_CAP_EXPRESS ; + pos =pcie_endpoint_cap_init (pci_dev,0 ); +assert (pos >0 ); + + pos =pci_add_capability (pci_dev,PCI_CAP_ID_PM ,0 ,PCI_PM_SIZEOF ); +assert (pos >0 ); + +/* +* Indicates that this function complies with revision 1.2 of the +* PCI Power Management Interface Specification. +*/ +pci_set_word (pci_dev->config + pos +PCI_PM_PMC ,0x3 ); + } +virtio_pci_bus_new (&proxy->bus,sizeof (proxy->bus), proxy);if (k->realize) { k->realize (proxy, errp);
@@ -1622,6 +1642,8 @@ static Property virtio_pci_properties[] = { VIRTIO_PCI_FLAG_DISABLE_LEGACY_BIT, false),DEFINE_PROP_BIT ("disable-modern" , VirtIOPCIProxy, flags,VIRTIO_PCI_FLAG_DISABLE_MODERN_BIT , true), +DEFINE_PROP_BIT ("x-disable-pcie" , VirtIOPCIProxy, flags, +VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT , false), DEFINE_PROP_END_OF_LIST(), };
hw/virtio/virtio-pci.h +2 -0
@@ -72,8 +72,10 @@ typedef struct VirtioBusClass VirtioPCIBusClass;/* virtio version flags */ #define VIRTIO_PCI_FLAG_DISABLE_LEGACY_BIT 2#define VIRTIO_PCI_FLAG_DISABLE_MODERN_BIT 3 +#define VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT 4#define VIRTIO_PCI_FLAG_DISABLE_LEGACY (1 << VIRTIO_PCI_FLAG_DISABLE_LEGACY_BIT)#define VIRTIO_PCI_FLAG_DISABLE_MODERN (1 << VIRTIO_PCI_FLAG_DISABLE_MODERN_BIT) +#define VIRTIO_PCI_FLAG_DISABLE_PCIE (1 << VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT) typedef struct { MSIMessage msg;
include/hw/compat.h +4 -0
@@ -6,6 +6,10 @@ .driver = "virtio-blk-device",\ .property = "scsi",\ .value = "true",\ + },{\ + .driver = "virtio-pci",\ + .property = "x-disable-pcie",\ + .value = "on",\ },#define HW_COMPAT_2_3 \
Signature · 1 line
2.1.0
Hi,
hw/virtio/virtio-pci.c static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) · line 1599
Quoted 13 lines · On Tue, 10 Nov 2015 13:41:29 +0200, [email protected] wrote:
The virtio devices are converted to PCI-Express
if they are plugged into a PCI-Express bus and
the 'modern' protocol is enabled.hw/virtio/virtio-pci.c +7
-0@@ -1592,6 +1592,26 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) +if (!(proxy->flags &VIRTIO_PCI_FLAG_DISABLE_PCIE ) + && !(proxy->flags &VIRTIO_PCI_FLAG_DISABLE_MODERN ) + &&pci_bus_is_express (pci_dev->bus) + && !pci_bus_is_root (pci_dev->bus)) { +int pos; + + pci_dev->cap_present |=QEMU_PCI_CAP_EXPRESS ;
Setting QEMU_PCI_CAP_EXPRESS here in 'virtio_pci_realize' is too late.
This is since 'pci_qdev_realize' (DeviceClass.realize of TYPE_PCI_DEVICE) is invoked prior the PCIDeviceClass's specific 'realize' method (virtio_pci_realize in this case).
During 'pci_qdev_realize' (specifically, within do_pci_register_device), the QEMU_PCI_CAP_EXPRESS gets tested indirectly, when pci_is_express and pci_config_size helpers are called.
For example: 'pci_config_alloc' uses 'pci_config_size' which relies on QEMU_PCI_CAP_EXPRESS property.
Since virtio_pci sets QEMU_PCI_CAP_EXPRESS *after* pci_qdev_realize has finished, we end up having an insufficient pci config space allocated for the virtio "pcie" device.
May I suggest the following:
- Expose 'pci_qdev_realize'
- Have 'virtio_pci_class_init' arm it's own dc->realize, which will first set 'QEMU_PCI_CAP_EXPRESS' flag as needed, and then call 'pci_qdev_realize'
- Now, in 'virtio_pci_realize' we may use 'pci_is_express' instead of directly checking the proxy->flags
If this sounds ok, I'll submit a fix.
Regards,
Shmulik
Quoted 27 lines · Shmulik Ladkani
Hi,
hw/virtio/virtio-pci.c static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) · line 1599
Quoted 13 lines · On Tue, 10 Nov 2015 13:41:29 +0200, [email protected] wrote:
The virtio devices are converted to PCI-Express
if they are plugged into a PCI-Express bus and
the 'modern' protocol is enabled.hw/virtio/virtio-pci.c +7
-0@@ -1592,6 +1592,26 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) +if (!(proxy->flags &VIRTIO_PCI_FLAG_DISABLE_PCIE ) + && !(proxy->flags &VIRTIO_PCI_FLAG_DISABLE_MODERN ) + &&pci_bus_is_express (pci_dev->bus) + && !pci_bus_is_root (pci_dev->bus)) { +int pos; + + pci_dev->cap_present |=QEMU_PCI_CAP_EXPRESS ;Setting
QEMU_PCI_CAP_EXPRESShere in 'virtio_pci_realize' is too late.This is since '
pci_qdev_realize' (DeviceClass.realize ofTYPE_PCI_DEVICE) is invoked prior the PCIDeviceClass's specific 'realize' method (virtio_pci_realizein this case).During '
pci_qdev_realize' (specifically, withindo_pci_register_device), theQEMU_PCI_CAP_EXPRESSgets tested indirectly, whenpci_is_expressandpci_config_sizehelpers are called.
Quoted 6 lines
For example: '
pci_config_alloc' uses 'pci_config_size' which relies onQEMU_PCI_CAP_EXPRESSproperty.Since
virtio_pcisetsQEMU_PCI_CAP_EXPRESS*after*pci_qdev_realizehas finished, we end up having an insufficient pci config space allocated for the virtio "pcie" device.
Hi,
Thanks for catching it. It is a good time for a fix since we don't have yet an actual PCIe express capability.
Quoted 10 lines
May I suggest the following:
- Expose 'pci_qdev_realize'
- Have 'virtio_pci_class_init' arm it's owndc->realize, which will first set 'QEMU_PCI_CAP_EXPRESS' flag as needed, and then call 'pci_qdev_realize'
- Now, in 'virtio_pci_realize' we may use 'pci_is_express' instead of directly checking theproxy->flagsIf this sounds ok, I'll submit a fix.
Give it a try, sure, but you don't need to expose pci_qdev_realize, you can 'hijack' parent's realize method before replacing it.
Thanks,
Marcel
Quoted 4 lines
Regards,
Shmulik
Hi,
Quoted 8 lines · On Tue, 1 Dec 2015 13:48:13 +0200, [email protected] wrote:
Quoted 8 lines
May I suggest the following:
- Expose 'pci_qdev_realize'
- Have 'virtio_pci_class_init' arm it's owndc->realize, which will first set 'QEMU_PCI_CAP_EXPRESS' flag as needed, and then call 'pci_qdev_realize'
- Now, in 'virtio_pci_realize' we may use 'pci_is_express' instead of directly checking theproxy->flags
Quoted 4 lines
Quoted 1 line
If this sounds ok, I'll submit a fix.
Give it a try, sure, but you don't need to expose
pci_qdev_realize, you can 'hijack' parent's realize method before replacing it.
Actually I caught this as I needed to do something very similar to vmxnet3.
Since we need this in 2 places, I thought it would be better to expose pci_qdev_realize (maybe name it better) and not store parent's original realize before replacing it.
WDYT?
Quoted 16 lines · Shmulik Ladkani
Hi,
Quoted 8 lines · On Tue, 1 Dec 2015 13:48:13 +0200, [email protected] wrote:
Quoted 8 lines
May I suggest the following:
- Expose 'pci_qdev_realize'
- Have 'virtio_pci_class_init' arm it's owndc->realize, which will first set 'QEMU_PCI_CAP_EXPRESS' flag as needed, and then call 'pci_qdev_realize'
- Now, in 'virtio_pci_realize' we may use 'pci_is_express' instead ofdirectly checking the proxy->flagsQuoted 4 lines
Quoted 1 line
If this sounds ok, I'll submit a fix.
Give it a try, sure, but you don't need to expose
pci_qdev_realize, you can 'hijack' parent's realize method before replacing it.
Quoted 9 lines
Actually I caught this as I needed to do something very similar to vmxnet3.
Since we need this in 2 places, I thought it would be better to expose
pci_qdev_realize(maybe name it better) and not store parent's original realize before replacing it.WDYT?
Personally I don't think pci_qdev_realize should be visible outside hw/pci/pci.c because "realize" should be private to object...
However I didn't see yet the vmxnet3 changes you propose. Maybe you can send an RFC and we'll check then.
Thanks,
Marcel
Hi,
Quoted 18 lines · On Tue, 1 Dec 2015 16:01:00 +0200, [email protected] wrote:
Quoted 17 lines · Shmulik Ladkani
Quoted 8 lines · On Tue, 1 Dec 2015 13:48:13 +0200, [email protected] wrote:
Quoted 8 lines
May I suggest the following:
- Expose 'pci_qdev_realize'
- Have 'virtio_pci_class_init' arm it's owndc->realize, which will first set 'QEMU_PCI_CAP_EXPRESS' flag as needed, and then call 'pci_qdev_realize'
- Now, in 'virtio_pci_realize' we may use 'pci_is_express' instead ofdirectly checking the proxy->flagsQuoted 4 lines
Quoted 1 line
If this sounds ok, I'll submit a fix.
Give it a try, sure, but you don't need to expose
pci_qdev_realize, you can 'hijack' parent's realize method before replacing it.Actually I caught this as I needed to do something very similar to vmxnet3.
Quoted 8 lines
Quoted 5 lines
Since we need this in 2 places, I thought it would be better to expose
pci_qdev_realize(maybe name it better) and not store parent's original realize before replacing it.WDYT?
Personally I don't think
pci_qdev_realizeshould be visible outsidehw/pci/pci.cbecause "realize" should be private to object...
Thanks I'll follow that path and submit a patch shortly.
Quoted 25 lines · Shmulik Ladkani
Hi,
Quoted 22 lines · On Tue, 1 Dec 2015 16:01:00 +0200, [email protected] wrote:
Quoted 21 lines · Shmulik Ladkani
Quoted 8 lines · On Tue, 1 Dec 2015 13:48:13 +0200, [email protected] wrote:
Quoted 8 lines
May I suggest the following:
- Expose 'pci_qdev_realize'
- Have 'virtio_pci_class_init' arm it's owndc->realize, which will first set 'QEMU_PCI_CAP_EXPRESS' flag as needed, and then call 'pci_qdev_realize'
- Now, in 'virtio_pci_realize' we may use 'pci_is_express' instead ofdirectly checking the proxy->flagsQuoted 4 lines
Quoted 1 line
If this sounds ok, I'll submit a fix.
Give it a try, sure, but you don't need to expose
pci_qdev_realize, you can 'hijack' parent's realize method before replacing it.Actually I caught this as I needed to do something very similar to vmxnet3.
Since we need this in 2 places, I thought it would be better to expose
pci_qdev_realize(maybe name it better) and not store parent's original realize before replacing it.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
WDYT?
Personally I don't think
pci_qdev_realizeshould be visible outsidehw/pci/pci.cbecause "realize" should be private to object...Thanks I'll follow that path and submit a patch shortly.
Thanks and please post the fix before our next rc.
If you don't have the time, please let me know and I'll submit a fix.
Thanks again!
Marcel