virtio_pci: Add SR-IOV support
Subsystems (4)
RFC PATCH v2 1/1 virtio_pci: Add SR-IOV support
Hardware-realized virtio-pci devices can implement SR-IOV, so this patch enables its use. The device in question is an upcoming Intel NIC that implements both a virtio-net PF and virtio-net VFs. These are hardware realizations of what has been up to now been a software interface.
The device in question has the following 4-part PCI IDs:
PF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 15fe
VF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 05fe
The patch needs no check for device ID, because the callback will never be made for devices that do not assert the capability or when run on a platform incapable of SR-IOV.
One reason for this patch is because the hardware requires the vendor ID of a VF to be the same as the vendor ID of the PF that created it. So it seemed logical to simply have a fully-functioning virtio-net PF create the VFs. This patch makes that possible.
---
Changes in V2:
- Simplified logic from previous version, removed added driver variable
- Disable SR-IOV on driver removal excapt when VFs are assigned
- Sent as RFC to virtio-dev, linux-pci, netdev, lkml and others
--- drivers/virtio/virtio_pci_common.c | 47 ++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
drivers/virtio/virtio_pci_common.c +47 -0
@@ -572,6 +572,47 @@ static int virtio_pci_probe(struct pci_dev *pci_dev,return rc; } +#ifdef CONFIG_PCI_IOV +static int virtio_pci_sriov_disable (struct pci_dev *pci_dev) +{ +/* If vfs are assigned we cannot shut down SR-IOV without causing +* issues, so just leave the hardware available. +*/ +if (pci_vfs_assigned (pci_dev)) { +dev_warn (&pci_dev->dev, +"Unloading driver while VFs are assigned - VFs will not be deallocated\n" ); +return -EPERM ; + } +pci_disable_sriov (pci_dev); +return 0 ; +} + +static int virtio_pci_sriov_enable (struct pci_dev *pci_dev,int num_vfs) +{ +int rc =0 ; + +if (pci_num_vf (pci_dev)) +return -EINVAL ; + + rc =pci_enable_sriov (pci_dev, num_vfs); +if (rc) { +dev_warn (&pci_dev->dev,"Failed to enable PCI sriov: %d\n" , rc); +return rc; + } +dev_info (&pci_dev->dev,"SR-IOV enabled with %d VFs\n" , num_vfs); +return num_vfs; +} + +static int virtio_pci_sriov_configure (struct pci_dev *dev,int num_vfs) +{ +if (num_vfs) +return virtio_pci_sriov_enable (dev, num_vfs); +if (!pci_num_vf (dev)) +return -EINVAL ; +return virtio_pci_sriov_disable (dev); +} +#endif /* CONFIG_PCI_IOV */ +static void virtio_pci_remove (struct pci_dev *pci_dev) {struct virtio_pci_device *vp_dev =pci_get_drvdata (pci_dev);
@@ -584,6 +625,9 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)else virtio_pci_modern_remove (vp_dev); +#ifdef CONFIG_PCI_IOV +virtio_pci_sriov_disable (pci_dev); +#endif pci_disable_device (pci_dev);put_device (dev); }
@@ -596,6 +640,9 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)#ifdef CONFIG_PM_SLEEP .driver.pm = &virtio_pci_pm_ops,#endif +#ifdef CONFIG_PCI_IOV + .sriov_configure = virtio_pci_sriov_configure, +#endif };module_pci_driver (virtio_pci_driver);
Can we move this into common code as a a generic_sriov_configure helper? Nothing is really virtio specific, and it seems like some other drivers could also use it, e.g. ena or nvme.
Quoted 5 lines · Christoph Hellwig
Can we move this into common code as a a
generic_sriov_configurehelper? Nothing is really virtio specific, and it seems like some other drivers could also use it, e.g. ena or nvme.
That seems like a good idea to me, especially if PCI developers concur.
Signature · 1 line
Mark Rustad, Networking Division, Intel Corporation
Quoted 10 lines · Mark Rustad
Hardware-realized virtio-pci devices can implement SR-IOV, so this patch enables its use. The device in question is an upcoming Intel NIC that implements both a virtio-net PF and virtio-net VFs. These are hardware realizations of what has been up to now been a software interface.
Quoted 4 lines
The device in question has the following 4-part PCI IDs:
PF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 15fe
VF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 05fe
Small mistake in the commit message. Red Hat (Qumranet) vendor ID is 1af4, virtio-net device ID is 1041. Should be:
PF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 15fe
VF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 05fe
Best regards,
Yan.
Quoted 99 lines
The patch needs no check for device ID, because the callback will never be made for devices that do not assert the capability or when run on a platform incapable of SR-IOV.
One reason for this patch is because the hardware requires the vendor ID of a VF to be the same as the vendor ID of the PF that created it. So it seemed logical to simply have a fully-functioning virtio-net PF create the VFs. This patch makes that possible.
Signed-off-by Mark Rustad <[email protected]>Reviewed-by Alexander Duyck <[email protected]>---
Changes in V2:
- Simplified logic from previous version, removed added driver variable
- Disable SR-IOV on driver removal excapt when VFs are assigned
- Sent as RFC to virtio-dev, linux-pci, netdev, lkml and others
---drivers/virtio/virtio_pci_common.c| 47 ++++++++++++++++++++++++++++++++++++1 file changed, 47 insertions(+)drivers/virtio/virtio_pci_common.c +47
-1@@ -572,6 +572,47 @@ static int virtio_pci_probe(struct pci_dev *pci_dev,return rc; } +#ifdef CONFIG_PCI_IOV +static int virtio_pci_sriov_disable (struct pci_dev *pci_dev) +{ +/* If vfs are assigned we cannot shut down SR-IOV without causing +* issues, so just leave the hardware available. +*/ +if (pci_vfs_assigned (pci_dev)) { +dev_warn (&pci_dev->dev, +"Unloading driver while VFs are assigned - VFs will not be deallocated\n" ); +return -EPERM ; + } +pci_disable_sriov (pci_dev); +return 0 ; +} + +static int virtio_pci_sriov_enable (struct pci_dev *pci_dev,int num_vfs) +{ +int rc =0 ; + +if (pci_num_vf (pci_dev)) +return -EINVAL ; + + rc =pci_enable_sriov (pci_dev, num_vfs); +if (rc) { +dev_warn (&pci_dev->dev,"Failed to enable PCI sriov: %d\n" , rc); +return rc; + } +dev_info (&pci_dev->dev,"SR-IOV enabled with %d VFs\n" , num_vfs); +return num_vfs; +} + +static int virtio_pci_sriov_configure (struct pci_dev *dev,int num_vfs) +{ +if (num_vfs) +return virtio_pci_sriov_enable (dev, num_vfs); +if (!pci_num_vf (dev)) +return -EINVAL ; +return virtio_pci_sriov_disable (dev); +} +#endif /* CONFIG_PCI_IOV */ +static void virtio_pci_remove (struct pci_dev *pci_dev) {struct virtio_pci_device *vp_dev =pci_get_drvdata (pci_dev);@@ -584,6 +625,9 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)else virtio_pci_modern_remove (vp_dev); +#ifdef CONFIG_PCI_IOV +virtio_pci_sriov_disable (pci_dev); +#endif pci_disable_device (pci_dev);put_device (dev); }@@ -596,6 +640,9 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)#ifdef CONFIG_PM_SLEEP .driver.pm = &virtio_pci_pm_ops,#endif +#ifdef CONFIG_PCI_IOV + .sriov_configure = virtio_pci_sriov_configure, +#endif };module_pci_driver (virtio_pci_driver); ---------------------------------------------------------------------To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 6 lines · Yan Vugenfirer
Small mistake in the commit message. Red Hat (Qumranet) vendor ID is 1af4, virtio-net device ID is 1041. Should be:
PF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 15fe
VF: vendor: 1af4 device: 1041 subvendor: 8086 subdevice: 05fe
You are so right. I will fix it before I send the updated patch that provides the generic helper that Christoph suggested. Thanks for catching my mislabeling.
--
Mark Rustad, [email protected]