virtio_pci: Add SR-IOV support
Subsystems (4)
RFC PATCH v3 1/1 virtio_pci: Add SR-IOV support
Hardware-realized virtio-pci devices can implement SR-IOV, so this patch enables its use. The device in question is an upcoming Intel NIC that implements both a virtio-net PF and virtio-net VFs. These are hardware realizations of what has been up to now been a software interface.
The device in question has the following 4-part PCI IDs:
PF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 15fe
VF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 05fe
The patch needs no check for device ID, because the callback will never be made for devices that do not assert the capability or when run on a platform incapable of SR-IOV.
One reason for this patch is because the hardware requires the vendor ID of a VF to be the same as the vendor ID of the PF that created it. So it seemed logical to simply have a fully-functioning virtio-net PF create the VFs. This patch makes that possible.
---
Changes in V3:
- Move most code to a new helper in pci/iov.c, pci_sriov_configure Changes in V2:
- Simplified logic from previous version, removed added driver variable
- Disable SR-IOV on driver removal excapt when VFs are assigned
- Sent as RFC to virtio-dev, linux-pci, netdev, lkml and others
--- drivers/pci/iov.c | 48 ++++++++++++++++++++++++++++++++++++ drivers/virtio/virtio_pci_common.c | 2 ++ include/linux/pci.h | 10 ++++++++ 3 files changed, 60 insertions(+)
drivers/pci/iov.c +48 -0
@@ -367,6 +367,54 @@ static void sriov_disable(struct pci_dev *dev)pci_iov_set_numvfs (dev,0 ); } +/** +* pci_sriov_disable - standard helper to disable SR-IOV +* @dev:the PCI PF device whose VFs are to be disabled +*/ +int pci_sriov_disable (struct pci_dev *dev) +{ +/* +* If vfs are assigned we cannot shut down SR-IOV without causing +* issues, so just leave the hardware available. +*/ +if (pci_vfs_assigned (dev)) { +pci_warn (&dev->dev, +"Cannot disable SR-IOV while VFs are assigned - VFs will not be deallocated\n" ); +return -EPERM ; + } +pci_disable_sriov (dev); +return 0 ; +} + +static int pci_sriov_enable (struct pci_dev *dev,int num_vfs) +{ +int rc; + +if (pci_num_vf (dev)) +return -EINVAL ; + + rc =pci_enable_sriov (dev, num_vfs); +if (rc) { +pci_warn (dev,"Failed to enable PCI sriov: %d\n" , rc); +return rc; + } +dev_info (dev,"SR-IOV enabled with %d VFs\n" , num_vfs); +return num_vfs; +} + +/** +* pci_sriov_configure - standard helper to configure SR-IOV +* @dev: the PCI PF device that is configuring SR-IOV +*/ +int pci_sriov_configure (struct pci_dev *dev,int num_vfs) +{ +if (num_vfs) +return pci_sriov_enable (dev, num_vfs); +if (!pci_num_vf (dev)) +return -EINVAL ; +return pci_sriov_disable (dev); +} +static int sriov_init (struct pci_dev *dev,int pos) {int i, bar64;
drivers/virtio/virtio_pci_common.c +2 -0
@@ -584,6 +584,7 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)else virtio_pci_modern_remove (vp_dev); +pci_sriov_disable (pci_dev);pci_disable_device (pci_dev);put_device (dev); }
@@ -596,6 +597,7 @@ static struct pci_driver virtio_pci_driver = {#ifdef CONFIG_PM_SLEEP .driver.pm = &virtio_pci_pm_ops, #endif + .sriov_configure = pci_sriov_configure, };module_pci_driver (virtio_pci_driver);
include/linux/pci.h +10 -0
@@ -1947,6 +1947,8 @@ int pci_iov_virtfn_devfn(struct pci_dev *dev, int id);int pci_enable_sriov (struct pci_dev *dev,int nr_virtfn);void pci_disable_sriov (struct pci_dev *dev); +int pci_sriov_disable (struct pci_dev *dev); +int pci_sriov_configure (struct pci_dev *dev,int num_vfs);int pci_iov_add_virtfn (struct pci_dev *dev,int id);void pci_iov_remove_virtfn (struct pci_dev *dev,int id);int pci_num_vf (struct pci_dev *dev);
@@ -1973,6 +1975,14 @@ static inline int pci_iov_add_virtfn(struct pci_dev *dev, int id)static inline void pci_iov_remove_virtfn (struct pci_dev *dev,int id) { }static inline void pci_disable_sriov (struct pci_dev *dev) { } +static inline int pci_sriov_disable (struct pci_dev *dev) +{ +return -ENODEV ; +} +static inline int pci_sriov_configure (struct pci_dev *dev,int num_vfs) +{ +return -ENODEV ; +}static inline int pci_num_vf (struct pci_dev *dev) {return 0 ; }static inline int pci_vfs_assigned (struct pci_dev *dev) {return 0 ; }
David Miller
maintainerFrom: Mark Rustad <[email protected]>
Date: Sun, 25 Feb 2018 19:19:11 -0800
Quoted 32 lines
drivers/pci/iov.c +24
-0@@ -367,6 +367,54 @@ static void sriov_disable(struct pci_dev *dev)pci_iov_set_numvfs (dev,0 ); } +/** +* pci_sriov_disable - standard helper to disable SR-IOV +* @dev:the PCI PF device whose VFs are to be disabled +*/ +int pci_sriov_disable (struct pci_dev *dev) +{ +/* +* If vfs are assigned we cannot shut down SR-IOV without causing +* issues, so just leave the hardware available. +*/ +if (pci_vfs_assigned (dev)) { +pci_warn (&dev->dev, +"Cannot disable SR-IOV while VFs are assigned - VFs will not be deallocated\n" ); +return -EPERM ; + } +pci_disable_sriov (dev); +return 0 ; +} + +static int pci_sriov_enable (struct pci_dev *dev,int num_vfs) +{ +int rc; + +if (pci_num_vf (dev))
drivers/pci/iov.c static void sriov_disable(struct pci_dev *dev) · line 403
Quoted 10 lines
drivers/pci/iov.c +10
-0+return -EINVAL ; + + rc =pci_enable_sriov (dev, num_vfs); +if (rc) { +pci_warn (dev,"Failed to enable PCI sriov: %d\n" , rc); +return rc; + } +dev_info (dev,"SR-IOV enabled with %d VFs\n" , num_vfs); +return num_vfs; +}
I don't like these helpers on many different levels.
The pci_num_vf() test in pci_sriov_enable() is redundant, the pci_enable_sriov() code path does that check and returns the same exact error code from sriov_enable().
Just call pci_enable_sriov() directly. The log message adds no value justifying an entirely new (and confusingly named) helper. If the log message is useful, add it to pci_enable_sriov().
Speaking of naming, is this stuff confusing or what? As a programmer what am I supposed to think when I consider what may be the difference between two interfaces, the only difference in naming is that two words are transposed?
pci_enable_sriov()
pci_sriov_enable()
pci_disable_sriov()
pci_sriov_disable()
?!?!?!?!
As per pci_sriov_disable() explicitly, all it does different is check for vf assignment and return failure.
If you want a little help that does that, name it appropriately.
pci_disable_sriov_if_unassigned()
So kill off pci_sriov_enable() helper completely, it is unnecessary, and rename the disable helper so that it says something meaningful to the reader.
Thanks.
Quoted 3 lines · David Miller
I don't like these helpers on many different levels.
<snip much embarrassment>
Quoted 3 lines
So kill off
pci_sriov_enable()helper completely, it is unnecessary, and rename the disable helper so that it says something meaningful to the reader.
Yes. Once pointed out, I completely agree with your comments and wish that I had seen those things myself.
Quoted 1 line
Thanks.
V3 was junk, but your comments apply to V4 as well, so please ignore it.
Thank you for your valuable review.
Signature · 1 line
Mark Rustad, Networking Division, Intel Corporation