Skip to content

RFC PCI: Add support for Scalable I/O Virtualization

Dimitri Daskalakis kvm linux-pci linux-s390 linuxppc-dev xen-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest12 patches 6 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/12PCI: Add helpers to identify SR-IOV PFs/VFs.include/linux/pci.h+12 -00 replies
2/12PCI: Convert iov.c to pci_is_sriov_* helpersdrivers/pci/iov.c+23 -230 replies
3/12PCI: Convert pci.h to pci_is_sriov_* helpersdrivers/pci/pci.h+1 -10 replies
4/12PCI: Convert arch/powerpc to pci_is_sriov_* helpersarch/powerpc/kernel/pci_dn.c arch/powerpc/platforms/powernv/pci-ioda.c arch/powerpc/platforms/powernv/pci-sriov.c +2 more+16 -160 replies
5/12PCI: Convert s390/pci/pci.c to pci_is_sriov_* helpersarch/s390/pci/pci.c+1 -10 replies
6/12PCI: Convert vfio_pci_core.c to pci_is_sriov_* helpersdrivers/vfio/pci/vfio_pci_core.c+6 -60 replies
7/12PCI: Convert xen-pciback and pci-driver to pci_is_sriov_* helpersdrivers/pci/pci-driver.c drivers/xen/xen-pciback/pci_stub.c+3 -31 reply
8/12PCI: Add is_sriov bit to struct pci_devarch/s390/pci/pci_iov.c drivers/pci/iov.c include/linux/pci.h+8 -20 replies
9/12PCI: Add helper to compute VF Routing ID to pci.hdrivers/pci/iov.c drivers/pci/pci.h+16 -40 replies
10/12PCI: Add Scalable I/O Virtualization data structure definitionsdrivers/pci/Kconfig drivers/pci/pci.h include/linux/pci.h +1 more+50 -20 replies
11/12PCI: Initialize and release SIOV capabilitydrivers/pci/Makefile drivers/pci/pci.h drivers/pci/probe.c +1 more+132 -00 replies
12/12PCI: Reserve bus range for SIOV devicesdrivers/pci/probe.c drivers/pci/siov.c+23 -20 replies

Reviews trailers per patch

1 of 12 patches has a Reviewed-by or Acked-by.

PatchJuergen Gross
1/12
2/12
3/12
4/12
5/12
6/12
7/12R-b
8/12
9/12
10/12
11/12
12/12

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [RFC 00/12] PCI: Add support for Scalable I/O Virtualization

From: Dimitri Daskalakis <[email protected]>

Scalable I/O Virtualization (SIOV) is the next-generation alternative to SR-IOV. The goal of SIOV is to support more virtual devices than SR-IOV can currently support, while relaxing many of the HW requirements of SR-IOV.

SIOV VFs are referred to as Scalable Device Interfaces (SDI). An SDI has a unique PCIe Routing ID (RID), but has no configuration space, BAR, or MSI-X table.

An overview can be found here:
https://pcisig.com/PCIExpress/ECN/Base/ScalableIOVirtualization

Since SDIs lack HW support, the complexity will fall on software (hypervisors, vmms, drivers, and/or firmware) to provide the same isolation guarantees for SIOV that SR-IOV has today.

This patch series is one step in that direction, allowing the PCI subsystem to discover the SIOV capability during enumeration. This was the minimum set of changes needed so I could test the SIOV feature of developmental HW in emulation. I have not tested a device that supports both SR-IOV and SIOV, but this combination is allowed per the spec.

SIOV has two ways to assign RIDs, strided (like SR-IOV) or software assigned. To support software RID assignment, you need to compute the RID allowlist after all PCI devices have been enumerated. I've deferred this complexity for now and only implemented strided RID assignment.

Patch 1 adds helpers to identify if a PF/VF is a SR-IOV PF/VF. The PF and VF bits within struct pci_dev should be agnostic of virtualization type. The helper uses the current logic which assumes any PF/VF is SR-IOV.

Patch 2-7 uses the new helpers throughout core. I didn't convert certain device drivers (drivers/net, drivers/gpu) because the devices will not suddenly start advertising the SIOV capability. These can be updated in the future if desired.

Patch 8 tightens the helpers introduced in patch 1 with a new is_sriov bit.

Patch 9 is a small refactor for computing VF RID which can be shared between SR-IOV and SIOV.

Patch 10-12 add SIOV definitions, capability detection, and bus reservation.

With this patchset core enumarates the SIOV capability and can identify SIOV PFs. But there is no central mechanism to allocate/manage SIOV VFs. To support device pass through, devices will need to add a vfio-mdev driver with IOMMUFD support (or something similar).

Dimitri Daskalakis (12):
PCI: Add helpers to identify SR-IOV PFs/VFs.
PCI: Convert iov.c to pci_is_sriov_* helpers
PCI: Convert pci.h to pci_is_sriov_* helpers
PCI: Convert arch/powerpc to pci_is_sriov_* helpers
PCI: Convert s390/pci/pci.c to pci_is_sriov_* helpers
PCI: Convert vfio_pci_core.c to pci_is_sriov_* helpers
PCI: Convert xen-pciback and pci-driver to pci_is_sriov_* helpers
PCI: Add is_sriov bit to struct pci_dev
PCI: Add helper to compute VF Routing ID to pci.h
PCI: Add Scalable I/O Virtualization data structure definitions
PCI: Initialize and release SIOV capability
PCI: Reserve bus range for SIOV devices

 arch/powerpc/kernel/pci_dn.c                 |   4 +-
 arch/powerpc/platforms/powernv/pci-ioda.c    |   6 +-
 arch/powerpc/platforms/powernv/pci-sriov.c   |  10 +-
 arch/powerpc/platforms/pseries/eeh_pseries.c |   8 +-
 arch/powerpc/platforms/pseries/setup.c       |   4 +-
 arch/s390/pci/pci.c                          |   2 +-
 arch/s390/pci/pci_iov.c                      |   1 +
 drivers/pci/Kconfig                          |  11 ++
 drivers/pci/Makefile                         |   1 +
 drivers/pci/iov.c                            |  58 ++++----
 drivers/pci/pci-driver.c                     |   4 +-
 drivers/pci/pci.h                            |  43 +++++-
 drivers/pci/probe.c                          |   6 +-
 drivers/pci/siov.c                           | 134 +++++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c             |  12 +-
 drivers/xen/xen-pciback/pci_stub.c           |   2 +-
 include/linux/pci.h                          |  29 +++-
 include/uapi/linux/pci_regs.h                |  12 +-
 18 files changed, 289 insertions(+), 58 deletions(-)
 create mode 100644 drivers/pci/siov.c
Signature · 1 line
2.52.0