Skip to content

powerpc: pSeries: vfio: iommu: Re-enable support for SPAPR TCE VFIO

Shivaprasad G Bhat kvm linuxppc-dev lkml 4 versions – Applied

Status from this message: Applied to powerpc/next.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest6 patches 1 replyApplied
  2. v36 patches 0 repliesSuperseded
  3. v2 RFC6 patches 6 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v1 RFC3 patches7 repliesSuperseded

Changelog from the cover letter

v3: https://lore.kernel.org/linuxppc-dev/[email protected]/
- Rebased to top of the tree.
- In the first patch, spapr_tce_table_group_ops is only used locally in
the pseries/iommu.c file. So, made it static.
- Fixed the test robot reported issues [email protected]
and [email protected]
- Updated the commit messages

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/6powerpc/iommu: Move pSeries specific functions to pseries/iommu.carch/powerpc/include/asm/iommu.h arch/powerpc/kernel/iommu.c arch/powerpc/platforms/pseries/iommu.c+149 -1480 replies
2/6powerpc/pseries/iommu: Fix the VFIO_IOMMU_SPAPR_TCE_GET_INFO ioctl outputarch/powerpc/platforms/pseries/iommu.c+67 -140 replies
3/6powerpc/pseries/iommu: Use the iommu table[0] for IOV VF's DDWarch/powerpc/platforms/pseries/iommu.c+7 -50 replies
4/6vfio/spapr: Always clear TCEs before unsetting the windowdrivers/vfio/vfio_iommu_spapr_tce.c+9 -40 replies
5/6powerpc/iommu: Move dev_has_iommu_table() to iommu.carch/powerpc/include/asm/iommu.h arch/powerpc/kernel/eeh.c arch/powerpc/kernel/iommu.c+23 -160 replies
6/6powerpc/iommu: Reimplement the iommu_table_group_ops for pSeriesarch/powerpc/include/asm/iommu.h arch/powerpc/kernel/iommu.c arch/powerpc/platforms/powernv/pci-ioda.c +1 more+543 -980 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCH v4 0/6] powerpc: pSeries: vfio: iommu: Re-enable support for SPAPR TCE VFIO

The patches reimplement the iommu table_group_ops for pSeries for VFIO SPAPR TCE sub-driver thereby bringing consistency with PowerNV implementation and getting rid of limitations/bugs which were emanating from these differences on the earlier approach on pSeries.

Structure of the patchset:
-------------------------
The first and fifth patches just code movements.

Second patch takes care of collecting the TCE and DDW information for the vfio_iommu_spapr_tce_ddw_info during probe.

Third patch fixes the convention of using table[1] for VFs on pSeries when used by the host driver.

Fourth patch fixes the VFIO to call TCE clear before unset window.

The last patch has the API reimplementations, please find the details on its commit description.

Testing:
-------
Tested with nested guest for NVME card, Mellanox multi-function card by attaching them to nested kvm guest running on a pSeries lpar.
Also vfio-test [2] by Alex Willamson, was forked and updated to add support for pSeries guest and used to test these patches[3].

Limitations/Known Issues:
------------------------
* The DMA window restrictions with SRIOV VF scenarios of having maximum 1 dma window is taken care in the current patches itself. However, the necessary changes required in vfio_iommu_spapr_tce_ddw_info to expose the default window being a 64-bit one and the qemu changes handle the same will be taken care in future versions.

References:
----------
[1] https://lore.kernel.org/linuxppc-dev/[email protected]/
[2] https://github.com/awilliam/tests
[3] https://github.com/nnmwebmin/vfio-ppc-tests/tree/vfio-ppc-ex

---
Changelog:
v3: https://lore.kernel.org/linuxppc-dev/[email protected]/
- Rebased to top of the tree.
- In the first patch, spapr_tce_table_group_ops is only used locally in
the pseries/iommu.c file. So, made it static.
- Fixed the test robot reported issues [email protected]
and [email protected]
- Updated the commit messages

v2: https://lore.kernel.org/linuxppc-dev/[email protected]/
- Rebased to upstream. So, required the explicit vmalloc.h inclusion as its
removed from the system header io.h now.
- Fixed the DLPAR hotplugged device assignment case. The dma window
property is backed up before removal. That copy is restored when required.
- Cleaned up bit more. Removed leftover debug prints and dump_stack()s.
- The warning at remap_pfn_range_notrack() during kvm guest boot is no longer
seen after the rebase.

v1: https://lore.kernel.org/linuxppc-dev/[email protected]/
- Rewrite as to stop borrowing the DMA windows and implemented
the table_group_ops for pSeries.
- Cover letter and Patch 6 has more details as this was a rewrite.

Shivaprasad G Bhat (6):
powerpc/iommu: Move pSeries specific functions to pseries/iommu.c powerpc/pseries/iommu: Fix the VFIO_IOMMU_SPAPR_TCE_GET_INFO ioctl output powerpc/pseries/iommu: Use the iommu table[0] for IOV VF's DDW vfio/spapr: Always clear TCEs before unsetting the window powerpc/iommu: Move dev_has_iommu_table() to iommu.c powerpc/iommu: Reimplement the iommu_table_group_ops for pSeries

 arch/powerpc/include/asm/iommu.h          |  14 +-
 arch/powerpc/kernel/eeh.c                 |  16 -
 arch/powerpc/kernel/iommu.c               | 170 +-----
 arch/powerpc/platforms/powernv/pci-ioda.c |   6 +-
 arch/powerpc/platforms/pseries/iommu.c    | 704 +++++++++++++++++++++-
 drivers/vfio/vfio_iommu_spapr_tce.c       |  13 +-
 6 files changed, 718 insertions(+), 205 deletions(-)

--
Signature