Skip to content

RFC iommu/arm-smmu-v3: Add NVIDIA implementation

Nicolin Chen kvm linux-arm-kernel linux-doc linux-iommu linux-tegra lkml llvm oe-kbuild-all 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 RFC Latest13 patches 5 repliesIn review
  2. v1 RFC2 patches 0 repliesSuperseded

Changes in v2 from the cover letter

* Added mdev interface support for hypervisor and VMs.
* Added preparational changes for mdev interface implementation.
* PATCH-12 Changed ->issue_cmdlist() to ->get_cmdq() for a better integration with recently merged ECMDQ-related changes.

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/13iommu: Add set_nesting_vmid/get_nesting_vmid functionsdrivers/iommu/iommu.c include/linux/iommu.h+25 -00 replies
2/13vfio: add VFIO_IOMMU_GET_VMID and VFIO_IOMMU_SET_VMIDdrivers/vfio/vfio.c include/uapi/linux/vfio.h+39 -00 replies
3/13vfio: Document VMID control for IOMMU VirtualizationDocumentation/driver-api/vfio.rst+34 -00 replies
4/13vfio: add set_vmid and get_vmid for vfio_iommu_type1drivers/vfio/vfio.c include/linux/vfio.h+14 -00 replies
5/13vfio/type1: Implement set_vmid and get_vmiddrivers/vfio/vfio_iommu_type1.c+25 -00 replies
6/13vfio/type1: Set/get VMID to/from iommu driverdrivers/vfio/vfio_iommu_type1.c+12 -00 replies
7/13iommu/arm-smmu-v3: Add shared VMID support for NESTINGdrivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h+60 -60 replies
8/13iommu/arm-smmu-v3: Add VMID alloc/free helpersdrivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h+13 -00 replies
9/13iommu/arm-smmu-v3: Pass dev pointer to arm_smmu_detach_devdrivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c+3 -30 replies
10/13iommu/arm-smmu-v3: Pass cmdq pointer in arm_smmu_cmdq_issue_cmdlist()drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c+8 -70 replies
11/13iommu/arm-smmu-v3: Add implementation infrastructuredrivers/iommu/arm/arm-smmu-v3/Makefile drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-impl.c drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +1 more+17 -10 replies
12/13iommu/arm-smmu-v3: Add support for NVIDIA CMDQ-Virtualization hwMAINTAINERS drivers/iommu/arm/arm-smmu-v3/Makefile drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-impl.c +3 more+463 -30 replies
13/13iommu/nvidia-smmu-v3: Add mdev interface supportdrivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h drivers/iommu/arm/arm-smmu-v3/nvidia-smmu-v3.c+825 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [RFC][PATCH v2 00/13] iommu/arm-smmu-v3: Add NVIDIA implementation

The SMMUv3 devices implemented in the Grace SoC support NVIDIA's custom CMDQ-Virtualization (CMDQV) hardware. Like the new ECMDQ feature first introduced in the ARM SMMUv3.3 specification, CMDQV adds multiple VCMDQ interfaces to supplement the single architected SMMU_CMDQ in an effort to reduce contention.

This series of patches add CMDQV support with its preparational changes:

* PATCH-1 to PATCH-8 are related to shared VMID feature: they are used first to improve TLB utilization, second to bind a shared VMID with a VCMDQ interface for hardware configuring requirement.

* PATCH-9 and PATCH-10 are to accommodate the NVIDIA implementation with the existing arm-smmu-v3 driver.

* PATCH-11 borrows the "implementation infrastructure" from the arm-smmu driver so later change can build upon it.

* PATCH-12 adds an initial NVIDIA implementation related to host feature, and also adds implementation specific ->device_reset() and ->get_cmdq() callback functions.

* PATCH-13 adds virtualization features using VFIO mdev interface, which allows user space hypervisor to map and get access to one of the VCMDQ interfaces of CMDQV module.

( Thinking that reviewers can get a better view of this implementation,
I am attaching QEMU changes here for reference purpose:

      https://github.com/nicolinc/qemu/commits/dev/cmdqv_v6.0.0-rc2

The branch has all preparational changes, while I'm still integrating device model and ARM-VIRT changes, and will push them these two days, although they might not be in a good shape of being sent to review yet )

Above all, I marked RFC for this series, as I feel that we may come up some better solution. So please kindly share your reviews and insights.

Thank you!

Changelog
v1->v2:
* Added mdev interface support for hypervisor and VMs.
* Added preparational changes for mdev interface implementation.
* PATCH-12 Changed ->issue_cmdlist() to ->get_cmdq() for a better integration with recently merged ECMDQ-related changes.

Nate Watterson (3):
iommu/arm-smmu-v3: Add implementation infrastructure iommu/arm-smmu-v3: Add support for NVIDIA CMDQ-Virtualization hw iommu/nvidia-smmu-v3: Add mdev interface support

Nicolin Chen (10):
iommu: Add set_nesting_vmid/get_nesting_vmid functions
vfio: add VFIO_IOMMU_GET_VMID and VFIO_IOMMU_SET_VMID
vfio: Document VMID control for IOMMU Virtualization
vfio: add set_vmid and get_vmid for vfio_iommu_type1 vfio/type1: Implement set_vmid and get_vmid
vfio/type1: Set/get VMID to/from iommu driver iommu/arm-smmu-v3: Add shared VMID support for NESTING iommu/arm-smmu-v3: Add VMID alloc/free helpers iommu/arm-smmu-v3: Pass dev pointer to arm_smmu_detach_dev iommu/arm-smmu-v3: Pass cmdq pointer in arm_smmu_cmdq_issue_cmdlist()

 Documentation/driver-api/vfio.rst             |   34 +
 MAINTAINERS                                   |    2 +
 drivers/iommu/arm/arm-smmu-v3/Makefile        |    2 +-
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-impl.c  |   15 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |  121 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   18 +
 .../iommu/arm/arm-smmu-v3/nvidia-smmu-v3.c    | 1249 +++++++++++++++++
 drivers/iommu/iommu.c                         |   20 +
 drivers/vfio/vfio.c                           |   25 +
 drivers/vfio/vfio_iommu_type1.c               |   37 +
 include/linux/iommu.h                         |    5 +
 include/linux/vfio.h                          |    2 +
 include/uapi/linux/vfio.h                     |   26 +
 13 files changed, 1537 insertions(+), 19 deletions(-)
 create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-impl.c
 create mode 100644 drivers/iommu/arm/arm-smmu-v3/nvidia-smmu-v3.c
Signature · 1 line
2.17.1