Skip to content

RFC Add SMMUv3 Stage 1 Support for XEN guests

Rahul Singh xen-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest21 patches 32 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/21xen/arm: smmuv3: Maintain a SID->device structurexen/drivers/passthrough/arm/smmu-v3.c xen/drivers/passthrough/arm/smmu-v3.h+118 -260 replies
2/21xen/arm: smmuv3: Add support for stage-1 and nested stage translationxen/drivers/passthrough/arm/smmu-v3.c xen/drivers/passthrough/arm/smmu-v3.h+92 -110 replies
3/21xen/arm: smmuv3: Alloc io_domain for each devicexen/drivers/passthrough/arm/smmu-v3.c+11 -20 replies
4/21xen/arm: vIOMMU: add generic vIOMMU frameworkxen/arch/arm/domain.c xen/arch/arm/domain_build.c xen/arch/arm/include/asm/viommu.h +4 more+149 -010 replies
5/21xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guestsxen/arch/arm/domain.c xen/arch/arm/include/asm/domain.h xen/arch/arm/include/asm/viommu.h +7 more+222 -22 replies
6/21xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config paramdocs/man/xl.cfg.5.pod.in tools/golang/xenlight/helpers.gen.go tools/golang/xenlight/types.gen.go +4 more+47 -03 replies
7/21xen/arm: vIOMMU: Add cmdline boot option "viommu = <boolean>"docs/misc/xen-command-line.pandoc xen/arch/arm/include/asm/viommu.h xen/drivers/passthrough/arm/viommu.c +1 more+30 -01 reply
8/21xen/arm: vsmmuv3: Add support for registers emulationxen/drivers/passthrough/arm/vsmmu-v3.c+281 -02 replies
9/21xen/arm: vsmmuv3: Add support for cmdqueue handlingxen/drivers/passthrough/arm/vsmmu-v3.c+101 -01 reply
10/21xen/arm: vsmmuv3: Add support for command CMD_CFGI_STExen/drivers/passthrough/arm/vsmmu-v3.c+148 -01 reply
11/21xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardwarexen/drivers/passthrough/arm/smmu-v3.c xen/drivers/passthrough/arm/smmu-v3.h xen/drivers/passthrough/arm/vsmmu-v3.c +1 more+112 -02 replies
12/21xen/arm: vsmmuv3: Add support for event queue and global errorxen/drivers/passthrough/arm/smmu-v3.h xen/drivers/passthrough/arm/vsmmu-v3.c xen/include/public/arch-arm.h+183 -50 replies
13/21xen/arm: vsmmuv3: Add "iommus" property node for dom0 devicesxen/arch/arm/domain_build.c+5 -20 replies
14/21xen/arm: vIOMMU: IOMMU device tree node for dom0xen/arch/arm/domain_build.c xen/arch/arm/include/asm/viommu.h+95 -01 reply
15/21xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0lessxen/arch/arm/domain_build.c xen/include/public/device_tree_defs.h+53 -01 reply
16/21arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxltools/libs/light/libxl_arm.c+39 -00 replies
17/21xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3tools/libs/light/libxl_arm.c xen/arch/arm/domain_build.c xen/drivers/passthrough/arm/vsmmu-v3.c+43 -20 replies
18/21xen/arm: iommu: skip the iommu-map property for PCI devicesxen/arch/arm/domain_build.c+12 -30 replies
19/21xen/arm: vsmmuv3: Add support to send stage-1 event to guestxen/drivers/passthrough/arm/smmu-v3.c xen/drivers/passthrough/arm/vsmmu-v3.c xen/drivers/passthrough/arm/vsmmu-v3.h+103 -20 replies
20/21libxl/arm: vIOMMU: Modify the partial device tree for iommustools/libs/light/libxl_arm.c+46 -10 replies
21/21xen/arm: vIOMMU: Modify the partial device tree for dom0lessxen/arch/arm/domain_build.c+30 -12 replies

Reviews trailers per patch

1 of 21 patches has a Reviewed-by or Acked-by.

PatchJonathan CameronEric AugerKeqian ZhuWill Deacon
1/21A-b R-b R-b A-b
2/21
3/21
4/21
5/21
6/21
7/21
8/21
9/21
10/21
11/21
12/21
13/21
14/21
15/21
16/21
17/21
18/21
19/21
20/21
21/21

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v1 thread

Cover letter [RFC PATCH 00/21] Add SMMUv3 Stage 1 Support for XEN guests

The SMMUv3 supports two stages of translation. Each stage of translation can be independently enabled. An incoming address is logically translated from VA to IPA in stage 1, then the IPA is input to stage 2 which translates the IPA to the output PA.

Stage 1 is intended to be used by a software entity to provide isolation or translation to buffers within the entity, for example DMA isolation within an OS. Stage 2 is intended to be available in systems supporting the Virtualization Extensions and is intended to virtualize device DMA to guest VM address spaces. When both stage 1 and stage 2 are enabled, the translation configuration is called nested.

Stage 1 translation support is required to provide isolation between different devices within OS. XEN already supports Stage 2 translation but there is no support for Stage 1 translation. The goal of this work is to support Stage 1 translation for XEN guests. Stage 1 has to be configured within the guest to provide isolation.

We cannot trust the guest OS to control the SMMUv3 hardware directly as compromised guest OS can corrupt the SMMUv3 configuration and make the system vulnerable. The guest gets the ownership of the stage 1 page tables and also owns stage 1 configuration structures. The XEN handles the root configuration structure (for security reasons), including the stage 2 configuration.

XEN will emulate the SMMUv3 hardware and exposes the virtual SMMUv3 to the guest. Guest can use the native SMMUv3 driver to configure the stage 1 translation. When the guest configures the SMMUv3 for Stage 1, XEN will trap the access and configure hardware.

SMMUv3 Driver(Guest OS) -> Configure the Stage-1 translation -> XEN trap access -> XEN SMMUv3 driver configure the HW.

SMMUv3 driver has to be updated to support the Stage-1 translation support based on work done by the KVM team to support Nested Stage translation:
https://github.com/eauger/linux/commits/v5.11-stallv12-2stage-v14
https://lwn.net/Articles/852299/

As the stage 1 translation is configured by XEN on behalf of the guest, translation faults encountered during the translation process need to be propagated up to the guest and re-injected into the guest. When the guest invalidates stage 1 related caches, invalidations must be forwarded to the SMMUv3 hardware.

This patch series is sent as RFC to get the initial feedback from the community. This patch series consists of 21 patches which is a big number for the reviewer to review the patches but to understand the feature end-to-end we thought of sending this as a big series. Once we will get initial feedback, we will divide the series into a small number of patches for review.

Jean-Philippe Brucker (1):
xen/arm: smmuv3: Maintain a SID->device structure

Rahul Singh (20):
xen/arm: smmuv3: Add support for stage-1 and nested stage translation xen/arm: smmuv3: Alloc io_domain for each device
xen/arm: vIOMMU: add generic vIOMMU framework
xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param xen/arm: vIOMMU: Add cmdline boot option "viommu = <boolean>" xen/arm: vsmmuv3: Add support for registers emulation xen/arm: vsmmuv3: Add support for cmdqueue handling xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware xen/arm: vsmmuv3: Add support for event queue and global error xen/arm: vsmmuv3: Add "iommus" property node for dom0 devices xen/arm: vIOMMU: IOMMU device tree node for dom0
xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3
xen/arm: iommu: skip the iommu-map property for PCI devices xen/arm: vsmmuv3: Add support to send stage-1 event to guest libxl/arm: vIOMMU: Modify the partial device tree for iommus xen/arm: vIOMMU: Modify the partial device tree for dom0less

 docs/man/xl.cfg.5.pod.in               |  11 +
 docs/misc/xen-command-line.pandoc      |   7 +
 tools/golang/xenlight/helpers.gen.go   |   2 +
 tools/golang/xenlight/types.gen.go     |   1 +
 tools/include/libxl.h                  |   5 +
 tools/libs/light/libxl_arm.c           | 121 +++-
 tools/libs/light/libxl_types.idl       |   6 +
 tools/xl/xl_parse.c                    |   9 +
 xen/arch/arm/domain.c                  |  18 +
 xen/arch/arm/domain_build.c            | 213 +++++-
 xen/arch/arm/include/asm/domain.h      |   4 +
 xen/arch/arm/include/asm/viommu.h      | 102 +++
 xen/drivers/passthrough/Kconfig        |  14 +
 xen/drivers/passthrough/arm/Makefile   |   2 +
 xen/drivers/passthrough/arm/smmu-v3.c  | 370 +++++++++--
 xen/drivers/passthrough/arm/smmu-v3.h  |  43 +-
 xen/drivers/passthrough/arm/viommu.c   |  87 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 887 +++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h |  32 +
 xen/include/public/arch-arm.h          |  14 +-
 xen/include/public/device_tree_defs.h  |   1 +
 xen/include/xen/iommu.h                |  14 +
 22 files changed, 1913 insertions(+), 50 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/viommu.h
 create mode 100644 xen/drivers/passthrough/arm/viommu.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h

--
2.25.1