soc: ti: Add and use PVU on K3-AM65 for DMA isolation
Subsystems (5)
v5 cover soc: ti: Add and use PVU on K3-AM65 for DMA isolation
Changes in v5:
- resolve review comments on pci-host bindings
- reduce DMA memory regions to 1 - swiotlb does not support more
- move activation into overlay (controlled via firmware)
- use ks_init_vmap helper instead of loop in
rework ks_init_restricted_dma
- add more comments to pci-keystone
- use 2 chained TLBs of PVU to support maximum of swiotlb (320 MB)
Changes in v4:
- reorder patch queue, moving all DTS changes to the back
- limit activation to IOT2050 Advanced variants
- move DMA pool to allow firmware-based expansion it up to 512M
Changes in v3:
- fix ti,am654-pvu.yaml according to review comments
- address review comments on ti,am65-pci-host.yaml
- differentiate between different compatibles in ti,am65-pci-host.yaml
- move pvu nodes to k3-am65-main.dtsi
- reorder patch series, pulling bindings and generic DT bits to the front
Changes in v2:
- fix dt_bindings_check issues (patch 1)
- address first review comments (patch 2)
- extend ti,am65-pci-host bindings for PVU (new patch 3)
Only few of the K3 SoCs have an IOMMU and, thus, can isolate the system against DMA-based attacks of external PCI devices. The AM65 is without an IOMMU, but it comes with something close to it: the Peripheral Virtualization Unit (PVU).
The PVU was originally designed to establish static compartments via a hypervisor, isolate those DMA-wise against each other and the host and even allow remapping of guest-physical addresses. But it only provides a static translation region, not page-granular mappings. Thus, it cannot be handled transparently like an IOMMU.
Now, to use the PVU for the purpose of isolated PCI devices from the Linux host, this series takes a different approach. It defines a restricted-dma-pool for the PCI host, using swiotlb to map all DMA buffers from a static memory carve-out. And to enforce that the devices actually follow this, a special PVU soc driver is introduced. The driver permits access to the GIC ITS and otherwise waits for other drivers that detect devices with constrained DMA to register pools with the PVU.
For the AM65, the first (and possibly only) driver where this is introduced is the pci-keystone host controller. Finally, this series provides a DT overlay for the IOT2050 Advanced devices (all have MiniPCIe or M.2 extension slots) to make use of this protection scheme. Application of this overlay will be handled by firmware.
Due to the cross-cutting nature of these changes, multiple subsystems are affected. However, I wanted to present the whole thing in one series to allow everyone to review with the complete picture in hands. If preferred, I can also split the series up, of course.
Jan
Jan Kiszka (7):
dt-bindings: soc: ti: Add AM65 peripheral virtualization unit
dt-bindings: PCI: ti,am65: Extend for use with PVU
soc: ti: Add IOMMU-like PVU driver
PCI: keystone: Add support for PVU-based DMA isolation on AM654
arm64: dts: ti: k3-am65-main: Add PVU nodes
arm64: dts: ti: k3-am65-main: Add VMAP registers to PCI root complexes
arm64: dts: ti: iot2050: Add overlay for DMA isolation for devices
behind PCI RC
.../bindings/pci/ti,am65-pci-host.yaml | 29 +- .../bindings/soc/ti/ti,am654-pvu.yaml | 51 ++ arch/arm64/boot/dts/ti/Makefile | 5 + arch/arm64/boot/dts/ti/k3-am65-main.dtsi | 38 +- ...am6548-iot2050-advanced-dma-isolation.dtso | 33 ++ drivers/pci/controller/dwc/pci-keystone.c | 108 ++++ drivers/soc/ti/Kconfig | 4 + drivers/soc/ti/Makefile | 1 + drivers/soc/ti/ti-pvu.c | 500 ++++++++++++++++++ include/linux/ti-pvu.h | 16 + 10 files changed, 778 insertions(+), 7 deletions(-) create mode 100644 Documentation/devicetree/bindings/soc/ti/ti,am654-pvu.yaml create mode 100644 arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-dma-isolation.dtso create mode 100644 drivers/soc/ti/ti-pvu.c create mode 100644 include/linux/ti-pvu.h
Signature · 1 line
2.43.0
PATCH v5 1/7 dt-bindings: soc: ti: Add AM65 peripheral virtualization unit
From: Jan Kiszka <[email protected]>
The PVU allows to define a limited set of mappings for incoming DMA requests to the system memory. It is not a real IOMMU, thus hooked up under the TI SoC bindings.
--- .../bindings/soc/ti/ti,am654-pvu.yaml | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 Documentation/devicetree/bindings/soc/ti/ti,am654-pvu.yaml
Documentation/devicetree/bindings/soc/ti/ti,am654-pvu.yaml new file +51 -0
@@ -0,0 +1,51 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +# Copyright (c) Siemens AG, 2024 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/soc/ti/ti,am654-pvu.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: TI AM654 Peripheral Virtualization Unit + +maintainers: + - Jan Kiszka <[email protected]> + +properties: + compatible: + enum: + - ti,am654-pvu + + reg: + maxItems: 2 + + reg-names: + items: + - const: cfg + - const: tlbif + + interrupts: + items: + - description: fault interrupt + + interrupt-names: + items: + - const: pvu + +required: + - compatible + - reg + - interrupts + - interrupt-names + +additionalProperties: false + +examples: + - | + iommu@30f80000 { + compatible = "ti,am654-pvu"; + reg = <0x30f80000 0x1000>, + <0x36000000 0x100000>; + reg-names = "cfg", "tlbif"; + interrupts-extended = <&intr_main_navss 390>; + interrupt-names = "pvu"; + };
Signature · 1 line
2.43.0
PATCH v5 2/7 dt-bindings: PCI: ti,am65: Extend for use with PVU
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optional unless the PVU shall be used for PCIe.
---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]
--- .../bindings/pci/ti,am65-pci-host.yaml | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +26 -3
@@ -20,14 +20,18 @@ properties: - ti,keystone-pcie reg: - maxItems: 4 + minItems: 4 + maxItems: 6 reg-names: + minItems: 4 items: - const: app - const: dbics - const: config - const: atu + - const: vmap_lp + - const: vmap_hp interrupts: maxItems: 1
@@ -83,13 +87,30 @@ if: compatible: enum: - ti,am654-pcie-rc + then: + properties: + memory-region: + maxItems: 1 + description: | + phandle to a restricted DMA pool to be used for all devices behind + this controller. The regions should be defined according to + reserved-memory/shared-dma-pool.yaml. + required: - dma-coherent - power-domains - msi-map - num-viewport +else: + properties: + reg: + maxItems: 4 + + reg-names: + maxItems: 4 + unevaluatedProperties: false examples:
@@ -104,8 +125,10 @@ examples: reg = <0x5500000 0x1000>, <0x5501000 0x1000>, <0x10000000 0x2000>, - <0x5506000 0x1000>; - reg-names = "app", "dbics", "config", "atu"; + <0x5506000 0x1000>, + <0x2900000 0x1000>, + <0x2908000 0x1000>; + reg-names = "app", "dbics", "config", "atu", "vmap_lp", "vmap_hp"; power-domains = <&k3_pds 120 TI_SCI_PD_EXCLUSIVE>; #address-cells = <3>; #size-cells = <2>;
Signature · 1 line
2.43.0
Re: [PATCH v5 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 47 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optional unless the PVU shall be used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +5
-1@@ -20,14 +20,18 @@ properties: - ti,keystone-pcie reg: - maxItems: 4 + minItems: 4 + maxItems: 6 reg-names: + minItems: 4 items: - const: app - const: dbics - const: config - const: atu + - const: vmap_lp + - const: vmap_hp interrupts: maxItems: 1
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 93
Quoted 7 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +3
-0@@ -83,13 +87,30 @@ if: compatible: enum: - ti,am654-pcie-rc + then: + properties: + memory-region:
I think I said it two times already. You must define properties in top-level. That's how we expect, that's how dtschema works (even if it works fine otherwise, it's not always that case), that's how almost all bindings are written.
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 98
Quoted 5 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +5
-0+ maxItems: 1 + description: | + phandle to a restricted DMA pool to be used for all devices behind + this controller. The regions should be defined according to + reserved-memory/shared-dma-pool.yaml.
Best regards,
Krzysztof
Re: [PATCH v5 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 45 lines · Krzysztof Kozlowski
Quoted 44 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optional unless the PVU shall be used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +5
-1@@ -20,14 +20,18 @@ properties: - ti,keystone-pcie reg: - maxItems: 4 + minItems: 4 + maxItems: 6 reg-names: + minItems: 4 items: - const: app - const: dbics - const: config - const: atu + - const: vmap_lp + - const: vmap_hp
Quoted 15 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 93
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +3
-0interrupts: maxItems: 1@@ -83,13 +87,30 @@ if: compatible: enum: - ti,am654-pcie-rc + then: + properties: + memory-region:I think I said it two times already. You must define properties in top-level. That's how we expect, that's how dtschema works (even if it works fine otherwise, it's not always that case), that's how almost all bindings are written.
Look, if you have such rules, also enhance the checker, or people like me will continue to work intuitively. Add reasoning along that as well, would help further to reduce your review effort. The current situation with rather fuzzy results from the checker and strange mechanisms inside (see my maxItems finding) is not very helpful IMHO.
I this concrete case, I would add this item top-level, just to set maxItems to 0 for ti,keystone-pcie? Not a pattern I'm finding anywhere. Or do we have to allow memory-regions for all compatibles now?
Sorry for all these iterations, but you should see from my questions and actions where the problems in the concepts are.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v5 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 62 lines · Jan Kiszka
Quoted 45 lines · Krzysztof Kozlowski
Quoted 44 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optional unless the PVU shall be used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +5
-1@@ -20,14 +20,18 @@ properties: - ti,keystone-pcie reg: - maxItems: 4 + minItems: 4 + maxItems: 6 reg-names: + minItems: 4 items: - const: app - const: dbics - const: config - const: atu + - const: vmap_lp + - const: vmap_hpQuoted 15 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 93
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +3
-0interrupts: maxItems: 1@@ -83,13 +87,30 @@ if: compatible: enum: - ti,am654-pcie-rc + then: + properties: + memory-region:I think I said it two times already. You must define properties in top-level. That's how we expect, that's how dtschema works (even if it works fine otherwise, it's not always that case), that's how almost all bindings are written.
Quoted 2 lines
Look, if you have such rules, also enhance the checker, or people like me will continue to work intuitively. Add reasoning along that as well,
That would be ideal, but I also asked to do this twice. It does not matter if dtschema or me tells you this, if you do not implement it.
Quoted 7 lines
would help further to reduce your review effort. The current situation with rather fuzzy results from the checker and strange mechanisms inside (see my maxItems finding) is not very helpful IMHO.
I this concrete case, I would add this item top-level, just to set maxItems to 0 for ti,keystone-pcie? Not a pattern I'm finding anywhere. Or do we have to allow memory-regions for all compatibles now?
Is it really not suitable for all the compatibles? Maybe these are quite different devices in such case?
But if it is not really suitable, then you can disallow it for other variants with :false. This is also explicitly documented in example-schema:
https://elixir.bootlin.com/linux/v5.19/source/Documentation/devicetree/bindings/example-schema.yaml#L212
Quoted 3 lines
Sorry for all these iterations, but you should see from my questions and actions where the problems in the concepts are.
Best regards,
Krzysztof
Re: [PATCH v5 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 77 lines · Krzysztof Kozlowski
Quoted 62 lines · Jan Kiszka
Quoted 45 lines · Krzysztof Kozlowski
Quoted 44 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optional unless the PVU shall be used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +5
-1@@ -20,14 +20,18 @@ properties: - ti,keystone-pcie reg: - maxItems: 4 + minItems: 4 + maxItems: 6 reg-names: + minItems: 4 items: - const: app - const: dbics - const: config - const: atu + - const: vmap_lp + - const: vmap_hpQuoted 15 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 93
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +3
-0interrupts: maxItems: 1@@ -83,13 +87,30 @@ if: compatible: enum: - ti,am654-pcie-rc + then: + properties: + memory-region:I think I said it two times already. You must define properties in top-level. That's how we expect, that's how dtschema works (even if it works fine otherwise, it's not always that case), that's how almost all bindings are written.
Quoted 2 lines
Look, if you have such rules, also enhance the checker, or people like me will continue to work intuitively. Add reasoning along that as well,
That would be ideal, but I also asked to do this twice. It does not matter if dtschema or me tells you this, if you do not implement it.
Quoted 7 lines
would help further to reduce your review effort. The current situation with rather fuzzy results from the checker and strange mechanisms inside (see my maxItems finding) is not very helpful IMHO.
I this concrete case, I would add this item top-level, just to set maxItems to 0 for ti,keystone-pcie? Not a pattern I'm finding anywhere. Or do we have to allow memory-regions for all compatibles now?
Quoted 7 lines
Is it really not suitable for all the compatibles? Maybe these are quite different devices in such case?
But if it is not really suitable, then you can disallow it for other variants with :false. This is also explicitly documented in example-schema:
https://elixir.bootlin.com/linux/v5.19/source/Documentation/devicetree/bindings/example-schema.yaml#L212
I will address this via documentation: The property is not hardware related but permits swiotlb. It can be applied to devices as well that have no hardware enforcement, unlike the am654.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
PATCH v5 3/7 soc: ti: Add IOMMU-like PVU driver
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The TI Peripheral Virtualization Unit (PVU) permits to define a limited set of mappings for DMA requests on the system memory. Unlike with an IOMMU, there is no fallback to a memory-backed page table, only a fixed set of register-backed TLBs. Emulating an IOMMU behavior appears to be the more fragile the more fragmentation of pending requests occur.
Therefore, this driver does not expose the PVU as an IOMMU. It rather introduces a simple, static interface to devices that are under restricted-dma-pool constraints. They can register their pools with the PVUs, enabling only those pools to work for DMA. As also MSI is issued as DMA, the PVU already register the related translator region of the AM654 as valid DMA target.
This driver is the essential building block for limiting DMA from untrusted devices to clearly defined memory regions in the absence of a real IOMMU (SMMU).
--- drivers/soc/ti/Kconfig | 4 + drivers/soc/ti/Makefile | 1 + drivers/soc/ti/ti-pvu.c | 500 ++++++++++++++++++++++++++++++++++++++++ include/linux/ti-pvu.h | 16 ++ 4 files changed, 521 insertions(+) create mode 100644 drivers/soc/ti/ti-pvu.c create mode 100644 include/linux/ti-pvu.h
drivers/soc/ti/Kconfig +4 -0
@@ -82,6 +82,10 @@ config TI_PRUSS processors on various TI SoCs. It's safe to say N here if you're not interested in the PRU or if you are unsure. +config TI_PVU + bool "TI Peripheral Virtualization Unit driver" + depends on ARCH_K3 && DMA_RESTRICTED_POOL + endif # SOC_TI config TI_SCI_INTA_MSI_DOMAIN
drivers/soc/ti/Makefile +1 -0
@@ -12,3 +12,4 @@ obj-$(CONFIG_TI_K3_RINGACC) += k3-ringacc.o obj-$(CONFIG_TI_K3_SOCINFO) += k3-socinfo.o obj-$(CONFIG_TI_PRUSS) += pruss.o obj-$(CONFIG_POWER_AVS_OMAP) += smartreflex.o +obj-$(CONFIG_TI_PVU) += ti-pvu.o
drivers/soc/ti/ti-pvu.c new file +500 -0
500 lines not on this page · raw message
include/linux/ti-pvu.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* TI Peripheral Virtualization Unit driver for static DMA isolation +* +* Copyright (c) 2024, Siemens AG +*/ + +#ifndef _LINUX_TI_PVU_H +#define _LINUX_TI_PVU_H + +#include <linux/ioport.h> + +int ti_pvu_create_region (unsigned int virt_id,const struct resource *region); +int ti_pvu_remove_region (unsigned int virt_id,const struct resource *region); + +#endif /* _LINUX_TI_PVU_H */
Signature · 1 line
2.43.0
PATCH v5 4/7 PCI: keystone: Add support for PVU-based DMA isolation on AM654
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The AM654 lacks an IOMMU, thus does not support isolating DMA requests from untrusted PCI devices to selected memory regions this way. Use static PVU-based protection instead. The PVU, when enabled, will only accept DMA requests that address previously configured regions.
Use the availability of a restricted-dma-pool memory region as trigger and register it as valid DMA target with the PVU. In addition, enable the mapping of requester IDs to VirtIDs in the PCI RC. Use only a single VirtID so far, catching all devices. This may be extended later on.
---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]
--- drivers/pci/controller/dwc/pci-keystone.c | 108 ++++++++++++++++++++++ 1 file changed, 108 insertions(+)
drivers/pci/controller/dwc/pci-keystone.c +108 -0
@@ -19,6 +19,7 @@#include <linux/mfd/syscon.h> #include <linux/msi.h> #include <linux/of.h> +#include <linux/of_address.h> #include <linux/of_irq.h> #include <linux/of_pci.h> #include <linux/phy/phy.h>
@@ -26,6 +27,7 @@#include <linux/regmap.h> #include <linux/resource.h> #include <linux/signal.h> +#include <linux/ti-pvu.h> #include "../../pci.h" #include "pcie-designware.h"
@@ -111,6 +113,16 @@#define PCI_DEVICE_ID_TI_AM654X 0xb00c +#define KS_PCI_VIRTID 0 + +#define PCIE_VMAP_xP_CTRL 0x0 +#define PCIE_VMAP_xP_REQID 0x4 +#define PCIE_VMAP_xP_VIRTID 0x8 + +#define PCIE_VMAP_xP_CTRL_EN BIT(0) + +#define PCIE_VMAP_xP_VIRTID_VID_MASK 0xfff +struct ks_pcie_of_data {enum dw_pcie_device_mode mode;const struct dw_pcie_host_ops *host_ops;
@@ -1125,6 +1137,96 @@ static const struct of_device_id ks_pcie_of_match[] = { { }, }; +#ifdef CONFIG_TI_PVU +static int ks_init_vmap (struct platform_device *pdev,const char *vmap_name) +{ +struct resource *res; +void __iomem *base; +u32 val; + + res =platform_get_resource_byname (pdev,IORESOURCE_MEM , vmap_name); + base =devm_pci_remap_cfg_resource (&pdev->dev, res); +if (IS_ERR (base)) +return PTR_ERR (base); + +writel (0 , base + PCIE_VMAP_xP_REQID); + + val =readl (base + PCIE_VMAP_xP_VIRTID); + val &= ~PCIE_VMAP_xP_VIRTID_VID_MASK; + val |=KS_PCI_VIRTID ; +writel (val, base + PCIE_VMAP_xP_VIRTID); + + val =readl (base + PCIE_VMAP_xP_CTRL); + val |= PCIE_VMAP_xP_CTRL_EN; +writel (val, base + PCIE_VMAP_xP_CTRL); + +return 0 ; +} + +static int ks_init_restricted_dma (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct of_phandle_iterator it; +struct resource phys; +int err; + +/* Only process the first restricted dma pool, more are not allowed */ +of_for_each_phandle (&it, err, dev->of_node,"memory-region" , +NULL ,0 ) { +if (of_device_is_compatible (it.node,"restricted-dma-pool" )) +break ; + } +if (err) +return err == -ENOENT ?0 : err; + + err =of_address_to_resource (it.node,0 , &phys); +if (err <0 ) { +dev_err (dev,"failed to parse memory region %pOF: %d\n" , + it.node, err); +return 0 ; + } + +/* Map all incoming requests on low and high prio port to virtID 0 */ + err =ks_init_vmap (pdev,"vmap_lp" ); +if (err) +return err; + err =ks_init_vmap (pdev,"vmap_hp" ); +if (err) +return err; + +/* +* Enforce DMA pool usage with the help of the PVU. +* Any request outside will be dropped and raise an error at the PVU. +*/ +return ti_pvu_create_region (KS_PCI_VIRTID , &phys); +} + +static void ks_release_restricted_dma (struct platform_device *pdev) +{ +struct of_phandle_iterator it; +struct resource phys; +int err; + +of_for_each_phandle (&it, err, pdev->dev.of_node,"memory-region" , +NULL ,0 ) { +if (of_device_is_compatible (it.node,"restricted-dma-pool" ) && +of_address_to_resource (it.node,0 , &phys) ==0 ) { +ti_pvu_remove_region (KS_PCI_VIRTID , &phys); +break ; + } + } +} +#else +static inline int ks_init_restricted_dma (struct platform_device *pdev) +{ +return 0 ; +} + +static inline void ks_release_restricted_dma (struct platform_device *pdev) +{ +} +#endif +static int ks_pcie_probe (struct platform_device *pdev) {const struct dw_pcie_host_ops *host_ops;
@@ -1273,6 +1375,10 @@ static int ks_pcie_probe(struct platform_device *pdev)if (ret <0 ) goto err_get_sync; + ret =ks_init_restricted_dma (pdev); +if (ret <0 ) + goto err_get_sync; +switch (mode) {case DW_PCIE_RC_TYPE :if (!IS_ENABLED (CONFIG_PCI_KEYSTONE_HOST )) {
@@ -1354,6 +1460,8 @@ static void ks_pcie_remove(struct platform_device *pdev)int num_lanes = ks_pcie->num_lanes;struct device *dev = &pdev->dev; +ks_release_restricted_dma (pdev); +pm_runtime_put (dev);pm_runtime_disable (dev);ks_pcie_disable_phy (ks_pcie);
Signature · 1 line
2.43.0
PATCH v5 5/7 arm64: dts: ti: k3-am65-main: Add PVU nodes
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Add nodes for the two PVUs of the AM65. Keep them disabled, though, because the board has to additionally define DMA pools and the devices to be isolated.
--- arch/arm64/boot/dts/ti/k3-am65-main.dtsi | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/arm64/boot/dts/ti/k3-am65-main.dtsi +20 -0
@@ -841,6 +841,26 @@ main_cpts_mux: refclk-mux { assigned-clock-parents = <&k3_clks 118 5>; }; }; + + ti_pvu0: iommu@30f80000 { + compatible = "ti,am654-pvu"; + reg = <0 0x30f80000 0 0x1000>, + <0 0x36000000 0 0x100000>; + reg-names = "cfg", "tlbif"; + interrupts-extended = <&intr_main_navss 390>; + interrupt-names = "pvu"; + status = "disabled"; + }; + + ti_pvu1: iommu@30f81000 { + compatible = "ti,am654-pvu"; + reg = <0 0x30f81000 0 0x1000>, + <0 0x36100000 0 0x100000>; + reg-names = "cfg", "tlbif"; + interrupts-extended = <&intr_main_navss 389>; + interrupt-names = "pvu"; + status = "disabled"; + }; }; main_gpio0: gpio@600000 {
Signature · 1 line
2.43.0
PATCH v5 6/7 arm64: dts: ti: k3-am65-main: Add VMAP registers to PCI root complexes
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Rewrap the long lines at this chance.
--- arch/arm64/boot/dts/ti/k3-am65-main.dtsi | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
arch/arm64/boot/dts/ti/k3-am65-main.dtsi +14 -4
@@ -895,8 +895,13 @@ main_gpio1: gpio@601000 { pcie0_rc: pcie@5500000 { compatible = "ti,am654-pcie-rc"; - reg = <0x0 0x5500000 0x0 0x1000>, <0x0 0x5501000 0x0 0x1000>, <0x0 0x10000000 0x0 0x2000>, <0x0 0x5506000 0x0 0x1000>; - reg-names = "app", "dbics", "config", "atu"; + reg = <0x0 0x5500000 0x0 0x1000>, + <0x0 0x5501000 0x0 0x1000>, + <0x0 0x10000000 0x0 0x2000>, + <0x0 0x5506000 0x0 0x1000>, + <0x0 0x2900000 0x0 0x1000>, + <0x0 0x2908000 0x0 0x1000>; + reg-names = "app", "dbics", "config", "atu", "vmap_lp", "vmap_hp"; power-domains = <&k3_pds 120 TI_SCI_PD_EXCLUSIVE>; #address-cells = <3>; #size-cells = <2>;
@@ -916,8 +921,13 @@ pcie0_rc: pcie@5500000 { pcie1_rc: pcie@5600000 { compatible = "ti,am654-pcie-rc"; - reg = <0x0 0x5600000 0x0 0x1000>, <0x0 0x5601000 0x0 0x1000>, <0x0 0x18000000 0x0 0x2000>, <0x0 0x5606000 0x0 0x1000>; - reg-names = "app", "dbics", "config", "atu"; + reg = <0x0 0x5600000 0x0 0x1000>, + <0x0 0x5601000 0x0 0x1000>, + <0x0 0x18000000 0x0 0x2000>, + <0x0 0x5606000 0x0 0x1000>, + <0x0 0x2910000 0x0 0x1000>, + <0x0 0x2918000 0x0 0x1000>; + reg-names = "app", "dbics", "config", "atu", "vmap_lp", "vmap_hp"; power-domains = <&k3_pds 121 TI_SCI_PD_EXCLUSIVE>; #address-cells = <3>; #size-cells = <2>;
Signature · 1 line
2.43.0
PATCH v5 7/7 arm64: dts: ti: iot2050: Add overlay for DMA isolation for devices behind PCI RC
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Reserve a 64M memory region and ensure that all PCI devices do their DMA only inside that region. This is configured via a restricted-dma-pool and enforced with the help of the first PVU.
Applying this isolation is not totally free in terms of overhead and memory consumption. It makes only sense for variants that support secure booting, and generally only when this is actually enable. Therefore model it as overlay that can be activated on demand. The firmware will take care of this via DT fixup during boot and will also provide a way to adjust the pool size.
--- arch/arm64/boot/dts/ti/Makefile | 5 +++ ...am6548-iot2050-advanced-dma-isolation.dtso | 33 +++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-dma-isolation.dtso
arch/arm64/boot/dts/ti/Makefile +5 -0
@@ -74,8 +74,10 @@ k3-am654-gp-evm-dtbs := k3-am654-base-board.dtb \ k3-am654-evm-dtbs := k3-am654-base-board.dtb k3-am654-icssg2.dtbo k3-am654-idk-dtbs := k3-am654-evm.dtb k3-am654-idk.dtbo k3-am654-pcie-usb2.dtbo k3-am6548-iot2050-advanced-m2-bkey-ekey-pcie-dtbs := k3-am6548-iot2050-advanced-m2.dtb \ + k3-am6548-iot2050-advanced-dma-isolation.dtbo \ k3-am6548-iot2050-advanced-m2-bkey-ekey-pcie.dtbo k3-am6548-iot2050-advanced-m2-bkey-usb3-dtbs := k3-am6548-iot2050-advanced-m2.dtb \ + k3-am6548-iot2050-advanced-dma-isolation.dtbo \ k3-am6548-iot2050-advanced-m2-bkey-usb3.dtbo dtb-$(CONFIG_ARCH_K3) += k3-am6528-iot2050-basic.dtb dtb-$(CONFIG_ARCH_K3) += k3-am6528-iot2050-basic-pg2.dtb
@@ -240,7 +242,10 @@ DTC_FLAGS_k3-am62p5-sk += -@ DTC_FLAGS_k3-am642-evm += -@ DTC_FLAGS_k3-am642-phyboard-electra-rdk += -@ DTC_FLAGS_k3-am642-tqma64xxl-mbax4xxl += -@ +DTC_FLAGS_k3-am6548-iot2050-advanced += -@ DTC_FLAGS_k3-am6548-iot2050-advanced-m2 += -@ +DTC_FLAGS_k3-am6548-iot2050-advanced-pg2 += -@ +DTC_FLAGS_k3-am6548-iot2050-advanced-sm += -@ DTC_FLAGS_k3-am68-sk-base-board += -@ DTC_FLAGS_k3-am69-sk += -@ DTC_FLAGS_k3-j721e-common-proc-board += -@
arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-dma-isolation.dtso new file +33 -0
@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * IOT2050, overlay for isolating DMA requests via PVU + * Copyright (c) Siemens AG, 2024 + * + * Authors: + * Jan Kiszka <[email protected]> + */ + +/dts-v1/; +/plugin/; + +&{/reserved-memory} { + #address-cells = <2>; + #size-cells = <2>; + + pci_restricted_dma_region: restricted-dma@c0000000 { + compatible = "restricted-dma-pool"; + reg = <0 0xc0000000 0 0x4000000>; + }; +}; + +&pcie0_rc { + memory-region = <&pci_restricted_dma_region>; +}; + +&pcie1_rc { + memory-region = <&pci_restricted_dma_region>; +}; + +&ti_pvu0 { + status = "okay"; +};
Signature · 1 line
2.43.0