soc: ti: Add and use PVU on K3-AM65 for DMA isolation
Subsystems (5)
v4 cover soc: ti: Add and use PVU on K3-AM65 for DMA isolation
Changes in v4:
- reorder patch queue, moving all DTS changes to the back
- limit activation to IOT2050 Advanced variants
- move DMA pool to allow firmware-based expansion it up to 512M
Changes in v3:
- fix ti,am654-pvu.yaml according to review comments
- address review comments on ti,am65-pci-host.yaml
- differentiate between different compatibles in ti,am65-pci-host.yaml
- move pvu nodes to k3-am65-main.dtsi
- reorder patch series, pulling bindings and generic DT bits to the front
Changes in v2:
- fix dt_bindings_check issues (patch 1)
- address first review comments (patch 2)
- extend ti,am65-pci-host bindings for PVU (new patch 3)
Only few of the K3 SoCs have an IOMMU and, thus, can isolate the system against DMA-based attacks of external PCI devices. The AM65 is without an IOMMU, but it comes with something close to it: the Peripheral Virtualization Unit (PVU).
The PVU was originally designed to establish static compartments via a hypervisor, isolate those DMA-wise against each other and the host and even allow remapping of guest-physical addresses. But it only provides a static translation region, not page-granular mappings. Thus, it cannot be handled transparently like an IOMMU.
Now, to use the PVU for the purpose of isolated PCI devices from the Linux host, this series takes a different approach. It defines a restricted-dma-pool for the PCI host, using swiotlb to map all DMA buffers from a static memory carve-out. And to enforce that the devices actually follow this, a special PVU soc driver is introduced. The driver permits access to the GIC ITS and otherwise waits for other drivers that detect devices with constrained DMA to register pools with the PVU.
For the AM65, the first (and possibly only) driver where this is introduced is the pci-keystone host controller. Finally, this series configures the IOT2050 devices (all have MiniPCIe or M.2 extension slots) to make use of this protection scheme.
Due to the cross-cutting nature of these changes, multiple subsystems are affected. However, I wanted to present the whole thing in one series to allow everyone to review with the complete picture in hands. If preferred, I can also split the series up, of course.
Jan
Jan Kiszka (7):
dt-bindings: soc: ti: Add AM65 peripheral virtualization unit
dt-bindings: PCI: ti,am65: Extend for use with PVU
soc: ti: Add IOMPU-like PVU driver
PCI: keystone: Add supported for PVU-based DMA isolation on AM654
arm64: dts: ti: k3-am65-main: Add PVU nodes
arm64: dts: ti: k3-am65-main: Add VMAP registers to PCI root complexes
arm64: dts: ti: iot2050: Enforce DMA isolation for devices behind PCI
RC on Advanced
.../bindings/pci/ti,am65-pci-host.yaml | 52 +- .../bindings/soc/ti/ti,am654-pvu.yaml | 51 ++ arch/arm64/boot/dts/ti/k3-am65-main.dtsi | 38 +- .../ti/k3-am6548-iot2050-advanced-common.dtsi | 21 +- drivers/pci/controller/dwc/pci-keystone.c | 101 ++++ drivers/soc/ti/Kconfig | 4 + drivers/soc/ti/Makefile | 1 + drivers/soc/ti/ti-pvu.c | 487 ++++++++++++++++++ include/linux/ti-pvu.h | 16 + 9 files changed, 754 insertions(+), 17 deletions(-) create mode 100644 Documentation/devicetree/bindings/soc/ti/ti,am654-pvu.yaml create mode 100644 drivers/soc/ti/ti-pvu.c create mode 100644 include/linux/ti-pvu.h
Signature · 1 line
2.43.0
PATCH v4 1/7 dt-bindings: soc: ti: Add AM65 peripheral virtualization unit
From: Jan Kiszka <[email protected]>
The PVU allows to define a limited set of mappings for incoming DMA requests to the system memory. It is not a real IOMMU, thus hooked up under the TI SoC bindings.
--- .../bindings/soc/ti/ti,am654-pvu.yaml | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 Documentation/devicetree/bindings/soc/ti/ti,am654-pvu.yaml
Documentation/devicetree/bindings/soc/ti/ti,am654-pvu.yaml new file +51 -0
@@ -0,0 +1,51 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +# Copyright (c) Siemens AG, 2024 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/soc/ti/ti,am654-pvu.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: TI AM654 Peripheral Virtualization Unit + +maintainers: + - Jan Kiszka <[email protected]> + +properties: + compatible: + enum: + - ti,am654-pvu + + reg: + maxItems: 2 + + reg-names: + items: + - const: cfg + - const: tlbif + + interrupts: + items: + - description: fault interrupt + + interrupt-names: + items: + - const: pvu + +required: + - compatible + - reg + - interrupts + - interrupt-names + +additionalProperties: false + +examples: + - | + iommu@30f80000 { + compatible = "ti,am654-pvu"; + reg = <0x30f80000 0x1000>, + <0x36000000 0x100000>; + reg-names = "cfg", "tlbif"; + interrupts-extended = <&intr_main_navss 390>; + interrupt-names = "pvu"; + };
Signature · 1 line
2.43.0
PATCH v4 2/7 dt-bindings: PCI: ti,am65: Extend for use with PVU
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]
--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +40 -12
@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcie - reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atu - interrupts: maxItems: 1
@@ -84,12 +74,48 @@ if: enum: - ti,am654-pcie-rc then: + properties: + reg: + minItems: 4 + maxItems: 6 + + reg-names: + minItems: 4 + items: + - const: app + - const: dbics + - const: config + - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region: + minItems: 1 + description: | + phandle to one or more restricted DMA pools to be used for all devices + behind this controller. The regions should be defined according to + reserved-memory/shared-dma-pool.yaml. + items: + maxItems: 1 + required: - dma-coherent - power-domains - msi-map - num-viewport +else: + properties: + reg: + maxItems: 4 + + reg-names: + items: + - const: app + - const: dbics + - const: config + - const: atu + unevaluatedProperties: false examples:
@@ -104,8 +130,10 @@ examples: reg = <0x5500000 0x1000>, <0x5501000 0x1000>, <0x10000000 0x2000>, - <0x5506000 0x1000>; - reg-names = "app", "dbics", "config", "atu"; + <0x5506000 0x1000>, + <0x2900000 0x1000>, + <0x2908000 0x1000>; + reg-names = "app", "dbics", "config", "atu", "vmap_lp", "vmap_hp"; power-domains = <&k3_pds 120 TI_SCI_PD_EXCLUSIVE>; #address-cells = <3>; #size-cells = <2>;
Signature · 1 line
2.43.0
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 11 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
The last line above seems to be accidentally modified when compared to the one at:
https://lore.kernel.org/r/[email protected]/
"Hence, describe the VMAP registers which are optionally configured whenever PVU is used for PCIe."
If you intended to modify it, then the sentence appears distorted.
Regards,
Siddharth.
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 13 lines · Siddharth Vadapalli
Quoted 11 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Quoted 8 lines
The last line above seems to be accidentally modified when compared to the one at:
https://lore.kernel.org/r/[email protected]/
"Hence, describe the VMAP registers which are optionally configured whenever PVU is used for PCIe."If you intended to modify it, then the sentence appears distorted.
Yeah, I wanted to make the original sentence clearer but failed:
"...which are optional unless the PVU shall be used for PCIe."
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
In reply to Jan KiszkaQuoted 29 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcie
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atu
Nothing improved here.
Quoted 15 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +7
-1- interrupts: maxItems: 1@@ -84,12 +74,48 @@ if: enum: - ti,am654-pcie-rc then: + properties: + reg: + minItems: 4 + maxItems: 6 + + reg-names: + minItems: 4
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 93
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +10
-0+ items: + - const: app + - const: dbics + - const: config + - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region: + minItems: 1
Missing maxItems
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 99
Quoted 6 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +6
-0+ description: | + phandle to one or more restricted DMA pools to be used for all devices + behind this controller. The regions should be defined according to + reserved-memory/shared-dma-pool.yaml. + items: + maxItems: 1
And this feels redundant.
Best regards,
Krzysztof
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 31 lines · Krzysztof Kozlowski
Quoted 30 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcie
Quoted 12 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 9 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNothing improved here.
Yes, explained the background to you. Sorry, if you do not address my replies, I'm lost with your feedback.
Quoted 20 lines
Quoted 20 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +11
-1- interrupts: maxItems: 1@@ -84,12 +74,48 @@ if: enum: - ti,am654-pcie-rc then: + properties: + reg: + minItems: 4 + maxItems: 6 + + reg-names: + minItems: 4 + items: + - const: app + - const: dbics + - const: config
Quoted 8 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 93
Quoted 6 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +6
-0+ - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region: + minItems: 1Missing maxItems
Same as above. Please address my answers.
Quoted 9 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 99
Quoted 7 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +6
-0+ description: | + phandle to one or more restricted DMA pools to be used for all devices + behind this controller. The regions should be defined according to + reserved-memory/shared-dma-pool.yaml. + items: + maxItems: 1And this feels redundant.
I can drop that if it's not needed. Unfortunately, schemas are far from being intuitive to me.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 45 lines · Jan Kiszka
Quoted 31 lines · Krzysztof Kozlowski
Quoted 30 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieQuoted 12 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 9 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNothing improved here.
Quoted 2 lines
Yes, explained the background to you. Sorry, if you do not address my replies, I'm lost with your feedback.
My magic ball could not figure out the problem, so did not provide the answer.
I gave you the exact code which illustrates how to do it. If you do it that way: it works. If you do it other way: it might not work. However without seeing anything, magic ball was silent, so I am not participating in game: would you be so kind to give more information so I won't waste my day in asking what is wrong.
Best regards,
Krzysztof
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 46 lines · Krzysztof Kozlowski
Quoted 45 lines · Jan Kiszka
Quoted 43 lines · Krzysztof Kozlowski
Quoted 29 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNothing improved here.
Quoted 8 lines
Quoted 2 lines
Yes, explained the background to you. Sorry, if you do not address my replies, I'm lost with your feedback.
My magic ball could not figure out the problem, so did not provide the answer.
I gave you the exact code which illustrates how to do it. If you do it that way: it works. If you do it other way: it might not work. However
The link you provided was unfortunately not self-explanatory because if I - apparently - do it like that example, I'm getting the errors below.
Quoted 3 lines
without seeing anything, magic ball was silent, so I am not participating in game: would you be so kind to give more information so I won't waste my day in asking what is wrong.
With my patch:
# make ... dtbs_check DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-ekey-pcie.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-usb3.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-sm.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am654-base-board.dtb
With this revert on top:
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +10 -12
@@ -19,6 +19,16 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcie + reg: + maxItems: 4 + + reg-names: + items: + - const: app + - const: dbics + - const: config + - const: atu + interrupts: maxItems: 1
@@ -104,18 +114,6 @@ then: - msi-map - num-viewport -else: - properties: - reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atu - unevaluatedProperties: false examples:
# make ... dtbs_check
DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb
.../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5500000: reg: [[0, 89128960, 0, 4096], [0, 89133056, 0, 4096], [0, 268435456, 0, 8192], [0, 89153536, 0, 4096], [0, 42991616, 0, 4096], [0, 43024384, 0, 4096]] is too long
from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml#
.../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5500000: reg-names: ['app', 'dbics', 'config', 'atu', 'vmap_lp', 'vmap_hp'] is too long
from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml#
.../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5600000: reg: [[0, 90177536, 0, 4096], [0, 90181632, 0, 4096], [0, 402653184, 0, 8192], [0, 90202112, 0, 4096], [0, 43057152, 0, 4096], [0, 43089920, 0, 4096]] is too long
from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml#
.../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5600000: reg-names: ['app', 'dbics', 'config', 'atu', 'vmap_lp', 'vmap_hp'] is too long
from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml#
DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic-pg2.dtb
...Which magic dtschema spell am I missing to make this work like you suggest?
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 85 lines · Jan Kiszka
Quoted 46 lines · Krzysztof Kozlowski
Quoted 45 lines · Jan Kiszka
Quoted 43 lines · Krzysztof Kozlowski
Quoted 29 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNothing improved here.
Quoted 8 lines
Quoted 2 lines
Yes, explained the background to you. Sorry, if you do not address my replies, I'm lost with your feedback.
My magic ball could not figure out the problem, so did not provide the answer.
I gave you the exact code which illustrates how to do it. If you do it that way: it works. If you do it other way: it might not work. However
The link you provided was unfortunately not self-explanatory because if I - apparently - do it like that example, I'm getting the errors below.
Quoted 3 lines
without seeing anything, magic ball was silent, so I am not participating in game: would you be so kind to give more information so I won't waste my day in asking what is wrong.
With my patch:
# make ... dtbs_check DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-ekey-pcie.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-usb3.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-sm.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am654-base-board.dtbWith this revert on top:
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,6 +19,16 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcie
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +9
-0+ reg: + maxItems: 4 + + reg-names: + items: + - const: app + - const: dbics + - const: config + - const: atu
There is nothing like that in that example.
https://elixir.bootlin.com/linux/v6.8/source/Documentation/devicetree/bindings/ufs/qcom,ufs.yaml#L44
Quoted 9 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +1
-1+ interrupts: maxItems: 1@@ -104,18 +114,6 @@ then: - msi-map - num-viewport -else:
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml then: · line 117
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-10- properties: - reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atu
Neither this.
Each case MUST be covered, look:
https://elixir.bootlin.com/linux/v6.8/source/Documentation/devicetree/bindings/ufs/qcom,ufs.yaml#L191
Quoted 19 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-1- unevaluatedProperties: false examples:# make ... dtbs_check DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb .../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5500000: reg: [[0, 89128960, 0, 4096], [0, 89133056, 0, 4096], [0, 268435456, 0, 8192], [0, 89153536, 0, 4096], [0, 42991616, 0, 4096], [0, 43024384, 0, 4096]] is too long from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml# .../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5500000: reg-names: ['app', 'dbics', 'config', 'atu', 'vmap_lp', 'vmap_hp'] is too long from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml# .../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5600000: reg: [[0, 90177536, 0, 4096], [0, 90181632, 0, 4096], [0, 402653184, 0, 8192], [0, 90202112, 0, 4096], [0, 43057152, 0, 4096], [0, 43089920, 0, 4096]] is too long from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml# .../arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb: pcie@5600000: reg-names: ['app', 'dbics', 'config', 'atu', 'vmap_lp', 'vmap_hp'] is too long from schema $id: http://devicetree.org/schemas/pci/ti,am65-pci-host.yaml# DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic-pg2.dtb ...
Quoted 2 lines
Which magic dtschema spell am I missing to make this work like you suggest?
follow the example. You do it entirely different so you have different result. Code works deterministically.
Best regards,
Krzysztof
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 122 lines · Krzysztof Kozlowski
Quoted 85 lines · Jan Kiszka
Quoted 46 lines · Krzysztof Kozlowski
Quoted 45 lines · Jan Kiszka
Quoted 43 lines · Krzysztof Kozlowski
Quoted 29 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNothing improved here.
Quoted 8 lines
Quoted 2 lines
Yes, explained the background to you. Sorry, if you do not address my replies, I'm lost with your feedback.
My magic ball could not figure out the problem, so did not provide the answer.
I gave you the exact code which illustrates how to do it. If you do it that way: it works. If you do it other way: it might not work. However
The link you provided was unfortunately not self-explanatory because if I - apparently - do it like that example, I'm getting the errors below.
Quoted 3 lines
without seeing anything, magic ball was silent, so I am not participating in game: would you be so kind to give more information so I won't waste my day in asking what is wrong.
With my patch:
# make ... dtbs_check DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-ekey-pcie.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-usb3.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-sm.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am654-base-board.dtbWith this revert on top:
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,6 +19,16 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +9
-0+ reg: + maxItems: 4 + + reg-names: + items: + - const: app + - const: dbics + - const: config + - const: atuThere is nothing like that in that example.
https://elixir.bootlin.com/linux/v6.8/source/Documentation/devicetree/bindings/ufs/qcom,ufs.yaml#L44Quoted 9 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +1
-1+ interrupts: maxItems: 1@@ -104,18 +114,6 @@ then: - msi-map - num-viewport -else:Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml then: · line 117
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-10- properties: - reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNeither this.
Quoted 2 lines
Each case MUST be covered, look:
https://elixir.bootlin.com/linux/v6.8/source/Documentation/devicetree/bindings/ufs/qcom,ufs.yaml#L191
Actually your case fits better another example from UFS, so take that one:
Best regards,
Krzysztof
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 123 lines · Krzysztof Kozlowski
Quoted 122 lines · Krzysztof Kozlowski
Quoted 85 lines · Jan Kiszka
Quoted 46 lines · Krzysztof Kozlowski
Quoted 45 lines · Jan Kiszka
Quoted 43 lines · Krzysztof Kozlowski
Quoted 29 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]--- .../bindings/pci/ti,am65-pci-host.yaml | 52 ++++++++++++++----- 1 file changed, 40 insertions(+), 12 deletions(-)Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,16 +19,6 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-9- reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNothing improved here.
Quoted 8 lines
Quoted 2 lines
Yes, explained the background to you. Sorry, if you do not address my replies, I'm lost with your feedback.
My magic ball could not figure out the problem, so did not provide the answer.
I gave you the exact code which illustrates how to do it. If you do it that way: it works. If you do it other way: it might not work. However
The link you provided was unfortunately not self-explanatory because if I - apparently - do it like that example, I'm getting the errors below.
Quoted 3 lines
without seeing anything, magic ball was silent, so I am not participating in game: would you be so kind to give more information so I won't waste my day in asking what is wrong.
With my patch:
# make ... dtbs_check DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6528-iot2050-basic-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-ekey-pcie.dtb OVL [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-m2-bkey-usb3.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-pg2.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-sm.dtb DTC [C] arch/arm64/boot/dts/ti/k3-am654-base-board.dtbWith this revert on top:
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-0@@ -19,6 +19,16 @@ properties: - ti,am654-pcie-rc - ti,keystone-pcieDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml properties: · line 30
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +9
-0+ reg: + maxItems: 4 + + reg-names: + items: + - const: app + - const: dbics + - const: config + - const: atuThere is nothing like that in that example.
https://elixir.bootlin.com/linux/v6.8/source/Documentation/devicetree/bindings/ufs/qcom,ufs.yaml#L44Quoted 9 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +1
-1+ interrupts: maxItems: 1@@ -104,18 +114,6 @@ then: - msi-map - num-viewport -else:Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml then: · line 117
Quoted 10 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +0
-10- properties: - reg: - maxItems: 4 - - reg-names: - items: - - const: app - - const: dbics - - const: config - - const: atuNeither this.
Quoted 7 lines
Quoted 2 lines
Each case MUST be covered, look:
https://elixir.bootlin.com/linux/v6.8/source/Documentation/devicetree/bindings/ufs/qcom,ufs.yaml#L191Actually your case fits better another example from UFS, so take that one:
Via lots of trial-and-error, I think I now understood the magic behind
this: You may reduce maxItems in a condition, but you cannot increase it. That's why all my intuition-based attempts failed before. Not sure if this is a specified property of the meta schema or just an oddity of the validator.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
In reply to Jan KiszkaDocumentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 92
Quoted 12 lines · Jan Kiszka
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +12
-0+ + reg-names: + minItems: 4 + items: + - const: app + - const: dbics + - const: config + - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region:
This also did not improve. You did not address any feedback from v3.
Missed feedback:
This *must* be defined in top-level.
I still think this must have some sort of maxItems. I accept your explanation that you could have multiple memory pools, but I don't think 2147000 pools is possible. Make it 4, 8 or 32.
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 99
Quoted 7 lines
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +7
-0+ minItems: 1 + description: | + phandle to one or more restricted DMA pools to be used for all devices + behind this controller. The regions should be defined according to + reserved-memory/shared-dma-pool.yaml. + items: + maxItems: 1
Best regards,
Krzysztof
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 16 lines · Krzysztof Kozlowski
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 92
Quoted 12 lines · Jan Kiszka
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +12
-0+ + reg-names: + minItems: 4 + items: + - const: app + - const: dbics + - const: config + - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region:This also did not improve. You did not address any feedback from v3.
Quoted 3 lines
Missed feedback:
This *must* be defined in top-level.
Answered in the other reply.
Quoted 3 lines
I still think this must have some sort of maxItems. I accept your explanation that you could have multiple memory pools, but I don't think 2147000 pools is possible. Make it 4, 8 or 32.
Can do - if you can explain the benefit of that.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Krzysztof Kozlowski
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 92
Quoted 12 lines · Jan Kiszka
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +12
-0+ + reg-names: + minItems: 4 + items: + - const: app + - const: dbics + - const: config + - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region:This also did not improve. You did not address any feedback from v3.
Quoted 5 lines
Quoted 3 lines
Missed feedback:
This *must* be defined in top-level.
Answered in the other reply.
I don't see anywhere answer, but regardless your binding is not an exception. Please follow rules for every binding, not expect something special here.
Best regards,
Krzysztof
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
In reply to Jan KiszkaQuoted 21 lines · Jan Kiszka
Quoted 16 lines · Krzysztof Kozlowski
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml if: · line 92
Quoted 12 lines · Jan Kiszka
Documentation/devicetree/bindings/pci/ti,am65-pci-host.yaml +12
-0+ + reg-names: + minItems: 4 + items: + - const: app + - const: dbics + - const: config + - const: atu + - const: vmap_lp + - const: vmap_hp + + memory-region:This also did not improve. You did not address any feedback from v3.
Quoted 3 lines
Missed feedback:
This *must* be defined in top-level.
Quoted 8 lines
Answered in the other reply.
Quoted 3 lines
I still think this must have some sort of maxItems. I accept your explanation that you could have multiple memory pools, but I don't think 2147000 pools is possible. Make it 4, 8 or 32.
Can do - if you can explain the benefit of that.
It turned out during further investigations that swiotlb actually only allows a single restricted DMA pool per device. So this discussion auto-resolves, bindings and code will be adjusted in v5.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 2/7] dt-bindings: PCI: ti,am65: Extend for use with PVU
In reply to Jan KiszkaQuoted 11 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The PVU on the AM65 SoC is capable of restricting DMA from PCIe devices to specific regions of host memory. Add the optional property "memory-regions" to point to such regions of memory when PVU is used.
Since the PVU deals with system physical addresses, utilizing the PVU with PCIe devices also requires setting up the VMAP registers to map the Requester ID of the PCIe device to the CBA Virtual ID, which in turn is mapped to the system physical address. Hence, describe the VMAP registers which are optionally unless the PVU shall used for PCIe.
s/optionally/optional/
s/shall used/should be/ ? (Not sure that's the sense you intend)
PATCH v4 3/7 soc: ti: Add IOMPU-like PVU driver
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The TI Peripheral Virtualization Unit (PVU) permits to define a limited set of mappings for DMA requests on the system memory. Unlike with an IOMMU, there is no fallback to a memory-backed page table, only a fixed set of register-backed TLBs. Emulating an IOMMU behavior appears to be the more fragile the more fragmentation of pending requests occur.
Therefore, this driver does not expose the PVU as an IOMMU. It rather introduces a simple, static interface to devices that are under restricted-dma-pool constraints. They can register their pools with the PVUs, enabling only those pools to work for DMA. As also MSI is issued as DMA, the PVU already register the related translator region of the AM654 as valid DMA target.
This driver is the essential building block for limiting DMA from untrusted devices to clearly defined memory regions in the absence of a real IOMMU (SMMU).
--- drivers/soc/ti/Kconfig | 4 + drivers/soc/ti/Makefile | 1 + drivers/soc/ti/ti-pvu.c | 487 ++++++++++++++++++++++++++++++++++++++++ include/linux/ti-pvu.h | 16 ++ 4 files changed, 508 insertions(+) create mode 100644 drivers/soc/ti/ti-pvu.c create mode 100644 include/linux/ti-pvu.h
drivers/soc/ti/Kconfig +4 -0
@@ -82,6 +82,10 @@ config TI_PRUSS processors on various TI SoCs. It's safe to say N here if you're not interested in the PRU or if you are unsure. +config TI_PVU + bool "TI Peripheral Virtualization Unit driver" + depends on ARCH_K3 && DMA_RESTRICTED_POOL + endif # SOC_TI config TI_SCI_INTA_MSI_DOMAIN
drivers/soc/ti/Makefile +1 -0
@@ -12,3 +12,4 @@ obj-$(CONFIG_TI_K3_RINGACC) += k3-ringacc.o obj-$(CONFIG_TI_K3_SOCINFO) += k3-socinfo.o obj-$(CONFIG_TI_PRUSS) += pruss.o obj-$(CONFIG_POWER_AVS_OMAP) += smartreflex.o +obj-$(CONFIG_TI_PVU) += ti-pvu.o
drivers/soc/ti/ti-pvu.c new file +487 -0
487 lines not on this page · raw message
include/linux/ti-pvu.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* TI Peripheral Virtualization Unit driver for static DMA isolation +* +* Copyright (c) 2024, Siemens AG +*/ + +#ifndef _LINUX_TI_PVU_H +#define _LINUX_TI_PVU_H + +#include <linux/ioport.h> + +int ti_pvu_create_region (unsigned int virt_id,const struct resource *region); +int ti_pvu_remove_region (unsigned int virt_id,const struct resource *region); + +#endif /* _LINUX_TI_PVU_H */
Signature · 1 line
2.43.0
PATCH v4 4/7 PCI: keystone: Add supported for PVU-based DMA isolation on AM654
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The AM654 lacks an IOMMU, thus does not support isolating DMA requests from untrusted PCI devices to selected memory regions this way. Use static PVU-based protection instead.
For this, we use the availability of restricted-dma-pool memory regions as trigger and register those as valid DMA targets with the PVU. In addition, we need to enable the mapping of requester IDs to VirtIDs in the PCI RC. We only use a single VirtID so far, catching all devices. This may be extended later on.
---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]
--- drivers/pci/controller/dwc/pci-keystone.c | 101 ++++++++++++++++++++++ 1 file changed, 101 insertions(+)
drivers/pci/controller/dwc/pci-keystone.c +101 -0
@@ -19,6 +19,7 @@#include <linux/mfd/syscon.h> #include <linux/msi.h> #include <linux/of.h> +#include <linux/of_address.h> #include <linux/of_irq.h> #include <linux/of_pci.h> #include <linux/phy/phy.h>
@@ -26,6 +27,7 @@#include <linux/regmap.h> #include <linux/resource.h> #include <linux/signal.h> +#include <linux/ti-pvu.h> #include "../../pci.h" #include "pcie-designware.h"
@@ -111,6 +113,16 @@#define PCI_DEVICE_ID_TI_AM654X 0xb00c +#define KS_PCI_VIRTID 0 + +#define PCIE_VMAP_xP_CTRL 0x0 +#define PCIE_VMAP_xP_REQID 0x4 +#define PCIE_VMAP_xP_VIRTID 0x8 + +#define PCIE_VMAP_xP_CTRL_EN BIT(0) + +#define PCIE_VMAP_xP_VIRTID_VID_MASK 0xfff +struct ks_pcie_of_data {enum dw_pcie_device_mode mode;const struct dw_pcie_host_ops *host_ops;
@@ -1125,6 +1137,89 @@ static const struct of_device_id ks_pcie_of_match[] = { { }, }; +#ifdef CONFIG_TI_PVU +static const char *ks_vmap_res[] = {"vmap_lp" ,"vmap_hp" }; + +static int ks_init_restricted_dma (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct of_phandle_iterator it; +bool init_vmap = false; +struct resource phys; +struct resource *res; +void __iomem *base; +unsigned int n; +u32 val; +int err; + +of_for_each_phandle (&it, err, dev->of_node,"memory-region" , +NULL ,0 ) { +if (!of_device_is_compatible (it.node,"restricted-dma-pool" )) +continue ; + + err =of_address_to_resource (it.node,0 , &phys); +if (err <0 ) { +dev_err (dev,"failed to parse memory region %pOF: %d\n" , + it.node, err); +continue ; + } + + err =ti_pvu_create_region (KS_PCI_VIRTID , &phys); +if (err <0 ) +return err; + + init_vmap = true; + } + +if (init_vmap) { +for (n =0 ; n <ARRAY_SIZE (ks_vmap_res); n++) { + res =platform_get_resource_byname (pdev,IORESOURCE_MEM , + ks_vmap_res[n]); + base =devm_pci_remap_cfg_resource (dev, res); +if (IS_ERR (base)) +return PTR_ERR (base); + +writel (0 , base + PCIE_VMAP_xP_REQID); + + val =readl (base + PCIE_VMAP_xP_VIRTID); + val &= ~PCIE_VMAP_xP_VIRTID_VID_MASK; + val |=KS_PCI_VIRTID ; +writel (val, base + PCIE_VMAP_xP_VIRTID); + + val =readl (base + PCIE_VMAP_xP_CTRL); + val |= PCIE_VMAP_xP_CTRL_EN; +writel (val, base + PCIE_VMAP_xP_CTRL); + } + } + +return 0 ; +} + +static void ks_release_restricted_dma (struct platform_device *pdev) +{ +struct of_phandle_iterator it; +struct resource phys; +int err; + +of_for_each_phandle (&it, err, pdev->dev.of_node,"memory-region" , +NULL ,0 ) { +if (of_device_is_compatible (it.node,"restricted-dma-pool" ) && +of_address_to_resource (it.node,0 , &phys) ==0 ) +ti_pvu_remove_region (KS_PCI_VIRTID , &phys); + + } +} +#else +static inline int ks_init_restricted_dma (struct platform_device *pdev) +{ +return 0 ; +} + +static inline void ks_release_restricted_dma (struct platform_device *pdev) +{ +} +#endif +static int ks_pcie_probe (struct platform_device *pdev) {const struct dw_pcie_host_ops *host_ops;
@@ -1273,6 +1368,10 @@ static int ks_pcie_probe(struct platform_device *pdev)if (ret <0 ) goto err_get_sync; + ret =ks_init_restricted_dma (pdev); +if (ret <0 ) + goto err_get_sync; +switch (mode) {case DW_PCIE_RC_TYPE :if (!IS_ENABLED (CONFIG_PCI_KEYSTONE_HOST )) {
@@ -1354,6 +1453,8 @@ static void ks_pcie_remove(struct platform_device *pdev)int num_lanes = ks_pcie->num_lanes;struct device *dev = &pdev->dev; +ks_release_restricted_dma (pdev); +pm_runtime_put (dev);pm_runtime_disable (dev);ks_pcie_disable_phy (ks_pcie);
Signature · 1 line
2.43.0
Re: [PATCH v4 4/7] PCI: keystone: Add supported for PVU-based DMA isolation on AM654
[+cc Kishon, just in case you have time/interest ;)]
Quoted 8 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The AM654 lacks an IOMMU, thus does not support isolating DMA requests from untrusted PCI devices to selected memory regions this way. Use static PVU-based protection instead.
For this, we use the availability of restricted-dma-pool memory regions as trigger and register those as valid DMA targets with the PVU.
I guess the implication is that DMA *outside* the restricted-dma-pool just gets dropped, and the Requester would see Completion Timeouts or something for reads?
Quoted 11 lines
In
addition, we need to enable the mapping of requester IDs to VirtIDs in the PCI RC. We only use a single VirtID so far, catching all devices. This may be extended later on.Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]
Regrettably we don't really have anybody taking care of pci-keystone.c (at least per MAINTAINERS).
Quoted 68 lines
--- drivers/pci/controller/dwc/pci-keystone.c | 101 ++++++++++++++++++++++ 1 file changed, 101 insertions(+)drivers/pci/controller/dwc/pci-keystone.c +35
-0@@ -19,6 +19,7 @@#include <linux/mfd/syscon.h> #include <linux/msi.h> #include <linux/of.h> +#include <linux/of_address.h> #include <linux/of_irq.h> #include <linux/of_pci.h> #include <linux/phy/phy.h> @@ -26,6 +27,7 @@#include <linux/regmap.h> #include <linux/resource.h> #include <linux/signal.h> +#include <linux/ti-pvu.h> #include "../../pci.h" #include "pcie-designware.h" @@ -111,6 +113,16 @@#define PCI_DEVICE_ID_TI_AM654X 0xb00c +#define KS_PCI_VIRTID 0 + +#define PCIE_VMAP_xP_CTRL 0x0 +#define PCIE_VMAP_xP_REQID 0x4 +#define PCIE_VMAP_xP_VIRTID 0x8 + +#define PCIE_VMAP_xP_CTRL_EN BIT(0) + +#define PCIE_VMAP_xP_VIRTID_VID_MASK 0xfff +struct ks_pcie_of_data {enum dw_pcie_device_mode mode;const struct dw_pcie_host_ops *host_ops;@@ -1125,6 +1137,89 @@ static const struct of_device_id ks_pcie_of_match[] = { { }, }; +#ifdef CONFIG_TI_PVU +static const char *ks_vmap_res[] = {"vmap_lp" ,"vmap_hp" }; + +static int ks_init_restricted_dma (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct of_phandle_iterator it; +bool init_vmap = false; +struct resource phys; +struct resource *res; +void __iomem *base; +unsigned int n; +u32 val; +int err; + +of_for_each_phandle (&it, err, dev->of_node,"memory-region" , +NULL ,0 ) { +if (!of_device_is_compatible (it.node,"restricted-dma-pool" )) +continue ; + + err =of_address_to_resource (it.node,0 , &phys); +if (err <0 ) { + dev_err(dev, "failed to parse memory region %pOF: %d\n",
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1172
Quoted 10 lines
drivers/pci/controller/dwc/pci-keystone.c +10
-0+ it.node, err); +continue ; + } + + err =ti_pvu_create_region (KS_PCI_VIRTID , &phys); +if (err <0 ) +return err; + + init_vmap = true; + }
if (!init_vmap)
return 0;
would unindent the following.
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1175
Quoted 3 lines
drivers/pci/controller/dwc/pci-keystone.c +3
-0+ +if (init_vmap) { +for (n =0 ; n <ARRAY_SIZE (ks_vmap_res); n++) {
Since the only use of ks_vmap_res is here, this might be more readable if there were a helper that would be called twice with the constant strings, e.g.,
helper(pdev, "vmap_lp"); helper(pdev, "vmap_hp");
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1177
Quoted 2 lines
drivers/pci/controller/dwc/pci-keystone.c +2
-0+ res =platform_get_resource_byname (pdev,IORESOURCE_MEM , + ks_vmap_res[n]);
Seems like we should check "res" for error before using it?
Quoted 4 lines
drivers/pci/controller/dwc/pci-keystone.c +4
-0+ base =devm_pci_remap_cfg_resource (dev, res); +if (IS_ERR (base)) +return PTR_ERR (base); +
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1191
Quoted 10 lines
drivers/pci/controller/dwc/pci-keystone.c +10
-0+writel (0 , base + PCIE_VMAP_xP_REQID); + + val =readl (base + PCIE_VMAP_xP_VIRTID); + val &= ~PCIE_VMAP_xP_VIRTID_VID_MASK; + val |=KS_PCI_VIRTID ; +writel (val, base + PCIE_VMAP_xP_VIRTID); + + val =readl (base + PCIE_VMAP_xP_CTRL); + val |= PCIE_VMAP_xP_CTRL_EN; +writel (val, base + PCIE_VMAP_xP_CTRL);
Since there's no explicit use of "restricted-dma-pool" elsewhere in this patch, I assume the setup above causes the controller to drop any DMA accesses outside that pool? I think a comment about how the controller behavior is being changed would be useful. Basically the same comment as for the commit log.
Would there be any value in a dmesg note about a restriction being enforced? Seems like it's dependent on both CONFIG_TI_PVU and some DT properties, and since those are invisible in the log, maybe a note would help understand/debug any issues?
Quoted 7 lines
drivers/pci/controller/dwc/pci-keystone.c +7
-0+ } + } + +return 0 ; +} + +static void ks_release_restricted_dma (struct platform_device *pdev)
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1208
Quoted 10 lines
drivers/pci/controller/dwc/pci-keystone.c +10
-0+{ +struct of_phandle_iterator it; +struct resource phys; +int err; + +of_for_each_phandle (&it, err, pdev->dev.of_node,"memory-region" , +NULL ,0 ) { +if (of_device_is_compatible (it.node,"restricted-dma-pool" ) && +of_address_to_resource (it.node,0 , &phys) ==0 ) +ti_pvu_remove_region (KS_PCI_VIRTID , &phys);
I guess it's not important to undo the PCIE_VMAP_xP_CTRL_EN and related setup that was done by ks_init_restricted_dma()?
Quoted 39 lines
drivers/pci/controller/dwc/pci-keystone.c +19
-0+ } +} +#else +static inline int ks_init_restricted_dma (struct platform_device *pdev) +{ +return 0 ; +} + +static inline void ks_release_restricted_dma (struct platform_device *pdev) +{ +} +#endif +static int ks_pcie_probe (struct platform_device *pdev) {const struct dw_pcie_host_ops *host_ops;@@ -1273,6 +1368,10 @@ static int ks_pcie_probe(struct platform_device *pdev)if (ret <0 ) goto err_get_sync; + ret =ks_init_restricted_dma (pdev); +if (ret <0 ) + goto err_get_sync; +switch (mode) {case DW_PCIE_RC_TYPE :if (!IS_ENABLED (CONFIG_PCI_KEYSTONE_HOST )) {@@ -1354,6 +1453,8 @@ static void ks_pcie_remove(struct platform_device *pdev)int num_lanes = ks_pcie->num_lanes;struct device *dev = &pdev->dev; +ks_release_restricted_dma (pdev); +pm_runtime_put (dev);pm_runtime_disable (dev);ks_pcie_disable_phy (ks_pcie);Signature · 1 line
2.43.0
Re: [PATCH v4 4/7] PCI: keystone: Add supported for PVU-based DMA isolation on AM654
Quoted 9 lines · Bjorn Helgaas
[+cc Kishon, just in case you have time/interest ;)]
Quoted 6 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The AM654 lacks an IOMMU, thus does not support isolating DMA requests from untrusted PCI devices to selected memory regions this way. Use static PVU-based protection instead.
Quoted 6 lines
Quoted 2 lines
For this, we use the availability of restricted-dma-pool memory regions as trigger and register those as valid DMA targets with the PVU.
I guess the implication is that DMA *outside* the restricted-dma-pool just gets dropped, and the Requester would see Completion Timeouts or something for reads?
I cannot tell what happens on the PCI bus in that case, maybe someone from TI can help out.
On the host side, the PVU will record an error and raise an interrupt which will make the driver report that to the kernel log. That's quite similar to what IOMMU drivers do on translation faults.
Quoted 108 lines
Quoted 6 lines
In
addition, we need to enable the mapping of requester IDs to VirtIDs in the PCI RC. We only use a single VirtID so far, catching all devices. This may be extended later on.Quoted 6 lines
Signed-off-by Jan Kiszka <[email protected]>---
CC: Lorenzo Pieralisi <[email protected]>
CC: "Krzysztof Wilczyński" <[email protected]>
CC: Bjorn Helgaas <[email protected]>
CC: [email protected]Regrettably we don't really have anybody taking care of
pci-keystone.c(at least perMAINTAINERS).Quoted 68 lines
--- drivers/pci/controller/dwc/pci-keystone.c | 101 ++++++++++++++++++++++ 1 file changed, 101 insertions(+)drivers/pci/controller/dwc/pci-keystone.c +35
-0@@ -19,6 +19,7 @@#include <linux/mfd/syscon.h> #include <linux/msi.h> #include <linux/of.h> +#include <linux/of_address.h> #include <linux/of_irq.h> #include <linux/of_pci.h> #include <linux/phy/phy.h> @@ -26,6 +27,7 @@#include <linux/regmap.h> #include <linux/resource.h> #include <linux/signal.h> +#include <linux/ti-pvu.h> #include "../../pci.h" #include "pcie-designware.h" @@ -111,6 +113,16 @@#define PCI_DEVICE_ID_TI_AM654X 0xb00c +#define KS_PCI_VIRTID 0 + +#define PCIE_VMAP_xP_CTRL 0x0 +#define PCIE_VMAP_xP_REQID 0x4 +#define PCIE_VMAP_xP_VIRTID 0x8 + +#define PCIE_VMAP_xP_CTRL_EN BIT(0) + +#define PCIE_VMAP_xP_VIRTID_VID_MASK 0xfff +struct ks_pcie_of_data {enum dw_pcie_device_mode mode;const struct dw_pcie_host_ops *host_ops;@@ -1125,6 +1137,89 @@ static const struct of_device_id ks_pcie_of_match[] = { { }, }; +#ifdef CONFIG_TI_PVU +static const char *ks_vmap_res[] = {"vmap_lp" ,"vmap_hp" }; + +static int ks_init_restricted_dma (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct of_phandle_iterator it; +bool init_vmap = false; +struct resource phys; +struct resource *res; +void __iomem *base; +unsigned int n; +u32 val; +int err; + +of_for_each_phandle (&it, err, dev->of_node,"memory-region" , +NULL ,0 ) { +if (!of_device_is_compatible (it.node,"restricted-dma-pool" )) +continue ; + + err =of_address_to_resource (it.node,0 , &phys); +if (err <0 ) { + dev_err(dev, "failed to parse memory region %pOF: %d\n",drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1172
Quoted 10 lines
drivers/pci/controller/dwc/pci-keystone.c +10
-0+ it.node, err); +continue ; + } + + err =ti_pvu_create_region (KS_PCI_VIRTID , &phys); +if (err <0 ) +return err; + + init_vmap = true; + }if (!init_vmap)
return 0;would unindent the following.
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1175
Quoted 3 lines
drivers/pci/controller/dwc/pci-keystone.c +3
-0+ +if (init_vmap) { +for (n =0 ; n <ARRAY_SIZE (ks_vmap_res); n++) {Since the only use of
ks_vmap_resis here, this might be more readable if there were a helper that would be called twice with the constant strings, e.g.,
Quoted 2 lines
helper(pdev, "vmap_lp"); helper(pdev, "vmap_hp");
OK.
Quoted 5 lines
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1177
Quoted 3 lines
drivers/pci/controller/dwc/pci-keystone.c +2
-0+ res =platform_get_resource_byname (pdev,IORESOURCE_MEM , + ks_vmap_res[n]);Seems like we should check "res" for error before using it?
Oh, unfinished constructions.
Quoted 13 lines
Quoted 13 lines
drivers/pci/controller/dwc/pci-keystone.c +12
-0+ base =devm_pci_remap_cfg_resource (dev, res); +if (IS_ERR (base)) +return PTR_ERR (base); + +writel (0 , base + PCIE_VMAP_xP_REQID); + + val =readl (base + PCIE_VMAP_xP_VIRTID); + val &= ~PCIE_VMAP_xP_VIRTID_VID_MASK; + val |=KS_PCI_VIRTID ; +writel (val, base + PCIE_VMAP_xP_VIRTID); + + val =readl (base + PCIE_VMAP_xP_CTRL);
Quoted 8 lines
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1191
Quoted 2 lines
drivers/pci/controller/dwc/pci-keystone.c +2
-0+ val |= PCIE_VMAP_xP_CTRL_EN; +writel (val, base + PCIE_VMAP_xP_CTRL);Since there's no explicit use of "restricted-dma-pool" elsewhere in this patch, I assume the setup above causes the controller to drop any DMA accesses outside that pool? I think a comment about how the controller behavior is being changed would be useful. Basically the same comment as for the commit log.
Right, this is what will happen. Will add some comment.
Quoted 5 lines
Would there be any value in a dmesg note about a restriction being enforced? Seems like it's dependent on both
CONFIG_TI_PVUand some DT properties, and since those are invisible in the log, maybe a note would help understand/debug any issues?
This is what you will see when there are reserved region and PVU in play:
keystone-pcie 5600000.pcie: assigned reserved memory node restricted-dma@c0000000 ti-pvu 30f80000.iommu: created TLB entry 0.2: 0xc0000000, psize 4 (0x02000000) ti-pvu 30f80000.iommu: created TLB entry 0.3: 0xc2000000, psize 4 (0x02000000)
...
ath9k 0000:01:00.0: assigned reserved memory node restricted-dma@c0000000
Quoted 13 lines
Quoted 13 lines
drivers/pci/controller/dwc/pci-keystone.c +12
-0+ } + } + +return 0 ; +} + +static void ks_release_restricted_dma (struct platform_device *pdev) +{ +struct of_phandle_iterator it; +struct resource phys; +int err; +
Quoted 9 lines
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1208
Quoted 5 lines
drivers/pci/controller/dwc/pci-keystone.c +5
-0+of_for_each_phandle (&it, err, pdev->dev.of_node,"memory-region" , +NULL ,0 ) { +if (of_device_is_compatible (it.node,"restricted-dma-pool" ) && +of_address_to_resource (it.node,0 , &phys) ==0 ) +ti_pvu_remove_region (KS_PCI_VIRTID , &phys);I guess it's not important to undo the
PCIE_VMAP_xP_CTRL_ENand related setup that was done byks_init_restricted_dma()?
Right, I didn't find a reason to do that.
Quoted 31 lines
Quoted 31 lines
drivers/pci/controller/dwc/pci-keystone.c +17
-0+ } +} +#else +static inline int ks_init_restricted_dma (struct platform_device *pdev) +{ +return 0 ; +} + +static inline void ks_release_restricted_dma (struct platform_device *pdev) +{ +} +#endif +static int ks_pcie_probe (struct platform_device *pdev) {const struct dw_pcie_host_ops *host_ops;@@ -1273,6 +1368,10 @@ static int ks_pcie_probe(struct platform_device *pdev)if (ret <0 ) goto err_get_sync; + ret =ks_init_restricted_dma (pdev); +if (ret <0 ) + goto err_get_sync; +switch (mode) {case DW_PCIE_RC_TYPE :if (!IS_ENABLED (CONFIG_PCI_KEYSTONE_HOST )) {@@ -1354,6 +1453,8 @@ static void ks_pcie_remove(struct platform_device *pdev)int num_lanes = ks_pcie->num_lanes;struct device *dev = &pdev->dev;
Quoted 8 lines
Quoted 8 lines
drivers/pci/controller/dwc/pci-keystone.c +2
-0+ks_release_restricted_dma (pdev); +pm_runtime_put (dev);pm_runtime_disable (dev);ks_pcie_disable_phy (ks_pcie);Signature · 1 line
2.43.0
Thanks,
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
Re: [PATCH v4 4/7] PCI: keystone: Add supported for PVU-based DMA isolation on AM654
Quoted 17 lines · Jan Kiszka
Quoted 9 lines · Bjorn Helgaas
[+cc Kishon, just in case you have time/interest ;)]
Quoted 6 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
The AM654 lacks an IOMMU, thus does not support isolating DMA requests from untrusted PCI devices to selected memory regions this way. Use static PVU-based protection instead.
Quoted 6 lines
Quoted 2 lines
For this, we use the availability of restricted-dma-pool memory regions as trigger and register those as valid DMA targets with the PVU.
I guess the implication is that DMA *outside* the restricted-dma-pool just gets dropped, and the Requester would see Completion Timeouts or something for reads?
Quoted 6 lines
I cannot tell what happens on the PCI bus in that case, maybe someone from TI can help out.
On the host side, the PVU will record an error and raise an interrupt which will make the driver report that to the kernel log. That's quite similar to what IOMMU drivers do on translation faults.
The main thing is that the DMA doesn't complete, as you mentioned below.
Quoted 13 lines
Quoted 5 lines
Since there's no explicit use of "restricted-dma-pool" elsewhere in this patch, I assume the setup above causes the controller to drop any DMA accesses outside that pool? I think a comment about how the controller behavior is being changed would be useful. Basically the same comment as for the commit log.
Right, this is what will happen. Will add some comment.
Quoted 4 lines
Would there be any value in a dmesg note about a restriction being enforced? Seems like it's dependent on both
CONFIG_TI_PVUand some DT properties, and since those are invisible in the log, maybe a note would help understand/debug any issues?
Quoted 8 lines
This is what you will see when there are reserved region and PVU in play:
keystone-pcie 5600000.pcie: assigned reserved memory node restricted-dma@c0000000 ti-pvu 30f80000.iommu: created TLB entry 0.2:
0xc0000000, psize 4 (0x02000000) ti-pvu 30f80000.iommu: created TLB entry 0.3:0xc2000000, psize 4 (0x02000000)
...
ath9k 0000:01:00.0: assigned reserved memory node restricted-dma@c0000000
Looks reasonable and solves my concern.
Quoted 11 lines
Quoted 9 lines
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1208
Quoted 5 lines
drivers/pci/controller/dwc/pci-keystone.c +5
-0+of_for_each_phandle (&it, err, pdev->dev.of_node,"memory-region" , +NULL ,0 ) { +if (of_device_is_compatible (it.node,"restricted-dma-pool" ) && +of_address_to_resource (it.node,0 , &phys) ==0 ) +ti_pvu_remove_region (KS_PCI_VIRTID , &phys);I guess it's not important to undo the
PCIE_VMAP_xP_CTRL_ENand related setup that was done byks_init_restricted_dma()?Right, I didn't find a reason to do that.
OK, as long as you considered it :)
Bjorn
Re: [PATCH v4 4/7] PCI: keystone: Add supported for PVU-based DMA isolation on AM654
In reply to Jan KiszkaQuoted 7 lines · Jan Kiszka
Quoted 4 lines · Bjorn Helgaas
drivers/pci/controller/dwc/pci-keystone.c static const struct of_device_id ks_pcie_of_match[] = { · line 1177
Quoted 2 lines
drivers/pci/controller/dwc/pci-keystone.c +2
-0+ res =platform_get_resource_byname (pdev,IORESOURCE_MEM , + ks_vmap_res[n]);Seems like we should check "res" for error before using it?
Oh, unfinished constructions.
In fact, devm_pci_remap_cfg_resource takes care of res == NULL.
Jan
Signature · 2 lines
Siemens AG, Technology Linux Expert Center
PATCH v4 5/7 arm64: dts: ti: k3-am65-main: Add PVU nodes
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Add nodes for the two PVUs of the AM65. Keep them disabled, though, because the board has to additionally define DMA pools and the devices to be isolated.
--- arch/arm64/boot/dts/ti/k3-am65-main.dtsi | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/arm64/boot/dts/ti/k3-am65-main.dtsi +20 -0
@@ -841,6 +841,26 @@ main_cpts_mux: refclk-mux { assigned-clock-parents = <&k3_clks 118 5>; }; }; + + ti_pvu0: iommu@30f80000 { + compatible = "ti,am654-pvu"; + reg = <0 0x30f80000 0 0x1000>, + <0 0x36000000 0 0x100000>; + reg-names = "cfg", "tlbif"; + interrupts-extended = <&intr_main_navss 390>; + interrupt-names = "pvu"; + status = "disabled"; + }; + + ti_pvu1: iommu@30f81000 { + compatible = "ti,am654-pvu"; + reg = <0 0x30f81000 0 0x1000>, + <0 0x36100000 0 0x100000>; + reg-names = "cfg", "tlbif"; + interrupts-extended = <&intr_main_navss 389>; + interrupt-names = "pvu"; + status = "disabled"; + }; }; main_gpio0: gpio@600000 {
Signature · 1 line
2.43.0
PATCH v4 6/7 arm64: dts: ti: k3-am65-main: Add VMAP registers to PCI root complexes
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Rewrap the long lines at this chance.
--- arch/arm64/boot/dts/ti/k3-am65-main.dtsi | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
arch/arm64/boot/dts/ti/k3-am65-main.dtsi +14 -4
@@ -895,8 +895,13 @@ main_gpio1: gpio@601000 { pcie0_rc: pcie@5500000 { compatible = "ti,am654-pcie-rc"; - reg = <0x0 0x5500000 0x0 0x1000>, <0x0 0x5501000 0x0 0x1000>, <0x0 0x10000000 0x0 0x2000>, <0x0 0x5506000 0x0 0x1000>; - reg-names = "app", "dbics", "config", "atu"; + reg = <0x0 0x5500000 0x0 0x1000>, + <0x0 0x5501000 0x0 0x1000>, + <0x0 0x10000000 0x0 0x2000>, + <0x0 0x5506000 0x0 0x1000>, + <0x0 0x2900000 0x0 0x1000>, + <0x0 0x2908000 0x0 0x1000>; + reg-names = "app", "dbics", "config", "atu", "vmap_lp", "vmap_hp"; power-domains = <&k3_pds 120 TI_SCI_PD_EXCLUSIVE>; #address-cells = <3>; #size-cells = <2>;
@@ -916,8 +921,13 @@ pcie0_rc: pcie@5500000 { pcie1_rc: pcie@5600000 { compatible = "ti,am654-pcie-rc"; - reg = <0x0 0x5600000 0x0 0x1000>, <0x0 0x5601000 0x0 0x1000>, <0x0 0x18000000 0x0 0x2000>, <0x0 0x5606000 0x0 0x1000>; - reg-names = "app", "dbics", "config", "atu"; + reg = <0x0 0x5600000 0x0 0x1000>, + <0x0 0x5601000 0x0 0x1000>, + <0x0 0x18000000 0x0 0x2000>, + <0x0 0x5606000 0x0 0x1000>, + <0x0 0x2910000 0x0 0x1000>, + <0x0 0x2918000 0x0 0x1000>; + reg-names = "app", "dbics", "config", "atu", "vmap_lp", "vmap_hp"; power-domains = <&k3_pds 121 TI_SCI_PD_EXCLUSIVE>; #address-cells = <3>; #size-cells = <2>;
Signature · 1 line
2.43.0
PATCH v4 7/7 arm64: dts: ti: iot2050: Enforce DMA isolation for devices behind PCI RC on Advanced
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Reserve a 64M memory region and ensure that all PCI devices do their DMA only inside that region. This is configured via a restricted-dma-pool and enforced with the help of the first PVU.
The pool size may be adjusted via DT fixup during boot by the firmware. The location is chosen to allow up to 512M for special needs without causing conflicts.
Applying this isolation which is not totally free in terms of overhead and memory consumption makes only sense for variants that support secure booting. These are the advanced-class variants.
--- .../ti/k3-am6548-iot2050-advanced-common.dtsi | 21 ++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
arch/arm64/boot/dts/ti/k3-am6548-iot2050-advanced-common.dtsi +20 -1
@@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * Copyright (c) Siemens AG, 2018-2021 + * Copyright (c) Siemens AG, 2018-2024 * * Authors: * Le Jin <[email protected]>
@@ -20,6 +20,25 @@ memory@80000000 { /* 2G RAM */ reg = <0x00000000 0x80000000 0x00000000 0x80000000>; }; + + reserved-memory { + pci_restricted_dma_region: restricted-dma@c0000000 { + compatible = "restricted-dma-pool"; + reg = <0 0xc0000000 0 0x4000000>; + }; + }; +}; + +&pcie0_rc { + memory-region = <&pci_restricted_dma_region>; +}; + +&pcie1_rc { + memory-region = <&pci_restricted_dma_region>; +}; + +&ti_pvu0 { + status = "okay"; }; &main_pmx0 {
Signature · 1 line
2.43.0