arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep
Subsystems (2)
PATCH 1/1 arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep
Support for msi-map in the pci-ep device tree binding was added in commit a6aed6b9c79e ("dt-bindings: PCI: pci-ep: Add support for iommu-map and msi-map").
The PCI endpoint doorbell feature was added in commit 1c3b002c6bf6 ("PCI:
endpoint: Add RC-to-EP doorbell support using platform MSI controller").
The PCI endpoint doorbell feature requires:
-An interrupt controller that implements GIC interrupt translation
services (ITS).
-msi-map being defined in the pcie endpoint device tree node. -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.
Add msi-map to the pcie3x4_ep device tree node.
With this, the pci endpoint kselftest doorbell test case passes:
# pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST TAP version 13 1..1 # Starting 1 tests from 1 test cases. # RUN pcie_ep_doorbell.DOORBELL_TEST ... # OK pcie_ep_doorbell.DOORBELL_TEST ok 1 pcie_ep_doorbell.DOORBELL_TEST # PASSED: 1 / 1 tests passed.
--- arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 + 1 file changed, 1 insertion(+)
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi +1 -0
@@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 { interrupt-names = "sys", "pmc", "msg", "legacy", "err", "dma0", "dma1", "dma2", "dma3"; max-link-speed = <3>; + msi-map = <0x0000 &its1 0x0000 0x1000>; num-lanes = <4>; phys = <&pcie30phy>; phy-names = "pcie-phy";
Signature · 7 lines
2.51.0 _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip
Quoted 36 lines · Niklas Cassel
Support for msi-map in the pci-ep device tree binding was added in commit
a6aed6b9c79e("dt-bindings: PCI: pci-ep: Add support for iommu-map and msi-map").The PCI endpoint doorbell feature was added in commit
1c3b002c6bf6("PCI:
endpoint: Add RC-to-EP doorbell support using platform MSI controller").The PCI endpoint doorbell feature requires:
(quoted diff) +0
-3-An interrupt controller that implements GIC interrupt translation services (ITS). -msi-map being defined in the pcie endpoint device tree node. -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.Add msi-map to the
pcie3x4_epdevice tree node.With this, the pci endpoint kselftest doorbell test case passes:
# pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST TAP version 13 1..1 # Starting 1 tests from 1 test cases. # RUN pcie_ep_doorbell.DOORBELL_TEST ... # OK pcie_ep_doorbell.DOORBELL_TEST ok 1 pcie_ep_doorbell.DOORBELL_TEST # PASSED: 1 / 1 tests passed.Cc Frank Li <[email protected]>Signed-off-by Niklas Cassel <[email protected]>--- arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 + 1 file changed, 1 insertion(+)arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi +0
-0
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi pcie3x4_ep: pcie-ep@fe150000 { · line 392
Quoted 4 lines
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi +1
-0@@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 { interrupt-names = "sys", "pmc", "msg", "legacy", "err", "dma0", "dma1", "dma2", "dma3"; max-link-speed = <3>; + msi-map = <0x0000 &its1 0x0000 0x1000>;
Not sure how your hardware map PCIe rid at EP side. If your hardware direct use RC's RID, difference host may use differecne RID. I remember you met similar problem when enable IOMMU.
Generally, Only 1 EP support, you can use msi-mask=<0> map all RID to a fixed value, But it also needs change glue layer configure to force to use the same RID.
I suppose this may not work if connected to second PCIe host slot.
Frank
Quoted 11 lines
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi +0
-1num-lanes = <4>; phys = <&pcie30phy>; phy-names = "pcie-phy"; --2.51.0
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
Quoted 32 lines · Frank Li
Quoted 31 lines · Niklas Cassel
Support for msi-map in the pci-ep device tree binding was added in commit
a6aed6b9c79e("dt-bindings: PCI: pci-ep: Add support for iommu-map and msi-map").The PCI endpoint doorbell feature was added in commit
1c3b002c6bf6("PCI:
endpoint: Add RC-to-EP doorbell support using platform MSI controller").The PCI endpoint doorbell feature requires:
(quoted diff) +0
-3-An interrupt controller that implements GIC interrupt translation services (ITS). -msi-map being defined in the pcie endpoint device tree node. -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.Add msi-map to the
pcie3x4_epdevice tree node.With this, the pci endpoint kselftest doorbell test case passes:
# pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST TAP version 13 1..1 # Starting 1 tests from 1 test cases. # RUN pcie_ep_doorbell.DOORBELL_TEST ... # OK pcie_ep_doorbell.DOORBELL_TEST ok 1 pcie_ep_doorbell.DOORBELL_TEST # PASSED: 1 / 1 tests passed.Cc Frank Li <[email protected]>Signed-off-by Niklas Cassel <[email protected]>--- arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 + 1 file changed, 1 insertion(+)
Quoted 13 lines
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi pcie3x4_ep: pcie-ep@fe150000 { · line 392
Quoted 9 lines
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi +1
-0@@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 { interrupt-names = "sys", "pmc", "msg", "legacy", "err", "dma0", "dma1", "dma2", "dma3"; max-link-speed = <3>; + msi-map = <0x0000 &its1 0x0000 0x1000>;Not sure how your hardware map PCIe rid at EP side. If your hardware direct use RC's RID, difference host may use differecne RID. I remember you met similar problem when enable IOMMU.
Yes, the Requester ID will be the BDF of the Root Complex which is sending the request to the endpoint.
Quoted 6 lines
Generally, Only 1 EP support, you can use msi-mask=<0> map all RID to a fixed value, But it also needs change glue layer configure to force to use the same RID.
I suppose this may not work if connected to second PCIe host slot.
I tested the patch with three different hosts.
On Intel and Rock5b host, this patch both worked fine:
ok 1 pcie_ep_doorbell.DOORBELL_TEST
On another ARM64 board, this patch did not work.
I tried adding msi-map-mask = <0>;
But still no luck.
I'm not sure what is missing in the glue driver, because AFAICT, the logic parsing the msi-map-mask is already in drivers/irqchip/irq-gic-its-msi-parent.c
Also, the device tree binding inDocumentation/devicetree/bindings/pci/pci-ep.yaml
Does mention that:
- Bits [2:0] for the function number (func)
- Bits [18:3] for the virtual function index (vfunc)
However, the EP cannot rely on Requester ID (RID) because the RID is determined by the PCI topology of the host system. Since the EP may be connected to different PCI hosts, the RID can vary between systems and is therefore not a reliable identifier.
The resulting device ID is computed as:
(func & 0x7) | (vfunc << 3)
The property is an arbitrary number of tuples of
(device-id-base, msi, msi-base,length).
Thus, I don't see that the Requester ID is used at all.
Anyway, considering that this patch does not work with all hosts, it is best to not merge this patch for now.
Kind regards,
Niklas
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
Quoted 63 lines · Niklas Cassel
Quoted 32 lines · Frank Li
Quoted 31 lines · Niklas Cassel
Support for msi-map in the pci-ep device tree binding was added in commit
a6aed6b9c79e("dt-bindings: PCI: pci-ep: Add support for iommu-map and msi-map").The PCI endpoint doorbell feature was added in commit
1c3b002c6bf6("PCI:
endpoint: Add RC-to-EP doorbell support using platform MSI controller").The PCI endpoint doorbell feature requires:
(quoted diff) +0
-3-An interrupt controller that implements GIC interrupt translation services (ITS). -msi-map being defined in the pcie endpoint device tree node. -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.Add msi-map to the
pcie3x4_epdevice tree node.With this, the pci endpoint kselftest doorbell test case passes:
# pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST TAP version 13 1..1 # Starting 1 tests from 1 test cases. # RUN pcie_ep_doorbell.DOORBELL_TEST ... # OK pcie_ep_doorbell.DOORBELL_TEST ok 1 pcie_ep_doorbell.DOORBELL_TEST # PASSED: 1 / 1 tests passed.Cc Frank Li <[email protected]>Signed-off-by Niklas Cassel <[email protected]>--- arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 + 1 file changed, 1 insertion(+)Quoted 13 lines
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi pcie3x4_ep: pcie-ep@fe150000 { · line 392
Quoted 9 lines
arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi +1
-0@@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 { interrupt-names = "sys", "pmc", "msg", "legacy", "err", "dma0", "dma1", "dma2", "dma3"; max-link-speed = <3>; + msi-map = <0x0000 &its1 0x0000 0x1000>;Not sure how your hardware map PCIe rid at EP side. If your hardware direct use RC's RID, difference host may use differecne RID. I remember you met similar problem when enable IOMMU.
Yes, the Requester ID will be the BDF of the Root Complex which is sending the request to the endpoint.
Quoted 6 lines
Generally, Only 1 EP support, you can use msi-mask=<0> map all RID to a fixed value, But it also needs change glue layer configure to force to use the same RID.
I suppose this may not work if connected to second PCIe host slot.
I tested the patch with three different hosts.
On Intel and Rock5b host, this patch both worked fine:
ok 1 pcie_ep_doorbell.DOORBELL_TEST
Quoted 8 lines
On another ARM64 board, this patch did not work.
I tried adding msi-map-mask = <0>;But still no luck.
I'm not sure what is missing in the glue driver, because AFAICT, the logic parsing the msi-map-mask is already indrivers/irqchip/irq-gic-its-msi-parent.c
The key problem is we don't know EP's RID (RC send out TLP package), which is transparent to software.
So layerscape have internal logic to map RID to fixed (predictable) stream ID, which is neccessary for EP function if use IOMMU or MSI, or you can capture RID from first cfg access from HOST.
Maybe there already have it in this SOC, suggest add it in future chips if not support yet.
Frank
Quoted 27 lines
Also, the device tree binding in
Documentation/devicetree/bindings/pci/pci-ep.yamlDoes mention that:
- Bits [2:0] for the function number (func)
- Bits [18:3] for the virtual function index (vfunc)However, the EP cannot rely on Requester ID (RID) because the RID is determined by the PCI topology of the host system. Since the EP may be connected to different PCI hosts, the RID can vary between systems and is therefore not a reliable identifier.
The resulting device ID is computed as:
(func &
0x7) | (vfunc << 3)The property is an arbitrary number of tuples of
(device-id-base, msi, msi-base,length).Thus, I don't see that the Requester ID is used at all.
Anyway, considering that this patch does not work with all hosts, it is best to not merge this patch for now.
Quoted 7 lines
Kind regards,
Niklas_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip