Skip to contentSkip to conversation

PCI: endpoint: Add Tegra194/234 BAR layout and pci_endpoint_test support

PATCH Applied Manikanta Maddireddy 2 participants 6 messages Mar 24 – Apr 4

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v3 cover PCI: endpoint: Add Tegra194/234 BAR layout and pci_endpoint_test support

This series is on top of https://lore.kernel.org/linux-pci/[email protected]/T/#u

This series wires up Tegra194 and Tegra234 PCI endpoint controllers to the shared PCI endpoint and test infrastructure:

1. Add a new reserved-region type for MSI-X (Table and PBA) so EPC drivers can describe hardware-owned MSI-X regions behind a BAR_RESERVED BAR.

2. Make Tegra194 BAR0 programmable and drop the 1MB fixed size so EPF drivers can use it with arbitrary sizes via the existing DBI2 BAR programmable path.

3. Expose Tegra194 BAR2 (MSI-X) and BAR4 (DMA) as 64-bit BAR_RESERVED with the appropriate reserved subregions (MSI-X 128KB, DMA 4KB), keeping CONSECUTIVE_BAR_TEST working while allowing the host to use these BARs.

4. Add Tegra194 and Tegra234 to the pci_endpoint_test device table so the endpoint test driver can bind and run on these controllers.

Tested with pci_endpoint_test on Tegra194/Tegra234 endpoint.

./pci_endpoint_test -f pci_ep_bar -f pci_ep_basic -v memcpy -T COPY_TEST -V dma TAP version 13
1..13
# Starting 13 tests from 8 test cases.
# RUN pci_ep_bar.BAR0.BAR_TEST ...
# OK pci_ep_bar.BAR0.BAR_TEST
ok 1 pci_ep_bar.BAR0.BAR_TEST

#  RUN           pci_ep_bar.BAR1.BAR_TEST ...
#      SKIP      BAR is disabled
#            OK  pci_ep_bar.BAR1.BAR_TEST
ok 2 pci_ep_bar.BAR1.BAR_TEST # SKIP BAR is disabled
#  RUN           pci_ep_bar.BAR2.BAR_TEST ...
#      SKIP      BAR is reserved
#            OK  pci_ep_bar.BAR2.BAR_TEST
ok 3 pci_ep_bar.BAR2.BAR_TEST # SKIP BAR is reserved
#  RUN           pci_ep_bar.BAR3.BAR_TEST ...
#      SKIP      BAR is disabled
#            OK  pci_ep_bar.BAR3.BAR_TEST
ok 4 pci_ep_bar.BAR3.BAR_TEST # SKIP BAR is disabled
#  RUN           pci_ep_bar.BAR4.BAR_TEST ...
#      SKIP      BAR is reserved
#            OK  pci_ep_bar.BAR4.BAR_TEST
ok 5 pci_ep_bar.BAR4.BAR_TEST # SKIP BAR is reserved
#  RUN           pci_ep_bar.BAR5.BAR_TEST ...
#      SKIP      BAR is disabled
#            OK  pci_ep_bar.BAR5.BAR_TEST

ok 6 pci_ep_bar.BAR5.BAR_TEST # SKIP BAR is disabled
# RUN pci_ep_basic.CONSECUTIVE_BAR_TEST ...
# OK pci_ep_basic.CONSECUTIVE_BAR_TEST
ok 7 pci_ep_basic.CONSECUTIVE_BAR_TEST
# RUN pci_ep_basic.LEGACY_IRQ_TEST ...
# OK pci_ep_basic.LEGACY_IRQ_TEST
ok 8 pci_ep_basic.LEGACY_IRQ_TEST

#  RUN           pci_ep_basic.MSI_TEST ...
#      SKIP      MSI17 is disabled
#            OK  pci_ep_basic.MSI_TEST

ok 9 pci_ep_basic.MSI_TEST # SKIP MSI17 is disabled
# RUN pci_ep_basic.MSIX_TEST ...
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X1
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X2
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X3
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X4
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X5
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X6
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X7
# pci_endpoint_test.c:146:MSIX_TEST:Expected 0 (0) == ret (-5)
# pci_endpoint_test.c:146:MSIX_TEST:Test failed for MSI-X8
# SKIP MSI-X9 is disabled
# OK pci_ep_basic.MSIX_TEST
ok 10 pci_ep_basic.MSIX_TEST # SKIP MSI-X9 is disabled
# RUN pci_ep_data_transfer.memcpy.READ_TEST ...
# OK pci_ep_data_transfer.memcpy.READ_TEST
ok 11 pci_ep_data_transfer.memcpy.READ_TEST
# RUN pci_ep_data_transfer.memcpy.WRITE_TEST ...
# OK pci_ep_data_transfer.memcpy.WRITE_TEST
ok 12 pci_ep_data_transfer.memcpy.WRITE_TEST
# RUN pci_ep_data_transfer.memcpy.COPY_TEST ...
# OK pci_ep_data_transfer.memcpy.COPY_TEST
ok 13 pci_ep_data_transfer.memcpy.COPY_TEST
# PASSED: 13 / 13 tests passed.
# 7 skipped test(s) detected. Consider enabling relevant config options to improve coverage. # Totals: pass:6 fail:0 xfail:0 xpass:0 skip:7 error:0

lspci output displays all three BARs with this series,

0005:01:00.0 Unassigned class [ff00]: NVIDIA Corporation Device 229b
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 165
IOMMU group: 13
Region 0: Memory at 2b28001000 (64-bit, non-prefetchable) [size=256]
Region 2: Memory at 2800000000 (64-bit, prefetchable) [size=128K]
Region 4: Memory at 2b28000000 (64-bit, non-prefetchable) [size=4K]

Manikanta Maddireddy (4):
PCI: endpoint: Add reserved region type for MSI-X Table and PBA
PCI: tegra194: Make BAR0 programmable and remove 1MB size limit
PCI: tegra194: Expose BAR2 (MSI-X) and BAR4 (DMA) as 64-bit
BAR_RESERVED
misc: pci_endpoint_test: Add Tegra194 and Tegra234 device table
entries

 drivers/misc/pci_endpoint_test.c           |  5 +++
 drivers/pci/controller/dwc/pcie-tegra194.c | 44 +++++++++++++++++++---
 include/linux/pci-epc.h                    |  4 ++
 3 files changed, 47 insertions(+), 6 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v3 1/4 PCI: endpoint: Add reserved region type for MSI-X Table and PBA

Add PCI_EPC_BAR_RSVD_MSIX_TBL_RAM and PCI_EPC_BAR_RSVD_MSIX_PBA_RAM to enum pci_epc_bar_rsvd_region_type so that Endpoint controllers can describe hardware-owned MSI-X Table and PBA (Pending Bit Array) regions behind a BAR_RESERVED BAR.

---
v3: Change 'PBA structures' to 'PBA structure'
v2: Split MSI-X pci_epc_bar_rsvd_region_type for both MSI-X table and PBA

 include/linux/pci-epc.h | 4 ++++
 1 file changed, 4 insertions(+)
include/linux/pci-epc.h +4 -0
index 334c2b7578d0..1eca1264815b 100644
@@ -211,6 +211,8 @@ enum pci_epc_bar_type {
 /**
  * enum pci_epc_bar_rsvd_region_type - type of a fixed subregion behind a BAR
  * @PCI_EPC_BAR_RSVD_DMA_CTRL_MMIO: Integrated DMA controller MMIO window
+ * @PCI_EPC_BAR_RSVD_MSIX_TBL_RAM: MSI-X table structure
+ * @PCI_EPC_BAR_RSVD_MSIX_PBA_RAM: MSI-X PBA structure
  *
  * BARs marked BAR_RESERVED are owned by the SoC/EPC hardware and must not be
  * reprogrammed by EPF drivers. Some of them still expose fixed subregions that
@@ -218,6 +220,8 @@ enum pci_epc_bar_type {
  */
 enum pci_epc_bar_rsvd_region_type {
 	PCI_EPC_BAR_RSVD_DMA_CTRL_MMIO = 0,
+	PCI_EPC_BAR_RSVD_MSIX_TBL_RAM,
+	PCI_EPC_BAR_RSVD_MSIX_PBA_RAM,
 };
 
 /**
Signature · 1 line
2.34.1
#2 lore raw

PATCH v3 2/4 PCI: tegra194: Make BAR0 programmable and remove 1MB size limit

In reply to Manikanta Maddireddy

The Tegra194/234 Endpoint does not support the Resizable BAR capability, but BAR0 can be programmed to different sizes via the DBI2 BAR registers in dw_pcie_ep_set_bar_programmable(). The BAR0 size is set once during initialization.

Remove the fixed 1MB limit from pci_epc_features so Endpoint function drivers can configure the BAR0 size they need.

---
v3: Rebased on pci/endpoint
v2: drop .type init for BAR_0

 drivers/pci/controller/dwc/pcie-tegra194.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -2
index 6881f0b94c73..c5381ffdf1eb 100644
@@ -1978,12 +1978,12 @@ static int tegra_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
 	return 0;
 }
 
+/* Tegra EP: BAR0 = 64-bit programmable BAR */
 static const struct pci_epc_features tegra_pcie_epc_features = {
 	DWC_EPC_COMMON_FEATURES,
 	.linkup_notifier = true,
 	.msi_capable = true,
-	.bar[BAR_0] = { .type = BAR_FIXED, .fixed_size = SZ_1M,
-			.only_64bit = true, },
+	.bar[BAR_0] = { .only_64bit = true, },
 	.bar[BAR_2] = { .type = BAR_DISABLED, },
 	.bar[BAR_3] = { .type = BAR_DISABLED, },
 	.bar[BAR_4] = { .type = BAR_DISABLED, },
Signature · 1 line
2.34.1
#3 lore raw

PATCH v3 3/4 PCI: tegra194: Expose BAR2 (MSI-X) and BAR4 (DMA) as 64-bit BAR_RESERVED

In reply to Manikanta Maddireddy

Tegra Endpoint exposes three 64-bit BARs at indices 0, 2, and 4:
- BAR0+BAR1: EPF test/data (programmable 64-bit BAR)
- BAR2+BAR3: MSI-X table (hardware-backed)
- BAR4+BAR5: DMA registers (hardware-backed)

Update tegra_pcie_epc_features so that BAR2 is BAR_RESERVED with PCI_EPC_BAR_RSVD_MSIX_TBL_RAM (64 KB) & PCI_EPC_BAR_RSVD_MSIX_PBA_RAM (64 KB) and BAR4 is BAR_RESERVED with PCI_EPC_BAR_RSVD_DMA_CTRL_MMIO (4KB). This keeps CONSECUTIVE_BAR_TEST working while allowing the host to use 64-bit BAR2 (MSI-X) and BAR4 (DMA).

---
v3: Rebased on pci/endpoint
v2: Split MSI-X table and PBA reserved region

 drivers/pci/controller/dwc/pcie-tegra194.c | 42 +++++++++++++++++++---
 1 file changed, 37 insertions(+), 5 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +37 -5
index c5381ffdf1eb..ea7a6256450c 100644
@@ -1978,16 +1978,48 @@ static int tegra_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
 	return 0;
 }
 
-/* Tegra EP: BAR0 = 64-bit programmable BAR */
+static const struct pci_epc_bar_rsvd_region tegra194_bar2_rsvd[] = {
+	{
+		/* MSI-X table structure */
+		.type = PCI_EPC_BAR_RSVD_MSIX_TBL_RAM,
+		.offset = 0x0,
+		.size = SZ_64K,
+	},
+	{
+		/* MSI-X PBA structure */
+		.type = PCI_EPC_BAR_RSVD_MSIX_PBA_RAM,
+		.offset = 0x10000,
+		.size = SZ_64K,
+	},
+};
+
+static const struct pci_epc_bar_rsvd_region tegra194_bar4_rsvd[] = {
+	{
+		/* DMA_CAP (BAR4: DMA Port Logic Structure) */
+		.type = PCI_EPC_BAR_RSVD_DMA_CTRL_MMIO,
+		.offset = 0x0,
+		.size = SZ_4K,
+	},
+};
+
+/* Tegra EP: BAR0 = 64-bit programmable BAR,  BAR2 = 64-bit MSI-X table, BAR4 = 64-bit DMA regs. */
 static const struct pci_epc_features tegra_pcie_epc_features = {
 	DWC_EPC_COMMON_FEATURES,
 	.linkup_notifier = true,
 	.msi_capable = true,
 	.bar[BAR_0] = { .only_64bit = true, },
-	.bar[BAR_2] = { .type = BAR_DISABLED, },
-	.bar[BAR_3] = { .type = BAR_DISABLED, },
-	.bar[BAR_4] = { .type = BAR_DISABLED, },
-	.bar[BAR_5] = { .type = BAR_DISABLED, },
+	.bar[BAR_2] = {
+		.type = BAR_RESERVED,
+		.only_64bit = true,
+		.nr_rsvd_regions = ARRAY_SIZE(tegra194_bar2_rsvd),
+		.rsvd_regions = tegra194_bar2_rsvd,
+	},
+	.bar[BAR_4] = {
+		.type = BAR_RESERVED,
+		.only_64bit = true,
+		.nr_rsvd_regions = ARRAY_SIZE(tegra194_bar4_rsvd),
+		.rsvd_regions = tegra194_bar4_rsvd,
+	},
 	.align = SZ_64K,
 };
 
Signature · 1 line
2.34.1
#4 lore raw

PATCH v3 4/4 misc: pci_endpoint_test: Add Tegra194 and Tegra234 device table entries

In reply to Manikanta Maddireddy

Add PCI device IDs for Tegra194 (0x1ad4) and Tegra234(0x229b) Endpoint controllers, so that pci_endpoint_test can bind and run on these controllers.

---
v3: Fix review comment in v2
v2: Remove pci_endpoint_test_data

 drivers/misc/pci_endpoint_test.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/misc/pci_endpoint_test.c +5 -0
index 55e128ed82f0..63f1e874719a 100644
@@ -114,6 +114,9 @@
 
 #define PCI_DEVICE_ID_ROCKCHIP_RK3588		0x3588
 
+#define PCI_DEVICE_ID_NVIDIA_TEGRA194_EP	0x1ad4
+#define PCI_DEVICE_ID_NVIDIA_TEGRA234_EP	0x229b
+
 #define PCI_ENDPOINT_TEST_BAR_SUBRANGE_NSUB	2
 
 static DEFINE_IDA(pci_endpoint_test_ida);
@@ -1438,6 +1441,8 @@ static const struct pci_device_id pci_endpoint_test_tbl[] = {
 	{ PCI_DEVICE(PCI_VENDOR_ID_ROCKCHIP, PCI_DEVICE_ID_ROCKCHIP_RK3588),
 	  .driver_data = (kernel_ulong_t)&rk3588_data,
 	},
+	{ PCI_DEVICE(PCI_VENDOR_ID_NVIDIA, PCI_DEVICE_ID_NVIDIA_TEGRA194_EP),},
+	{ PCI_DEVICE(PCI_VENDOR_ID_NVIDIA, PCI_DEVICE_ID_NVIDIA_TEGRA234_EP),},
 	{ }
 };
 MODULE_DEVICE_TABLE(pci, pci_endpoint_test_tbl);
Signature · 1 line
2.34.1
#5 lore raw
In reply to Manikanta Maddireddy
Quoted 9 lines · Manikanta Maddireddy

This series is on top of https://lore.kernel.org/linux-pci/[email protected]/T/#u

This series wires up Tegra194 and Tegra234 PCI endpoint controllers to the shared PCI endpoint and test infrastructure:

1. Add a new reserved-region type for MSI-X (Table and PBA) so EPC drivers can describe hardware-owned MSI-X regions behind a BAR_RESERVED BAR.

[...]

Applied, thanks!

[1/4] PCI: endpoint: Add reserved region type for MSI-X Table and PBA

      commit: 5f352433ea39171e19fbb3a7e18d983510176854
[2/4] PCI: tegra194: Make BAR0 programmable and remove 1MB size limit
      commit: 5aec1f18b326ddc455ae9d9d0f5394efc20eee9b
[3/4] PCI: tegra194: Expose BAR2 (MSI-X) and BAR4 (DMA) as 64-bit BAR_RESERVED
      commit: 12a22fb38cbba1991a3f174912b343cdb6afa186
[4/4] misc: pci_endpoint_test: Add Tegra194 and Tegra234 device table entries
      commit: 5ab7a225888baa5474def18ba3b0a298d27e6ba0

Best regards,

Signature · 1 line
Manivannan Sadhasivam <[email protected]>