Skip to contentSkip to conversation

ARM: RPi5: Enable PCIe

PATCH In review Torsten Duwe 3 participants 12 messages Jun 1 – Jun 14

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

v5 cover ARM: RPi5: Enable PCIe

Basic PCIe support for the RPi5. This is the prerequisite for PCIe slot adapters / HATs and NVMe storage. Those might require their own supporting drivers or changes. It is also the prerequisite for all peripherials residing on the RP1 I/O controller, like USB, network, GPIOs and all serial ports beyond the debug connector.

Changes from v4:
- fixed remaining style issues, no functional changes
- patches 3&6 Reviewed-by Neil Armstrong

Changes from v3:
- differentiate only between bcm2711 and -12. No possible future extensions for other chips. In particular, keep EXT_CFG_INDEX and EXT_CFG_DATA offsets fixed, no functions in the cfg struct that are the same for both, like bridge_sw_init_set and rc_mode.
- pass error codes through the newly introduced reset call chain
- some patches Reviewed-by Neil Armstrong

Changes from v2:
- fix wrong subject prefixes
- make reset drivers only found on the RPi5 depend on ARCH_BCM283X
- some patches Reviewed-by Peter Robinson

Changes from v1:
- fixed inbound window rewrite, which broke RPi4
- series Tested-by Pedro Falcato

Andrea della Porta (1):
pci: brcmstb: Fix PCIe bus numbers

Torsten Duwe (8):
ARM: bcm283x: Add bcm2712 PCIe memory window
pci: brcmstb: Support different variants using a cfg struct
reset: Add RPi5 brcmstb reset facilities
reset: Add RPi5 rescal reset facilities
pci: brcmstb: Get and use bridge and rescal reset properties
pci: brcmstb: Fix iBAR size calculation
pci: brcmstb: rework iBAR handling
pci: brcmstb: Adapt to AXI bridge

 .../mach-bcm283x/include/mach/acpi/bcm2711.h  |   6 +
 arch/arm/mach-bcm283x/init.c                  |  10 +-
 configs/rpi_arm64_defconfig                   |   2 +
 drivers/pci/pcie_brcmstb.c                    | 389 ++++++++++++++++--
 drivers/reset/Kconfig                         |  16 +
 drivers/reset/Makefile                        |   2 +
 drivers/reset/reset-brcmstb-rescal.c          | 103 +++++
 drivers/reset/reset-brcmstb.c                 |  97 +++++
 8 files changed, 589 insertions(+), 36 deletions(-)
 create mode 100644 drivers/reset/reset-brcmstb-rescal.c
 create mode 100644 drivers/reset/reset-brcmstb.c
Signature · 1 line
2.54.0
#1 lore raw

PATCH v5 1/9 ARM: bcm283x: Add bcm2712 PCIe memory window

From: Torsten Duwe <[email protected]>

Add a mapping region for the PCIe bus address spaces to the BCM2712 memory controller setup. Generously merging the PCIe address spaces works sufficiently well for a boot loader.

---
 arch/arm/mach-bcm283x/init.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
arch/arm/mach-bcm283x/init.c +9 -1
index 7a1de22e0ae..7a2faaa4de6 100644
@@ -18,7 +18,7 @@
 #ifdef CONFIG_ARM64
 #include <asm/armv8/mmu.h>
 
-#define MEM_MAP_MAX_ENTRIES (4)
+#define MEM_MAP_MAX_ENTRIES (5)
 
 static struct mm_region bcm283x_mem_map[MEM_MAP_MAX_ENTRIES] = {
 	{
@@ -83,6 +83,14 @@ static struct mm_region bcm2712_mem_map[MEM_MAP_MAX_ENTRIES] = {
 		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
 			 PTE_BLOCK_NON_SHARE |
 			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		/* Whole PCIe section */
+		.virt = 0x1800000000UL,
+		.phys = 0x1800000000UL,
+		.size = 0x0800000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
 	}, {
 		/* SoC bus */
 		.virt = 0x107c000000UL,
Signature · 1 line
2.54.0
#2 lore raw

PATCH v5 2/9 pci: brcmstb: Fix PCIe bus numbers

In reply to Torsten Duwe

From: Andrea della Porta <[email protected]>

The linux kernel assigns a new domain for every Root Complex where bus numbering starts from 0 for each domain. U-Boot does not have domains and uses a flattened bus numbering scheme instead. This means that any device or bridge on the second enumerated RC will receive a bus number equal to the last assigned one +1. This bus number contributes to the address written into the index register, which will select the configuration space to be read. Compensate for this contribution by subtracting the base bus number.

---
 drivers/pci/pcie_brcmstb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/pcie_brcmstb.c +1 -1
index f089c48f028..47c0802df23 100644
@@ -125,7 +125,7 @@ static int brcm_pcie_config_address(const struct udevice *dev, pci_dev_t bdf,
 				    uint offset, void **paddress)
 {
 	struct brcm_pcie *pcie = dev_get_priv(dev);
-	unsigned int pci_bus = PCI_BUS(bdf);
+	unsigned int pci_bus = PCI_BUS(bdf) - dev_seq(dev);
 	unsigned int pci_dev = PCI_DEV(bdf);
 	unsigned int pci_func = PCI_FUNC(bdf);
 	int idx;
Signature · 1 line
2.54.0
#3 lore raw

PATCH v5 3/9 pci: brcmstb: Support different variants using a cfg struct

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

The Linux kernel driver already had support for multiple hardware variants when the bcm2712 was added (see e.g. linux commit 10dbedad3c818 which is the last in a longer set of changes). This patch brings in this required infrastructure and adds a differentiation between 2711 and 2712 register layouts on top.

---
 .../mach-bcm283x/include/mach/acpi/bcm2711.h  |   5 +
 drivers/pci/pcie_brcmstb.c                    | 104 ++++++++++++++++--
 2 files changed, 99 insertions(+), 10 deletions(-)
arch/arm/mach-bcm283x/include/mach/acpi/bcm2711.h +5 -0
index a86875b1833..5171c593c72 100644
@@ -70,6 +70,7 @@
 #define PCIE_MISC_RC_BAR2_CONFIG_HI               0x4038
 #define PCIE_MISC_RC_BAR3_CONFIG_LO               0x403c
 #define  RC_BAR3_CONFIG_LO_SIZE_MASK                0x1f
+#define PCIE_MISC_PCIE_CTRL			  0x4064
 #define PCIE_MISC_PCIE_STATUS                     0x4068
 #define  STATUS_PCIE_PORT_MASK                      0x80
 #define  STATUS_PCIE_PORT_SHIFT                        7
@@ -108,6 +109,10 @@
 
 #define PCIE_RGR1_SW_INIT_1                   0x9210
 #define PCIE_EXT_CFG_INDEX                    0x9000
+#define  RGR1_SW_INIT_1_PERST_MASK			0x1
+#define  RGR1_SW_INIT_1_PERSTB_MASK			0x4
+#define  RGR1_SW_INIT_1_INIT_MASK			0x2
+
 /* A small window pointing at the ECAM of the device selected by CFG_INDEX */
 #define PCIE_EXT_CFG_DATA                     0x8000
 
drivers/pci/pcie_brcmstb.c +94 -10

165 lines not on this page · raw message

Signature · 1 line
2.54.0
#4 lore raw

PATCH v5 4/9 reset: Add RPi5 brcmstb reset facilities

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

A driver for Broadcom reset controllers ported from linux/drivers/reset/reset-brcmstb.c to U-Boot.

---
 configs/rpi_arm64_defconfig   |  1 +
 drivers/reset/Kconfig         |  8 +++
 drivers/reset/Makefile        |  1 +
 drivers/reset/reset-brcmstb.c | 97 +++++++++++++++++++++++++++++++++++
 4 files changed, 107 insertions(+)
 create mode 100644 drivers/reset/reset-brcmstb.c
configs/rpi_arm64_defconfig +1 -0
index 69e8e72c5d7..153d7ed301e 100644
@@ -44,6 +44,7 @@ CONFIG_BCMGENET=y
 CONFIG_PCI_BRCMSTB=y
 CONFIG_PINCTRL=y
 # CONFIG_PINCTRL_GENERIC is not set
+CONFIG_RESET_BRCMSTB=y
 CONFIG_DM_RNG=y
 CONFIG_RNG_IPROC200=y
 # CONFIG_REQUIRE_SERIAL_CONSOLE is not set
drivers/reset/Kconfig +8 -0
index 2fd91d6299c..8f4c9e29baa 100644
@@ -64,6 +64,14 @@ config RESET_BCM6345
 	help
 	  Support reset controller on BCM6345.
 
+config RESET_BRCMSTB
+	depends on ARCH_BCM283X
+	bool "Generic Reset controller driver for Broadcom"
+	help
+	  This enables reset controller for Broadcom devices.
+	  If you wish to use reset resources managed by the Broadcom
+	  Reset Controller, say Y here. Otherwise, say N.
+
 config RESET_UNIPHIER
 	bool "Reset controller driver for UniPhier SoCs"
 	depends on ARCH_UNIPHIER
drivers/reset/Makefile +1 -0
index ee5b009d134..ebf3d7425a4 100644
@@ -13,6 +13,7 @@ obj-$(CONFIG_RESET_AIROHA) += reset-airoha.o
 obj-$(CONFIG_RESET_TI_SCI) += reset-ti-sci.o
 obj-$(CONFIG_RESET_HSDK) += reset-hsdk.o
 obj-$(CONFIG_RESET_BCM6345) += reset-bcm6345.o
+obj-$(CONFIG_RESET_BRCMSTB) += reset-brcmstb.o
 obj-$(CONFIG_RESET_UNIPHIER) += reset-uniphier.o
 obj-$(CONFIG_RESET_AST2500) += reset-ast2500.o
 obj-$(CONFIG_RESET_AST2600) += reset-ast2600.o
drivers/reset/reset-brcmstb.c new file +97 -0

97 lines not on this page · raw message

Signature · 1 line
2.54.0
#5 lore raw

PATCH v5 5/9 reset: Add RPi5 rescal reset facilities

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

A driver for Broadcom rescal reset controllers ported from linux/drivers/reset/reset-brcmstb-rescal.c to U-Boot.

---
 configs/rpi_arm64_defconfig          |   1 +
 drivers/reset/Kconfig                |   8 +++
 drivers/reset/Makefile               |   1 +
 drivers/reset/reset-brcmstb-rescal.c | 103 +++++++++++++++++++++++++++
 4 files changed, 113 insertions(+)
 create mode 100644 drivers/reset/reset-brcmstb-rescal.c
configs/rpi_arm64_defconfig +1 -0
index 153d7ed301e..38af5029403 100644
@@ -45,6 +45,7 @@ CONFIG_PCI_BRCMSTB=y
 CONFIG_PINCTRL=y
 # CONFIG_PINCTRL_GENERIC is not set
 CONFIG_RESET_BRCMSTB=y
+CONFIG_RESET_BRCMSTB_RESCAL=y
 CONFIG_DM_RNG=y
 CONFIG_RNG_IPROC200=y
 # CONFIG_REQUIRE_SERIAL_CONSOLE is not set
drivers/reset/Kconfig +8 -0
index 8f4c9e29baa..e6c23368428 100644
@@ -72,6 +72,14 @@ config RESET_BRCMSTB
 	  If you wish to use reset resources managed by the Broadcom
 	  Reset Controller, say Y here. Otherwise, say N.
 
+config RESET_BRCMSTB_RESCAL
+	depends on ARCH_BCM283X
+	bool "Generic Rescal Reset controller driver for Broadcom"
+	help
+	  Support rescal reset controller on Broadcom.
+	  If you wish to use reset resources managed by the Broadcom
+	  Reset Controller, say Y here. Otherwise, say N.
+
 config RESET_UNIPHIER
 	bool "Reset controller driver for UniPhier SoCs"
 	depends on ARCH_UNIPHIER
drivers/reset/Makefile +1 -0
index ebf3d7425a4..c7a9da3268d 100644
@@ -14,6 +14,7 @@ obj-$(CONFIG_RESET_TI_SCI) += reset-ti-sci.o
 obj-$(CONFIG_RESET_HSDK) += reset-hsdk.o
 obj-$(CONFIG_RESET_BCM6345) += reset-bcm6345.o
 obj-$(CONFIG_RESET_BRCMSTB) += reset-brcmstb.o
+obj-$(CONFIG_RESET_BRCMSTB_RESCAL) += reset-brcmstb-rescal.o
 obj-$(CONFIG_RESET_UNIPHIER) += reset-uniphier.o
 obj-$(CONFIG_RESET_AST2500) += reset-ast2500.o
 obj-$(CONFIG_RESET_AST2600) += reset-ast2600.o
drivers/reset/reset-brcmstb-rescal.c new file +103 -0

103 lines not on this page · raw message

Signature · 1 line
2.54.0
#6 lore raw

PATCH v5 6/9 pci: brcmstb: Get and use bridge and rescal reset properties

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

Check whether the device tree has nodes for the two reset controls and use them if so.

---
 drivers/pci/pcie_brcmstb.c | 71 +++++++++++++++++++++++++++++++++++---
 1 file changed, 66 insertions(+), 5 deletions(-)
drivers/pci/pcie_brcmstb.c +66 -5
index 5cb39dfd136..012b501078a 100644
@@ -21,6 +21,7 @@
 #include <linux/bitfield.h>
 #include <linux/log2.h>
 #include <linux/iopoll.h>
+#include <reset.h>
 
 /* PCIe parameters */
 #define BRCM_NUM_PCIE_OUT_WINS				4
@@ -79,6 +80,8 @@ struct brcm_pcie {
 
 	int			gen;
 	bool			ssc;
+	struct reset_ctl	rescal;
+	struct reset_ctl	bridge_reset;
 	const struct brcm_pcie_cfg_data *pcie_cfg;
 };
 
@@ -143,14 +146,58 @@ static void brcm_pcie_perst_set_2712(struct brcm_pcie *pcie, u32 val)
 	writel(tmp, pcie->base + PCIE_MISC_PCIE_CTRL);
 }
 
-static void brcm_pcie_bridge_sw_init_set(struct brcm_pcie *pcie, u32 val)
+static int brcm_pcie_get_resets_dt(struct udevice *dev)
 {
+	struct brcm_pcie *pcie = dev_get_priv(dev);
+	int ret;
+
+	ret = reset_get_by_name(dev, "rescal", &pcie->rescal);
+	if (ret) {
+		printf("Unable to get rescal reset\n");
+		return ret;
+	}
+
+	ret = reset_get_by_name(dev, "bridge", &pcie->bridge_reset);
+	if (ret)
+		printf("Unable to get bridge reset\n");
+
+	return ret;
+}
+
+static int brcm_pcie_do_reset(struct udevice *dev)
+{
+	struct brcm_pcie *pcie = dev_get_priv(dev);
+	int ret;
+
+	ret = reset_deassert(&pcie->rescal);
+	if (ret)
+		printf("failed to deassert 'rescal'\n");
+	return ret;
+}
+
+static int brcm_pcie_bridge_sw_init_set(struct brcm_pcie *pcie, u32 val)
+{
+	int ret = 0;
+
+	if (reset_valid(&pcie->bridge_reset))
+	{
+		if (val)
+			ret = reset_assert(&pcie->bridge_reset);
+		else
+			ret = reset_deassert(&pcie->bridge_reset);
+		if (ret)
+			log_err("failed to %sassert bridge reset, err=%d\n",
+				val ? "" : "de", ret);
+		return ret;
+	}
+
 	if (val)
 		setbits_le32(pcie->base + pcie->pcie_cfg->offsets[RGR1_SW_INIT_1],
 			     RGR1_SW_INIT_1_INIT_MASK);
 	else
 		clrbits_le32(pcie->base + pcie->pcie_cfg->offsets[RGR1_SW_INIT_1],
 			     RGR1_SW_INIT_1_INIT_MASK);
+	return 0;
 }
 
 /**
@@ -405,16 +452,25 @@ static int brcm_pcie_probe(struct udevice *dev)
 	int num_out_wins = 0;
 	u64 rc_bar2_offset, rc_bar2_size;
 	unsigned int scb_size_val;
-	int i, ret;
+	int i, ret = 0;
 	u16 nlw, cls, lnksta;
 	u32 tmp;
 
+	/*
+	 * Ensure rescal reset for BCM2712 is really disabled.
+	 */
+	if (pcie->pcie_cfg->type == BCM2712)
+		ret = brcm_pcie_do_reset(dev);
+	if (ret)
+		return ret;
 	/*
 	 * Reset the bridge, assert the fundamental reset. Note for some SoCs,
 	 * e.g. BCM7278, the fundamental reset should not be asserted here.
 	 * This will need to be changed when support for other SoCs is added.
 	 */
-	brcm_pcie_bridge_sw_init_set(pcie, 1);
+	ret = brcm_pcie_bridge_sw_init_set(pcie, 1);
+	if (ret)
+		return ret;
 	if (pcie->pcie_cfg->type != BCM2712)
 		pcie->pcie_cfg->perst_set(pcie, 1);
 	/*
@@ -424,8 +480,9 @@ static int brcm_pcie_probe(struct udevice *dev)
 	udelay(100);
 
 	/* Take the bridge out of reset */
-	brcm_pcie_bridge_sw_init_set(pcie, 0);
-
+	ret = brcm_pcie_bridge_sw_init_set(pcie, 0);
+	if (ret)
+		return ret;
 	clrbits_le32(base + pcie->pcie_cfg->offsets[PCIE_HARD_DEBUG],
 		     PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
 
@@ -607,6 +664,10 @@ static int brcm_pcie_of_to_plat(struct udevice *dev)
 		pcie->gen = max_link_speed;
 
 	pcie->pcie_cfg = (const struct brcm_pcie_cfg_data *)dev_get_driver_data(dev);
+
+	if (pcie->pcie_cfg->type == BCM2712)
+		return brcm_pcie_get_resets_dt(dev);
+
 	return 0;
 }
 
Signature · 1 line
2.54.0
#7 lore raw

PATCH v5 7/9 pci: brcmstb: Fix iBAR size calculation

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

Fix inbound window size calculation, like Linux commit 25a98c7270156.

---
 drivers/pci/pcie_brcmstb.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/pcie_brcmstb.c +2 -2
index 012b501078a..164d8e2b7fe 100644
@@ -101,8 +101,8 @@ static int brcm_pcie_encode_ibar_size(u64 size)
 	if (log2_in >= 12 && log2_in <= 15)
 		/* Covers 4KB to 32KB (inclusive) */
 		return (log2_in - 12) + 0x1c;
-	else if (log2_in >= 16 && log2_in <= 37)
-		/* Covers 64KB to 32GB, (inclusive) */
+	else if (log2_in >= 16 && log2_in <= 36)
+		/* Covers 64KB to 64GB, (inclusive) */
 		return log2_in - 15;
 
 	/* Something is awry so disable */
Signature · 1 line
2.54.0
#8 lore raw

PATCH v5 8/9 pci: brcmstb: rework iBAR handling

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

Rework the setup of inbound PCIe windows: use the convenience functions from Linux kernel commit ae6476c6de187 to calculate the BAR offsets and factor out the setup code into a separate function.

The Linux kernel first allocates and populates an array of inbound_win[] and sets the BARs from it later, while U-Boot does it all on the fly, in one go, so the code is not 1:1 comparable.

---
 drivers/pci/pcie_brcmstb.c | 150 ++++++++++++++++++++++++++++++++-----
 1 file changed, 131 insertions(+), 19 deletions(-)
drivers/pci/pcie_brcmstb.c +131 -19
index 164d8e2b7fe..624376fe235 100644
@@ -50,6 +50,29 @@
 #define SSC_STATUS_PLL_LOCK_MASK			0x800
 #define SSC_STATUS_PLL_LOCK_SHIFT			11
 
+#define PCIE_RC_PL_PHY_CTL_15				0x184c
+#define PCIE_RC_PL_PHY_CTL_15_DIS_PLL_PD_MASK		0x400000
+#define PCIE_RC_PL_PHY_CTL_15_PM_CLK_PERIOD_MASK	0xff
+
+#define PCIE_MISC_UBUS_CTRL				0x40a4
+#define  PCIE_MISC_UBUS_CTRL_UBUS_PCIE_REPLY_ERR_DIS_MASK	BIT(13)
+#define  PCIE_MISC_UBUS_CTRL_UBUS_PCIE_REPLY_DECERR_DIS_MASK	BIT(19)
+#define PCIE_MISC_AXI_READ_ERROR_DATA			0x4170
+#define PCIE_MISC_UBUS_TIMEOUT				0x40A8
+#define PCIE_MISC_RC_CONFIG_RETRY_TIMEOUT		0x405c
+#define PCIE_MISC_RC_BAR4_CONFIG_LO			0x40d4
+#define PCIE_MISC_RC_BAR4_CONFIG_HI			0x40d8
+#define PCIE_MISC_UBUS_BAR_CONFIG_REMAP_HI_MASK		0xff
+#define PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_HI		0x4110
+#define PCIE_MISC_UBUS_BAR_CONFIG_REMAP_ENABLE		0x1
+#define PCIE_MISC_UBUS_BAR_CONFIG_REMAP_LO_MASK		0xfffff000
+#define PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_LO		0x410c
+
+#define PCIE_MISC_UBUS_BAR1_CONFIG_REMAP		0x40ac
+#define PCIE_MISC_UBUS_BAR2_CONFIG_REMAP		0x40b4
+#define  PCIE_MISC_UBUS_BAR2_CONFIG_REMAP_ACCESS_ENABLE_MASK	BIT(0)
+#define  MISC_CTRL_PCIE_RCB_MPS_MODE_MASK		0x400
+
 enum {
 	RGR1_SW_INIT_1,
 	PCIE_HARD_DEBUG,
@@ -441,17 +464,105 @@ static void brcm_pcie_set_outbound_win(struct brcm_pcie *pcie,
 	writel(tmp, base + PCIE_MEM_WIN0_LIMIT_HI(win));
 }
 
+static u32 brcm_bar_reg_offset(int bar)
+{
+	if (bar <= 3)
+		return PCIE_MISC_RC_BAR1_CONFIG_LO + 8 * (bar - 1);
+	else
+		return PCIE_MISC_RC_BAR4_CONFIG_LO + 8 * (bar - 4);
+}
+
+static u32 brcm_ubus_reg_offset(int bar)
+{
+	if (bar <= 3)
+		return PCIE_MISC_UBUS_BAR1_CONFIG_REMAP + 8 * (bar - 1);
+	else
+		return PCIE_MISC_UBUS_BAR4_CONFIG_REMAP_LO + 8 * (bar - 4);
+}
+
+/*
+ * Round size up to the next power of two, as required by
+ * brcm_pcie_encode_ibar_size().  If size is already a power of two
+ * fls64(size - 1) still gives the correct result because the hardware
+ * encodes the exponent, not the raw value.
+ */
+static u64 brcm_ibar_round_size(u64 size)
+{
+	return 1ULL << fls64(size - 1);
+}
+
+static void brcm_pcie_set_inbound_windows(struct udevice *dev)
+{
+	struct brcm_pcie *pcie = dev_get_priv(dev);
+	void __iomem *base = pcie->base;
+	bool is_2712 = (pcie->pcie_cfg->type == BCM2712);
+	int i, ibar_no, ret;
+	u32 tmp;
+
+	ibar_no = 0;
+	/* pre-2712 chips leave the first entry empty */
+	if (pcie->pcie_cfg->type != BCM2712)
+		ibar_no++;
+
+	/* program inbound windows from OF property "dma-regions" */
+	for (i = 0; i < 7; i++, ibar_no++) {
+		u64 bar_cpu, bar_size, bar_pci;
+		struct pci_region region;
+		int ubus_bar_offset, rc_bar_offset;
+
+		ret = pci_get_dma_regions(dev, &region, i);
+		if (ret)	/* no region #i? Then we're done. */
+			break;
+		ubus_bar_offset = brcm_ubus_reg_offset(ibar_no + 1);
+		rc_bar_offset = brcm_bar_reg_offset(ibar_no + 1);
+
+		bar_pci = region.bus_start;
+		bar_cpu = region.phys_start;
+		bar_size = region.size;
+
+		if (is_2712) {
+			/* BCM2712: BAR holds raw PCI address; UBUS remap
+			 * registers supply the CPU-side translation. */
+			tmp = lower_32_bits(bar_pci);
+			u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(bar_size),
+					  RC_BAR2_CONFIG_LO_SIZE_MASK);
+			writel(tmp, base + rc_bar_offset);
+			writel(upper_32_bits(bar_pci), base + rc_bar_offset + 4);
+
+			tmp = lower_32_bits(bar_cpu) &
+					PCIE_MISC_UBUS_BAR_CONFIG_REMAP_LO_MASK;
+			tmp |= PCIE_MISC_UBUS_BAR_CONFIG_REMAP_ENABLE;
+			writel(tmp, base + ubus_bar_offset);
+
+			tmp = upper_32_bits(bar_cpu) &
+				PCIE_MISC_UBUS_BAR_CONFIG_REMAP_HI_MASK;
+			writel(tmp, base + ubus_bar_offset + 4);
+		} else {
+			/* Pre-BCM2712 (e.g. BCM2711 / RPi4): the BAR config
+			 * register holds the offset (bus_start - phys_start),
+			 * not the raw PCI address.  The size must be rounded
+			 * up to the next power of two before encoding. */
+			u64 bar_offset = bar_pci - bar_cpu;
+			u64 bar_size_po2 = brcm_ibar_round_size(bar_size);
+
+			tmp = lower_32_bits(bar_offset);
+			u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(bar_size_po2),
+					  RC_BAR2_CONFIG_LO_SIZE_MASK);
+			writel(tmp, base + rc_bar_offset);
+			writel(upper_32_bits(bar_offset), base + rc_bar_offset + 4);
+			/* UBUS remap registers are not used on pre-2712 hardware. */
+		}
+	}
+}
+
 static int brcm_pcie_probe(struct udevice *dev)
 {
 	struct udevice *ctlr = pci_get_controller(dev);
 	struct pci_controller *hose = dev_get_uclass_priv(ctlr);
 	struct brcm_pcie *pcie = dev_get_priv(dev);
 	void __iomem *base = pcie->base;
-	struct pci_region region;
 	bool ssc_good = false;
 	int num_out_wins = 0;
-	u64 rc_bar2_offset, rc_bar2_size;
-	unsigned int scb_size_val;
 	int i, ret = 0;
 	u16 nlw, cls, lnksta;
 	u32 tmp;
@@ -496,23 +607,22 @@ static int brcm_pcie_probe(struct udevice *dev)
 			MISC_CTRL_CFG_READ_UR_MODE_MASK |
 			MISC_CTRL_MAX_BURST_SIZE_128);
 
-	pci_get_dma_regions(dev, &region, 0);
-	rc_bar2_offset = region.bus_start - region.phys_start;
-	rc_bar2_size = 1ULL << fls64(region.size - 1);
-
-	tmp = lower_32_bits(rc_bar2_offset);
-	u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(rc_bar2_size),
-			  RC_BAR2_CONFIG_LO_SIZE_MASK);
-	writel(tmp, base + PCIE_MISC_RC_BAR2_CONFIG_LO);
-	writel(upper_32_bits(rc_bar2_offset),
-	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
-
-	scb_size_val = rc_bar2_size ?
-		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
-
 	tmp = readl(base + PCIE_MISC_MISC_CTRL);
-	u32p_replace_bits(&tmp, scb_size_val,
-			  MISC_CTRL_SCB0_SIZE_MASK);
+	if (pcie->pcie_cfg->type == BCM2712) {
+		/* BCM2712: fixed 32GB SCB0 window */
+		u32p_replace_bits(&tmp, 20, MISC_CTRL_SCB0_SIZE_MASK);
+	} else {
+		/* Pre-BCM2712: size SCB0 to match the actual DMA region.
+		 * rc_bar2_size must be a power of two; ilog2(size) - 15
+		 * gives the hardware encoding (e.g. 1GB -> 15). */
+		struct pci_region region;
+		u64 rc_bar2_size;
+
+		pci_get_dma_regions(dev, &region, 0);
+		rc_bar2_size = brcm_ibar_round_size(region.size);
+		u32p_replace_bits(&tmp, rc_bar2_size ? ilog2(rc_bar2_size) - 15 : 0xf,
+				  MISC_CTRL_SCB0_SIZE_MASK);
+	}
 	writel(tmp, base + PCIE_MISC_MISC_CTRL);
 
 	/* Disable the PCIe->GISB memory window (RC_BAR1) */
@@ -529,6 +639,8 @@ static int brcm_pcie_probe(struct udevice *dev)
 	/* Clear any interrupts we find on boot */
 	writel(0xffffffff, base + PCIE_MSI_INTR2_CLR);
 
+	brcm_pcie_set_inbound_windows(dev);
+
 	if (pcie->gen)
 		brcm_pcie_set_gen(pcie, pcie->gen);
 
Signature · 1 line
2.54.0
#9 lore raw

PATCH v5 9/9 pci: brcmstb: Adapt to AXI bridge

In reply to Torsten Duwe

From: Torsten Duwe <[email protected]>

Fix-ups for the BCM root complex when it is located behind an AXI bridge and clocked with 54MHz. Some are from kernel commit 377bced88c326, some where picked by Oleksii off a now-stale older branch. All reworked for the simpler setup code in U-Boot.

---
 .../mach-bcm283x/include/mach/acpi/bcm2711.h  |  1 +
 drivers/pci/pcie_brcmstb.c                    | 64 ++++++++++++++++++-
 2 files changed, 64 insertions(+), 1 deletion(-)
arch/arm/mach-bcm283x/include/mach/acpi/bcm2711.h +1 -0
index 5171c593c72..c72b47e1b10 100644
@@ -54,6 +54,7 @@
 #define  MISC_CTRL_CFG_READ_UR_MODE_MASK          0x2000
 #define  MISC_CTRL_MAX_BURST_SIZE_MASK            0x300000
 #define  MISC_CTRL_MAX_BURST_SIZE_128             0x0
+#define  MISC_CTRL_MAX_BURST_SIZE_128_2712		0x100000
 #define  MISC_CTRL_SCB0_SIZE_MASK                 0xf8000000
 
 #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO          0x400c
drivers/pci/pcie_brcmstb.c +63 -1

87 lines not on this page · raw message

Signature · 1 line
2.54.0
#10 lore raw
In reply to Torsten Duwe

Hi Torsten,

Quoted 53 lines · Torsten Duwe

Basic PCIe support for the RPi5. This is the prerequisite for PCIe slot adapters / HATs and NVMe storage. Those might require their own supporting drivers or changes. It is also the prerequisite for all peripherials residing on the RP1 I/O controller, like USB, network, GPIOs and all serial ports beyond the debug connector.

Changes from v4:
- fixed remaining style issues, no functional changes
- patches 3&6 Reviewed-by Neil Armstrong

Changes from v3:
- differentiate only between bcm2711 and -12. No possible future extensions for other chips. In particular, keep EXT_CFG_INDEX and EXT_CFG_DATA offsets fixed, no functions in the cfg struct that are the same for both, like bridge_sw_init_set and rc_mode.
- pass error codes through the newly introduced reset call chain
- some patches Reviewed-by Neil Armstrong

Changes from v2:
- fix wrong subject prefixes
- make reset drivers only found on the RPi5 depend on ARCH_BCM283X
- some patches Reviewed-by Peter Robinson

Changes from v1:
- fixed inbound window rewrite, which broke RPi4
- series Tested-by Pedro Falcato

Andrea della Porta (1):
pci: brcmstb: Fix PCIe bus numbers

Torsten Duwe (8):
ARM: bcm283x: Add bcm2712 PCIe memory window
pci: brcmstb: Support different variants using a cfg struct
reset: Add RPi5 brcmstb reset facilities
reset: Add RPi5 rescal reset facilities
pci: brcmstb: Get and use bridge and rescal reset properties
pci: brcmstb: Fix iBAR size calculation
pci: brcmstb: rework iBAR handling
pci: brcmstb: Adapt to AXI bridge

 .../mach-bcm283x/include/mach/acpi/bcm2711.h  |   6 +
 arch/arm/mach-bcm283x/init.c                  |  10 +-
 configs/rpi_arm64_defconfig                   |   2 +
 drivers/pci/pcie_brcmstb.c                    | 389 ++++++++++++++++--
 drivers/reset/Kconfig                         |  16 +
 drivers/reset/Makefile                        |   2 +
 drivers/reset/reset-brcmstb-rescal.c          | 103 +++++
 drivers/reset/reset-brcmstb.c                 |  97 +++++
 8 files changed, 589 insertions(+), 36 deletions(-)
 create mode 100644 drivers/reset/reset-brcmstb-rescal.c
 create mode 100644 drivers/reset/reset-brcmstb.c
Quoted 3 lines

--
2.54.0

I see that this series has been applied to -next although I didn't see any 'applied' email here.

Do you know how I could try this out, please? I have a GeeekPi N07 PCIe M.2 M-Key NVMe. What sort of changes might I need to make?

Thanks,
Simon

#11 lore raw
Quoted 57 lines · Simon Glass

Hi Torsten,

Quoted 53 lines · Torsten Duwe

Basic PCIe support for the RPi5. This is the prerequisite for PCIe slot adapters / HATs and NVMe storage. Those might require their own supporting drivers or changes. It is also the prerequisite for all peripherials residing on the RP1 I/O controller, like USB, network, GPIOs and all serial ports beyond the debug connector.

Changes from v4:
- fixed remaining style issues, no functional changes
- patches 3&6 Reviewed-by Neil Armstrong

Changes from v3:
- differentiate only between bcm2711 and -12. No possible future extensions for other chips. In particular, keep EXT_CFG_INDEX and EXT_CFG_DATA offsets fixed, no functions in the cfg struct that are the same for both, like bridge_sw_init_set and rc_mode.
- pass error codes through the newly introduced reset call chain
- some patches Reviewed-by Neil Armstrong

Changes from v2:
- fix wrong subject prefixes
- make reset drivers only found on the RPi5 depend on ARCH_BCM283X
- some patches Reviewed-by Peter Robinson

Changes from v1:
- fixed inbound window rewrite, which broke RPi4
- series Tested-by Pedro Falcato

Andrea della Porta (1):
pci: brcmstb: Fix PCIe bus numbers

Torsten Duwe (8):
ARM: bcm283x: Add bcm2712 PCIe memory window
pci: brcmstb: Support different variants using a cfg struct
reset: Add RPi5 brcmstb reset facilities
reset: Add RPi5 rescal reset facilities
pci: brcmstb: Get and use bridge and rescal reset properties
pci: brcmstb: Fix iBAR size calculation
pci: brcmstb: rework iBAR handling
pci: brcmstb: Adapt to AXI bridge

 .../mach-bcm283x/include/mach/acpi/bcm2711.h  |   6 +
 arch/arm/mach-bcm283x/init.c                  |  10 +-
 configs/rpi_arm64_defconfig                   |   2 +
 drivers/pci/pcie_brcmstb.c                    | 389 ++++++++++++++++--
 drivers/reset/Kconfig                         |  16 +
 drivers/reset/Makefile                        |   2 +
 drivers/reset/reset-brcmstb-rescal.c          | 103 +++++
 drivers/reset/reset-brcmstb.c                 |  97 +++++
 8 files changed, 589 insertions(+), 36 deletions(-)
 create mode 100644 drivers/reset/reset-brcmstb-rescal.c
 create mode 100644 drivers/reset/reset-brcmstb.c
Quoted 6 lines
Quoted 3 lines

--
2.54.0

I see that this series has been applied to -next although I didn't see any 'applied' email here.

That was probably me not replying and just marking it as applied in patchwork. It's in master I believe.

Quoted 2 lines

Do you know how I could try this out, please? I have a GeeekPi N07 PCIe M.2 M-Key NVMe. What sort of changes might I need to make?

You need another series [1], which currently needs a rebase, and it should mostly "just work".

[1] https://patchwork.ozlabs.org/project/uboot/list/?series=503410