Add AMD Versal CPM/QDMA PCIe support
Subsystems (4)
cover Add AMD Versal CPM/QDMA PCIe support
This series adds PCIe root port support for the AMD Versal CPM4, CPM5 and QDMA PL (soft IP) host bridges, and enables it along with NVMe storage on the xilinx_versal_virt platform.
Patch 1 adds a generic mmap config-space write helper for host bridges that only decode 32-bit accesses.
Patches 2 and 3 add the QDMA PL and CPM4/CPM5 host bridge drivers, respectively.
Patch 4 maps the CPM5 ECAM windows as device memory so config space accesses are not cached or reordered.
Patch 5 wires everything up in the versal_virt defconfig and adds NVMe as a distro boot target.
Pranav Sanwal (5):
pci: Add pci_generic_mmap_write_config32() helper
pci: Add support for AMD QDMA PL PCIe root port
pci: Add support for CPM4 and CPM5 PCIe controllers
arm64: versal: Add MMU mappings for CPM5 ECAM config space
arm64: versal: Enable PCIe/NVMe support and add NVMe boot target
MAINTAINERS | 10 + arch/arm/mach-versal/cpu.c | 20 ++ configs/xilinx_versal_virt_defconfig | 6 + drivers/pci/Kconfig | 19 ++ drivers/pci/Makefile | 2 + drivers/pci/pci-uclass.c | 42 ++++ drivers/pci/pcie_amd_cpm.c | 363 +++++++++++++++++++++++++++ drivers/pci/pcie_amd_qdma_pl.c | 205 +++++++++++++++ include/configs/xilinx_versal.h | 7 + include/pci.h | 26 ++ 10 files changed, 700 insertions(+) create mode 100644 drivers/pci/pcie_amd_cpm.c create mode 100644 drivers/pci/pcie_amd_qdma_pl.c
Signature · 1 line
2.43.7
PATCH 1/5 pci: Add pci_generic_mmap_write_config32() helper
Some memory-mapped PCI host bridges only decode 32-bit accesses to their configuration space and either drop or corrupt neighbouring bytes on a narrower access. Because such hardware only ever reliably commits a full 32-bit transaction, the only safe way to perform a narrower write is to synthesize it in software: read the containing word, patch in the requested bits, and write the whole word back.
Add pci_generic_mmap_write_config32(), a counterpart to pci_generic_mmap_write_config(), that does exactly this: it transparently promotes 8-bit and 16-bit writes to a 32-bit read-modify-write of the word containing the requested offset, leaving 32-bit writes untouched. This mirrors Linux's pci_generic_config_write32() (drivers/pci/access.c), used for the same class of host bridge.
--- drivers/pci/pci-uclass.c | 42 ++++++++++++++++++++++++++++++++++++++++ include/pci.h | 26 +++++++++++++++++++++++++ 2 files changed, 68 insertions(+)
drivers/pci/pci-uclass.c +42 -0
@@ -618,6 +618,48 @@ int pci_generic_mmap_write_config( } } +int pci_generic_mmap_write_config32 ( +const struct udevice *bus, +int (*addr_f)(const struct udevice *bus,pci_dev_t bdf,uint offset, +void **addrp), +pci_dev_t bdf, +uint offset, +ulong value, +enum pci_size_t size) +{ +u32 shift, mask, tmp; +void *address; + +if (addr_f (bus, bdf,ALIGN_DOWN (offset,4 ), &address) <0 ) { +debug ("%s: failed to get config address for offset 0x%x\n" , + __func__, offset); +return 0 ; + } + +switch (size) { +case PCI_SIZE_32 : +writel (value, address); +return 0 ; +case PCI_SIZE_8 : +case PCI_SIZE_16 : +/* +* Some host bridges only decode 32-bit accesses to their +* config space and silently corrupt neighbouring bytes on a +* narrower write. Widen the access to a 32-bit +* read-modify-write of the word containing the requested +* offset. +*/ + shift = (offset %4 ) *BITS_PER_BYTE ; + mask =pci_get_ff (size) << shift; + tmp =readl (address) & ~mask; + tmp |= (value << shift) & mask; +writel (tmp, address); +return 0 ; +default : +return -EINVAL ; + } +} + int pci_generic_mmap_read_config( const struct udevice *bus, int (*addr_f)(const struct udevice *bus, pci_dev_t bdf, uint offset,
include/pci.h +26 -0
@@ -1151,6 +1151,32 @@ int pci_generic_mmap_write_config(ulong value, enum pci_size_t size); +/** +* pci_generic_mmap_write_config32() - Generic helper for writing to +* memory-mapped PCI configuration space that only supports 32-bit accesses. +* @bus: Pointer to the PCI bus +* @addr_f: Callback for calculating the config space address +* @bdf: Identifies the PCI device to access +* @offset: The offset into the device's configuration space +* @value: The value to write +* @size: Indicates the size of access to perform +* +* Like pci_generic_mmap_write_config(), but for host bridges whose +* configuration space does not decode 8-bit or 16-bit accesses correctly. +* Accesses narrower than 32 bits are promoted to a 32-bit +* read-modify-write of the word containing @offset. +* +* Return: 0 on success, else -EINVAL +*/ +int pci_generic_mmap_write_config32 ( +const struct udevice *bus, +int (*addr_f)(const struct udevice *bus,pci_dev_t bdf,uint offset, +void **addrp), +pci_dev_t bdf, +uint offset, +ulong value, +enum pci_size_t size); +/** * pci_generic_mmap_read_config() - Generic helper for reading from * memory-mapped PCI configuration space.
Signature · 1 line
2.43.7
PATCH 2/5 pci: Add support for AMD QDMA PL PCIe root port
In reply to Pranav SanwalAdd PCIe host bridge driver for AMD QDMA PL (Programmable Logic) soft IP PCIe5 controller. This driver enables PCIe root port functionality for QDMA IP instantiated in the PL fabric.
--- MAINTAINERS | 5 + drivers/pci/Kconfig | 10 ++ drivers/pci/Makefile | 1 + drivers/pci/pcie_amd_qdma_pl.c | 205 +++++++++++++++++++++++++++++++++ 4 files changed, 221 insertions(+) create mode 100644 drivers/pci/pcie_amd_qdma_pl.c
MAINTAINERS +5 -0
@@ -71,6 +71,11 @@ F: include/alist.h F: lib/alist.c F: test/lib/alist.c +AMD QDMA PL PCIE DRIVER +M: Pranav Sanwal <[email protected]> +S: Maintained +F: drivers/pci/pcie_amd_qdma_pl.c + AMD VERSAL2 PCIE DRIVER M: Pranav Sanwal <[email protected]> S: Maintained
drivers/pci/Kconfig +10 -0
@@ -444,6 +444,16 @@ config PCIE_XILINX_NWL Say 'Y' here if you want support for Xilinx / AMD NWL PCIe controller as Root Port. +config PCIE_AMD_QDMA_PL + bool "AMD QDMA PL PCIe host bridge support" + depends on ARCH_VERSAL + select SYS_PCI_64BIT + help + Say 'Y' here if you want support for the Xilinx/AMD QDMA + PL PCIe host bridge. The controller is a Soft IP instantiated + in the PL fabric which can act as a PCIe Root Port. It provides + ECAM-based configuration space access for device enumeration. + config PCIE_PLDA_COMMON bool
drivers/pci/Makefile +1 -0
@@ -54,6 +54,7 @@ obj-$(CONFIG_PCIE_OCTEON) += pcie_octeon.o obj-$(CONFIG_PCIE_DW_SIFIVE) += pcie_dw_sifive.o obj-$(CONFIG_PCIE_UNIPHIER) += pcie_uniphier.o obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o +obj-$(CONFIG_PCIE_AMD_QDMA_PL) += pcie_amd_qdma_pl.o obj-$(CONFIG_PCIE_PLDA_COMMON) += pcie_plda_common.o obj-$(CONFIG_PCIE_STARFIVE_JH7110) += pcie_starfive_jh7110.o obj-$(CONFIG_PCIE_DW_AMD) += pcie_dw_amd.o
drivers/pci/pcie_amd_qdma_pl.c new file +205 -0
205 lines not on this page · raw message
Signature · 1 line
2.43.7
PATCH 3/5 pci: Add support for CPM4 and CPM5 PCIe controllers
In reply to Pranav SanwalAdd a host controller driver for the AMD Versal CPM4 and CPM5 PCIe bridges, selected via the DT compatible string and enabled with CONFIG_PCIE_AMD_CPM. The driver maps the bridge/ECAM regions, services ECAM-based config space accesses, masks off all local interrupts, and brings up the root port bridge. CPM5 variants also support an optional PERST#-driven IP reset sequence ahead of link bring-up.
--- MAINTAINERS | 5 + drivers/pci/Kconfig | 9 + drivers/pci/Makefile | 1 + drivers/pci/pcie_amd_cpm.c | 363 +++++++++++++++++++++++++++++++++++++ 4 files changed, 378 insertions(+) create mode 100644 drivers/pci/pcie_amd_cpm.c
MAINTAINERS +5 -0
@@ -71,6 +71,11 @@ F: include/alist.h F: lib/alist.c F: test/lib/alist.c +AMD CPM PCIE DRIVER +M: Pranav Sanwal <[email protected]> +S: Maintained +F: drivers/pci/pcie_amd_cpm.c + AMD QDMA PL PCIE DRIVER M: Pranav Sanwal <[email protected]> S: Maintained
drivers/pci/Kconfig +9 -0
@@ -444,6 +444,15 @@ config PCIE_XILINX_NWL Say 'Y' here if you want support for Xilinx / AMD NWL PCIe controller as Root Port. +config PCIE_AMD_CPM + bool "AMD Versal CPM PCIe host bridge support" + depends on ARCH_VERSAL + depends on DM_GPIO + select SYS_PCI_64BIT + help + Say 'Y' here if you want support for the AMD Versal + CPM PCIe host bridge. + config PCIE_AMD_QDMA_PL bool "AMD QDMA PL PCIe host bridge support" depends on ARCH_VERSAL
drivers/pci/Makefile +1 -0
@@ -54,6 +54,7 @@ obj-$(CONFIG_PCIE_OCTEON) += pcie_octeon.o obj-$(CONFIG_PCIE_DW_SIFIVE) += pcie_dw_sifive.o obj-$(CONFIG_PCIE_UNIPHIER) += pcie_uniphier.o obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o +obj-$(CONFIG_PCIE_AMD_CPM) += pcie_amd_cpm.o obj-$(CONFIG_PCIE_AMD_QDMA_PL) += pcie_amd_qdma_pl.o obj-$(CONFIG_PCIE_PLDA_COMMON) += pcie_plda_common.o obj-$(CONFIG_PCIE_STARFIVE_JH7110) += pcie_starfive_jh7110.o
drivers/pci/pcie_amd_cpm.c new file +363 -0
363 lines not on this page · raw message
Signature · 1 line
2.43.7
PATCH 4/5 arm64: versal: Add MMU mappings for CPM5 ECAM config space
In reply to Pranav SanwalCPM5 Host0 and Host1 each expose PCIe config space (ECAM) inside their 4 GiB NoC address slot, at 0x600000000 and 0x700000000 respectively. Per the DT "cfg" reg and the PCIe ECAM spec (256 bus numbers x 1 MiB per bus), each window is 256 MiB. Config space accesses must not be cached, reordered, or speculatively issued, so map each ECAM window as Device memory (MT_DEVICE_NGNRNE).
Rather than mapping 0x600000000-0xe00000000 only when CONFIG_PCIE_AMD_CPM is enabled, add it as a baseline Normal/cacheable entry unconditionally: this NoC/PL address space is shared by more than CPM (PL fabric AXI space, other NoC-attached peripherals, etc.), so it needs coverage regardless of which PCIe controllers are built in. The two CPM5 ECAM sub-ranges are then overlaid as Device memory on top of that baseline only when PCIE_AMD_CPM is enabled.
Update VERSAL_MEM_MAP_USED to match: 3 entries when PCIE_AMD_CPM is disabled, 5 when it's enabled (+2 ECAM overlays).
--- arch/arm/mach-versal/cpu.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/arm/mach-versal/cpu.c +20 -0
@@ -20,7 +20,11 @@DECLARE_GLOBAL_DATA_PTR ; +#if CONFIG_IS_ENABLED (PCIE_AMD_CPM ) +#define VERSAL_MEM_MAP_USED 5 +#else #define VERSAL_MEM_MAP_USED 3 +#endif #define DRAM_BANKS CONFIG_NR_DRAM_BANKS
@@ -55,6 +59,22 @@ static struct mm_region versal_mem_map[VERSAL_MEM_MAP_MAX] = { .attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | PTE_BLOCK_NON_SHARE | PTE_BLOCK_PXN | PTE_BLOCK_UXN +#if CONFIG_IS_ENABLED (PCIE_AMD_CPM ) + }, { + .virt = 0x600000000UL, + .phys = 0x600000000UL, + .size = 0x010000000UL, + .attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | + PTE_BLOCK_NON_SHARE | + PTE_BLOCK_PXN | PTE_BLOCK_UXN + }, { + .virt = 0x700000000UL, + .phys = 0x700000000UL, + .size = 0x010000000UL, + .attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | + PTE_BLOCK_NON_SHARE | + PTE_BLOCK_PXN | PTE_BLOCK_UXN +#endif } };
Signature · 1 line
2.43.7
PATCH 5/5 arm64: versal: Enable PCIe/NVMe support and add NVMe boot target
In reply to Pranav SanwalEnable the AMD Versal CPM/QDMA PCIe host controllers and NVMe storage in the virtual platform defconfig.
Add BOOT_TARGET_DEVICES_NVME to xilinx_versal.h so NVMe is included in the distro boot target list when CONFIG_NVME is enabled.
--- configs/xilinx_versal_virt_defconfig | 6 ++++++ include/configs/xilinx_versal.h | 7 +++++++ 2 files changed, 13 insertions(+)
configs/xilinx_versal_virt_defconfig +6 -0
@@ -16,6 +16,7 @@ CONFIG_SYS_LOAD_ADDR=0x8000000 CONFIG_ENV_OFFSET_REDUND=0x7F00000 CONFIG_CMD_FRU=y CONFIG_DEFINE_TCM_OCM_MMAP=y +CONFIG_PCI=y CONFIG_SYS_MEMTEST_START=0x00000000 CONFIG_SYS_MEMTEST_END=0x00001000 CONFIG_REMAKE_ELF=y
@@ -47,9 +48,11 @@ CONFIG_CMD_CLK=y CONFIG_CMD_DFU=y CONFIG_CMD_GPIO=y CONFIG_CMD_I2C=y +CONFIG_CMD_LSBLK=y CONFIG_CMD_MMC=y CONFIG_MMC_SPEED_MODE_SET=y CONFIG_CMD_MTD=y +CONFIG_CMD_PCI=y CONFIG_CMD_SF_TEST=y CONFIG_CMD_USB=y CONFIG_CMD_NFS=y
@@ -135,6 +138,9 @@ CONFIG_PHY_NATSEMI=y CONFIG_PHY_REALTEK=y CONFIG_PHY_TI_DP83867=y CONFIG_PHY_FIXED=y +CONFIG_NVME_PCI=y +CONFIG_PCIE_AMD_CPM=y +CONFIG_PCIE_AMD_QDMA_PL=y CONFIG_POWER_DOMAIN=y CONFIG_ZYNQMP_POWER_DOMAIN=y CONFIG_RESET_ZYNQMP=y
include/configs/xilinx_versal.h +7 -0
@@ -118,10 +118,17 @@#define BOOTENV_DEV_NAME_USB_THOR (devtypeu, devtypel, instance) \ "" +#if defined(CONFIG_NVME ) +# define BOOT_TARGET_DEVICES_NVME (func) func(NVME, nvme, 0) +#else +# define BOOT_TARGET_DEVICES_NVME (func) +#endif +#define BOOT_TARGET_DEVICES (func) \ BOOT_TARGET_DEVICES_JTAG(func) \ BOOT_TARGET_DEVICES_MMC(func) \ BOOT_TARGET_DEVICES_XSPI(func) \ + BOOT_TARGET_DEVICES_NVME(func) \ BOOT_TARGET_DEVICES_USB_DFU(func) \ BOOT_TARGET_DEVICES_USB_THOR(func) \ BOOT_TARGET_DEVICES_USB(func) \
Signature · 1 line
2.43.7
Quoted 31 lines · Pranav Sanwal
This series adds PCIe root port support for the AMD Versal CPM4, CPM5 and QDMA PL (soft IP) host bridges, and enables it along with NVMe storage on the
xilinx_versal_virtplatform.Patch 1 adds a generic mmap config-space write helper for host bridges that only decode 32-bit accesses.
Patches 2 and 3 add the QDMA PL and CPM4/CPM5 host bridge drivers, respectively.
Patch 4 maps the CPM5 ECAM windows as device memory so config space accesses are not cached or reordered.
Patch 5 wires everything up in theversal_virtdefconfig and adds NVMe as a distro boot target.Pranav Sanwal (5):
pci: Addpci_generic_mmap_write_config32()helper
pci: Add support for AMD QDMA PL PCIe root port
pci: Add support for CPM4 and CPM5 PCIe controllers
arm64: versal: Add MMU mappings for CPM5 ECAM config space
arm64: versal: Enable PCIe/NVMe support and add NVMe boot targetMAINTAINERS | 10 + arch/arm/mach-versal/cpu.c | 20 ++ configs/xilinx_versal_virt_defconfig | 6 + drivers/pci/Kconfig | 19 ++ drivers/pci/Makefile | 2 + drivers/pci/pci-uclass.c | 42 ++++ drivers/pci/pcie_amd_cpm.c | 363 +++++++++++++++++++++++++++ drivers/pci/pcie_amd_qdma_pl.c | 205 +++++++++++++++ include/configs/xilinx_versal.h | 7 + include/pci.h | 26 ++ 10 files changed, 700 insertions(+)
Quoted 3 lines
create mode 100644
drivers/pci/pcie_amd_cpm.ccreate mode 100644drivers/pci/pcie_amd_qdma_pl.c
Applied.
M