Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B012AC7619A for ; Mon, 27 Mar 2023 01:04:05 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 194E186031; Mon, 27 Mar 2023 03:04:02 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=starfivetech.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: by phobos.denx.de (Postfix, from userid 109) id 9616B8603E; Mon, 27 Mar 2023 03:03:58 +0200 (CEST) Received: from fd01.gateway.ufhost.com (fd01.gateway.ufhost.com [61.152.239.71]) by phobos.denx.de (Postfix) with ESMTP id 9B2DA86031 for ; Mon, 27 Mar 2023 03:03:50 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=starfivetech.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=minda.chen@starfivetech.com Received: from EXMBX166.cuchost.com (unknown [175.102.18.54]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "EXMBX166", Issuer "EXMBX166" (not verified)) by fd01.gateway.ufhost.com (Postfix) with ESMTP id 9FFB424E2A5; Mon, 27 Mar 2023 09:03:47 +0800 (CST) Received: from EXMBX171.cuchost.com (172.16.6.91) by EXMBX166.cuchost.com (172.16.6.76) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Mon, 27 Mar 2023 09:03:47 +0800 Received: from [192.168.125.108] (113.72.145.117) by EXMBX171.cuchost.com (172.16.6.91) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Mon, 27 Mar 2023 09:03:46 +0800 Message-ID: Date: Mon, 27 Mar 2023 09:03:46 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: [PATCH v2 1/3] starfive: pci: Add StarFive JH7110 pcie driver Content-Language: en-US To: =?UTF-8?Q?Pali_Roh=c3=a1r?= CC: Simon Glass , Stefan Roese , Andrew Scull , Mark Kettenis , , Rick Chen , Leo , Mason Huo , Yanhong Wang , Leyfoon Tan , References: <20230308054833.95730-1-minda.chen@starfivetech.com> <20230308054833.95730-2-minda.chen@starfivetech.com> <20230308073159.iomklf3lybpzq45b@pali> <45bb46dd-1b1b-ff47-8ae3-48dfb2b48644@starfivetech.com> <20230310174236.6dp55bxkwcmvt6ej@pali> <7955a9f9-a63f-b47f-11c1-1a694b979ec2@starfivetech.com> <20230323181938.durxufzm25gvfibq@pali> <62427848-c850-a92a-be48-f64bfca339bc@starfivetech.com> <20230325123111.yyqbtoqrnfrvmko7@pali> From: Minda Chen In-Reply-To: <20230325123111.yyqbtoqrnfrvmko7@pali> Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [113.72.145.117] X-ClientProxiedBy: EXCAS065.cuchost.com (172.16.6.25) To EXMBX171.cuchost.com (172.16.6.91) X-YovoleRuleAgent: yovoleflag Content-Transfer-Encoding: quoted-printable X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean On 2023/3/25 20:31, Pali Roh=C3=A1r wrote: > On Friday 24 March 2023 18:57:33 Minda Chen wrote: >> On 2023/3/24 2:19, Pali Roh=C3=A1r wrote: >> > On Thursday 23 March 2023 18:51:38 Minda Chen wrote: >> >> On 2023/3/11 1:42, Pali Roh=C3=A1r wrote: >> >> > On Friday 10 March 2023 18:36:44 Minda Chen wrote: >> >> >> On 2023/3/8 15:31, Pali Roh=C3=A1r wrote: >> >> >> > Hello! See few comments below. >> >> >> >=20 >> >> >> > On Wednesday 08 March 2023 13:48:31 Minda Chen wrote: >> >> >> >> From: Mason Huo >> >> >> >>=20 >> >> >> >> Add pcie driver for StarFive JH7110, the driver depends on >> >> >> >> starfive gpio, pinctrl, clk and reset driver to do init. >> >> >> >>=20 >> >> >> >> Several devices are tested: >> >> >> >> a) M.2 NVMe SSD >> >> >> >> b) Realtek 8169 Ethernet adapter. >> >> >> >>=20 >> >> >> >> Signed-off-by: Mason Huo >> >> >> >> Signed-off-by: Minda Chen >> >> >> >> --- >> >> >> >> drivers/pci/Kconfig | 11 + >> >> >> >> drivers/pci/Makefile | 1 + >> >> >> >> drivers/pci/pcie_starfive_jh7110.c | 478 +++++++++++++++++++= ++++++++++ >> >> >> >> 3 files changed, 490 insertions(+) >> >> >> >> create mode 100644 drivers/pci/pcie_starfive_jh7110.c >> >> >> >>=20 >> >> >> >> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig >> >> >> >> index ef328d2652..e7b0ff5bc3 100644 >> >> >> >> --- a/drivers/pci/Kconfig >> >> >> >> +++ b/drivers/pci/Kconfig >> >> >> >> @@ -374,4 +374,15 @@ config PCIE_UNIPHIER >> >> >> >> Say Y here if you want to enable PCIe controller support = on >> >> >> >> UniPhier SoCs. >> >> >> >> =20 >> >> >> >> +config PCIE_STARFIVE_JH7110 >> >> >> >> + bool "Enable Starfive JH7110 PCIe driver" >> >> >> >> + depends on STARFIVE_JH7110 >> >> >> >> + depends on PINCTRL_STARFIVE_JH7110 >> >> >> >> + depends on CLK_JH7110 >> >> >> >> + depends on RESET_JH7110 >> >> >> >> + default y >> >> >> >> + help >> >> >> >> + Say Y here if you want to enable PCIe controller support = on >> >> >> >> + StarFive JH7110 SoC. >> >> >> >> + >> >> >> >> endif >> >> >> >> diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile >> >> >> >> index 49506e7ba5..bbe3323bb5 100644 >> >> >> >> --- a/drivers/pci/Makefile >> >> >> >> +++ b/drivers/pci/Makefile >> >> >> >> @@ -49,3 +49,4 @@ obj-$(CONFIG_PCI_OCTEONTX) +=3D pci_octeont= x.o >> >> >> >> obj-$(CONFIG_PCIE_OCTEON) +=3D pcie_octeon.o >> >> >> >> obj-$(CONFIG_PCIE_DW_SIFIVE) +=3D pcie_dw_sifive.o >> >> >> >> obj-$(CONFIG_PCIE_UNIPHIER) +=3D pcie_uniphier.o >> >> >> >> +obj-$(CONFIG_PCIE_STARFIVE_JH7110) +=3D pcie_starfive_jh7110= .o >> >> >> >> diff --git a/drivers/pci/pcie_starfive_jh7110.c b/drivers/pci= /pcie_starfive_jh7110.c >> >> >> >> new file mode 100644 >> >> >> >> index 0000000000..5ccef1ef02 >> >> >> >> --- /dev/null >> >> >> >> +++ b/drivers/pci/pcie_starfive_jh7110.c >> >> >> >> @@ -0,0 +1,478 @@ >> >> >> >> +// SPDX-License-Identifier: GPL-2.0+ >> >> >> >> +/* >> >> >> >> + * StarFive PLDA PCIe host controller driver >> >> >> >> + * >> >> >> >> + * Copyright (c) 2023 Starfive, Inc. >> >> >> >> + * Author: Mason Huo >> >> >> >> + * >> >> >> >> + */ >> >> >> >> + >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> +#include >> >> >> >> + >> >> >> >> +DECLARE_GLOBAL_DATA_PTR; >> >> >> >> + >> >> >> >> +#define GEN_SETTINGS 0x80 >> >> >> >> +#define PCIE_PCI_IDS 0x9C >> >> >> >> +#define PCIE_WINROM 0xFC >> >> >> >> +#define PMSG_SUPPORT_RX 0x3F0 >> >> >> >> +#define PCI_MISC 0xB4 >> >> >> >> + >> >> >> >> +#define PLDA_EP_ENABLE 0 >> >> >> >> +#define PLDA_RP_ENABLE 1 >> >> >> >> + >> >> >> >> +#define IDS_REVISION_ID 0x02 >> >> >> >> +#define IDS_PCI_TO_PCI_BRIDGE 0x060400 >> >> >> >> +#define IDS_CLASS_CODE_SHIFT 8 >> >> >> >=20 >> >> >> > Please do not duplicate standard PCI macros and constants. In = U-Boot >> >> >> > they are already available in include/pci_ids.h header file. >> >> >> >=20 >> >> >> ok >> >> >> >> +#define PREF_MEM_WIN_64_SUPPORT BIT(3) >> >> >> >> +#define PMSG_LTR_SUPPORT BIT(2) >> >> >> >> +#define PLDA_FUNCTION_DIS BIT(15) >> >> >> >> +#define PLDA_FUNC_NUM 4 >> >> >> >> +#define PLDA_PHY_FUNC_SHIFT 9 >> >> >> >> + >> >> >> >> +#define XR3PCI_ATR_AXI4_SLV0 0x800 >> >> >> >> +#define XR3PCI_ATR_SRC_ADDR_LOW 0x0 >> >> >> >> +#define XR3PCI_ATR_SRC_ADDR_HIGH 0x4 >> >> >> >> +#define XR3PCI_ATR_TRSL_ADDR_LOW 0x8 >> >> >> >> +#define XR3PCI_ATR_TRSL_ADDR_HIGH 0xc >> >> >> >> +#define XR3PCI_ATR_TRSL_PARAM 0x10 >> >> >> >> +#define XR3PCI_ATR_TABLE_OFFSET 0x20 >> >> >> >> +#define XR3PCI_ATR_MAX_TABLE_NUM 8 >> >> >> >> + >> >> >> >> +#define XR3PCI_ATR_SRC_WIN_SIZE_SHIFT 1 >> >> >> >> +#define XR3PCI_ATR_SRC_ADDR_MASK GENMASK(31, 12) >> >> >> >> +#define XR3PCI_ATR_TRSL_ADDR_MASK GENMASK(31, 12) >> >> >> >> +#define XR3_PCI_ECAM_SIZE 28 >> >> >> >> +#define XR3PCI_ATR_TRSL_DIR BIT(22) >> >> >> >> +/* IDs used in the XR3PCI_ATR_TRSL_PARAM */ >> >> >> >> +#define XR3PCI_ATR_TRSLID_PCIE_MEMORY 0x0 >> >> >> >> +#define XR3PCI_ATR_TRSLID_PCIE_CONFIG 0x1 >> >> >> >> + >> >> >> >> +#define ECAM_BUS_SHIFT 20 >> >> >> >> +#define ECAM_DEV_SHIFT 15 >> >> >> >> +#define ECAM_FUNC_SHIFT 12 >> >> >> >=20 >> >> >> > Please do not implement duplicate PCIe ECAM code. U-Boot and a= lso Linux >> >> >> > kernel already provides PCIE_ECAM_OFFSET() macro. >> >> >> >=20 >> >> >> >> +/* Secondary bus number offset in config space */ >> >> >> >> +#define PCI_SECONDARY_BUS 0x19 >> >> >> >=20 >> >> >> > Unused. >> >> >> >=20 >> >> >> ok >> >> >> >> + >> >> >> >> +/* system control */ >> >> >> >> +#define STG_SYSCON_K_RP_NEP_MASK BIT(8) >> >> >> >> +#define STG_SYSCON_AXI4_SLVL_ARFUNC_MASK GENMASK(22, 8) >> >> >> >> +#define STG_SYSCON_AXI4_SLVL_ARFUNC_SHIFT 8 >> >> >> >> +#define STG_SYSCON_AXI4_SLVL_AWFUNC_MASK GENMASK(14, 0) >> >> >> >> +#define STG_SYSCON_CLKREQ_MASK BIT(22) >> >> >> >> +#define STG_SYSCON_CKREF_SRC_SHIFT 18 >> >> >> >> +#define STG_SYSCON_CKREF_SRC_MASK GENMASK(19, 18) >> >> >> >> + >> >> >> >> +struct starfive_pcie { >> >> >> >> + struct udevice *dev; >> >> >> >> + >> >> >> >> + void __iomem *reg_base; >> >> >> >> + void __iomem *cfg_base; >> >> >> >> + >> >> >> >> + struct regmap *regmap; >> >> >> >> + u32 stg_arfun; >> >> >> >> + u32 stg_awfun; >> >> >> >> + u32 stg_rp_nep; >> >> >> >> + >> >> >> >> + struct clk_bulk clks; >> >> >> >> + struct reset_ctl_bulk rsts; >> >> >> >> + >> >> >> >> + int atr_table_num; >> >> >> >> + int first_busno; >> >> >> >> +}; >> >> >> >> + >> >> >> >> +static int starfive_pcie_addr_valid(pci_dev_t bdf, int first= _busno) >> >> >> >> +{ >> >> >> >> + if ((PCI_BUS(bdf) =3D=3D first_busno) && (PCI_DEV(bdf) > 0)= ) >> >> >> >> + return 0; >> >> >> >> + if ((PCI_BUS(bdf) =3D=3D first_busno + 1) && (PCI_DEV(bdf) = > 0)) >> >> >> >> + return 0; >> >> >> >=20 >> >> >> > If fisrt_busno is the bus of the root port then first_busno+1 = does not >> >> >> > have to bus of the PCIe device on the other end of the link fr= om the >> >> >> > root port. See for example pci_mvebu.c how this logic is handl= ed. >> >> >> > You probably want to also limit access to non-zero function of= root port >> >> >> > unless you have multifunction root-port device. >> >> >> >=20 >> >> >> root port limitation can follow the pci_mvebu.c >> >> >=20 >> >> > It can follow, but question is why it should follow pci_mvebu.c. = Marvell >> >> > PCIe controller does not use ECAM, instead it uses Intel PCI conf= 1 >> >> > mechanism and moreover it contains HW bugs which are workarounded= in >> >> > that filter code. >> >> >=20 >> >> starfive_pcie_addr_valid limit the access the non-zero device. >> >>=20 >> >> The limitaion reason is list below >> >> For PCIe bus of JH7110 SoC limitation, The PCIe host bridge of j= ust support one device. >> >>=20 >> >> (The simple topic graph)=20 >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D JH7110 SoC =3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> [Processor] >> >> | >> >> (Root Complex) >> >> | >> >> ------------------------------ >> >> [Host Bridge Domain 0000] >> >> | >> >> (Bus 0) >> >> | >> >> [Bus 0 Device 0] >> >> |=20 >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D JH7110 SoC =3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> |=09 >> >> (BUS 1)=09 >> >> | >> >> [BUS 1 Device 0] >> >>=20 >> >> To avoid access non-zero device config space, add this limitation. >> >=20 >> > Ok. The hierarchy is quite standard. But question is why you need to= do >> > explicit filtering in the driver? ECAM should correctly work also wh= en >> > you ask non-existent device (e.g. by returning all-ones). >> >=20 >> > Is the driver working fine also without filtering? >> >=20 >> >> And I can see designware PCIe code (pcie_dw_common.c) have the same >> >> code logic(pcie_dw_addr_valid). >> >=20 >> > In mvebu is explicit filtering because of HW bug which cause that HW >> > ignores device number. And so device 0 is aliased to device 2, 3, 4.= .. >> > So to prevent kernel to see tons of non-existent devices, mvebu driv= er >> > has to filter access. >> >=20 >> > But normally any special logic should not be needed unless there is = a HW >> > bug. HW just normally returns to kernel that there is no other devic= e. >> >=20 >> > mvebu driver is not something useful for taking as a template driver= for >> > building a new driver because it contains tons of code for workaroun= ding >> > real HW issues without which other kernel (and userspace) PCIe parts= do >> > not work. >> >=20 >> Uboot PCIe can not work well. If without filtering, uboot get duplicat= e >> devices. device 0 is aliased to device 2, 3, 4...31 >=20 > Ok, so you are seeing same issue as on mvebu HW which I described above= , > right? And which bus is affected by this issue? Root (zero) bus? Or the > bus behind it (1)? >=20 bus 1. >> >> >> >> + >> >> >> >> + return 1; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_off_conf(pci_dev_t bdf, uint offset= ) >> >> >> >> +{ >> >> >> >> + unsigned int bus =3D PCI_BUS(bdf); >> >> >> >> + unsigned int dev =3D PCI_DEV(bdf); >> >> >> >> + unsigned int func =3D PCI_FUNC(bdf); >> >> >> >> + >> >> >> >> + return (bus << ECAM_BUS_SHIFT) | (dev << ECAM_DEV_SHIFT) | >> >> >> >> + (func << ECAM_FUNC_SHIFT) | offset; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static bool starfive_pcie_hide_rc_bar(pci_dev_t bdf, int off= set) >> >> >> >> +{ >> >> >> >> + /* Only root port 1 is in use */ >> >> >> >> + if ((PCI_BUS(bdf) =3D=3D 0) && >> >> >> >=20 >> >> >> > This looks like an incorrect check for the bus of he root port= . >> >> >> > See for example pci_mvebu.c where logic should be correct. >> >> >> >=20 >> >> >> > I guess you wanted to use: PCI_BUS(bdf)-dev_seq(udev) >> >> >> > [1] This is a work around method. Our PCIe support 64bit. But = the config >> >> >> space is 16KB. If no limitation here, uboot will set high 32 bit= base address to 0xfffffff after init. >> >> >=20 >> >> > I just do not understand. BAR configures PCIe MEM space mapping a= nd it >> >> > is fully independent of the PCIe config space. I do not see reaso= n for >> >> > limitation. >> >> >=20 >> >> >> But this will make kernel pcie failed. This value will be not cl= eared. >> >> >> While kernel read the base address first. kernel pcie can not wo= rk. >> >> >> And our IC we can not reset the PCIe to clear this value. >> >> >> I dont know how to fix it expcept this method. >> >> >=20 >> >> > Well, this sounds like a bug in kernel, which should be fixed or = at >> >> > least reported. >> >> >> And if in the u-boot is workaround for particular kernel bug the= n it >> >> > should be properly documented with information in which kernel ve= rsion >> >> > was bug fixed. Otherwise u-boot code stay in non-maintenanceble s= tate. >> >> > >> >> I am not sure whether it is a kernel issue. For this are workaroun= d codes, >> >> I will remove this function (starfive_pcie_hide_rc_bar) in next ver= sion. Remove this codes uboot PCIe still can work.=20 >> >=20 >> > If everything is working fine then I do not see reason for having it >> > there. >> >=20 >> >> >> >> + (offset =3D=3D PCI_BASE_ADDRESS_0 || offset =3D=3D PCI_= BASE_ADDRESS_1)) >> >> >> >> + return true; >> >> >> >> + >> >> >> >> + return false; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_conf_address(const struct udevice *= udev, pci_dev_t bdf, >> >> >> >> + uint offset, void **paddr) >> >> >> >> +{ >> >> >> >> + struct starfive_pcie *priv =3D dev_get_priv(udev); >> >> >> >> + int where =3D starfive_pcie_off_conf(bdf, offset); >> >> >> >> + >> >> >> >> + if (!starfive_pcie_addr_valid(bdf, priv->first_busno)) >> >> >> >> + return -EINVAL; >> >> >> >=20 >> >> >> > Is this check needed? Does it mean that ECAM returns bogus inf= ormation >> >> >> > for disconnected devices, that it needs to be manually filtere= d? (Which >> >> >> > is against PCIe spec). >> >> >> >=20 >> >> >> I found nvme device can not work if the non-zero function of roo= t port >> >> >> without limitation. But I will try to fix it. Maybe I can remove= this. >> >> Please ignore this reply. I have replied wrong message. I am sorry = about this >> >> It is not 'non-zero function'. I should said 'non-zero device'. The= check reason >> >> see starfive_pcie_addr_valid reply.=20 >> >> > So you have multifunction root port device (if you are talking ab= out >> >> > non-zero function)? And what does it mean that NVMe device cannot= work? >> >> > Can you connect your device to second function of root port if it= works? >> >> > >> >> Please ignore previous reply. And the test NVMe devices do not supp= ort >> >> multi-function. >> >> >> >> + >> >> >> >> + *paddr =3D (void *)(priv->cfg_base + where); >> >> >> >> + return 0; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_config_read(const struct udevice *u= dev, pci_dev_t bdf, >> >> >> >> + uint offset, ulong *valuep, >> >> >> >> + enum pci_size_t size) >> >> >> >> +{ >> >> >> >> + /* Make sure the LAST TLP is finished, before reading vendo= r ID. */ >> >> >> >> + if (offset =3D=3D PCI_VENDOR_ID) >> >> >> >> + mdelay(20); >> >> >> >=20 >> >> >> > This looks strange. There is some bug / errata that reading ve= ndor id >> >> >> > from ECAM does not work correctly? >> >> >> >=20 >> >> >> I will remove this. It is not a errata of reading the vendor ID.= =20 >> >> >> >> + >> >> >> >> + return pci_generic_mmap_read_config(udev, starfive_pcie_con= f_address, >> >> >> >> + bdf, offset, valuep, size); >> >> >> >> +} >> >> >> >> + >> >> >> >> +int starfive_pcie_config_write(struct udevice *udev, pci_dev= _t bdf, >> >> >> >> + uint offset, ulong value, >> >> >> >> + enum pci_size_t size) >> >> >> >> +{ >> >> >> >> + if (starfive_pcie_hide_rc_bar(bdf, offset)) >> >> >> >> + return 0; >> >> >> >=20 >> >> >> > Why you are ignoring writes to some offset, but not ignoring r= eads? And >> >> >> > ignoring results in no-error (return value 0)? >> >> >> >The same as [1] >> >> >=20 >> >> > But this does not explain why is result ignored. I'm just lost wh= at is >> >> > aim or expected result here. >> >> >=20 >> >> Since it is a workaround codes for booting kernel, (Writing root po= rt base0 and base 1 >> >> register cause kernel PCIe driver can not work, So just limtation w= riting base0 and bas1). >> >> I will remove starfive_pcie_hide_rc_bar as previous said. >> >> >> >> + >> >> >> >> + return pci_generic_mmap_write_config(udev, starfive_pcie_co= nf_address, >> >> >> >> + bdf, offset, value, size); >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_set_atr_entry(struct starfive_pcie = *priv, phys_addr_t src_addr, >> >> >> >> + phys_addr_t trsl_addr, size_t window_size, >> >> >> >> + int trsl_param) >> >> >> >> +{ >> >> >> >> + void __iomem *base =3D >> >> >> >> + priv->reg_base + XR3PCI_ATR_AXI4_SLV0; >> >> >> >> + >> >> >> >> + /* Support AXI4 Slave 0 Address Translation Tables 0-7. */ >> >> >> >> + if (priv->atr_table_num >=3D XR3PCI_ATR_MAX_TABLE_NUM) { >> >> >> >> + dev_err(priv->dev, "ATR table number %d exceeds max num\n"= , >> >> >> >> + priv->atr_table_num); >> >> >> >> + return -EINVAL; >> >> >> >> + } >> >> >> >> + base +=3D XR3PCI_ATR_TABLE_OFFSET * priv->atr_table_num; >> >> >> >> + priv->atr_table_num++; >> >> >> >> + >> >> >> >> + /* X3PCI_ATR_SRC_ADDR_LOW: >> >> >> >> + * - bit 0: enable entry, >> >> >> >> + * - bits 1-6: ATR window size: total size in bytes: 2^(A= TR_WSIZE + 1) >> >> >> >> + * - bits 7-11: reserved >> >> >> >> + * - bits 12-31: start of source address >> >> >> >> + */ >> >> >> >> + writel((lower_32_bits(src_addr) & XR3PCI_ATR_SRC_ADDR_MASK)= | >> >> >> >> + (fls(window_size) - 1) << XR3PCI_ATR_SRC_WIN_SIZE_SHIFT |= 1, >> >> >> >> + base + XR3PCI_ATR_SRC_ADDR_LOW); >> >> >> >> + writel(upper_32_bits(src_addr), base + XR3PCI_ATR_SRC_ADDR_= HIGH); >> >> >> >> + writel((lower_32_bits(trsl_addr) & XR3PCI_ATR_TRSL_ADDR_MAS= K), >> >> >> >> + base + XR3PCI_ATR_TRSL_ADDR_LOW); >> >> >> >> + writel(upper_32_bits(trsl_addr), base + XR3PCI_ATR_TRSL_ADD= R_HIGH); >> >> >> >> + writel(trsl_param, base + XR3PCI_ATR_TRSL_PARAM); >> >> >> >> + >> >> >> >> + dev_dbg(priv->dev, "ATR entry: 0x%010llx %s 0x%010llx [0x%0= 10llx] (param: 0x%06x)\n", >> >> >> >> + src_addr, (trsl_param & XR3PCI_ATR_TRSL_DIR) ? "<-" : "->"= , >> >> >> >> + trsl_addr, (u64)window_size, trsl_param); >> >> >> >> + return 0; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_atr_init(struct starfive_pcie *priv= ) >> >> >> >> +{ >> >> >> >> + struct udevice *ctlr =3D pci_get_controller(priv->dev); >> >> >> >> + struct pci_controller *hose =3D dev_get_uclass_priv(ctlr); >> >> >> >> + int i, ret; >> >> >> >> + >> >> >> >> + /* As the two host bridges in JH7110 soc have the same defa= ult >> >> >> >> + * address translation table, this cause the second root po= rt can't >> >> >> >> + * access it's host bridge config space correctly. >> >> >> >> + * To workaround, config the ATR of host bridge config spac= e by SW. >> >> >> >> + */ >> >> >> >> + >> >> >> >> + ret =3D starfive_pcie_set_atr_entry(priv, >> >> >> >> + (phys_addr_t)priv->cfg_base, >> >> >> >> + 0, >> >> >> >> + 1 << XR3_PCI_ECAM_SIZE, >> >> >> >> + XR3PCI_ATR_TRSLID_PCIE_CONFIG); >> >> >> >> + if (ret) >> >> >> >> + return ret; >> >> >> >> + >> >> >> >> + for (i =3D 0; i < hose->region_count; i++) { >> >> >> >> + if (hose->regions[i].flags =3D=3D PCI_REGION_SYS_MEMORY) >> >> >> >> + continue; >> >> >> >> + >> >> >> >> + /* Only support identity mappings. */ >> >> >> >> + if (hose->regions[i].bus_start !=3D >> >> >> >> + hose->regions[i].phys_start) >> >> >> >> + return -EINVAL; >> >> >> >> + >> >> >> >> + ret =3D starfive_pcie_set_atr_entry(priv, >> >> >> >> + hose->regions[i].phys_start, >> >> >> >> + hose->regions[i].bus_start, >> >> >> >> + hose->regions[i].size, >> >> >> >> + XR3PCI_ATR_TRSLID_PCIE_MEMORY); >> >> >> >> + if (ret) >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + return 0; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_get_syscon(struct udevice *dev) >> >> >> >> +{ >> >> >> >> + struct starfive_pcie *priv =3D dev_get_priv(dev); >> >> >> >> + struct udevice *syscon; >> >> >> >> + struct ofnode_phandle_args syscfg_phandle; >> >> >> >> + u32 cells[4]; >> >> >> >> + int ret; >> >> >> >> + >> >> >> >> + /* get corresponding syscon phandle */ >> >> >> >> + ret =3D dev_read_phandle_with_args(dev, "starfive,stg-sysco= n", NULL, 0, 0, >> >> >> >> + &syscfg_phandle); >> >> >> >> + >> >> >> >> + if (ret < 0) { >> >> >> >> + dev_err(dev, "Can't get syscfg phandle: %d\n", ret); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + ret =3D uclass_get_device_by_ofnode(UCLASS_SYSCON, syscfg_p= handle.node, >> >> >> >> + &syscon); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Unable to find syscon device (%d)\n", ret); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + priv->regmap =3D syscon_get_regmap(syscon); >> >> >> >> + if (!priv->regmap) { >> >> >> >> + dev_err(dev, "Unable to find regmap\n"); >> >> >> >> + return -ENODEV; >> >> >> >> + } >> >> >> >> + >> >> >> >> + /* get syscon register offset */ >> >> >> >> + ret =3D dev_read_u32_array(dev, "starfive,stg-syscon", >> >> >> >> + cells, ARRAY_SIZE(cells)); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Get syscon register err %d\n", ret); >> >> >> >> + return -EINVAL; >> >> >> >> + } >> >> >> >> + >> >> >> >> + dev_dbg(dev, "Get syscon values: %x, %x, %x\n", >> >> >> >> + cells[1], cells[2], cells[3]); >> >> >> >> + priv->stg_arfun =3D cells[1]; >> >> >> >> + priv->stg_awfun =3D cells[2]; >> >> >> >> + priv->stg_rp_nep =3D cells[3]; >> >> >> >> + >> >> >> >> + return 0; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_parse_dt(struct udevice *dev) >> >> >> >> +{ >> >> >> >> + struct starfive_pcie *priv =3D dev_get_priv(dev); >> >> >> >> + int ret; >> >> >> >> + >> >> >> >> + priv->reg_base =3D (void *)dev_read_addr_name(dev, "reg"); >> >> >> >> + if (priv->reg_base =3D=3D (void __iomem *)FDT_ADDR_T_NONE) = { >> >> >> >> + dev_err(dev, "Missing required reg address range\n"); >> >> >> >> + return -EINVAL; >> >> >> >> + } >> >> >> >> + >> >> >> >> + priv->cfg_base =3D (void *)dev_read_addr_name(dev, "config"= ); >> >> >> >> + if (priv->cfg_base =3D=3D (void __iomem *)FDT_ADDR_T_NONE) = { >> >> >> >> + dev_err(dev, "Missing required config address range"); >> >> >> >> + return -EINVAL; >> >> >> >> + } >> >> >> >> + >> >> >> >> + ret =3D starfive_pcie_get_syscon(dev); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Can't get syscon: %d\n", ret); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + ret =3D reset_get_bulk(dev, &priv->rsts); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Can't get reset: %d\n", ret); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + ret =3D clk_get_bulk(dev, &priv->clks); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Can't get clock: %d\n", ret); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + return 0; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_init_port(struct udevice *dev) >> >> >> >> +{ >> >> >> >> + int ret, i; >> >> >> >> + unsigned int value; >> >> >> >> + struct starfive_pcie *priv =3D dev_get_priv(dev); >> >> >> >> + >> >> >> >> + ret =3D clk_enable_bulk(&priv->clks); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Failed to enable clks (ret=3D%d)\n", ret); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + ret =3D reset_deassert_bulk(&priv->rsts); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Failed to deassert resets (ret=3D%d)\n", ret= ); >> >> >> >> + goto err_deassert_clk; >> >> >> >> + } >> >> >> >> + >> >> >> >> + ret =3D pinctrl_select_state(dev, "perst-active"); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Set perst-active pinctrl failed: %d\n", ret)= ; >> >> >> >> + goto err_deassert_reset; >> >> >> >> + } >> >> >> >> + >> >> >> >> + /* Disable physical functions except #0 */ >> >> >> >> + for (i =3D 1; i < PLDA_FUNC_NUM; i++) { >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_arfun, >> >> >> >> + STG_SYSCON_AXI4_SLVL_ARFUNC_MASK, >> >> >> >> + (i << PLDA_PHY_FUNC_SHIFT) << >> >> >> >> + STG_SYSCON_AXI4_SLVL_ARFUNC_SHIFT); >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_awfun, >> >> >> >> + STG_SYSCON_AXI4_SLVL_AWFUNC_MASK, >> >> >> >> + i << PLDA_PHY_FUNC_SHIFT); >> >> >> >> + >> >> >> >> + value =3D readl(priv->reg_base + PCI_MISC); >> >> >> >> + value |=3D PLDA_FUNCTION_DIS; >> >> >> >> + writel(value, priv->reg_base + PCI_MISC); >> >> >> >> + } >> >> >> >> + >> >> >> >> + /* Disable physical functions */ >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_arfun, >> >> >> >> + STG_SYSCON_AXI4_SLVL_ARFUNC_MASK, >> >> >> >> + 0); >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_awfun, >> >> >> >> + STG_SYSCON_AXI4_SLVL_AWFUNC_MASK, >> >> >> >> + 0); >> >> >> >> + >> >> >> >> + /* Enable root port */ >> >> >> >> + value =3D readl(priv->reg_base + GEN_SETTINGS); >> >> >> >> + value |=3D PLDA_RP_ENABLE; >> >> >> >> + writel(value, priv->reg_base + GEN_SETTINGS); >> >> >> >> + >> >> >> >> + /* PCIe PCI Standard Configuration Identification Settings.= */ >> >> >> >> + value =3D (IDS_PCI_TO_PCI_BRIDGE << IDS_CLASS_CODE_SHIFT) |= IDS_REVISION_ID; >> >> >> >> + writel(value, priv->reg_base + PCIE_PCI_IDS); >> >> >> >> + >> >> >> >> + /* The LTR message forwarding of PCIe Message Reception was= set by core >> >> >> >> + * as default, but the forward id & addr are also need to b= e reset. >> >> >> >> + * If we do not disable LTR message forwarding here, or set= a legal >> >> >> >> + * forwarding address, the kernel will get stuck after this= driver probe. >> >> >> >> + * To workaround, disable the LTR message forwarding suppor= t on >> >> >> >> + * PCIe Message Reception. >> >> >> >> + */ >> >> >> >> + value =3D readl(priv->reg_base + PMSG_SUPPORT_RX); >> >> >> >> + value &=3D ~PMSG_LTR_SUPPORT; >> >> >> >> + writel(value, priv->reg_base + PMSG_SUPPORT_RX); >> >> >> >> + >> >> >> >> + /* Prefetchable memory window 64-bit addressing support */ >> >> >> >> + value =3D readl(priv->reg_base + PCIE_WINROM); >> >> >> >> + value |=3D PREF_MEM_WIN_64_SUPPORT; >> >> >> >> + writel(value, priv->reg_base + PCIE_WINROM); >> >> >> >> + >> >> >> >> + starfive_pcie_atr_init(priv); >> >> >> >> + >> >> >> >> + /* Ensure that PERST has been asserted for at least 300 ms = */ >> >> >> >> + mdelay(300); >> >> >> >> + ret =3D pinctrl_select_state(dev, "perst-default"); >> >> >> >> + if (ret) { >> >> >> >> + dev_err(dev, "Set perst-default pinctrl failed: %d\n", ret= ); >> >> >> >> + return ret; >> >> >> >> + } >> >> >> >> + >> >> >> >> + return 0; >> >> >> >> + >> >> >> >> +err_deassert_clk: >> >> >> >> + clk_disable_bulk(&priv->clks); >> >> >> >> +err_deassert_reset: >> >> >> >> + reset_assert_bulk(&priv->rsts); >> >> >> >> + return ret; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static int starfive_pcie_probe(struct udevice *dev) >> >> >> >> +{ >> >> >> >> + struct starfive_pcie *priv =3D dev_get_priv(dev); >> >> >> >> + int ret; >> >> >> >> + >> >> >> >> + priv->atr_table_num =3D 0; >> >> >> >> + priv->dev =3D dev; >> >> >> >> + priv->first_busno =3D dev_seq(dev); >> >> >> >> + >> >> >> >> + ret =3D starfive_pcie_parse_dt(dev); >> >> >> >> + if (ret) >> >> >> >> + return ret; >> >> >> >> + >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_rp_nep, >> >> >> >> + STG_SYSCON_K_RP_NEP_MASK, >> >> >> >> + STG_SYSCON_K_RP_NEP_MASK); >> >> >> >> + >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_awfun, >> >> >> >> + STG_SYSCON_CKREF_SRC_MASK, >> >> >> >> + 2 << STG_SYSCON_CKREF_SRC_SHIFT); >> >> >> >> + >> >> >> >> + regmap_update_bits(priv->regmap, >> >> >> >> + priv->stg_awfun, >> >> >> >> + STG_SYSCON_CLKREQ_MASK, >> >> >> >> + STG_SYSCON_CLKREQ_MASK); >> >> >> >> + >> >> >> >> + ret =3D starfive_pcie_init_port(dev); >> >> >> >> + if (ret) >> >> >> >> + return ret; >> >> >> >> + >> >> >> >> + dev_err(dev, "Starfive PCIe bus probed.\n"); >> >> >> >> + >> >> >> >> + return 0; >> >> >> >> +} >> >> >> >> + >> >> >> >> +static const struct dm_pci_ops starfive_pcie_ops =3D { >> >> >> >> + .read_config =3D starfive_pcie_config_read, >> >> >> >> + .write_config =3D starfive_pcie_config_write, >> >> >> >> +}; >> >> >> >> + >> >> >> >> +static const struct udevice_id starfive_pcie_ids[] =3D { >> >> >> >> + { .compatible =3D "starfive,jh7110-pcie" }, >> >> >> >> + { } >> >> >> >> +}; >> >> >> >> + >> >> >> >> +U_BOOT_DRIVER(starfive_pcie_drv) =3D { >> >> >> >> + .name =3D "starfive_7110_pcie", >> >> >> >> + .id =3D UCLASS_PCI, >> >> >> >> + .of_match =3D starfive_pcie_ids, >> >> >> >> + .ops =3D &starfive_pcie_ops, >> >> >> >> + .probe =3D starfive_pcie_probe, >> >> >> >> + .priv_auto =3D sizeof(struct starfive_pcie), >> >> >> >> +}; >> >> >> >> --=20 >> >> >> >> 2.17.1 >> >> >> >>=20