pci, thunder: Add Cavium Thunder PCIe host controller
cover pci, thunder: Add Cavium Thunder PCIe host controller
From: Robert Richter <[email protected]>
This patches add support for the Cavium Thunder PCIe host controller.
Patches base on following branches:
git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci pci/liviu-generic-v11
git://linux-arm.org/linux-ld for-upstream/pci_arm64_v11
Additionally a fix is required for liviu-generic-v11 to allow the generation of mem-mapped pci ranges:
https://lkml.org/lkml/2014/9/22/150
It also requires Marc's gicv3-its support to work.
The first patch introduces the host controller driver. Patch 2 to 4 are DT changes including documentation. Patch 5 and 6 enable PCI support for arm64 defconfig.
Robert Richter (2):
arm64, defconfig: Enable PCI
pci, thunder: Enable Cavium Thunder PCIe host controller
Sunil Goutham (3):
pci, thunder: Add support for Thunder PCIe host controller pci, thunder: Add PCIe host controller devicetree bindings pci, thunder: Document PCIe host controller devicetree bindings
Tirumalesh Chalamarla (1):
GICv3: Add ITS entry to THUNDER dts
.../devicetree/bindings/pci/cavium,thunder-pci.txt | 32 +++ arch/arm64/Kconfig | 15 ++ arch/arm64/boot/dts/thunder-88xx.dtsi | 58 +++++ drivers/pci/host/Kconfig | 8 + drivers/pci/host/Makefile | 1 + drivers/pci/host/pcie-thunder.c | 246 +++++++++++++++++++++ include/linux/pci_ids.h | 2 + 7 files changed, 362 insertions(+) create mode 100644 Documentation/devicetree/bindings/pci/cavium,thunder-pci.txt create mode 100644 drivers/pci/host/pcie-thunder.c
Signature · 1 line
2.1.0
PATCH 1/6 pci, thunder: Add support for Thunder PCIe host controller
From: Sunil Goutham <[email protected]>
This patch adds support for PCI host controller of Cavium Thunder SoCs.
--- drivers/pci/host/Kconfig | 8 ++ drivers/pci/host/Makefile | 1 + drivers/pci/host/pcie-thunder.c | 246 ++++++++++++++++++++++++++++++++++++++++ include/linux/pci_ids.h | 2 + 4 files changed, 257 insertions(+) create mode 100644 drivers/pci/host/pcie-thunder.c
drivers/pci/host/Kconfig +8 -0
@@ -63,4 +63,12 @@ config PCIE_SPEAR13XX help Say Y here if you want PCIe support on SPEAr13XX SoCs. +config PCI_THUNDER + bool "Thunder PCIe host controller" + depends on ARM64 || COMPILE_TEST + depends on OF_PCI + depends on PCI_MSI + help + Say Y here if you want internal PCI support on Thunder SoC. + endmenu
drivers/pci/host/Makefile +1 -0
@@ -8,3 +8,4 @@ obj-$(CONFIG_PCI_RCAR_GEN2) += pci-rcar-gen2.o obj-$(CONFIG_PCI_RCAR_GEN2_PCIE) += pcie-rcar.o obj-$(CONFIG_PCI_HOST_GENERIC) += pci-host-generic.o obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o +obj-$(CONFIG_PCI_THUNDER) += pcie-thunder.o
drivers/pci/host/pcie-thunder.c new file +246 -0
246 lines not on this page · raw message
include/linux/pci_ids.h +2 -0
@@ -2322,6 +2322,8 @@#define PCI_DEVICE_ID_ALTIMA_AC9100 0x03ea#define PCI_DEVICE_ID_ALTIMA_AC1003 0x03eb +#define PCI_VENDOR_ID_CAVIUM 0x177d +#define PCI_VENDOR_ID_BELKIN 0x1799#define PCI_DEVICE_ID_BELKIN_F5D7010V7 0x701f
Signature · 1 line
2.1.0
Re: [PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller
Quoted 4 lines · On Wednesday 24 September 2014 17:37:43 Robert Richter wrote:
From: Sunil Goutham <[email protected]>
This patch adds support for PCI host controller of Cavium Thunder SoCs.
I had expected this hardware to be SBSA compliant. Why do you need a hardware specific driver, is this a workaround for buggy hardware or just noncompliant?
drivers/pci/host/pcie-thunder.c line 69
Quoted 11 lines
drivers/pci/host/pcie-thunder.c new file +11
-0+/* +* All PCIe devices in Thunder have fixed resources, shouldn't be reassigned. +* Also claim the device's valid resources to set 'res->parent' hierarchy. +*/ +static void pci_dev_resource_fixup (struct pci_dev *dev) +{ +struct resource *res; +int resno; + +for (resno =0 ; resno <PCI_NUM_RESOURCES ; resno++) + dev->resource[resno].flags |=IORESOURCE_PCI_FIXED ;
You have listed these as relocatable in DT, why do you have to mark them as nonrelocatable here?
Quoted 26 lines
drivers/pci/host/pcie-thunder.c new file +26
-0+static void __iomem *thunder_pcie_cfg_base (struct thunder_pcie *pcie, +unsigned int bus,unsigned int devfn) +{ +return pcie->cfg_base + ((bus <<THUNDER_PCIE_BUS_SHIFT ) + | (PCI_SLOT (devfn) <<THUNDER_PCIE_DEV_SHIFT ) + | (PCI_FUNC (devfn) <<THUNDER_PCIE_FUNC_SHIFT )); +} + +static int thunder_pcie_read_config (struct pci_bus *bus,unsigned int devfn, +int reg,int size,u32 *val) +{ +struct thunder_pcie *pcie = bus->sysdata; +void __iomem *addr; +unsigned int busnr = bus->number; + +if (busnr >255 || devfn >255 || reg >4095 ) +return PCIBIOS_DEVICE_NOT_FOUND ; + + addr =thunder_pcie_cfg_base (pcie, busnr, devfn) + reg; + +switch (size) { +case 1 : + *val =readb (addr); +break ; +case 2 : + *val =readw (addr);
drivers/pci/host/pcie-thunder.c line 116
Quoted 10 lines
drivers/pci/host/pcie-thunder.c new file +10
-0+break ; +case 4 : + *val =readl (addr); +break ; +default : +return PCIBIOS_BAD_REGISTER_NUMBER ; + } + +return PCIBIOS_SUCCESSFUL ; +}
This looks roughly ECAM compliant, are you sure you need a private implementation?
Quoted 8 lines
drivers/pci/host/pcie-thunder.c new file +8
-0+static int thunder_pcie_msi_enable (struct thunder_pcie *pcie, +struct pci_bus *bus) +{ +struct device_node *msi_node; + + msi_node =of_parse_phandle (pcie->node,"msi-parent" ,0 ); +if (!msi_node) +return -ENODEV ;
drivers/pci/host/pcie-thunder.c line 169
Quoted 10 lines
drivers/pci/host/pcie-thunder.c new file +10
-0+ + pcie->msi =of_pci_find_msi_chip_by_node (msi_node); +if (!pcie->msi) +return -ENODEV ; + + pcie->msi->dev = pcie->dev; + bus->msi = pcie->msi; + +return 0 ; +}
This is probably something we should add to the generic host driver as well, so it can work with SBSA compliant implementations that come with an MSI controller. Maybe move it into common code so it can be shared with that driver.
Arnd
Re: [PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller
Quoted 9 lines · Arnd Bergmann
Quoted 4 lines · On Wednesday 24 September 2014 17:37:43 Robert Richter wrote:
From: Sunil Goutham <[email protected]>
This patch adds support for PCI host controller of Cavium Thunder SoCs.
I had expected this hardware to be SBSA compliant. Why do you need a hardware specific driver, is this a workaround for buggy hardware or just noncompliant?
Patches welcome to pci-host-generic.c :) Lorenzo already has code to port it to Liviu's new API, so do shout if it's not suitable for your needs.
Quoted 15 lines
Quoted 15 lines
drivers/pci/host/pcie-thunder.c new file +15
-0+static int thunder_pcie_msi_enable (struct thunder_pcie *pcie, +struct pci_bus *bus) +{ +struct device_node *msi_node; + + msi_node =of_parse_phandle (pcie->node,"msi-parent" ,0 ); +if (!msi_node) +return -ENODEV ; + + pcie->msi =of_pci_find_msi_chip_by_node (msi_node); +if (!pcie->msi) +return -ENODEV ; + + pcie->msi->dev = pcie->dev; + bus->msi = pcie->msi;
Quoted 8 lines
drivers/pci/host/pcie-thunder.c line 169
Quoted 3 lines
drivers/pci/host/pcie-thunder.c new file +3
-0+ +return 0 ; +}This is probably something we should add to the generic host driver as well, so it can work with SBSA compliant implementations that come with an MSI controller. Maybe move it into common code so it can be shared with that driver.
Agreed. I've been carrying something similar [1] (based on a hacked-up version of bios32, so not bothered to post it) whilst I've been waiting for the arm64 core PCI code to get merged.
Will
Re: [PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller
Will/Arnd
Thanks for the comments.
There is another patch submitted for adding MSI controller to Generic driver.
https://lkml.org/lkml/2014/9/28/150
Will go through these and comeback.
Quoted 49 lines · Will Deacon
Quoted 9 lines · Arnd Bergmann
Quoted 4 lines · On Wednesday 24 September 2014 17:37:43 Robert Richter wrote:
From: Sunil Goutham <[email protected]>
This patch adds support for PCI host controller of Cavium Thunder SoCs.
I had expected this hardware to be SBSA compliant. Why do you need a hardware specific driver, is this a workaround for buggy hardware or just noncompliant?
Patches welcome to
pci-host-generic.c:) Lorenzo already has code to port it to Liviu's new API, so do shout if it's not suitable for your needs.Quoted 15 lines
Quoted 15 lines
drivers/pci/host/pcie-thunder.c new file +15
-0+static int thunder_pcie_msi_enable (struct thunder_pcie *pcie, +struct pci_bus *bus) +{ +struct device_node *msi_node; + + msi_node =of_parse_phandle (pcie->node,"msi-parent" ,0 ); +if (!msi_node) +return -ENODEV ; + + pcie->msi =of_pci_find_msi_chip_by_node (msi_node); +if (!pcie->msi) +return -ENODEV ; + + pcie->msi->dev = pcie->dev; + bus->msi = pcie->msi;Quoted 8 lines
drivers/pci/host/pcie-thunder.c line 169
Quoted 3 lines
drivers/pci/host/pcie-thunder.c new file +3
-0+ +return 0 ; +}This is probably something we should add to the generic host driver as well, so it can work with SBSA compliant implementations that come with an MSI controller. Maybe move it into common code so it can be shared with that driver.
Agreed. I've been carrying something similar [1] (based on a hacked-up version of bios32, so not bothered to post it) whilst I've been waiting for the arm64 core PCI code to get merged.
Will
[1] https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/commit/drivers/pci/host/pci-host-generic.c?h=iommu/pci&id=b719acf062ceccfbd79ee7b1ae0b7904ea4da27e
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/6 GICv3: Add ITS entry to THUNDER dts
In reply to Robert RichterFrom: Tirumalesh Chalamarla <[email protected]>
The PCIe host controller uses MSIs provided by GICv3 ITS. Enable it on Thunder SoCs by adding an entry to DT.
--- arch/arm64/boot/dts/thunder-88xx.dtsi | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/arm64/boot/dts/thunder-88xx.dtsi +9 -0
@@ -376,10 +376,19 @@ gic0: interrupt-controller at 8010,00000000 { compatible = "arm,gic-v3"; #interrupt-cells = <3>; + #address-cells = <2>; + #size-cells = <2>; + ranges; interrupt-controller; reg = <0x8010 0x00000000 0x0 0x010000>, /* GICD */ <0x8010 0x80000000 0x0 0x600000>; /* GICR */ interrupts = <1 9 0xf04>; + + its: gic-its at 8010,00020000 { + compatible = "arm,gic-v3-its"; + msi-controller; + reg = <0x8010 0x20000 0x0 0x200000>; + }; }; uaa0: serial at 87e0,24000000 {
Signature · 1 line
2.1.0
Re: [PATCH 2/6] GICv3: Add ITS entry to THUNDER dts
Hi Marc,
Any feed back on this.
Do you want me to submit this as a separate patch?so that it is easy for getting acceptance.
Thanks,
Tirumalesh.
Quoted 40 lines · Robert Richter
From: Tirumalesh Chalamarla <[email protected]>
The PCIe host controller uses MSIs provided by GICv3 ITS. Enable it on Thunder SoCs by adding an entry to DT.
Signed-off-by Tirumalesh Chalamarla <[email protected]>Signed-off-by Robert Richter <[email protected]>---
arch/arm64/boot/dts/thunder-88xx.dtsi| 9 +++++++++1 file changed, 9 insertions(+)arch/arm64/boot/dts/thunder-88xx.dtsi +9
-0@@ -376,10 +376,19 @@ gic0: interrupt-controller at 8010,00000000 { compatible = "arm,gic-v3"; #interrupt-cells = <3>; + #address-cells = <2>; + #size-cells = <2>; + ranges; interrupt-controller; reg = <0x8010 0x00000000 0x0 0x010000>, /* GICD */ <0x8010 0x80000000 0x0 0x600000>; /* GICR */ interrupts = <1 9 0xf04>; + + its: gic-its at 8010,00020000 { + compatible = "arm,gic-v3-its"; + msi-controller; + reg = <0x8010 0x20000 0x0 0x200000>; + }; }; uaa0: serial at 87e0,24000000 {Signature · 1 line
2.1.0
Re: [PATCH 2/6] GICv3: Add ITS entry to THUNDER dts
Quoted 4 lines · Tirumalesh
Hi Marc,
Any feed back on this.
Do you want me to submit this as a separate patch?so that it is easy for getting acceptance.
It looks good to me, so you can add my Acked-by on it.
I think you should repost it and get it merged through arm-soc.
Thanks,
M.
Quoted 44 lines
Thanks,
Tirumalesh.Quoted 40 lines · Robert Richter
From: Tirumalesh Chalamarla <[email protected]>
The PCIe host controller uses MSIs provided by GICv3 ITS. Enable it on Thunder SoCs by adding an entry to DT.
Signed-off-by Tirumalesh Chalamarla <[email protected]>Signed-off-by Robert Richter <[email protected]>---
arch/arm64/boot/dts/thunder-88xx.dtsi| 9 +++++++++1 file changed, 9 insertions(+)arch/arm64/boot/dts/thunder-88xx.dtsi +9
-0@@ -376,10 +376,19 @@ gic0: interrupt-controller at 8010,00000000 { compatible = "arm,gic-v3"; #interrupt-cells = <3>; + #address-cells = <2>; + #size-cells = <2>; + ranges; interrupt-controller; reg = <0x8010 0x00000000 0x0 0x010000>, /* GICD */ <0x8010 0x80000000 0x0 0x600000>; /* GICR */ interrupts = <1 9 0xf04>; + + its: gic-its at 8010,00020000 { + compatible = "arm,gic-v3-its"; + msi-controller; + reg = <0x8010 0x20000 0x0 0x200000>; + }; }; uaa0: serial at 87e0,24000000 {Signature · 1 line
2.1.0
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH 3/6 pci, thunder: Add PCIe host controller devicetree bindings
In reply to Robert RichterFrom: Sunil Goutham <[email protected]>
This patch adds the PCIe host controller entry for Cavium Thunder SoCs to the devicetree. There are 4 internal PCI controllers available.
--- arch/arm64/boot/dts/thunder-88xx.dtsi | 49 +++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
arch/arm64/boot/dts/thunder-88xx.dtsi +49 -0
@@ -407,4 +407,53 @@ clock-names = "apb_pclk"; }; }; + + pcie0 at 0x8480,00000000 { + compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + }; + + pcie1 at 0x8490,00000000 { + compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8490 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8310 0x00000000 0x8310 0x00000000 0x00 0x10000000>, /* mem ranges */ + <0x03000000 0x8100 0x00000000 0x8100 0x00000000 0x80 0x00000000>; + }; + + pcie2 at 0x84a0,00000000 { + compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x84a0 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8320 0x00000000 0x8320 0x00000000 0x00 0x10000000>, /* mem ranges */ + <0x03000000 0x8430 0x00000000 0x8430 0x00000000 0x01 0x00000000>; + }; + + pcie3 at 0x84b0,00000000 { + compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x84b0 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8330 0x00000000 0x8330 0x00000000 0x00 0x10000000>, /* mem ranges */ + <0x03000000 0x8180 0x00000000 0x8180 0x00000000 0x80 0x00000000>; + }; };
Signature · 1 line
2.1.0
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 2 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
+ pcie0 at 0x8480,00000000 {
The name should be pci, not pci0.
arch/arm64/boot/dts/thunder-88xx.dtsi line 422
Quoted 11 lines
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
The interrupt-map and interrupt-map-mask properties are required for PCI, otherwise you can't do LSI interrupts.
If your hardware can support it, you should also list I/O space and prefetchable memory spaces. Can you explain why you have multiple non-prefetchable ranges?
Arnd
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 5 lines · Arnd Bergmann
Quoted 2 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
+ pcie0 at 0x8480,00000000 {
The name should be pci, not pci0.
Thanks, will change.
Quoted 13 lines
arch/arm64/boot/dts/thunder-88xx.dtsi line 422
Quoted 12 lines
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };
Quoted 6 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
The interrupt-map and interrupt-map-mask properties are required for PCI, otherwise you can't do LSI interrupts.
This PCI controller supports only MSIx interrupts which are edge triggered.
Quoted 3 lines
If your hardware can support it, you should also list I/O space and prefetchable memory spaces. Can you explain why you have multiple non-prefetchable ranges?
Our hardware is an ECAM based host controller and doesn't support I/O and prefetchable memory spaces.
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.
Kernel's SRIOV (pci/iov.c) is expecting 'resource->parent' hierarchy to be set, otherwise doesn't
enable SRIOV device. So, here multiple non-prefetchable ranges of root bus aid in resource claiming and setting res->parent hierarchy.
We do call "pci_claim_resource" in controller driver code. "[PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller."
Quoted 6 lines
Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 14 lines · On Wednesday 24 September 2014 23:34:04 Sunil Kovvuri wrote:
Quoted 13 lines · Arnd Bergmann
Quoted 11 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };
Quoted 8 lines
Quoted 6 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
The interrupt-map and interrupt-map-mask properties are required for PCI, otherwise you can't do LSI interrupts.
This PCI controller supports only MSIx interrupts which are edge triggered.
Interesting, so it's not PCI compliant then? I assume this will be fixed in the production version of the silicon, right?
Having no support for interrupts mean that the majority of PCI device drivers will fail.
Quoted 8 lines
Quoted 2 lines
If your hardware can support it, you should also list I/O space and prefetchable memory spaces. Can you explain why you have multiple non-prefetchable ranges?
Our hardware is an ECAM based host controller and doesn't support I/O and prefetchable memory spaces.
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.
I think you need to mark the ones that are nonrelocatable with flag 0x80000000, otherwise the PCI core might decide to reassign them.
Quoted 4 lines
Kernel's SRIOV (
pci/iov.c) is expecting 'resource->parent' hierarchy to be set, otherwise doesn't
enable SRIOV device. So, here multiple non-prefetchable ranges of root bus aid in resource claiming and settingres->parenthierarchy.
I don't understand. Isn't that just a bug in the code that you are working around with the DT. Have you tried fixing the code instead?
Quoted 2 lines
We do call "
pci_claim_resource" in controller driver code. "[PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller."
My guess is that you are using the wrong interface here. Isn't the normal request_resource() in the host driver enough?
Arnd
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 22 lines · Arnd Bergmann
Quoted 21 lines · On Wednesday 24 September 2014 23:34:04 Sunil Kovvuri wrote:
Quoted 19 lines · Arnd Bergmann
arch/arm64/boot/dts/thunder-88xx.dtsi line 422
Quoted 11 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
The interrupt-map and interrupt-map-mask properties are required for PCI, otherwise you can't do LSI interrupts.
Quoted 7 lines
Quoted 1 line
This PCI controller supports only MSIx interrupts which are edge triggered.
Interesting, so it's not PCI compliant then? I assume this will be fixed in the production version of the silicon, right?
Having no support for interrupts mean that the majority of PCI device drivers will fail.
This controller is for on-board PCI devices and all of them do support MSIx interrupts.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
If your hardware can support it, you should also list I/O space and prefetchable memory spaces. Can you explain why you have multiple non-prefetchable ranges?
Quoted 8 lines
Quoted 5 lines
Our hardware is an ECAM based host controller and doesn't support I/O and prefetchable memory spaces.
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.I think you need to mark the ones that are nonrelocatable with flag
0x80000000, otherwise the PCI core might decide to reassign them.
Is this flag part of DT pci node properties ?
I am using IORESOURCE_PCI_FIXED flag. Its there in other patches of the same series.
Quoted 8 lines
Quoted 5 lines
Kernel's SRIOV (
pci/iov.c) is expecting 'resource->parent' hierarchy to be set, otherwise doesn't
enable SRIOV device. So, here multiple non-prefetchable ranges of root bus aid in resource claiming and settingres->parenthierarchy.I don't understand. Isn't that just a bug in the code that you are working around with the DT. Have you tried fixing the code instead?
I tried but wasn't sure if its going to impact existing SRIOV devices. Will have a deeper look again .
Quoted 6 lines
Quoted 3 lines
We do call "
pci_claim_resource" in controller driver code. "[PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller."My guess is that you are using the wrong interface here. Isn't the normal
request_resource()in the host driver enough?
Isn't that host driver calls "request_resource" only for resources of root port. i.e requesting from iomem_resource/ioport_resource. Here i am referring to SRIOV devices enumerated upon scan of root port.
Quoted 2 lines
Arnd
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 25 lines · On Thursday 25 September 2014 00:37:00 Sunil Kovvuri wrote:
Quoted 24 lines · Arnd Bergmann
Quoted 22 lines · On Wednesday 24 September 2014 23:34:04 Sunil Kovvuri wrote:
Quoted 13 lines · Arnd Bergmann
arch/arm64/boot/dts/thunder-88xx.dtsi line 422
Quoted 11 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };Quoted 6 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
The interrupt-map and interrupt-map-mask properties are required for PCI, otherwise you can't do LSI interrupts.
This PCI controller supports only MSIx interrupts which are edge triggered.
Quoted 8 lines
Quoted 5 lines
Interesting, so it's not PCI compliant then? I assume this will be fixed in the production version of the silicon, right?
Having no support for interrupts mean that the majority of PCI device drivers will fail.
This controller is for on-board PCI devices and all of them do support MSIx interrupts.
But in general, booting with "pci=nomsi" should still work, at least for debugging purposes. It's hard to believe the chip designers forgot to connect the one wire.
You said that the PCI host is SBSA compliant, so it must according to section 8.4 have the legacy interrupts wired up to SPI lines in the GIC.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
If your hardware can support it, you should also list I/O space and prefetchable memory spaces. Can you explain why you have multiple non-prefetchable ranges?
Our hardware is an ECAM based host controller and doesn't support I/O and prefetchable memory spaces.
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.
Quoted 6 lines
Quoted 2 lines
I think you need to mark the ones that are nonrelocatable with flag
0x80000000, otherwise the PCI core might decide to reassign them.Is this flag part of DT pci node properties ?
I am usingIORESOURCE_PCI_FIXEDflag. Its there in other patches of the same series.
Ah, right. I checked the source code again and it seems that we don't handle this right at the moment. I think a range that has the nonrelocatable flag set should be used for IORESOURCE_PCI_FIXED mappings without any host specific code, but that needs to be implemented in common code.
Quoted 10 lines
Quoted 7 lines
Quoted 4 lines
Kernel's SRIOV (
pci/iov.c) is expecting 'resource->parent' hierarchy to be set, otherwise doesn't
enable SRIOV device. So, here multiple non-prefetchable ranges of root bus aid in resource claiming and settingres->parenthierarchy.I don't understand. Isn't that just a bug in the code that you are working around with the DT. Have you tried fixing the code instead?
I tried but wasn't sure if its going to impact existing SRIOV devices. Will have a deeper look again .
IIRC others have reported problems in the same area before, and that turned out to be a resource that was not registered correctly. What do you see in /proc/iomem without your workaround?
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
We do call "
pci_claim_resource" in controller driver code. "[PATCH 1/6] pci, thunder: Add support for Thunder PCIe host controller."My guess is that you are using the wrong interface here. Isn't the normal
request_resource()in the host driver enough?Isn't that host driver calls "
request_resource" only for resources of root port. i.e requesting from iomem_resource/ioport_resource. Here i am referring to SRIOV devices enumerated upon scan of root port.
The device resources should be nested within the bus resources, as you would e.g. find on my PC system:
dc000000-dcffffff : PCI Bus 0000:40
dcc00000-dcefffff : PCI Bus 0000:41
dcce0000-dccfffff : 0000:41:00.0
dcd00000-dcefffff : PCI Bus 0000:42
dcd00000-dcdfffff : PCI Bus 0000:45
dcdef800-dcdeffff : 0000:45:00.0
dcdef800-dcdeffff : ahci
ArndBjorn Helgaas
maintainerRe: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · On Thursday 25 September 2014 00:37:00 Sunil Kovvuri wrote:
Quoted 8 lines · Arnd Bergmann
Quoted 3 lines · On Wednesday 24 September 2014 23:34:04 Sunil Kovvuri wrote:
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.
I think you need to mark the ones that are nonrelocatable with flag
0x80000000, otherwise the PCI core might decide to reassign them.
Quoted 8 lines
Quoted 3 lines
Is this flag part of DT pci node properties ?
I am usingIORESOURCE_PCI_FIXEDflag. Its there in other patches of the same series.Ah, right. I checked the source code again and it seems that we don't handle this right at the moment. I think a range that has the nonrelocatable flag set should be used for
IORESOURCE_PCI_FIXEDmappings without any host specific code, but that needs to be implemented in common code.
What connection do you envision between nonrelocatable ranges and IORESOURCE_PCI_FIXED? I don't know what a nonrelocatable range is, but for IORESOURCE_PCI_FIXED, all I intend is that the PCI core should not try to assign a different address, e.g., because the BAR is read-only or because it's a legacy IDE/VGA/etc. range for which there is no BAR at all.
Bjorn
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 21 lines · Bjorn Helgaas
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · On Thursday 25 September 2014 00:37:00 Sunil Kovvuri wrote:
Quoted 8 lines · Arnd Bergmann
Quoted 3 lines · On Wednesday 24 September 2014 23:34:04 Sunil Kovvuri wrote:
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.
I think you need to mark the ones that are nonrelocatable with flag
0x80000000, otherwise the PCI core might decide to reassign them.Quoted 8 lines
Quoted 3 lines
Is this flag part of DT pci node properties ?
I am usingIORESOURCE_PCI_FIXEDflag. Its there in other patches of the same series.Ah, right. I checked the source code again and it seems that we don't handle this right at the moment. I think a range that has the nonrelocatable flag set should be used for
IORESOURCE_PCI_FIXEDmappings without any host specific code, but that needs to be implemented in common code.
Quoted 6 lines
What connection do you envision between nonrelocatable ranges and
IORESOURCE_PCI_FIXED? I don't know what a nonrelocatable range is, but forIORESOURCE_PCI_FIXED, all I intend is that the PCI core should not try to assign a different address, e.g., because the BAR is read-only or because it's a legacy IDE/VGA/etc. range for which there is no BAR at all.
I think that is exactly the definition of the nonrelocatable flag in http://www.openfirmware.org/1275/bindings/pci/pci2_1.pdf
The example given in section 11.1.2 is for a VGA device that has some relocatable BARs and some nonrelocatable BARs.
Arnd
Bjorn Helgaas
maintainerRe: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 29 lines · Arnd Bergmann
Quoted 21 lines · Bjorn Helgaas
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · On Thursday 25 September 2014 00:37:00 Sunil Kovvuri wrote:
Quoted 8 lines · Arnd Bergmann
Quoted 3 lines · On Wednesday 24 September 2014 23:34:04 Sunil Kovvuri wrote:
All on-board PCI devices connected to this PCI controller have fixed resources and doesn't have to be allocated/reassigned. Some of these devices are SRIOV based.
I think you need to mark the ones that are nonrelocatable with flag
0x80000000, otherwise the PCI core might decide to reassign them.Quoted 8 lines
Quoted 3 lines
Is this flag part of DT pci node properties ?
I am usingIORESOURCE_PCI_FIXEDflag. Its there in other patches of the same series.Ah, right. I checked the source code again and it seems that we don't handle this right at the moment. I think a range that has the nonrelocatable flag set should be used for
IORESOURCE_PCI_FIXEDmappings without any host specific code, but that needs to be implemented in common code.Quoted 6 lines
What connection do you envision between nonrelocatable ranges and
IORESOURCE_PCI_FIXED? I don't know what a nonrelocatable range is, but forIORESOURCE_PCI_FIXED, all I intend is that the PCI core should not try to assign a different address, e.g., because the BAR is read-only or because it's a legacy IDE/VGA/etc. range for which there is no BAR at all.
Quoted 5 lines
I think that is exactly the definition of the nonrelocatable flag in http://www.openfirmware.org/1275/bindings/pci/pci2_1.pdf
The example given in section 11.1.2 is for a VGA device that has some relocatable BARs and some nonrelocatable BARs.
OK. You said "a range that has the nonrelocatable flag set should be used for IORESOURCE_PCI_FIXED mappings." I thought you meant that the range was a bridge window, and somehow PCI_FIXED BARs should be put in that window.
But maybe you meant that nonrelocatable ranges should have the IORESOURCE_PCI_FIXED bit set in their struct resources. That would mean we couldn't move the window, but we could put relocatable BARs inside the window.
What needs to be implemented? Just the code that would set IORESOURCE_PCI_FIXED for nonrelocatable ranges?
Bjorn
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 5 lines · Bjorn Helgaas
OK. You said "a range that has the nonrelocatable flag set should be used for
IORESOURCE_PCI_FIXEDmappings." I thought you meant that the range was a bridge window, and somehowPCI_FIXEDBARs should be put in that window.
Quoted 7 lines
But maybe you meant that nonrelocatable ranges should have the
IORESOURCE_PCI_FIXEDbit set in their struct resources. That would mean we couldn't move the window, but we could put relocatable BARs inside the window.What needs to be implemented? Just the code that would set
IORESOURCE_PCI_FIXEDfor nonrelocatable ranges?
It might be more complex than I thought. Let's see what the original hack does:
+/*
+ * All PCIe devices in Thunder have fixed resources, shouldn't be reassigned.
+ * Also claim the device's valid resources to set 'res->parent' hierarchy.
+ */
+static void pci_dev_resource_fixup(struct pci_dev *dev)
+{
+ struct resource *res;
+ int resno;
+
+ for (resno = 0; resno < PCI_NUM_RESOURCES; resno++)
+ dev->resource[resno].flags |= IORESOURCE_PCI_FIXED;
+
+ for (resno = 0; resno < PCI_BRIDGE_RESOURCES; resno++) {
+ res = &dev->resource[resno];
+ if (res->parent || !(res->flags & IORESOURCE_MEM))
+ continue;
+ pci_claim_resource(dev, resno);
+ }
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID,
+ pci_dev_resource_fixup);
This actually looks harmful, because it means that any kernel that contains the thunder host controller driver will do the above for any device made by Cavium, whether it's connected to this bridge or not.
What I think we want instead is to mark any resource whose parent resource is IORESOURCE_PCI_FIXED to have the same flag, and mark the PCI host controller resources IORESOURCE_PCI_FIXED when the nonrelocatable flag is set, and that should all be done in core code, not in a driver fixup.
The part that still looks weird is the pci_claim_resource() that Sunil mentioned. This is currently done for resources that do not have a parent, but AFAICT all PCI device resources should have a parent that connects it to the upstream bridge.
Arnd
Bjorn Helgaas
maintainerRe: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 39 lines · Arnd Bergmann
Quoted 5 lines · Bjorn Helgaas
OK. You said "a range that has the nonrelocatable flag set should be used for
IORESOURCE_PCI_FIXEDmappings." I thought you meant that the range was a bridge window, and somehowPCI_FIXEDBARs should be put in that window.Quoted 7 lines
But maybe you meant that nonrelocatable ranges should have the
IORESOURCE_PCI_FIXEDbit set in their struct resources. That would mean we couldn't move the window, but we could put relocatable BARs inside the window.What needs to be implemented? Just the code that would set
IORESOURCE_PCI_FIXEDfor nonrelocatable ranges?It might be more complex than I thought. Let's see what the original hack does:
(quoted diff) +21
-0+/* + * All PCIe devices in Thunder have fixed resources, shouldn't be reassigned. + * Also claim the device's valid resources to set 'res->parent' hierarchy. + */ +static void pci_dev_resource_fixup(struct pci_dev *dev) +{ + struct resource *res; + int resno; + + for (resno = 0; resno < PCI_NUM_RESOURCES; resno++) + dev->resource[resno].flags |= IORESOURCE_PCI_FIXED; + + for (resno = 0; resno < PCI_BRIDGE_RESOURCES; resno++) { + res = &dev->resource[resno]; + if (res->parent || !(res->flags & IORESOURCE_MEM)) + continue; + pci_claim_resource(dev, resno); + } +} +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, + pci_dev_resource_fixup);
Quoted 3 lines
This actually looks harmful, because it means that any kernel that contains the thunder host controller driver will do the above for any device made by Cavium, whether it's connected to this bridge or not.
I agree, that looks broken.
Quoted 5 lines
What I think we want instead is to mark any resource whose parent resource is
IORESOURCE_PCI_FIXEDto have the same flag, and mark the PCI host controller resourcesIORESOURCE_PCI_FIXEDwhen the nonrelocatable flag is set, and that should all be done in core code, not in a driver fixup.
The PCI host controller resources are host bridge windows, right? I don't think it matters whether they have IORESOURCE_PCI_FIXED set, because the bridge is not itself a PCI device, and PCI resource assignment treats the host bridge windows as fixed.
That doesn't imply that the PCI device resources *inside* the windows need to be fixed, though. Regular BARs can be moved around inside the window.
Sunil said "All on-board PCI devices connected to this PCI controller have fixed resources..." That sounds like these on-board PCI devices are non-compliant because their BARs don't work per spec. But that doesn't sound right, because we wouldn't be able to size them.
Quoted 4 lines
The part that still looks weird is the
pci_claim_resource()that Sunil mentioned. This is currently done for resources that do not have a parent, but AFAICT all PCI device resources should have a parent that connects it to the upstream bridge.
Ideally the pci_claim_resource() would be in the core, not in arch code, but it's a bit of a hodge-podge right now.
Bjorn
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
In reply to Arnd BergmannQuoted 5 lines · Arnd Bergmann
Quoted 2 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
+ pcie0 at 0x8480,00000000 {
The name should be pci, not pci0.
And the address should be "@848000000000". There was some confusion about when the comma should be used and it is only if you have things like PCI bus, device, function for the addressing.
Quoted 2 lines
Quoted 2 lines
+ compatible = "cavium,thunder-pcie";
A bit too generic as many of the Cavium bindings have been.
Rob
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Thanks for pointing correction ROB.
Will fix it.
Quoted 22 lines · Rob Herring
Quoted 5 lines · Arnd Bergmann
Quoted 2 lines · On Wednesday 24 September 2014 17:37:45 Robert Richter wrote:
+ pcie0 at 0x8480,00000000 {
The name should be pci, not pci0.
And the address should be "@848000000000". There was some confusion about when the comma should be used and it is only if you have things like PCI bus, device, function for the addressing.
Quoted 2 lines
Quoted 2 lines
+ compatible = "cavium,thunder-pcie";
A bit too generic as many of the Cavium bindings have been.
Rob
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
In reply to Arnd BergmannQuoted 12 lines · Arnd Bergmann
arch/arm64/boot/dts/thunder-88xx.dtsi line 422
Quoted 11 lines
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };
Quoted 3 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
Liviu's code assigns a unique id to the domain if missing, see of_pci_get_domain_nr(). So I don't think we need to add a "pci-domain" property here.
Liviu's DT implementation that assigns a unique number differs a bit from ACPI which states: "If _SEG [aka domain number] does not exist, OSPM assumes that all PCI bus segments are in PCI Segment Group 0."
Maybe of_pci_get_domain_nr() should be similar to ACPI. If there are multiple root bridges, the "pci-domain" property could be forced instead.
-Robert
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 13 lines · Robert Richter
Quoted 12 lines · Arnd Bergmann
Quoted 11 lines
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };
Quoted 7 lines
Quoted 3 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
Liviu's code assigns a unique id to the domain if missing, see
of_pci_get_domain_nr(). So I don't think we need to add a "pci-domain" property here.
Not anymore! That function is gone in v12 onwards. What is in -next has a new function called of_get_pci_domain_nr() (slight name change) but that only gets the value set in the "linux,pci-domain" property of the device node. It is the choice of the host bridge driver to use that function or to use pci_get_new_domain_nr() which *does* generate an unique id every time it gets called.
Quoted 8 lines
Liviu's DT implementation that assigns a unique number differs a bit from ACPI which states: "If
_SEG[aka domain number] does not exist, OSPM assumes that all PCI bus segments are in PCI Segment Group 0."Maybe
of_pci_get_domain_nr()should be similar to ACPI. If there are multiple root bridges, the "pci-domain" property could be forced instead.
Indeed. But the enforcing is left as an exercise to the host bridge implementor for the moment.
Best regards,
Liviu
Quoted 3 lines
-Robert
Signature · 8 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
?\_(?)_/?Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 22 lines · Liviu Dudau
Quoted 13 lines · Robert Richter
Quoted 12 lines · Arnd Bergmann
Quoted 11 lines
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };Quoted 7 lines
Quoted 3 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
Liviu's code assigns a unique id to the domain if missing, see
of_pci_get_domain_nr(). So I don't think we need to add a "pci-domain" property here.
Quoted 6 lines
Not anymore! That function is gone in v12 onwards. What is in -next has a new function called
of_get_pci_domain_nr()(slight name change) but that only gets the value set in the "linux,pci-domain" property of the device node. It is the choice of the host bridge driver to use that function or to usepci_get_new_domain_nr()which *does* generate an unique id every time it gets called.
I am quite confused a bit on which is the latest code base now. I was looking into Bjorn's pci/host-generic and there is a different implemetation:
----
/**
* This function will try to obtain the host bridge domain number by
* using of_alias_get_id() call with "pci-domain" as a stem. If that
* fails, a local allocator will be used. The local allocator can
* be requested to return a new domain_nr if the information is missing
* from the device tree.
*
* @node: device tree node with the domain information
* @allocate_if_missing: if DT lacks information about the domain nr,
* allocate a new number.
*
* Returns the associated domain number from DT, or a new domain number
* if DT information is missing and @allocate_if_missing is true. If
* @allocate_if_missing is false then the last allocated domain number
* will be returned.
*/
int of_pci_get_domain_nr(struct device_node *node, bool allocate_if_missing)
{
int domain;
domain = atomic_read(&of_domain_nr);
if (domain == -1) {
/* first run, get max defined domain nr in device tree */
domain = of_get_max_pci_domain_nr();
/* then set the start value for allocator to be max + 1 */
atomic_set(&of_domain_nr, domain + 1);
}
domain = of_alias_get_id(node, "pci-domain");
if (domain == -ENODEV) {
domain = atomic_read(&of_domain_nr);
if (allocate_if_missing)
atomic_inc(&of_domain_nr);
}
return domain;
}EXPORT_SYMBOL_GPL(of_pci_get_domain_nr);
----
This differs also from v13. Please check.
It would be good to have a stable code base to work with, so I would prefer incremental patches on top of Bjorn's pci/host-generic.
Quoted 10 lines
Quoted 7 lines
Liviu's DT implementation that assigns a unique number differs a bit from ACPI which states: "If
_SEG[aka domain number] does not exist, OSPM assumes that all PCI bus segments are in PCI Segment Group 0."Maybe
of_pci_get_domain_nr()should be similar to ACPI. If there are multiple root bridges, the "pci-domain" property could be forced instead.Indeed. But the enforcing is left as an exercise to the host bridge implementor for the moment.
Right, can be added later.
Thanks,
-Robert
Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 71 lines · Robert Richter
Quoted 22 lines · Liviu Dudau
Quoted 13 lines · Robert Richter
Quoted 12 lines · Arnd Bergmann
Quoted 11 lines
arch/arm64/boot/dts/thunder-88xx.dtsi +11
-0+ compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem ranges */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + };Quoted 7 lines
Quoted 3 lines
If you claim the entire 0-255 bus range, I think you should also specify a domain, otherwise it's not predictable which domain you get.
Liviu's code assigns a unique id to the domain if missing, see
of_pci_get_domain_nr(). So I don't think we need to add a "pci-domain" property here.Quoted 6 lines
Not anymore! That function is gone in v12 onwards. What is in -next has a new function called
of_get_pci_domain_nr()(slight name change) but that only gets the value set in the "linux,pci-domain" property of the device node. It is the choice of the host bridge driver to use that function or to usepci_get_new_domain_nr()which *does* generate an unique id every time it gets called.I am quite confused a bit on which is the latest code base now. I was looking into Bjorn's pci/host-generic and there is a different implemetation:
----
/**
* This function will try to obtain the host bridge domain number by
* usingof_alias_get_id()call with "pci-domain" as a stem. If that
* fails, a local allocator will be used. The local allocator can
* be requested to return a newdomain_nrif the information is missing
* from the device tree.
*
* @node: device tree node with the domain information
* @allocate_if_missing: if DT lacks information about the domain nr,
* allocate a new number.
*
* Returns the associated domain number from DT, or a new domain number
* if DT information is missing and @allocate_if_missing is true. If
* @allocate_if_missing is false then the last allocated domain number
* will be returned.
*/
intof_pci_get_domain_nr(struct device_node *node, bool allocate_if_missing){ int domain; domain = atomic_read(&of_domain_nr); if (domain == -1) { /* first run, get max defined domain nr in device tree */ domain = of_get_max_pci_domain_nr(); /* then set the start value for allocator to be max + 1 */ atomic_set(&of_domain_nr, domain + 1); } domain = of_alias_get_id(node, "pci-domain"); if (domain == -ENODEV) { domain = atomic_read(&of_domain_nr); if (allocate_if_missing) atomic_inc(&of_domain_nr); } return domain; }
Quoted 4 lines
EXPORT_SYMBOL_GPL(of_pci_get_domain_nr);
----This differs also from v13. Please check.
I'm not sure what you are looking at. Commit 41e5c0f81d3e does look like my v13. Not sure why you are still seeing this v11 version.
Quoted 3 lines
It would be good to have a stable code base to work with, so I would prefer incremental patches on top of Bjorn's pci/host-generic.
Agreed, but from what I can see from my side pci/host-generic and next have the same versions, so there should not be any confusion.
Best regards,
Liviu
Quoted 22 lines
Quoted 11 lines
Quoted 8 lines
Liviu's DT implementation that assigns a unique number differs a bit from ACPI which states: "If
_SEG[aka domain number] does not exist, OSPM assumes that all PCI bus segments are in PCI Segment Group 0."Maybe
of_pci_get_domain_nr()should be similar to ACPI. If there are multiple root bridges, the "pci-domain" property could be forced instead.Indeed. But the enforcing is left as an exercise to the host bridge implementor for the moment.
Right, can be added later.
Thanks,
-Robert
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 8 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
?\_(?)_/?Re: [PATCH 3/6] pci, thunder: Add PCIe host controller devicetree bindings
Quoted 10 lines · Liviu Dudau
Quoted 6 lines · Robert Richter
On 07.10.14 16:01:49, Liviu Dudau wrote:
I am quite confused a bit on which is the latest code base now. I was looking into Bjorn's pci/host-generic and there is a different implemetation:This differs also from v13. Please check.
I'm not sure what you are looking at. Commit
41e5c0f81d3edoes look like my v13. Not sure why you are still seeing this v11 version.
Right, my bad. Sorry for the noise.
-Robert
PATCH 4/6 pci, thunder: Document PCIe host controller devicetree bindings
In reply to Robert RichterFrom: Sunil Goutham <[email protected]>
This patch adds documentation for the devicetree bindings used by the Thunder PCI host controller.
--- .../devicetree/bindings/pci/cavium,thunder-pci.txt | 32 ++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 Documentation/devicetree/bindings/pci/cavium,thunder-pci.txt
Documentation/devicetree/bindings/pci/cavium,thunder-pci.txt new file +32 -0
@@ -0,0 +1,32 @@ +* Cavium Thunder PCIe interface + +Required properties: +- compatible: should contain "cavium,thunder-pcie" to identify the core. +- device_type: set to "pci" +- #address-cells: set to <3> +- #size-cells: set to <2> +- #stream-id-cells: set to <1> +- bus-range: PCI bus numbers covered +- reg: base address and length of the pcie configuration space. +- ranges: ranges for the PCI memory regions. +- msi-parent: Link to the hardware entity that serves as the Message + Signaled Interrupt controller for this PCI controller. + +Example: + +SoC specific DT Entry: + + pcie0: pcie0 at 0x848000000000 { + compatible = "cavium,thunder-pcie"; + device_type = "pci"; + msi-parent = <&its>; + bus-range = <0 255>; + #size-cells = <2>; + #address-cells = <3>; + #stream-id-cells = <1>; + reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */ + ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>, /* mem */ + <0x03000000 0x8300 0x00000000 0x8300 0x00000000 0x80 0x00000000>, + <0x03000000 0x87e0 0x00000000 0x87e0 0x00000000 0x01 0x00000000>; + }; +
Signature · 1 line
2.1.0
PATCH 5/6 arm64, defconfig: Enable PCI
In reply to Robert RichterFrom: Robert Richter <[email protected]>
Enable the PCI subsystem in defconfig. There will be more and more systems with pci support, so default enable it. This also implicitly enables the MSI feature.
--- arch/arm64/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/Kconfig +4 -0
@@ -163,11 +163,15 @@ config ARM_AMBA config PCI bool "PCI support" + default y help This feature enables support for PCIe bus system. If you say Y here, the kernel will include drivers and infrastructure code to support PCIe bus devices. +config PCI_MSI + def_bool PCI + config PCI_DOMAINS def_bool PCI
Signature · 1 line
2.1.0
Re: [PATCH 5/6] arm64, defconfig: Enable PCI
arch/arm64/Kconfig config ARM_AMBA · line 176
Quoted 13 lines · On Wednesday 24 September 2014 17:37:47 Robert Richter wrote:
config PCI
bool "PCI support"arch/arm64/Kconfig +4
-0+ default y help This feature enables support for PCIe bus system. If you say Y here, the kernel will include drivers and infrastructure code to support PCIe bus devices. +config PCI_MSI + def_bool PCI + config PCI_DOMAINS def_bool PCI
There is already a PCI_MSI symbol in drivers/pci/Kconfig. Just select that from the PCI symbol above rather than defining a second one.
Arnd
Re: [PATCH 5/6] arm64, defconfig: Enable PCI
Quoted 9 lines · Arnd Bergmann
Quoted 8 lines · On Wednesday 24 September 2014 17:37:47 Robert Richter wrote:
config PCI
bool "PCI support"
+ default y
help
This feature enables support for PCIe bus system. If you say Y here, the kernel will include drivers and infrastructure code to support PCIe bus devices.
Quoted 8 lines
arch/arm64/Kconfig config ARM_AMBA · line 176
Quoted 5 lines
arch/arm64/Kconfig +3
-0+config PCI_MSI + def_bool PCI + config PCI_DOMAINS def_bool PCIThere is already a
PCI_MSIsymbol indrivers/pci/Kconfig. Just select that from the PCI symbol above rather than defining a second one.
The intention is not to have a second definition, instead this should enable the default value just for arm64. Thus I put it to arch/arm64/Kconfig. Otherwise it would be enabled per default on all archs.
We could have used select in config ARM64, but I tried to avoid using select due to the dependency issue and instead implement this with default-y/depends-on. Doing so it can be manually disabled too.
-Robert
Catalin Marinas
maintainerRe: [PATCH 5/6] arm64, defconfig: Enable PCI
Quoted 19 lines · Robert Richter
Quoted 9 lines · Arnd Bergmann
Quoted 8 lines · On Wednesday 24 September 2014 17:37:47 Robert Richter wrote:
config PCI
bool "PCI support"
+ default y
help
This feature enables support for PCIe bus system. If you say Y here, the kernel will include drivers and infrastructure code to support PCIe bus devices.Quoted 8 lines
arch/arm64/Kconfig config ARM_AMBA · line 176
Quoted 5 lines
arch/arm64/Kconfig +3
-0+config PCI_MSI + def_bool PCI + config PCI_DOMAINS def_bool PCIThere is already a
PCI_MSIsymbol indrivers/pci/Kconfig. Just select that from the PCI symbol above rather than defining a second one.
Quoted 8 lines
The intention is not to have a second definition, instead this should enable the default value just for arm64. Thus I put it to
arch/arm64/Kconfig. Otherwise it would be enabled per default on all archs.We could have used select in config ARM64, but I tried to avoid using select due to the dependency issue and instead implement this with default-y/depends-on. Doing so it can be manually disabled too.
It may be better to simply set PCI and PCI_MSI as defaults in defconfig rather than "default y" and an additional config PCI_MSI entry.
Signature · 1 line
Catalin
Re: [PATCH 5/6] arm64, defconfig: Enable PCI
Quoted 20 lines · On Wednesday 24 September 2014 18:10:48 Catalin Marinas wrote:
Quoted 19 lines · Robert Richter
Quoted 17 lines · Arnd Bergmann
arch/arm64/Kconfig config ARM_AMBA · line 176
Quoted 13 lines · On Wednesday 24 September 2014 17:37:47 Robert Richter wrote:
config PCI
bool "PCI support"arch/arm64/Kconfig +4
-0+ default y help This feature enables support for PCIe bus system. If you say Y here, the kernel will include drivers and infrastructure code to support PCIe bus devices. +config PCI_MSI + def_bool PCI + config PCI_DOMAINS def_bool PCIThere is already a
PCI_MSIsymbol indrivers/pci/Kconfig. Just select that from the PCI symbol above rather than defining a second one.
Quoted 4 lines
Quoted 4 lines
The intention is not to have a second definition, instead this should enable the default value just for arm64. Thus I put it to
arch/arm64/Kconfig. Otherwise it would be enabled per default on all archs.
I don't think other architectures actually see the PCI symbol that is defined in the arm64 Kconfig file, but I might be wrong.
Quoted 3 lines
Quoted 3 lines
We could have used select in config ARM64, but I tried to avoid using select due to the dependency issue and instead implement this with default-y/depends-on. Doing so it can be manually disabled too.
How about adding a new symbol like
config ARM64_PCI
def_bool PCI
select PCI_MSI
Quoted 2 lines
It may be better to simply set PCI and
PCI_MSIas defaults in defconfig rather than "default y" and an additional configPCI_MSIentry.
Selecting PCI_MSI unconditionally seems fine to me. We won't have any ARM64 systems that only do classic PCI, and MSI is mandatory for PCIe.
Arnd
Catalin Marinas
maintainerRe: [PATCH 5/6] arm64, defconfig: Enable PCI
Quoted 21 lines · Arnd Bergmann
Quoted 20 lines · On Wednesday 24 September 2014 18:10:48 Catalin Marinas wrote:
Quoted 19 lines · Robert Richter
Quoted 17 lines · Arnd Bergmann
arch/arm64/Kconfig config ARM_AMBA · line 176
Quoted 13 lines · On Wednesday 24 September 2014 17:37:47 Robert Richter wrote:
config PCI
bool "PCI support"arch/arm64/Kconfig +4
-0+ default y help This feature enables support for PCIe bus system. If you say Y here, the kernel will include drivers and infrastructure code to support PCIe bus devices. +config PCI_MSI + def_bool PCI + config PCI_DOMAINS def_bool PCIThere is already a
PCI_MSIsymbol indrivers/pci/Kconfig. Just select that from the PCI symbol above rather than defining a second one.
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
The intention is not to have a second definition, instead this should enable the default value just for arm64. Thus I put it to
arch/arm64/Kconfig. Otherwise it would be enabled per default on all archs.I don't think other architectures actually see the PCI symbol that is defined in the arm64
Kconfigfile, but I might be wrong.
Other archs wouldn't see the symbols in arch/arm64/Kconfig.
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
We could have used select in config ARM64, but I tried to avoid using select due to the dependency issue and instead implement this with default-y/depends-on. Doing so it can be manually disabled too.
How about adding a new symbol like
config
ARM64_PCI
def_boolPCI
selectPCI_MSI
How is this different from just selecting PCI_MSI in config PCI in arch/arm64/Kconfig? I don't see what another symbol brings.
Quoted 5 lines
Quoted 2 lines
It may be better to simply set PCI and
PCI_MSIas defaults in defconfig rather than "default y" and an additional configPCI_MSIentry.Selecting
PCI_MSIunconditionally seems fine to me. We won't have any ARM64 systems that only do classic PCI, and MSI is mandatory for PCIe.
OK then, we can go with a select.
Signature · 1 line
Catalin
Re: [PATCH 5/6] arm64, defconfig: Enable PCI
Quoted 11 lines · On Thursday 25 September 2014 10:35:45 Catalin Marinas wrote:
Quoted 8 lines · Arnd Bergmann
Quoted 4 lines · On Wednesday 24 September 2014 18:10:48 Catalin Marinas wrote:
Quoted 4 lines
The intention is not to have a second definition, instead this should enable the default value just for arm64. Thus I put it to
arch/arm64/Kconfig. Otherwise it would be enabled per default on all archs.I don't think other architectures actually see the PCI symbol that is defined in the arm64
Kconfigfile, but I might be wrong.Other archs wouldn't see the symbols in
arch/arm64/Kconfig.
Ok.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
We could have used select in config ARM64, but I tried to avoid using select due to the dependency issue and instead implement this with default-y/depends-on. Doing so it can be manually disabled too.
Quoted 8 lines
Quoted 5 lines
How about adding a new symbol like
config
ARM64_PCI
def_boolPCI
selectPCI_MSIHow is this different from just selecting
PCI_MSIin config PCI inarch/arm64/Kconfig? I don't see what another symbol brings.
That suggestion was just in case I was wrong above. If the symbol is indeed hidden elsewhere, we don't need it.
Arnd
PATCH 6/6 pci, thunder: Enable Cavium Thunder PCIe host controller
In reply to Robert RichterFrom: Robert Richter <[email protected]>
Enable Cavium Thunder PCIe host controller driver.
IOV is enabled per default if drivers with IOV support are enabled. IOV can be disabled.
--- arch/arm64/Kconfig | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/arm64/Kconfig +11 -0
@@ -172,6 +172,14 @@ config PCI config PCI_MSI def_bool PCI +config NO_PCI_IOV + def_bool PCI + depends on !PCI_THUNDER + +config PCI_IOV + def_bool PCI + depends on !NO_PCI_IOV + config PCI_DOMAINS def_bool PCI
@@ -185,6 +193,9 @@ source "drivers/pci/Kconfig" source "drivers/pci/pcie/Kconfig" source "drivers/pci/hotplug/Kconfig" +config PCI_THUNDER + def_bool PCI + endmenu menu "Kernel Features"
Signature · 1 line
2.1.0
Catalin Marinas
maintainerRe: [PATCH 6/6] pci, thunder: Enable Cavium Thunder PCIe host controller
Quoted 17 lines · Robert Richter
From: Robert Richter <[email protected]>
Enable Cavium Thunder PCIe host controller driver.
IOV is enabled per default if drivers with IOV support are enabled. IOV can be disabled.
Signed-off-by Robert Richter <[email protected]>--- arch/arm64/Kconfig | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/arm64/Kconfig +0
-0
arch/arm64/Kconfig config PCI · line 181
Quoted 10 lines
arch/arm64/Kconfig +7
-0@@ -172,6 +172,14 @@ config PCI config PCI_MSI def_bool PCI +config NO_PCI_IOV + def_bool PCI + depends on !PCI_THUNDER + +config PCI_IOV + def_bool PCI + depends on !NO_PCI_IOV
Does this work with single image where drivers may have different PCI_IOV needs?
Signature · 1 line
Catalin