PCI: dwc: Add support to handle prefetchable memory separately
cover PCI: dwc: Add support to handle prefetchable memory separately
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understood that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU mapping. This is certainly true for Tegra194 and what I heard from our HW engineers is that it should generally be true for any DWC based implementation also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm the same so that this particular patch won't cause any regressions for other DWC based platforms.
Vidya Sagar (2):
PCI: dwc: Add support to handle prefetchable memory separately
PCI: dwc: Use ATU region to map prefetchable memory region
.../pci/controller/dwc/pcie-designware-host.c | 46 ++++++++++++++----- drivers/pci/controller/dwc/pcie-designware.c | 6 ++- drivers/pci/controller/dwc/pcie-designware.h | 8 +++- 3 files changed, 45 insertions(+), 15 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/2 PCI: dwc: Add support to handle prefetchable memory separately
Add required structure members to struct pcie_port to handle prefetchable memory aperture separately from non-prefetchable memory aperture so that any dependency on the order of their appearance in the 'ranges' property of the respective PCIe device tree node can be removed.
--- .../pci/controller/dwc/pcie-designware-host.c | 26 ++++++++++++------- drivers/pci/controller/dwc/pcie-designware.h | 4 +++ 2 files changed, 21 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-host.c +17 -9
@@ -363,13 +363,23 @@ int dw_pcie_host_init(struct pcie_port *pp) pp->io_base =pci_pio_to_address (pp->io->start);break ;case IORESOURCE_MEM : - pp->mem = win->res; - pp->mem->name ="MEM" ; - mem_size =resource_size (pp->mem); -if (upper_32_bits (mem_size)) -dev_warn (dev,"MEM resource size exceeds max for 32 bits\n" ); - pp->mem_size = mem_size; - pp->mem_bus_addr = pp->mem->start - win->offset; +if (win->res->flags &IORESOURCE_PREFETCH ) { + pp->prefetch = win->res; + pp->prefetch->name ="PREFETCH" ; + pp->prefetch_base = pp->prefetch->start; + pp->prefetch_size =resource_size (pp->prefetch); + pp->perfetch_bus_addr = pp->prefetch->start - + win->offset; + }else { + pp->mem = win->res; + pp->mem->name ="MEM" ; + pp->mem_base = pp->mem->start; + mem_size =resource_size (pp->mem); +if (upper_32_bits (mem_size)) +dev_warn (dev,"MEM resource size exceeds max for 32 bits\n" ); + pp->mem_size = mem_size; + pp->mem_bus_addr = pp->mem->start - win->offset; + }break ;case 0 : pp->cfg = win->res;
@@ -394,8 +404,6 @@ int dw_pcie_host_init(struct pcie_port *pp) } } - pp->mem_base = pp->mem->start; -if (!pp->va_cfg0_base) { pp->va_cfg0_base =devm_pci_remap_cfgspace (dev, pp->cfg0_base, pp->cfg0_size);
drivers/pci/controller/dwc/pcie-designware.h +4 -0
@@ -186,9 +186,13 @@ struct pcie_port {u64 mem_base;phys_addr_t mem_bus_addr;u32 mem_size; +u64 prefetch_base; +phys_addr_t perfetch_bus_addr; +u64 prefetch_size;struct resource *cfg;struct resource *io;struct resource *mem; +struct resource *prefetch;struct resource *busn;int irq;const struct dw_pcie_host_ops *ops;
Signature · 1 line
2.17.1
Rob Herring
reviewerQuoted 60 lines · Vidya Sagar
Add required structure members to struct
pcie_portto handle prefetchable memory aperture separately from non-prefetchable memory aperture so that any dependency on the order of their appearance in the 'ranges' property of the respective PCIe device tree node can be removed.Signed-off-by Vidya Sagar <[email protected]>--- .../pci/controller/dwc/pcie-designware-host.c | 26 ++++++++++++------- drivers/pci/controller/dwc/pcie-designware.h | 4 +++ 2 files changed, 21 insertions(+), 9 deletions(-)drivers/pci/controller/dwc/pcie-designware-host.c +17
-9@@ -363,13 +363,23 @@ int dw_pcie_host_init(struct pcie_port *pp) pp->io_base =pci_pio_to_address (pp->io->start);break ;case IORESOURCE_MEM : - pp->mem = win->res; - pp->mem->name ="MEM" ; - mem_size =resource_size (pp->mem); -if (upper_32_bits (mem_size)) -dev_warn (dev,"MEM resource size exceeds max for 32 bits\n" ); - pp->mem_size = mem_size; - pp->mem_bus_addr = pp->mem->start - win->offset; +if (win->res->flags &IORESOURCE_PREFETCH ) { + pp->prefetch = win->res; + pp->prefetch->name ="PREFETCH" ; + pp->prefetch_base = pp->prefetch->start; + pp->prefetch_size =resource_size (pp->prefetch); + pp->perfetch_bus_addr = pp->prefetch->start - + win->offset; + }else { + pp->mem = win->res; + pp->mem->name ="MEM" ; + pp->mem_base = pp->mem->start; + mem_size =resource_size (pp->mem); +if (upper_32_bits (mem_size)) +dev_warn (dev,"MEM resource size exceeds max for 32 bits\n" ); + pp->mem_size = mem_size; + pp->mem_bus_addr = pp->mem->start - win->offset; + }break ;case 0 : pp->cfg = win->res;@@ -394,8 +404,6 @@ int dw_pcie_host_init(struct pcie_port *pp) } } - pp->mem_base = pp->mem->start; -if (!pp->va_cfg0_base) { pp->va_cfg0_base =devm_pci_remap_cfgspace (dev, pp->cfg0_base, pp->cfg0_size);drivers/pci/controller/dwc/pcie-designware.h +0
-0
drivers/pci/controller/dwc/pcie-designware.h struct pcie_port { · line 191
Quoted 6 lines
drivers/pci/controller/dwc/pcie-designware.h +3
-0@@ -186,9 +186,13 @@ struct pcie_port {u64 mem_base;phys_addr_t mem_bus_addr;u32 mem_size; +u64 prefetch_base; +phys_addr_t perfetch_bus_addr; +u64 prefetch_size;
There's no reason to store these for all eternity as they are used in one place and already stored as resources in bridge->windows.
I have a patch series removing most of this that I will post in a few days. There's a WIP branch, pci-dw-config-access, in my kernel.org tree. Mostly you just need the bridge ptr which is isn't currently saved in pcie_port.
Rob
PATCH 2/2 PCI: dwc: Use ATU region to map prefetchable memory region
In reply to Vidya SagarUse ATU region-3 to setup mapping for prefetchable memory region. It also modifies the code to consume an ATU region for mapping non-prefetchable or prefetchable memory regions only if the CPU address and PCIe bus addresses are not equal as there is no need to use ATU mapping if there is a 1:1 mapping between CPU address and PCIe bus address.
--- .../pci/controller/dwc/pcie-designware-host.c | 20 ++++++++++++++++--- drivers/pci/controller/dwc/pcie-designware.c | 6 ++++-- drivers/pci/controller/dwc/pcie-designware.h | 4 +++- 3 files changed, 24 insertions(+), 6 deletions(-)
drivers/pci/controller/dwc/pcie-designware-host.c +17 -3
@@ -701,13 +701,27 @@ void dw_pcie_setup_rc(struct pcie_port *pp)* we should not program the ATU here. */ if (!pp->ops->rd_other_conf) { -dw_pcie_prog_outbound_atu (pci,PCIE_ATU_REGION_INDEX0 , -PCIE_ATU_TYPE_MEM , pp->mem_base, - pp->mem_bus_addr, pp->mem_size); +if (pp->mem_base != pp->mem_bus_addr) { +dw_pcie_prog_outbound_atu (pci,PCIE_ATU_REGION_INDEX0 , +PCIE_ATU_TYPE_MEM , + pp->mem_base, + pp->mem_bus_addr, + pp->mem_size); + }if (pci->num_viewport >2 )dw_pcie_prog_outbound_atu (pci,PCIE_ATU_REGION_INDEX2 ,PCIE_ATU_TYPE_IO , pp->io_base, pp->io_bus_addr, pp->io_size); +if (pp->prefetch_base != pp->perfetch_bus_addr && + pci->num_viewport >=4 ) { +dw_pcie_prog_outbound_atu (pci,PCIE_ATU_REGION_INDEX3 , +PCIE_ATU_TYPE_MEM , + pp->prefetch_base, + pp->perfetch_bus_addr, + pp->prefetch_size); + }else { +dev_warn (pci->dev,"Insufficient ATU regions to map Prefetchable memory\n" ); + } }dw_pcie_wr_own_conf (pp,PCI_BASE_ADDRESS_0 ,4 ,0 );
drivers/pci/controller/dwc/pcie-designware.c +4 -2
@@ -241,7 +241,7 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,static void dw_pcie_prog_outbound_atu_unroll (struct dw_pcie *pci,int index,int type,u64 cpu_addr, -u64 pci_addr,u32 size) +u64 pci_addr,u64 size) {u32 retries, val;u64 limit_addr = cpu_addr + size -1 ;
@@ -259,6 +259,8 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index,dw_pcie_writel_ob_unroll (pci, index,PCIE_ATU_UNR_UPPER_TARGET ,upper_32_bits (pci_addr));dw_pcie_writel_ob_unroll (pci, index,PCIE_ATU_UNR_REGION_CTRL1 , +upper_32_bits (size -1 ) ? + type |PCIE_ATU_INCREASE_REGION_SIZE : type);dw_pcie_writel_ob_unroll (pci, index,PCIE_ATU_UNR_REGION_CTRL2 ,PCIE_ATU_ENABLE );
@@ -279,7 +281,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index, }void dw_pcie_prog_outbound_atu (struct dw_pcie *pci,int index,int type, -u64 cpu_addr,u64 pci_addr,u32 size) +u64 cpu_addr,u64 pci_addr,u64 size) {u32 retries, val;
drivers/pci/controller/dwc/pcie-designware.h +3 -1
@@ -72,10 +72,12 @@#define PCIE_ATU_VIEWPORT 0x900#define PCIE_ATU_REGION_INBOUND BIT(31)#define PCIE_ATU_REGION_OUTBOUND 0 +#define PCIE_ATU_REGION_INDEX3 0x3#define PCIE_ATU_REGION_INDEX2 0x2#define PCIE_ATU_REGION_INDEX1 0x1#define PCIE_ATU_REGION_INDEX0 0x0#define PCIE_ATU_CR1 0x904 +#define PCIE_ATU_INCREASE_REGION_SIZE BIT(13)#define PCIE_ATU_TYPE_MEM 0x0#define PCIE_ATU_TYPE_IO 0x2#define PCIE_ATU_TYPE_CFG0 0x4
@@ -294,7 +296,7 @@ void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);int dw_pcie_wait_for_link (struct dw_pcie *pci);void dw_pcie_prog_outbound_atu (struct dw_pcie *pci,int index,int type,u64 cpu_addr,u64 pci_addr, -u32 size); +u64 size);int dw_pcie_prog_inbound_atu (struct dw_pcie *pci,int index,int bar,u64 cpu_addr,enum dw_pcie_as_type as_type); void dw_pcie_disable_atu(struct dw_pcie *pci, int index,
Signature · 1 line
2.17.1
Quoted 11 lines · Vidya Sagar
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understood
Quoted 8 lines
that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU mapping. This is certainly true for Tegra194 and what I heard from our HW engineers is that it should generally be true for any DWC based implementation also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm the same so that this particular patch won't cause any regressions for other DWC based platforms.
Hi Vidya,
Unfortunately due to the COVID-19 lockdown, I can't access my development prototype setup to test your patch.
It might take some while until I get the possibility to get access to it again.
-Gustavo
Quoted 12 lines
Vidya Sagar (2):
PCI: dwc: Add support to handle prefetchable memory separately
PCI: dwc: Use ATU region to map prefetchable memory region.../pci/controller/dwc/pcie-designware-host.c | 46 ++++++++++++++----- drivers/pci/controller/dwc/pcie-designware.c | 6 ++- drivers/pci/controller/dwc/pcie-designware.h | 8 +++- 3 files changed, 45 insertions(+), 15 deletions(-)Signature · 1 line
2.17.1
Quoted 25 lines · Gustavo Pimentel
External email: Use caution opening links or attachments
Quoted 11 lines · Vidya Sagar
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understoodQuoted 8 lines
that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU mapping. This is certainly true for Tegra194 and what I heard from our HW engineers is that it should generally be true for any DWC based implementation also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm the same so that this particular patch won't cause any regressions for other DWC based platforms.
Quoted 6 lines
Hi Vidya,
Unfortunately due to the COVID-19 lockdown, I can't access my development prototype setup to test your patch.
It might take some while until I get the possibility to get access to it again.
Hi Gustavo,
Did you find time to check this?
Adding Kishon and Alan as well to take a look at this and verify on their platforms if possible.
Thanks,
Vidya Sagar
Quoted 17 lines
-Gustavo
Quoted 12 lines
Vidya Sagar (2):
PCI: dwc: Add support to handle prefetchable memory separately
PCI: dwc: Use ATU region to map prefetchable memory region.../pci/controller/dwc/pcie-designware-host.c | 46 ++++++++++++++----- drivers/pci/controller/dwc/pcie-designware.c | 6 ++- drivers/pci/controller/dwc/pcie-designware.h | 8 +++- 3 files changed, 45 insertions(+), 15 deletions(-)--
2.17.1
Quoted 30 lines · Vidya Sagar
Quoted 27 lines · Gustavo Pimentel
External email: Use caution opening links or attachments
Quoted 11 lines · Vidya Sagar
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understoodQuoted 8 lines
that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU mapping. This is certainly true for Tegra194 and what I heard from our HW engineers is that it should generally be true for any DWC based implementation also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm the same so that this particular patch won't cause any regressions for other DWC based platforms.Hi Vidya,
Quoted 8 lines
Quoted 4 lines
Unfortunately due to the COVID-19 lockdown, I can't access my development prototype setup to test your patch.
It might take some while until I get the possibility to get access to it again.Hi Gustavo,
Did you find time to check this?
Adding Kishon and Alan as well to take a look at this and verify on their platforms if possible.
My site is still in lockdown, there is no date to return to the office. Sorry.
-Gustavo
Quoted 21 lines
Thanks,
Vidya SagarQuoted 17 lines
-Gustavo
Quoted 12 lines
Vidya Sagar (2):
PCI: dwc: Add support to handle prefetchable memory separately
PCI: dwc: Use ATU region to map prefetchable memory region.../pci/controller/dwc/pcie-designware-host.c | 46 ++++++++++++++----- drivers/pci/controller/dwc/pcie-designware.c | 6 ++- drivers/pci/controller/dwc/pcie-designware.h | 8 +++- 3 files changed, 45 insertions(+), 15 deletions(-)--
2.17.1
Quoted 30 lines · Vidya Sagar
Quoted 27 lines · Gustavo Pimentel
External email: Use caution opening links or attachments
Quoted 11 lines · Vidya Sagar
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understoodQuoted 8 lines
that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU mapping. This is certainly true for Tegra194 and what I heard from our HW engineers is that it should generally be true for any DWC based implementation also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm the same so that this particular patch won't cause any regressions for other DWC based platforms.Hi Vidya,
Quoted 7 lines
Quoted 4 lines
Unfortunately due to the COVID-19 lockdown, I can't access my development prototype setup to test your patch.
It might take some while until I get the possibility to get access to it again.Hi Gustavo,
Did you find time to check this?
Adding Kishon and Alan as well to take a look at this and verify on their platforms if possible.
Hi Kishon and Alan, did you find time to verify this on your respective platforms?
Thanks,
Vidya Sagar
Quoted 21 lines
Thanks,
Vidya SagarQuoted 17 lines
-Gustavo
Quoted 12 lines
Vidya Sagar (2):
PCI: dwc: Add support to handle prefetchable memory separately
PCI: dwc: Use ATU region to map prefetchable memory region
.../pci/controller/dwc/pcie-designware-host.c| 46 ++++++++++++++----- drivers/pci/controller/dwc/pcie-designware.c | 6 ++- drivers/pci/controller/dwc/pcie-designware.h | 8 +++-3 files changed, 45 insertions(+), 15 deletions(-)Signature · 1 line
2.17.1
Quoted 46 lines · Vidya Sagar
Quoted 43 lines · Vidya Sagar
Quoted 40 lines · Gustavo Pimentel
External email: Use caution opening links or attachments
Quoted 20 lines · Vidya Sagar
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node
separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU
addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by
checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understood
that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU
mapping. This is certainly true for Tegra194 and what I heard from our HWQuoted 8 lines
engineers is that it should generally be true for any DWC based implementation
also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm
the same so that this particular patch won't cause any regressions for other
DWC based platforms.Hi Vidya,
Unfortunately due to the COVID-19 lockdown, I can't access my development prototype setup to test your patch.
It might take some while until I get the possibility to get access to it again.
Quoted 6 lines
Quoted 4 lines
Hi Gustavo,
Did you find time to check this?
Adding Kishon and Alan as well to take a look at this and verify on their platforms if possible.Hi Kishon and Alan, did you find time to verify this on your respective platforms?
Yes please. I would like to merge this code, in preparation for that to happen mind rebasing the series against my pci/dwc branch with Rob's suggested changes implemented ?
Thanks a lot,
Lorenzo
Quoted 54 lines · Lorenzo Pieralisi
External email: Use caution opening links or attachments
Quoted 50 lines · Vidya Sagar
Quoted 47 lines · Vidya Sagar
Quoted 34 lines · Gustavo Pimentel
External email: Use caution opening links or attachments
Quoted 20 lines · Vidya Sagar
In this patch series,
Patch-1
adds required infrastructure to deal with prefetchable memory region information coming from 'ranges' property of the respective device-tree node
separately from non-prefetchable memory region information. Patch-2
Adds support to use ATU region-3 for establishing the mapping between CPU
addresses and PCIe bus addresses.
It also changes the logic to determine whether mapping is required or not by
checking both CPU address and PCIe bus address for both prefetchable and non-prefetchable regions. If the addresses are same, then, it is understood
that 1:1 mapping is in place and there is no need to setup ATU mapping whereas if the addresses are not the same, then, there is a need to setup ATU
mapping. This is certainly true for Tegra194 and what I heard from our HWQuoted 8 lines
engineers is that it should generally be true for any DWC based implementation
also.
Hence, I request Synopsys folks (Jingoo Han & Gustavo Pimentel ??) to confirm
the same so that this particular patch won't cause any regressions for other
DWC based platforms.Quoted 6 lines
Hi Vidya,
Unfortunately due to the COVID-19 lockdown, I can't access my development prototype setup to test your patch.
It might take some while until I get the possibility to get access to it again.Hi Gustavo,
Did you find time to check this?
Adding Kishon and Alan as well to take a look at this and verify on their platforms if possible.
Quoted 6 lines
Quoted 2 lines
Hi Kishon and Alan, did you find time to verify this on your respective platforms?
Yes please. I would like to merge this code, in preparation for that to happen mind rebasing the series against my pci/dwc branch with Rob's suggested changes implemented ?
Hi,
Apologies for the delay in reply. I was on leave and couldn't really look into it.
I pushed a new patch on top of your pci/dwc branch at http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/
@Rob and @Lorenzo, please review it.
Since I changed the subject, I pushed it as a new patch and not as V2 of the previous patch set. I hope this is fine.
Thanks,
Vidya Sagar
Quoted 4 lines
Thanks a lot,
Lorenzo