: IO translation with designware PCIe controller
Hi Arnd / Jingoo,
Sorry for not able to follow earlier discussion you had on this topic while initial reviews of patches from Jingoo.
I have few doubts that how will it actually work.
As per my understanding under current implementation:
1. Physical IO addresses are in the range of 0x1000 to 0xfffff.
2. They are mapped to fixed virtual address 0xFEE00000 using pci_ioremap_io.
3. IO resource addresses to any PCIe device will be allocated in above range.
4. Driver for that PCIe device will write in the above range only for IO transaction to the device.
5. As per current IO translation programming, its one to one translation. It means PCIe translation unit will have both input and output address in the range of 0x1000 to 0xfffff.
Did I miss something, or if above statements are correct, then what I do not understand is how can designware PCIe translation unit accept input address in the range of 0x1000 to 0xfffff?
For example in SPEAr1340, physically RAM is mapped on above addresses. PCIe address translation unit can accept address only in the range of core addresses which are assigned to PCIe RC ie 0x80000000-0x8FFFFFFF.
Regards
Pratyush
Quoted 20 lines · Pratyush Anand
Hi Arnd / Jingoo,
Sorry for not able to follow earlier discussion you had on this topic while initial reviews of patches from Jingoo.
I have few doubts that how will it actually work.
As per my understanding under current implementation:
1. Physical IO addresses are in the range of
0x1000to0xfffff.
2. They are mapped to fixed virtual address0xFEE00000usingpci_ioremap_io.
3. IO resource addresses to any PCIe device will be allocated in above range.
4. Driver for that PCIe device will write in the above range only for IO transaction to the device.
5. As per current IO translation programming, its one to one translation. It means PCIe translation unit will have both input and output address in the range of0x1000to0xfffff.
Quoted 3 lines
Did I miss something, or if above statements are correct, then what I do not understand is how can designware PCIe translation unit accept input address in the range of
0x1000to0xfffff?
Hi Pratyush,
I think the part that you are missing is that there are four, not two address spaces involved:
1. The "bus" I/O port address, in the range between 0x1000 and 0xffff (65535). This is the address used in data frames in the actual bus transaction going over the PCIe wires. If you have more than one PCIe host bridge, each of them can normally have a range of up to 65536 addresses. The bus addresses get written into BAR registers of the device.
2. The Linux I/O port numbers, between 0x00000 and 0xfffff (1048575). This is a logical number space that aggregates all bus I/O port numbers. The first 4096 ports are normally reserved for ISA devices and are not available for PCIe resources. Each PCI or PCIe host bridge can have a 0x10000 byte naturally aligned range in here. The common case is that you have only one host bridge using the range 0x1000-0xffff. If you have more than one, there may be an offset between them, e.g. the second PCIe host may have io_offset set to 0x10000, which means that its bus range 0x1000-0xffff gets translated to Linux port number 0x11000-0x1ffff. Linux port numbers are visible e.g. in /proc/ioport and /dev/port
3. The CPU physical address space: Each PCIe host normally has its bus I/O port space mapped into PCU visible addresses, either hardwired or (in case of dw_pcie) set up using an outbound translation window. These addresses are what get passed to pci_ioremap_io.
4. The CPU virtual address space: This is an implementation detail of the kernel, which results in the Linux I/O port numbers 0x0-0xfffff to be mapped at 0xfee00000-0xfeefffff.
Quoted 3 lines
For example in SPEAr1340, physically RAM is mapped on above addresses. PCIe address translation unit can accept address only in the range of core addresses which are assigned to PCIe RC ie 0x80000000-0x8FFFFFFF.
Since this is a physical address, it corresponds to address space 3 in my list above, and the address you pick here is what you pass to pci_ioremap_io.
Arnd
Hi Arnd,
Thanks a lot for detailed explanation.
Quoted 30 lines · Arnd Bergmann
Quoted 20 lines · Pratyush Anand
Hi Arnd / Jingoo,
Sorry for not able to follow earlier discussion you had on this topic while initial reviews of patches from Jingoo.
I have few doubts that how will it actually work.
As per my understanding under current implementation:
1. Physical IO addresses are in the range of
0x1000to0xfffff.
2. They are mapped to fixed virtual address0xFEE00000usingpci_ioremap_io.
3. IO resource addresses to any PCIe device will be allocated in above range.
4. Driver for that PCIe device will write in the above range only for IO transaction to the device.
5. As per current IO translation programming, its one to one translation. It means PCIe translation unit will have both input and output address in the range of0x1000to0xfffff.Quoted 3 lines
Did I miss something, or if above statements are correct, then what I do not understand is how can designware PCIe translation unit accept input address in the range of
0x1000to0xfffff?Hi Pratyush,
I think the part that you are missing is that there are four, not two address spaces involved:
Quoted 6 lines
1. The "bus" I/O port address, in the range between
0x1000and0xffff(65535). This is the address used in data frames in the actual bus transaction going over the PCIe wires. If you have more than one PCIe host bridge, each of them can normally have a range of up to 65536 addresses. The bus addresses get written into BAR registers of the device.
Correct. Got this point. So this address should be output of designware outbound address translation unit.
Quoted 11 lines
2. The Linux I/O port numbers, between
0x00000and0xfffff(1048575). This is a logical number space that aggregates all bus I/O port numbers. The first 4096 ports are normally reserved for ISA devices and are not available for PCIe resources. Each PCI or PCIe host bridge can have a0x10000byte naturally aligned range in here. The common case is that you have only one host bridge using the range 0x1000-0xffff. If you have more than one, there may be an offset between them, e.g. the second PCIe host may haveio_offsetset to0x10000, which means that its bus range 0x1000-0xffff gets translated to Linux port number 0x11000-0x1ffff. Linux port numbers are visible e.g. in/proc/ioportand/dev/port
Ok, got it. To know more, which binding function maps this logical number space to address space in 4?
Quoted 5 lines
3. The CPU physical address space: Each PCIe host normally has its bus I/O port space mapped into PCU visible addresses, either hardwired or (in case of
dw_pcie) set up using an outbound translation window. These addresses are what get passed topci_ioremap_io.
OK.
Quoted 5 lines
4. The CPU virtual address space: This is an implementation detail of the kernel, which results in the Linux I/O port numbers 0x0-0xfffff to be mapped at 0xfee00000-0xfeefffff.
Quoted 7 lines
Quoted 3 lines
For example in SPEAr1340, physically RAM is mapped on above addresses. PCIe address translation unit can accept address only in the range of core addresses which are assigned to PCIe RC ie 0x80000000-0x8FFFFFFF.
Since this is a physical address, it corresponds to address space 3 in my list above, and the address you pick here is what you pass to
pci_ioremap_io.
This is what I was expecting. But currently designware driver does not pass this address to pci_ioremap_io.
OK.. We will fix it and will send patch.
Regards
Pratyush
Quoted 2 lines
Arnd
Quoted 10 lines · Pratyush Anand
Quoted 7 lines
1. The "bus" I/O port address, in the range between
0x1000and0xffff(65535). This is the address used in data frames in the actual bus transaction going over the PCIe wires. If you have more than one PCIe host bridge, each of them can normally have a range of up to 65536 addresses. The bus addresses get written into BAR registers of the device.Correct. Got this point. So this address should be output of designware outbound address translation unit.
Right.
Quoted 11 lines
Quoted 10 lines
2. The Linux I/O port numbers, between
0x00000and0xfffff(1048575). This is a logical number space that aggregates all bus I/O port numbers. The first 4096 ports are normally reserved for ISA devices and are not available for PCIe resources. Each PCI or PCIe host bridge can have a0x10000byte naturally aligned range in here. The common case is that you have only one host bridge using the range 0x1000-0xffff. If you have more than one, there may be an offset between them, e.g. the second PCIe host may haveio_offsetset to0x10000, which means that its bus range 0x1000-0xffff gets translated to Linux port number 0x11000-0x1ffff. Linux port numbers are visible e.g. in/proc/ioportand/dev/port
Quoted 2 lines
Ok, got it. To know more, which binding function maps this logical number space to address space in 4?
There is just a linear offset between the two: PCI_IO_VIRT_BASE. This offset is applied in the __io() macro when converting from a port number to a pointer in various places in the kernel (inb/outb as well as pci_iomap/ioport_map), and the offset is added in pci_ioremap_io() when creating the page table entries.
Quoted 8 lines
Quoted 8 lines
4. The CPU virtual address space: This is an implementation detail of the kernel, which results in the Linux I/O port numbers 0x0-0xfffff to be mapped at 0xfee00000-0xfeefffff.
Quoted 3 lines
For example in SPEAr1340, physically RAM is mapped on above addresses. PCIe address translation unit can accept address only in the range of core addresses which are assigned to PCIe RC ie 0x80000000-0x8FFFFFFF.
Quoted 7 lines
Quoted 3 lines
Since this is a physical address, it corresponds to address space 3 in my list above, and the address you pick here is what you pass to
pci_ioremap_io.This is what I was expecting. But currently designware driver does not pass this address to
pci_ioremap_io.
OK.. We will fix it and will send patch.
I think it does handle this correctly, look at
static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
{
...
if (global_io_offset < SZ_1M && pp->config.io_size > 0) {
sys->io_offset = global_io_offset - pp->config.io_bus_addr;
pci_ioremap_io(sys->io_offset, pp->io.start);
global_io_offset += SZ_64K;
pci_add_resource_offset(&sys->resources, &pp->io,
sys->io_offset);
}
...
}I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node in DT.
Arnd
Hi Arnd,
Quoted 3 lines · Arnd Bergmann
Quoted 1 line · Pratyush Anand
Quoted 1 line
[...]
Quoted 27 lines
Quoted 11 lines
Quoted 7 lines
Quoted 3 lines
For example in SPEAr1340, physically RAM is mapped on above addresses. PCIe address translation unit can accept address only in the range of core addresses which are assigned to PCIe RC ie 0x80000000-0x8FFFFFFF.
Since this is a physical address, it corresponds to address space 3 in my list above, and the address you pick here is what you pass to
pci_ioremap_io.This is what I was expecting. But currently designware driver does not pass this address to
pci_ioremap_io.
OK.. We will fix it and will send patch.I think it does handle this correctly, look at
static int
dw_pcie_setup(int nr, struct pci_sys_data *sys){ ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node in DT.
May be not exactly. pp->io is the realio, and it is passed correctly to pci_add_resource_offset. But, as you had also
said that pci_ioremap_io will receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.
drivers/pci/host/pcie-designware.c +2 -2
@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);
@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Regards
Pratyush
Quoted 2 lines
Arnd
Quoted 15 lines · Pratyush Anand
Quoted 15 lines
I think it does handle this correctly, look at
static int
dw_pcie_setup(int nr, struct pci_sys_data *sys){ ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }
Quoted 8 lines
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node in DT.
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also
said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.
I see. I think you are right.
Quoted 13 lines
drivers/pci/host/pcie-designware.c +1
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 409
Quoted 7 lines
drivers/pci/host/pcie-designware.c +0
-1@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
This looks correct to me and it seems to also fix a bug in dw_pcie_prog_viewport_io_outbound if I read this correctly.
drivers/pci/host/pcie-designware.c static int dw_pcie_setup(int nr, struct pci_sys_data *sys) · line 673
Quoted 9 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
I think there is still a related bug in here: we should pass global_io_offset rather than sys->io_offset to pci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from 0x1000 but the raw BAR values would contain the value you have chosen.
Arnd
Quoted 46 lines · Arnd Bergmann
Quoted 15 lines · Pratyush Anand
Quoted 15 lines
I think it does handle this correctly, look at
static int
dw_pcie_setup(int nr, struct pci_sys_data *sys){ ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }Quoted 8 lines
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node in DT.
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also
said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.I see. I think you are right.
Quoted 19 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;
Quoted 4 lines
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 409
Quoted 1 line
drivers/pci/host/pcie-designware.c +0
-0pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,This looks correct to me and it seems to also fix a bug in
dw_pcie_prog_viewport_io_outboundif I read this correctly.
Yes, now outbound viewport for IO translation will get correct input address.
Quoted 19 lines
drivers/pci/host/pcie-designware.c static int dw_pcie_setup(int nr, struct pci_sys_data *sys) · line 673
Quoted 10 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);I think there is still a related bug in here: we should pass
global_io_offsetrather thansys->io_offsettopci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from0x1000but the raw BAR values would contain the value you have chosen.
OK.
@ Jingoo, Mohit
Is it possible for you to test following patch.
drivers/pci/host/pcie-designware.c +2 -2
@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);
@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Regards
Pratyush
Quoted 2 lines
Arnd
Quoted 75 lines · Pratyush Anand
Quoted 46 lines · Arnd Bergmann
Quoted 15 lines · Pratyush Anand
Quoted 15 lines
I think it does handle this correctly, look at
static int
dw_pcie_setup(int nr, struct pci_sys_data *sys){ ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }Quoted 8 lines
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node in DT.
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also
said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.I see. I think you are right.
Quoted 19 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;Quoted 4 lines
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 409
Quoted 1 line
drivers/pci/host/pcie-designware.c +0
-0pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,This looks correct to me and it seems to also fix a bug in
dw_pcie_prog_viewport_io_outboundif I read this correctly.Yes, now outbound viewport for IO translation will get correct input address.
Quoted 19 lines
drivers/pci/host/pcie-designware.c static int dw_pcie_setup(int nr, struct pci_sys_data *sys) · line 673
Quoted 10 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);I think there is still a related bug in here: we should pass
global_io_offsetrather thansys->io_offsettopci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from0x1000but the raw BAR values would contain the value you have chosen.
Quoted 5 lines
OK.
@ Jingoo, Mohit
Is it possible for you to test following patch.
Hi Pratyush Anand,
This patch works properly on Exynos platform with two different NICs. Also, I am tracing this discussion.
Thank you for sending your patch.
Best regards,
Jingoo Han
Quoted 30 lines
drivers/pci/host/pcie-designware.c +2
-2@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Hello Pratyush,
Quoted 88 lines
-----Original Message-----
From: Jingoo Han [mailto:[email protected]]
Sent: Tuesday, December 10, 2013 10:55 AM
To: Pratyush ANAND; 'Arnd Bergmann'; Mohit KUMAR DCG
Cc: 'Marek Vasut'; 'Richard Zhu'; 'Kishon Vijay Abraham I'; linux- [email protected]; 'Tim Harvey'; 'Jingoo Han'
Subject: Re: [Query/Discussion]: IO translation with designware PCIe controllerQuoted 16 lines · Pratyush Anand
Quoted 15 lines · Arnd Bergmann
Quoted 14 lines · Pratyush Anand
Quoted 14 lines
I think it does handle this correctly, look at
static int dw_pcie_setup(int nr, struct pci_sys_data *sys) { ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node
in DT.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.Quoted 8 lines
Quoted 8 lines
I see. I think you are right.
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port
Quoted 6 lines
drivers/pci/host/pcie-designware.c +0
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port*pp)
Quoted 7 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 384
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-0+ global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
+404,6 @@ int
__initdw_pcie_host_init(struct pcie_port *pp)drivers/pci/host/pcie-designware.c +0
-1pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;Quoted 28 lines
Quoted 4 lines
Quoted 1 line
pp->va_cfg0_base= devm_ioremap(pp->dev,pp->cfg0_base,This looks correct to me and it seems to also fix a bug in
dw_pcie_prog_viewport_io_outboundif I read this correctly.Yes, now outbound viewport for IO translation will get correct input address.
Quoted 20 lines
Quoted 11 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,
sys->io_offset);I think there is still a related bug in here: we should pass
global_io_offsetrather thansys->io_offsettopci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from0x1000
Quoted 5 lines
but the raw BAR values would contain the value you have chosen.
Quoted 4 lines
OK.
@ Jingoo, Mohit
- Now its working properly for SPEAr1310 tested with directly connected EP(xhc cards) as well as EP Connected through switch(Lecroy PTC in AIC mode). Without this patch my EP devices were not working when connected through Switch.
Thanks
Mohit
Quoted 46 lines
Quoted 2 lines
Is it possible for you to test following patch.
Hi Pratyush Anand,
This patch works properly on Exynos platform with two different NICs. Also, I am tracing this discussion.
Thank you for sending your patch.Best regards,
Jingoo HanQuoted 7 lines
drivers/pci/host/pcie-designware.c +0
-0drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 384
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp)
Quoted 8 lines
@@ int
__initdw_pcie_host_init(struct pcie_port *pp)drivers/pci/host/pcie-designware.c +0
-1pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base, @@ -667,7
Quoted 9 lines
+667,7 @@ static int
dw_pcie_setup(int nr, struct pci_sys_data *sys)drivers/pci/host/pcie-designware.c +1
-1if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Hi Tim,
Quoted 92 lines · Mohit KUMAR DCG
Hello Pratyush,
Quoted 90 lines
-----Original Message-----
From: Jingoo Han [mailto:[email protected]]
Sent: Tuesday, December 10, 2013 10:55 AM
To: Pratyush ANAND; 'Arnd Bergmann'; Mohit KUMAR DCG
Cc: 'Marek Vasut'; 'Richard Zhu'; 'Kishon Vijay Abraham I'; linux- [email protected]; 'Tim Harvey'; 'Jingoo Han'
Subject: Re: [Query/Discussion]: IO translation with designware PCIe controllerQuoted 16 lines · Pratyush Anand
Quoted 15 lines · Arnd Bergmann
Quoted 14 lines · Pratyush Anand
Quoted 14 lines
I think it does handle this correctly, look at
static int dw_pcie_setup(int nr, struct pci_sys_data *sys) { ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node
in DT.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.Quoted 8 lines
Quoted 8 lines
I see. I think you are right.
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port
Quoted 6 lines
drivers/pci/host/pcie-designware.c +0
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port*pp)
Quoted 7 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 384
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-0+ global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
+404,6 @@ int
__initdw_pcie_host_init(struct pcie_port *pp)drivers/pci/host/pcie-designware.c +0
-1pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;Quoted 28 lines
Quoted 4 lines
Quoted 1 line
pp->va_cfg0_base= devm_ioremap(pp->dev,pp->cfg0_base,This looks correct to me and it seems to also fix a bug in
dw_pcie_prog_viewport_io_outboundif I read this correctly.Yes, now outbound viewport for IO translation will get correct input address.
Quoted 20 lines
Quoted 11 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,
sys->io_offset);I think there is still a related bug in here: we should pass
global_io_offsetrather thansys->io_offsettopci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from0x1000but the raw BAR values would contain the value you have chosen.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
OK.
@ Jingoo, Mohit
- Now its working properly for SPEAr1310 tested with directly connected EP(xhc cards) as well as EP Connected through switch(Lecroy PTC in AIC mode). Without this patch my EP devices were not working when connected through Switch.
Can you also check if this patch resolves your issue, and you are able to work with switch without commenting dw_pcie_prog_viewport_io_outbound.
Regards
Pratyush
Quoted 50 lines
Thanks
MohitQuoted 46 lines
Quoted 2 lines
Is it possible for you to test following patch.
Hi Pratyush Anand,
This patch works properly on Exynos platform with two different NICs. Also, I am tracing this discussion.
Thank you for sending your patch.Best regards,
Jingoo HanQuoted 7 lines
drivers/pci/host/pcie-designware.c +0
-0drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 384
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp)
Quoted 8 lines
@@ int
__initdw_pcie_host_init(struct pcie_port *pp)drivers/pci/host/pcie-designware.c +0
-1pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base, @@ -667,7
Quoted 9 lines
+667,7 @@ static int
dw_pcie_setup(int nr, struct pci_sys_data *sys)drivers/pci/host/pcie-designware.c +1
-1if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Quoted 82 lines · Mohit KUMAR DCG
Quoted 81 lines · Jingoo Han
Quoted 16 lines · Pratyush Anand
Quoted 15 lines · Arnd Bergmann
Quoted 14 lines · Pratyush Anand
Quoted 14 lines
I think it does handle this correctly, look at
static int dw_pcie_setup(int nr, struct pci_sys_data *sys) { ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node
in DT.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.Quoted 8 lines
Quoted 8 lines
I see. I think you are right.
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port
Quoted 6 lines
drivers/pci/host/pcie-designware.c +0
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port*pp)
Quoted 7 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port · line 384
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-0+ global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
+404,6 @@ int
__initdw_pcie_host_init(struct pcie_port *pp)drivers/pci/host/pcie-designware.c +0
-1pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;Quoted 28 lines
Quoted 4 lines
Quoted 1 line
pp->va_cfg0_base= devm_ioremap(pp->dev,pp->cfg0_base,This looks correct to me and it seems to also fix a bug in
dw_pcie_prog_viewport_io_outboundif I read this correctly.Yes, now outbound viewport for IO translation will get correct input address.
Quoted 20 lines
Quoted 11 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,
sys->io_offset);I think there is still a related bug in here: we should pass
global_io_offsetrather thansys->io_offsettopci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from0x1000but the raw BAR values would contain the value you have chosen.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
OK.
@ Jingoo, Mohit
- Now its working properly for SPEAr1310 tested with directly connected EP(xhc cards) as well as EP Connected through switch(Lecroy PTC in AIC mode). Without this patch my EP devices were not working
when
connected through Switch.
Hi Marek, Richard,
How about testing the following Pratyush's patch on imx6 platform with the Pericom PI7C9X2G303EL PCIe switch?
As Mohit tested, it looks to resolve the i.MX6 problem when the PCIe switch is connected to the i.MX6 platform.
Best regards,
Jingoo Han
[.....]
drivers/pci/host/pcie-designware.c +2 -2
@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);
@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Hi Jinhoo:
Quoted 108 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jingoo Han
Sent: Tuesday, December 10, 2013 3:03 PM
To: 'Marek Vasut'; Zhu Richard-R65037
Cc: 'Mohit KUMAR DCG'; 'Pratyush ANAND'; 'Arnd Bergmann'; 'Kishon Vijay Abraham I'; [email protected]; 'Tim Harvey'; 'Jingoo Han'
Subject: Re: [Query/Discussion]: IO translation with designware PCIe controllerQuoted 86 lines · Mohit KUMAR DCG
Quoted 9 lines · Jingoo Han
Quoted 8 lines · Pratyush Anand
Quoted 7 lines · Arnd Bergmann
Quoted 6 lines · Pratyush Anand
Quoted 6 lines
I think it does handle this correctly, look at
static int dw_pcie_setup(int nr, struct pci_sys_data *sys) { ... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp-config.io_bus_addr;
Quoted 75 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ... }I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node
in DT.
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.I see. I think you are right.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c +0
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct
pcie_port*pp)
Quoted 7 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c int __init dw_pcie_host_init(struct pcie_port *pp) · line 384
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-0+ global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
+404,6 @@ int
__initdw_pcie_host_init(struct pcie_port *pp)drivers/pci/host/pcie-designware.c +0
-1pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;Quoted 29 lines
Quoted 4 lines
Quoted 1 line
pp->va_cfg0_base= devm_ioremap(pp->dev,pp->cfg0_base,This looks correct to me and it seems to also fix a bug in
dw_pcie_prog_viewport_io_outboundif I read this correctly.Yes, now outbound viewport for IO translation will get correct input address.
Quoted 21 lines
Quoted 11 lines
drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,
sys->io_offset);I think there is still a related bug in here: we should pass
global_io_offsetrather thansys->io_offsettopci_ioremap_io, so we map the new window into the first available spot in the Linux view of the I/O space, rather than passing a number that might be zero for any bus, if the 'ranges' are set up to have an identity mapping between Linux I/O spaces and PCI I/O spaces. You should be able to verify this by setting the I/O range for the bus to a random 4KB multiple in DT and observe that Linux start allocating ports from0x1000but the raw BAR values would contain the value you have chosen.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
OK.
@ Jingoo, Mohit
- Now its working properly for SPEAr1310 tested with directly connected EP(xhc cards) as well as EP Connected through switch(Lecroy PTC in AIC mode). Without this patch my EP devices were not working when connected through Switch.
Hi Marek, Richard,
Quoted 9 lines
How about testing the following Pratyush's patch on imx6 platform with the Pericom PI7C9X2G303EL PCIe switch?
As Mohit tested, it looks to resolve the i.MX6 problem when the PCIe switch is connected to the i.MX6 platform.
Best regards,
Jingoo Han
[Richard] Sorry to reply late, I went to hospital in the past two days. This patch is great, I tested at my side, it works, thanks a lot.
Quoted 34 lines
[.....]
drivers/pci/host/pcie- renamed from drivers/pci/host/pcie-designware.c +0
-0
designware.c
index be6ce30..b83f5e8 100644drivers/pci/host/pcie-designware.c +2
-1@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) { of_pci_range_to_resource(&range, np, &pp->mem); @@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base, @@ -667,7 +667,7@@ static int
dw_pcie_setup(int nr, struct pci_sys_data *sys)drivers/pci/host/pcie-designware.c +1
-2if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset); --
Quoted 4 lines
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
Best Regards
Richard Zhu
Quoted 81 lines · Pratyush Anand
Hi Arnd,
Quoted 1 line · Arnd Bergmann
On Friday 06 December 2013, Pratyush Anand wrote:
[...]
Quoted 34 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
For example in SPEAr1340, physically RAM is mapped on above addresses. PCIe address translation unit can accept address only in the range of core addresses which are assigned to PCIe RC ie 0x80000000-0x8FFFFFFF.
Quoted 7 lines
Quoted 3 lines
Since this is a physical address, it corresponds to address space 3 in my list above, and the address you pick here is what you pass to
pci_ioremap_io.This is what I was expecting. But currently designware driver does not pass this address to
pci_ioremap_io.
OK.. We will fix it and will send patch.I think it does handle this correctly, look at
static int
dw_pcie_setup(int nr, struct pci_sys_data *sys){... if (global_io_offset < SZ_1M && pp->config.io_size > 0) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; pci_ioremap_io(sys->io_offset, pp->io.start); global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset); } ...Quoted 4 lines
}
I believe this does the right thing, but you have to put the correct translation into the 'ranges' property of the host bridge node in DT.
May be not exactly.
pp->iois the realio, and it is passed correctly topci_add_resource_offset. But, as you had also
said thatpci_ioremap_iowill receive cpu physical address space as input, therefore I think following modification will be needed to work io transaction properly.(quoted diff) +0
-0
b/drivers/pci/host/pcie-designware.cindex be6ce30..cf68632 100644drivers/pci/host/pcie-designware.c +2
-2@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,
Quoted 1 line
sys->io_offset);
Tim, can you test if this patch fixes your SKY2 IOspace problem please ?
Quoted 1 line · Marek Vasut
On Monday, December 09, 2013 at 08:12:42 AM, Pratyush Anand wrote:
[.....]
Quoted 24 lines
Quoted 24 lines
(quoted diff) +0
-0
b/drivers/pci/host/pcie-designware.cindex be6ce30..cf68632 100644drivers/pci/host/pcie-designware.c +1
-1@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
Quoted 9 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-1if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,
sys->io_offset);Tim, can you test if this patch fixes your SKY2 IOspace problem please ?
Above mentioned patch is NOT the latest patch. Pratyush Anand already submitted the next patch as below.
Tim,
Would you test the following patch on i.MX6 platform?
drivers/pci/host/pcie-designware.c +2 -2
@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);
@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Best regards,
Jingoo Han
Quoted 76 lines · Jingoo Han
Quoted 1 line · Marek Vasut
On Monday, December 09, 2013 at 08:12:42 AM, Pratyush Anand wrote:
[.....]
Quoted 24 lines
Quoted 24 lines
(quoted diff) +0
-0
b/drivers/pci/host/pcie-designware.cindex be6ce30..cf68632 100644drivers/pci/host/pcie-designware.c +1
-1@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)Quoted 9 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-1if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,sys->io_offset);Tim, can you test if this patch fixes your SKY2 IOspace problem please ?
Above mentioned patch is NOT the latest patch. Pratyush Anand already submitted the next patch as below.
Tim,
Would you test the following patch on i.MX6 platform?drivers/pci/host/pcie-designware.c +2
-2@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Quoted 3 lines
Best regards,
Jingoo Han
Yes, this patch resolves the issue and now I can use devices with io resources behind a bridge attached to the IMX6.
Thanks everyone for your efforts!
Tim
Quoted 103 lines · Tim Harvey
Quoted 102 lines · Jingoo Han
Quoted 1 line · Marek Vasut
On Monday, December 09, 2013 at 08:12:42 AM, Pratyush Anand wrote:
[.....]
Quoted 40 lines
Quoted 40 lines
(quoted diff) +0
-0
b/drivers/pci/host/pcie-designware.cindex be6ce30..cf68632 100644drivers/pci/host/pcie-designware.c +1
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr;} if (restype == IORESOURCE_MEM) { of_pci_range_to_resource(&range, np, &pp->mem);drivers/pci/host/pcie-designware.c +0
-1@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base);Quoted 6 lines
Quoted 4 lines
global_io_offset += SZ_64K; pci_add_resource_offset(&sys->resources, &pp->io, sys->io_offset);Tim, can you test if this patch fixes your SKY2 IOspace problem please ?
Above mentioned patch is NOT the latest patch. Pratyush Anand already submitted the next patch as below.
Tim,
Would you test the following patch on i.MX6 platform?(quoted diff) +0
-0
b/drivers/pci/host/pcie-designware.cindex be6ce30..b83f5e8 100644drivers/pci/host/pcie-designware.c +1
-0@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr;} if (restype == IORESOURCE_MEM) { of_pci_range_to_resource(&range, np, &pp->mem);drivers/pci/host/pcie-designware.c +0
-1@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start;pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,drivers/pci/host/pcie-designware.c +1
-1@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Quoted 7 lines
Quoted 2 lines
Best regards,
Jingoo HanYes, this patch resolves the issue and now I can use devices with io resources behind a bridge attached to the IMX6.
Thanks everyone for your efforts!
This is awesome, thanks guys!
Quoted 77 lines · Tim Harvey
Quoted 76 lines · Jingoo Han
Quoted 1 line · Marek Vasut
On Monday, December 09, 2013 at 08:12:42 AM, Pratyush Anand wrote:
[.....]
Quoted 24 lines
Quoted 24 lines
(quoted diff) +0
-0
b/drivers/pci/host/pcie-designware.cindex be6ce30..cf68632 100644drivers/pci/host/pcie-designware.c +1
-1@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)Quoted 9 lines
Quoted 7 lines
drivers/pci/host/pcie-designware.c +1
-1if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (sys->io_offset, pp->io_base); global_io_offset +=SZ_64K ; pci_add_resource_offset(&sys->resources, &pp->io,sys->io_offset);Tim, can you test if this patch fixes your SKY2 IOspace problem please ?
Above mentioned patch is NOT the latest patch. Pratyush Anand already submitted the next patch as below.
Tim,
Would you test the following patch on i.MX6 platform?drivers/pci/host/pcie-designware.c +2
-2@@ -378,6 +378,7 @@ int __init dw_pcie_host_init(struct pcie_port *pp) + global_io_offset); pp->config.io_size =resource_size (&pp->io); pp->config.io_bus_addr = range.pci_addr; + pp->io_base = range.cpu_addr; }if (restype ==IORESOURCE_MEM ) {of_pci_range_to_resource (&range, np, &pp->mem);@@ -403,7 +404,6 @@ int __init dw_pcie_host_init(struct pcie_port *pp) pp->cfg0_base = pp->cfg.start; pp->cfg1_base = pp->cfg.start + pp->config.cfg0_size; - pp->io_base = pp->io.start; pp->mem_base = pp->mem.start; pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,@@ -667,7 +667,7 @@ static int dw_pcie_setup(int nr, struct pci_sys_data *sys)if (global_io_offset <SZ_1M && pp->config.io_size >0 ) { sys->io_offset = global_io_offset - pp->config.io_bus_addr; -pci_ioremap_io (sys->io_offset, pp->io.start); +pci_ioremap_io (global_io_offset, pp->io_base); global_io_offset +=SZ_64K ;pci_add_resource_offset (&sys->resources, &pp->io, sys->io_offset);
Quoted 8 lines
Quoted 3 lines
Best regards,
Jingoo HanYes, this patch resolves the issue and now I can use devices with io resources behind a bridge attached to the IMX6.
Thanks everyone for your efforts!
Wow, happy ending! :-)
I was deeply moved.
Pratyush Anand,
Would you re-send this patch with commit message? You did a good job! I really appreciate your patch.
Best regards,
Jingoo Han