Reverting old hack

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

Hi Ralf,

Quoted 6 lines · On Sat, 20 Feb 2010 12:31:34 +0100 Ralf Baechle <[email protected]> wrote:

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

Quoted 12 lines · Yoichi Yuasa

Hi Ralf,

Quoted 6 lines · On Sat, 20 Feb 2010 12:31:34 +0100 Ralf Baechle <[email protected]> wrote:

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

Quoted 8 lines

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

Thanks for trying that out.

I'd like to understand the PCI architecture of Cobalt better. Would you mind turning on CONFIG_PCI_DEBUG and posting the dmesg log?

The purpose of IORESOURCE_PCI_FIXED is to say that we shouldn't reassign the device resources. But the pcibios_fixup_device_resources() loop isn't *moving* devices, it's merely converting PCI bus addresses to CPU addresses. So I suspect that the IORESOURCE_PCI_FIXED test there is merely covering up another bug.

For example, maybe 00:09.1 is an internal device that's behind a different host bridge that does no address conversion, or maybe the host bridge just handles that device specially. If something like that is happening, pcibios_bus_to_resource() should be broken for that device, and fixing that should allow us to remove the IORESOURCE_PCI_FIXED test.

Quoted 2 lines

I'd like to understand the PCI architecture of Cobalt better. Would you mind turning on CONFIG_PCI_DEBUG and posting the dmesg log?

If you want to know what happen, you can see my old e-mail.

http://marc.info/?l=linux-kernel&m=118792430424186&w=2

Quoted 6 lines · Yoichi Yuasa

Quoted 2 lines

I'd like to understand the PCI architecture of Cobalt better. Would you mind turning on CONFIG_PCI_DEBUG and posting the dmesg log?

If you want to know what happen, you can see my old e-mail.

http://marc.info/?l=linux-kernel&m=118792430424186&w=2

There's not much detail there. It would save me a lot of time if you could collect the complete dmesg log, /proc/iomem, and /proc/ioports.

Hi Bjorn,

Quoted 4 lines · On Mon, 22 Feb 2010 13:55:28 -0700 Bjorn Helgaas <[email protected]> wrote:

Quoted 3 lines · Yoichi Yuasa

Quoted 2 lines

I'd like to understand the PCI architecture of Cobalt better. Would you mind turning on CONFIG_PCI_DEBUG and posting the dmesg log?

Quoted 6 lines

Quoted 3 lines

If you want to know what happen, you can see my old e-mail.

http://marc.info/?l=linux-kernel&m=118792430424186&w=2

There's not much detail there. It would save me a lot of time if you could collect the complete dmesg log, /proc/iomem, and /proc/ioports.

It cannot boot without old hack.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7]

Quoted 9 lines · Yoichi Yuasa

Hi Bjorn,

Quoted 4 lines · On Mon, 22 Feb 2010 13:55:28 -0700 Bjorn Helgaas <[email protected]> wrote:

Quoted 3 lines · Yoichi Yuasa

Quoted 2 lines

I'd like to understand the PCI architecture of Cobalt better. Would you mind turning on CONFIG_PCI_DEBUG and posting the dmesg log?

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

If you want to know what happen, you can see my old e-mail.

http://marc.info/?l=linux-kernel&m=118792430424186&w=2

There's not much detail there. It would save me a lot of time if you could collect the complete dmesg log, /proc/iomem, and /proc/ioports.

It cannot boot without old hack.

I know; I meant that the information from a kernel with the old hack would be useful. But I think I'm starting to understand anyway.

The Linux I/O port number space is defined here:

    static struct resource cobalt_io_resource = {
        .start  = 0x1000,
        .end    = GT_DEF_PCI0_IO_SIZE - 1,  /* 0x1ffffff */

[As an aside, I'm not sure 0x1000 is the correct start -- for example, I think Linux I/O port 0x1f0 is forwarded by the host bridge.]

The corresponding PCI I/O port numbers are determined by the PCI I/O decoder address, so I agree that we need the io_offset to convert between the Linux port numbers and ports that appear on the PCI bus.

I think the IDE device is a problem because pci_setup_device() fills in legacy resources with ports 0x1f0-0x1f7, etc. We expect those resources to contain PCI bus addresses at this point, but we could never see those addresses on the Cobalt PCI bus (we would only see things in the range 0x10000000-0x11ffffff).

When we convert 0x1f0 with pcibios_bus_to_resource() (or with pcibios_fixup_device_resources() without the IORESOURCE_PCI_FIXED hack), we get 0x1f0 + 0xf0000000 == 0xf00001f0, when we want 0x1f0 instead.

Quoted 1 line

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7]

I still don't know the best way to fix this, but does this make sense so far?

Hi Ralf,

Quoted 6 lines · On Sat, 20 Feb 2010 12:31:34 +0100 Ralf Baechle <[email protected]> wrote:

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

Quoted 12 lines · Yoichi Yuasa

Hi Ralf,

Quoted 6 lines · On Sat, 20 Feb 2010 12:31:34 +0100 Ralf Baechle <[email protected]> wrote:

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

Quoted 8 lines

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

I think the problem is that cobalt_io_resource contains PCI bus addresses when it should contain CPU addresses:

    static struct resource cobalt_io_resource = {
        .start  = 0x1000,
        .end    = GT_DEF_PCI0_IO_SIZE - 1,
        .name   = "PCI I/O",
        .flags  = IORESOURCE_IO,
    };

I think .start and .end need to be adjusted by GT_DEF_PCI0_IO_BASE.

Hi Ralf,

Quoted 6 lines · On Sat, 20 Feb 2010 12:31:34 +0100 Ralf Baechle <[email protected]> wrote:

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

Quoted 12 lines · Yoichi Yuasa

Hi Ralf,

Quoted 6 lines · On Sat, 20 Feb 2010 12:31:34 +0100 Ralf Baechle <[email protected]> wrote:

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

Quoted 8 lines

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

I think Cobalt needs something like the patch below, because I think in your working system, pata_via is using I/O port 0x1f0, not 0xf00001f0. That means the the port the driver sees in the pci_dev resource is identical to the port number that appears on the PCI bus, so there is no io_offset.

There are a few other places that may set non-zero io_offset values:
bcm1480, bcm1480ht. txx9_alloc_pci_controller(), bridge_probe(), and octeon_pcie_setup(). I don't know whether they have similar issues.

commit 7378269220d477118257d898bec9173743675f5e

Author: Bjorn Helgaas <[email protected]>
Date:   Sat Feb 20 07:52:29 2010 -0700

[MIPS] remove Cobalt I/O space offset

On Cobalt, "inb(x)" produces an I/O port access to port "x" on the PCI bus, which means the io_offset is zero and CPU (resource) addresses are identical to PCI bus addresses. Correcting this means we can remove the IORESOURCE_PCI_FIXED check from pcibios_fixup_device_resources().

The io_map_base is used internally by pci_iomap(), inb(), and other I/O port access functions to generate an MMIO access to the address that produces the desired I/O port PCI transaction.

[Cobalt plat_mem_setup() does this:
set_io_port_base(CKSEG1ADDR(GT_DEF_PCI0_IO_BASE));
rather than using cobalt_pci_controller.io_map_base, but the value's the same, and I don't know enough to clean that up.]

See http://lkml.org/lkml/2007/7/29/27

Signed-off-by Bjorn Helgaas <[email protected]>

arch/mips/cobalt/pci.c +0 -0

index cfce7af..84aa205 100644

Quoted 7 lines

Quoted 6 lines

Below 9f7670e4ddd940d95e48997c2da51614e5fde2cf, an old hack which I committed in December '07 I think mostly for Cobalt machines. This is now getting in the way - in fact the whole loop in pcibios_fixup_device_resources() may have to go. So I wonder if this old hack is still necessary. Only testing can answer so I'm going to put a patch to revert this into the -queue tree for 2.6.34.

Quoted 8 lines

It is still necessary for Cobalt.
I got the following IDE resource errors.

pata_via 0000:00:09.1: BAR 0: can't reserve [io 0xf00001f0-0xf00001f7] pata_via 0000:00:09.1: failed to request/iomap BARs for port 0 (errno=-16) pata_via 0000:00:09.1: BAR 2: can't reserve [io 0xf0000170-0xf0000177] pata_via 0000:00:09.1: failed to request/iomap BARs for port 1 (errno=-16) pata_via 0000:00:09.1: no available native port

I think Cobalt needs something like the patch below, because I think in your working system, pata_via is using I/O port 0x1f0, not 0xf00001f0. That means the the port the driver sees in the pci_dev resource is identical to the port number that appears on the PCI bus, so there is no io_offset.

There are a few other places that may set non-zero io_offset values:
bcm1480, bcm1480ht. txx9_alloc_pci_controller(), bridge_probe(), and octeon_pcie_setup(). I don't know whether they have similar issues.

It's a while since I last looked into this but here's how things afair are working on a MIPS-based Cobalt system.

The system is based on a MIPS processor and a GT-64111 system controller. Addresses within a certain CPU address range are passed to the PCI bus as I/O cycles without address cycles. Since memory is starting at CPU address zero (and has to because of the processors used), that address window has to get mapped somewhere else. So a CPU access to some virtual address gets translated to physical address 0xf00001f0. The GT-64111 passes this to the PCI bus as I/O port address 0xf00001f0. Finally the VT82C586 chip which only decodes the low 16 bits drops treats this as an I/O port space address 0x1f0.

Quoted 12 lines · Ralf Baechle

It's a while since I last looked into this but here's how things afair are working on a MIPS-based Cobalt system.

The system is based on a MIPS processor and a GT-64111 system controller. Addresses within a certain CPU address range are passed to the PCI bus as I/O cycles without address cycles. Since memory is starting at CPU address zero (and has to because of the processors used), that address window has to get mapped somewhere else. So a CPU access to some virtual address gets translated to physical address 0xf00001f0. The GT-64111 passes this to the PCI bus as I/O port address 0xf00001f0. Finally the VT82C586 chip which only decodes the low 16 bits drops treats this as an I/O port space address 0x1f0.

Yoichi, can you try the patch below? I think this is basically the approach Ben suggested long ago:

    http://marc.info/?l=linux-kernel&m=119733290624544&w=2

Quoted 2 lines

approach Ben suggested long ago:

    http://marc.info/?l=linux-kernel&m=119733290624544&w=2

It works fine with 2.6.34 queue tree.
pci.c change is already committed by Ralf.

Quoted 5 lines · Yoichi Yuasa

Quoted 2 lines

approach Ben suggested long ago:

    http://marc.info/?l=linux-kernel&m=119733290624544&w=2

It works fine with 2.6.34 queue tree.
pci.c change is already committed by Ralf.

Which I just dropped from queue. To keep the tree bisectable removal of the old hack and adding the fixup should be done in the same patch so I'd go for Bjorn's patch.

There is another somewhat theoretical correctness issue. Because the VIA SuperIO chip only decodes 24 bits of address space but port address space currently being configured as 32MB there is the theoretical possibility of I/O port addresses that alias with legacy addresses getting allocated.

Quoted 5 lines

There is another somewhat theoretical correctness issue. Because the VIA SuperIO chip only decodes 24 bits of address space but port address space currently being configured as 32MB there is the theoretical possibility of I/O port addresses that alias with legacy addresses getting allocated.

Does this mean my comment:

(quoted diff) +2 -0

+        * but the VT82C586 IDE controller does respond at 0x100001f0 because
+        * it only decodes the low 16 bits of the address.

should say "24 bits" instead of "16 bits"?

The complicated solution is to reserve all address range that potencially could cause such aliases. But with the PCI spec limiting port allocations for devices to a maximum of 256 bytes 16MB of port address space already is way more than one would ever expect to be used so I suggest to just limit the port address space to 16MB.

Could you test the patch below?

Quoted 5 lines · Yoichi Yuasa

Quoted 2 lines

approach Ben suggested long ago:

    http://marc.info/?l=linux-kernel&m=119733290624544&w=2

It works fine with 2.6.34 queue tree.
pci.c change is already committed by Ralf.

Which I just dropped from queue. To keep the tree bisectable removal of the old hack and adding the fixup should be done in the same patch so I'd go for Bjorn's patch.

There is another somewhat theoretical correctness issue. Because the VIA SuperIO chip only decodes 24 bits of address space but port address space currently being configured as 32MB there is the theoretical possibility of I/O port addresses that alias with legacy addresses getting allocated.

The complicated solution is to reserve all address range that potencially could cause such aliases. But with the PCI spec limiting port allocations for devices to a maximum of 256 bytes 16MB of port address space already is way more than one would ever expect to be used so I suggest to just limit the port address space to 16MB.

Could you test the patch below?

It has no problem.

Yoichi, can you try the patch below? I think this is basically the approach Ben suggested long ago:

    http://marc.info/?l=linux-kernel&m=119733290624544&w=2

Quoted 3 lines · Bjorn Helgaas

Yoichi, can you try the patch below? I think this is basically the approach Ben suggested long ago:

    http://marc.info/?l=linux-kernel&m=119733290624544&w=2

No Signed-off-by ...