Skip to contentSkip to conversation

No PCI_AUTO in 2.6...

DISCUSSION wlacey 4 participants 9 messages Dec 11, 2004 – Dec 16, 2004 MIPS

lore
Share & tools
lore

wlacey

author
#0 lore raw

Might someone be willing to share a bit knowledge with me?

I've transitioned to the 2.6.10 kernel and I'm having a difficult time understanding what things I must do different to get my pci slots probed as before in 2.4. At this point I'm well aware the 2.6 is not a drop in replacement for 2.4 but what is the a general approach to getting something like PCI_AUTO capability in 2.6 what steps must I take and is there document describing them.

I call register_pci_controller() but the bus is never scanned becasue pcibios_init() fails out with... "Skipping PCI bus scan due to resource conflict"

Any hints/clues/breadcrumbs for the starving?

Thanks,
W

#1 lore raw
Quoted 3 lines · wlacey

Might someone be willing to share a bit knowledge with me?

I've transitioned to the 2.6.10 kernel and I'm having a difficult time understanding what things I must do different to get my pci slots probed as before in 2.4. At this point I'm well aware the 2.6 is not a drop in replacement for 2.4 but what is the a general approach to getting something like PCI_AUTO capability in 2.6 what steps must I take and is there document describing them.

PCI_AUTO was really, really broken code. It only works for some subset of systems, won't handle hot plugging (Cardbus!), PCI-PCI bridges and many other cases. The world became a better place on the day when it got removed.

Quoted 2 lines

I call register_pci_controller() but the bus is never scanned becasue pcibios_init() fails out with... "Skipping PCI bus scan due to resource conflict"

The new PCI code by default will fully configure a PCI bus hierarchy. For this to work it needs to know which memory address range and which I/O port address range are available for assignment. This information is passed in the struct pci_controller argument of register_pci_controller(). Here's a simplified example from arch/mips/sni/setup.c:

static struct resource sni_io_resource = {
        "PCIMT IO MEM", 0x00001000UL, 0x03bfffffUL, IORESOURCE_IO,
};
static struct resource sni_mem_resource = {
        "PCIMT PCI MEM", 0x10000000UL, 0xffffffffUL, IORESOURCE_MEM
};

extern struct pci_ops sni_pci_ops;

static struct pci_controller sni_controller = {

        .pci_ops        = &sni_pci_ops,
        .mem_resource   = &sni_mem_resource,
        .mem_offset     = 0x10000000UL,
        .io_resource    = &sni_io_resource,
        .io_offset      = 0x00000000UL

};

That is PCI memory is in the address range of 0x10000000UL - 0xffffffffUL and I/O ports in the range 0x00001000UL - 0x03bfffffUL. The io_offset rsp. mem_offset values say how much needs to be added rsp. subtracted when converting a PCI bus address into a physical address. Often these values are either the same a the resource's start address or zero.

I hope that explains things a little ...

Ralf

#2 lore raw
Quoted 9 lines · Ralf Baechle

Here's a simplified example from arch/mips/sni/setup.c:

static struct resource sni_io_resource = {
        "PCIMT IO MEM", 0x00001000UL, 0x03bfffffUL, IORESOURCE_IO,
};
static struct resource sni_mem_resource = {
        "PCIMT PCI MEM", 0x10000000UL, 0xffffffffUL, IORESOURCE_MEM
};

I think it's more descriptive to call them "<foo> PCI I/O" and "<foo> PCI
MEM", respectively, to make it clearer the former expresses I/O port addresses (not the associated memory address range for accesses to be forwarded as PCI I/O cycles) and that both are PCI bus spaces.

Also for most PCI systems I/O port space should really start at 0 (for
"legacy" devices being decoded by the PCI-ISA bridge if there's one in the system), but generic code braindamage prevents it currently. I think there was only about a single PCI chipset that had a "reversed" architecture and sort-of bridged PCI over ISA -- the Intel i82420EX for the i486 processor. It would need a separate ISA I/O resource for a correct view of the system.

Quoted 5 lines

That is PCI memory is in the address range of 0x10000000UL - 0xffffffffUL and I/O ports in the range 0x00001000UL - 0x03bfffffUL. The io_offset rsp. mem_offset values say how much needs to be added rsp. subtracted when converting a PCI bus address into a physical address. Often these values are either the same a the resource's start address or zero.

Things start being tricky once you have to use such an offset for DMA
transfers as well...

Maciej

#3 lore raw
Quoted 15 lines · Maciej W. Rozycki
Quoted 9 lines

Here's a simplified example from arch/mips/sni/setup.c:

static struct resource sni_io_resource = {
        "PCIMT IO MEM", 0x00001000UL, 0x03bfffffUL, IORESOURCE_IO,
};
static struct resource sni_mem_resource = {
        "PCIMT PCI MEM", 0x10000000UL, 0xffffffffUL, IORESOURCE_MEM
};

I think it's more descriptive to call them "<foo> PCI I/O" and "<foo> PCI
MEM", respectively, to make it clearer the former expresses I/O port addresses (not the associated memory address range for accesses to be forwarded as PCI I/O cycles) and that both are PCI bus spaces.

Quoted 7 lines

Also for most PCI systems I/O port space should really start at 0 (for
"legacy" devices being decoded by the PCI-ISA bridge if there's one in the system), but generic code braindamage prevents it currently. I think there was only about a single PCI chipset that had a "reversed" architecture and sort-of bridged PCI over ISA -- the Intel i82420EX for the i486 processor. It would need a separate ISA I/O resource for a correct view of the system.

In the above case, the I/O port resource is starting at 0x1000 because the range of 0x0 - 0xfff contains a few legacy devices. The way the SNI code is initializing sni_io_resource ensures all the legacy devices are properly listed in the iomem_resource and /proc/ioports and the kernel will never place any I/O resource for a PCI card in the legacy port range. As the result we'll get:

[root@tbird root]# cat /proc/ioports
00000000-0000001f : dma1
00000020-0000003f : pic1
00000040-0000005f : timer
00000060-0000006f : keyboard
00000070-00000077 : rtc
00000080-0000008f : dma page reg
000000a0-000000bf : pic2
000000c0-000000df : dma2
000002f8-000002ff : serial
000003c0-000003df : vga+
000003f8-000003ff : serial
00000cfc-00000cff : PCI config data
00001000-03bfffff : PCIMT IO MEM
00001000-000010ff : 0000:00:01.0

    00001000-000010ff : sym53c8xx
  00001400-0000141f : 0000:00:02.0
    00001400-0000141f : pcnet32_probe_pci

[root@tbird root]#

This makes it look like the legacy ports are not behind the PCI bus which actually they are but that's a minor nit, it gets the address space allocation to work right.

Quoted 8 lines
Quoted 5 lines

That is PCI memory is in the address range of 0x10000000UL - 0xffffffffUL and I/O ports in the range 0x00001000UL - 0x03bfffffUL. The io_offset rsp. mem_offset values say how much needs to be added rsp. subtracted when converting a PCI bus address into a physical address. Often these values are either the same a the resource's start address or zero.

Things start being tricky once you have to use such an offset for DMA
transfers as well...

True, but that's dealt with elsewhere. And I never claimed the mess that PCI used to be in 2.4 has yet been fully cleaned up yet, more work to do.

Ralf

#4 lore raw
Quoted 8 lines · Ralf Baechle
Quoted 7 lines

Also for most PCI systems I/O port space should really start at 0 (for
"legacy" devices being decoded by the PCI-ISA bridge if there's one in the system), but generic code braindamage prevents it currently. I think there was only about a single PCI chipset that had a "reversed" architecture and sort-of bridged PCI over ISA -- the Intel i82420EX for the i486 processor. It would need a separate ISA I/O resource for a correct view of the system.

Quoted 5 lines

In the above case, the I/O port resource is starting at 0x1000 because the range of 0x0 - 0xfff contains a few legacy devices. The way the SNI code is initializing sni_io_resource ensures all the legacy devices are properly listed in the iomem_resource and /proc/ioports and the kernel will never place any I/O resource for a PCI card in the legacy port range.

I know and I consider it a bug. The correct way would be setting the
start at 0 and if avoiding the first kB of space was necessary, setting PCIBIOS_MIN_IO to 0x1000.

Quoted 17 lines

As the result we'll get:

[root@tbird root]# cat /proc/ioports
00000000-0000001f : dma1
00000020-0000003f : pic1
00000040-0000005f : timer
00000060-0000006f : keyboard
00000070-00000077 : rtc
00000080-0000008f : dma page reg
000000a0-000000bf : pic2
000000c0-000000df : dma2
000002f8-000002ff : serial
000003c0-000003df : vga+
000003f8-000003ff : serial
00000cfc-00000cff : PCI config data
00001000-03bfffff : PCIMT IO MEM
00001000-000010ff : 0000:00:01.0

Quoted 8 lines
    00001000-000010ff : sym53c8xx
  00001400-0000141f : 0000:00:02.0
    00001400-0000141f : pcnet32_probe_pci

[root@tbird root]#

This makes it look like the legacy ports are not behind the PCI bus which actually they are but that's a minor nit, it gets the address space allocation to work right.

I think it's going to matter if a PCI-ISA bridge is behind one or more
PCI-PCI bridges. I have an idea of a fix, but my initial approach haven't worked particularly well and I've had no time to dig into it further. Actually, on systems with PCI I think reserving legacy resources should happen only after they have been discovered (you don't need any PC/AT or ISA devices for a PCI configuration space scan), but that may be tough, so I've thought of an alternative.

Also the map of legacy resources reserved is actually incomplete -- PIC's
ELCR is missing for example.

Quoted 5 lines
Quoted 2 lines

Things start being tricky once you have to use such an offset for DMA
transfers as well...

True, but that's dealt with elsewhere. And I never claimed the mess that PCI used to be in 2.4 has yet been fully cleaned up yet, more work to do.

Sure, I'm more or less happy about the new code. I'm just warning about
possible pitfalls.

There is one more problem with the PCI resource manager -- it messes with
BARs of host bridges without asking for permission (which is often a no-no, as any messing with host bridges in general). I have a patch for it I'm currently trying to push to the PCI maintainer. See the thread at "http://www.uwsg.iu.edu/hypermail/linux/kernel/0412.1/0484.html" if interested. It's already being done correctly for i386 (some would probably argue that's what's most important) and ppc which use their own resource managers instead of the generic one.

Maciej

#5 lore raw
Quoted 3 lines · Maciej W. Rozycki

I know and I consider it a bug. The correct way would be setting the
start at 0 and if avoiding the first kB of space was necessary, setting PCIBIOS_MIN_IO to 0x1000.

PCIBIOS_MIN_IO is the same value for all busses. That can turn out a bit kludgy so I'm not much of a friend of it.

Quoted 11 lines
Quoted 3 lines

This makes it look like the legacy ports are not behind the PCI bus which actually they are but that's a minor nit, it gets the address space allocation to work right.

I think it's going to matter if a PCI-ISA bridge is behind one or more
PCI-PCI bridges. I have an idea of a fix, but my initial approach haven't worked particularly well and I've had no time to dig into it further. Actually, on systems with PCI I think reserving legacy resources should happen only after they have been discovered (you don't need any PC/AT or ISA devices for a PCI configuration space scan), but that may be tough, so I've thought of an alternative.

The evil about legacy devices is "they're just there". That is you have to know about their existence and in some case can't even do real probing or the device will react in really nasty ways.

The existence of an ELCR register is not documented for this machine. Anyway, the approach I've choosen is safe even we it exists.

Quoted 12 lines

Also the map of legacy resources reserved is actually incomplete -- PIC's
ELCR is missing for example.

Quoted 5 lines
Quoted 2 lines

Things start being tricky once you have to use such an offset for DMA
transfers as well...

True, but that's dealt with elsewhere. And I never claimed the mess that PCI used to be in 2.4 has yet been fully cleaned up yet, more work to do.

Sure, I'm more or less happy about the new code. I'm just warning about
possible pitfalls.

Quoted 8 lines

There is one more problem with the PCI resource manager -- it messes with
BARs of host bridges without asking for permission (which is often a no-no, as any messing with host bridges in general). I have a patch for it I'm currently trying to push to the PCI maintainer. See the thread at "http://www.uwsg.iu.edu/hypermail/linux/kernel/0412.1/0484.html" if interested. It's already being done correctly for i386 (some would probably argue that's what's most important) and ppc which use their own resource managers instead of the generic one.

That's a problem on many system controllers such as the GT-64120, all the other Galileo / Marvell stuff and more. The hackish solution of doing it in the pci_ops we're currently isn't a good one.

A comment on your patch. For most configurations it's ok and certainly better than hacking the pci_ops. At least on the GT-64120 it's possible to configure the device as a memory device and that will make your patch fail. Also there is the case where the GT-64120 or similar system controllers are used on a PCI card. For the CPU on the card it'll be the host bridge; for the host system just yet another PCI device.

Ralf

#6 lore raw
Quoted 5 lines · Ralf Baechle
Quoted 3 lines

I know and I consider it a bug. The correct way would be setting the
start at 0 and if avoiding the first kB of space was necessary, setting PCIBIOS_MIN_IO to 0x1000.

PCIBIOS_MIN_IO is the same value for all busses. That can turn out a bit

??? It's a variable and the IP32 port already modifies it.

Quoted 1 line

kludgy so I'm not much of a friend of it.

Probably the best way of dealing with an incomplete resource setup.
There is nothing wrong with setting BARs in the first kB of I/O space, but this way you may overlay an unregistered ISA-bridge device with another one. Of course it's PCI-ISA bridges that should have BARs for their resources, but I guess it's not going to happen. Though perhaps they could be created artificially in our PCI layer...

Quoted 11 lines
Quoted 7 lines

I think it's going to matter if a PCI-ISA bridge is behind one or more
PCI-PCI bridges. I have an idea of a fix, but my initial approach haven't worked particularly well and I've had no time to dig into it further. Actually, on systems with PCI I think reserving legacy resources should happen only after they have been discovered (you don't need any PC/AT or ISA devices for a PCI configuration space scan), but that may be tough, so I've thought of an alternative.

The evil about legacy devices is "they're just there". That is you have to know about their existence and in some case can't even do real probing or the device will react in really nasty ways.

No need to probe. The PCI-ISA bridge should reserve the entire
motherboard I/O range of 0x00 - 0xff plus any other non-ISA one that it knows it uses. ISA I/O ranges (with bits 8 or 9 non-zero) are already excluded implicitly.

Quoted 1 line

The existence of an ELCR register is not documented for this machine.

What kind of PCI-ISA bridge is it? You need ELCR whenever you want to
route PCI interrupts to some of i8259A inputs to set the trigger mode correctly. It's an EISA invention -- with the original i8259A you could only set up the trigger mode globally per chip (using ICW1).

Quoted 1 line

Anyway, the approach I've choosen is safe even we it exists.

Have you tried with a PCI-PCI bridge in the middle? Or do you have a
theoretical proof our code handles it right? ;-)

Quoted 9 lines
Quoted 8 lines

There is one more problem with the PCI resource manager -- it messes with
BARs of host bridges without asking for permission (which is often a no-no, as any messing with host bridges in general). I have a patch for it I'm currently trying to push to the PCI maintainer. See the thread at "http://www.uwsg.iu.edu/hypermail/linux/kernel/0412.1/0484.html" if interested. It's already being done correctly for i386 (some would probably argue that's what's most important) and ppc which use their own resource managers instead of the generic one.

Quoted 3 lines

That's a problem on many system controllers such as the GT-64120, all the other Galileo / Marvell stuff and more. The hackish solution of doing it in the pci_ops we're currently isn't a good one.

Sure -- I have a couple of reverts for these hacks ready.

Quoted 3 lines

A comment on your patch. For most configurations it's ok and certainly better than hacking the pci_ops. At least on the GT-64120 it's possible to configure the device as a memory device and that will make your patch

That's intended. If it's not a host bridge (implying it's an option
device/card) it should not be treated as such.

Quoted 3 lines

fail. Also there is the case where the GT-64120 or similar system controllers are used on a PCI card. For the CPU on the card it'll be the host bridge; for the host system just yet another PCI device.

Well in case of PCI cards they really should have their class codes set
to something different from a host bridge. AFAIK PCI is not meant as a star interconnect between hosts -- it's meant to have a tree topology rooted at a host bridge (that may be a bit different with the introduction of HyperTransport and its dual-hosted chains, but it's probably too early to worry about it). Of course you may have multiple host bridge devices in a system with peer host bridges, but then you just have multiple PCI buses with no path connecting them.

Anyway if a setup with host bridges as option devices really existed, it
should probably be handled by the CPU and the associated firmware running behind such bridges and if that was impossible, then by the driver for the device. Do you think such a setup is probable? How about putting such an option into an i386-based system? It would skip BARs on host bridges anyway. ;-)

Maciej

#7 lore raw
In reply to Ralf Baechle
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · Ralf Baechle

I know and I consider it a bug. The correct way would be setting the start at 0 and if avoiding the first kB of space was necessary, setting PCIBIOS_MIN_IO to 0x1000.

ralf> PCIBIOS_MIN_IO is the same value for all busses. That can turn ralf> out a bit kludgy so I'm not much of a friend of it.

BTW, yenta_socket driver uses PCIBIOS_MIN_IO and PCIBIOS_MIN_MEM, so these variables should be exported?

arch/mips/pci/pci.c +2 -0
@@ -294,6 +294,8 @@
 
 #ifdef CONFIG_HOTPLUG
 EXPORT_SYMBOL(pcibios_resource_to_bus);
+EXPORT_SYMBOL(PCIBIOS_MIN_IO);
+EXPORT_SYMBOL(PCIBIOS_MIN_MEM);
 #endif
 
 char *pcibios_setup(char *str)

---
Atsushi Nemoto

#8 lore raw
Quoted 2 lines · Atsushi Nemoto

BTW, yenta_socket driver uses PCIBIOS_MIN_IO and PCIBIOS_MIN_MEM, so these variables should be exported?

Yes.

Ralf