Skip to contentSkip to conversation

Linux running on a PCI Option device?

DISCUSSION Greg Ungerer 5 participants 13 messages Feb 9, 2006 – Mar 6, 2006

lore
Share & tools
lore
#0 lore raw

Hi Jon,

Quoted 1 line · Jon Ringle

I am working on a new board that will have Linux running on an xscale processor. This board will be a PCI Option device. I currently have a IXDP465 eval board which has a PCI Option connector that I will use for prototyping. From what I can tell so far, Linux wants to scan the PCI bus for devices as if it is the PCI host. Is there any provision in Linux so that it can take on the role of a PCI option rather than a PCI host?

Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in the function ixp4xx_pci_preinit().

It does a check on whether the PCI bus is configured as HOST or not. I don't know if that code support is enough for it all to work right though (I certainly haven't tried it on either the 425 or 465...)

Regards
Greg

------------------------------------------------------------------------

Greg Ungerer  --  Chief Software Dude       EMAIL:     [email protected]
SnapGear -- a CyberGuard Company            PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com
#1 lore raw
Quoted 10 lines · Greg Ungerer

Hi Jon,

Quoted 6 lines · Jon Ringle

I am working on a new board that will have Linux running on an xscale processor. This board will be a PCI Option device. I currently have a IXDP465 eval board which has a PCI Option connector that I will use for prototyping. From what I can tell so far, Linux wants to scan the PCI bus for devices as if it is the PCI host. Is there any provision in Linux so that it can take on the role of a PCI option rather than a PCI host?

Quoted 6 lines

Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in the function ixp4xx_pci_preinit().

It does a check on whether the PCI bus is configured as HOST or not. I don't know if that code support is enough for it all to work right though (I certainly haven't tried it on either the 425 or 465...)

Something that I don't quite understand is how I'm supposed to make vendor Id information available to the PCI host. Any ideas there?

TIA,

Jon

#2 lore raw

Hi Jon,

Quoted 12 lines · Jon Ringle
Quoted 11 lines · Greg Ungerer
Quoted 6 lines · Jon Ringle

I am working on a new board that will have Linux running on an xscale processor. This board will be a PCI Option device. I currently have a IXDP465 eval board which has a PCI Option connector that I will use for prototyping. From what I can tell so far, Linux wants to scan the PCI bus for devices as if it is the PCI host. Is there any provision in Linux so that it can take on the role of a PCI option rather than a PCI host?

Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in the function ixp4xx_pci_preinit().

Quoted 6 lines
Quoted 3 lines

It does a check on whether the PCI bus is configured as HOST or not. I don't know if that code support is enough for it all to work right though (I certainly haven't tried it on either the 425 or 465...)

Something that I don't quite understand is how I'm supposed to make vendor Id information available to the PCI host. Any ideas there?

No, sorry, no idea. I haven't looked at it in that much detail.

Regards
Greg

------------------------------------------------------------------------

Greg Ungerer  --  Chief Software Dude       EMAIL:     [email protected]
SnapGear -- a CyberGuard Company            PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com
#3 lore raw
In reply to Greg Ungerer
Quoted 10 lines · Greg Ungerer

Hi Jon,

Quoted 6 lines · Jon Ringle

I am working on a new board that will have Linux running on an xscale processor. This board will be a PCI Option device. I currently have a IXDP465 eval board which has a PCI Option connector that I will use for prototyping. From what I can tell so far, Linux wants to scan the PCI bus for devices as if it is the PCI host. Is there any provision in Linux so that it can take on the role of a PCI option rather than a PCI host?

Quoted 6 lines

Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in the function ixp4xx_pci_preinit().

It does a check on whether the PCI bus is configured as HOST or not. I don't know if that code support is enough for it all to work right though (I certainly haven't tried it on either the 425 or 465...)

When I have the IXDP465 in PCI Option mode, Linux still writes to pci configuration space which confuses the heck out of the PCI Host (Windows 2003). What do I need to do in order to have Linux work as a PCI option but still not mess with the pci configuration, and leave that task to the PCI Host?

Jon

#4 lore raw
Quoted 18 lines · Jon Ringle
Quoted 10 lines · Greg Ungerer

Hi Jon,

Quoted 6 lines · Jon Ringle

I am working on a new board that will have Linux running on an xscale processor. This board will be a PCI Option device. I currently have a IXDP465 eval board which has a PCI Option connector that I will use for prototyping. From what I can tell so far, Linux wants to scan the PCI bus for devices as if it is the PCI host. Is there any provision in Linux so that it can take on the role of a PCI option rather than a PCI host?

Quoted 6 lines

Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in the function ixp4xx_pci_preinit().

It does a check on whether the PCI bus is configured as HOST or not. I don't know if that code support is enough for it all to work right though (I certainly haven't tried it on either the 425 or 465...)

Quoted 7 lines

When I have the IXDP465 in PCI Option mode, Linux still writes to pci configuration space which confuses the heck out of the PCI Host (Windows 2003). What do I need to do in order to have Linux work as a PCI option but still not mess with the pci configuration, and leave that task to the PCI Host?

Jon

But anything on the PCI bus will get written, at least to find out the length of the address space. Please read about the PCI System Architecture. There are several writes that are mandatory. If somebody is attempting to configure the PCI devices, the following will occur.

(1) The BIOS will find some available address-space and put it into any base-address register that has memory-space enabled in the command register.

(2) The BIOS will find some available I/O space and put it into a base-address register, too.

This all occurs long before any OS is booted. These writes will occur. Then, the addresses may be re-written by any OS including Linux. Additional writes will occur when the device is enabled and when the cache-line size or the latency timer is written.

If the device is a host, and you don't want the BIOS to hack with it, then you need to turn off writes from the the host-bus side. This is something your FPGA designer needs to provide.

Usually, option boards use a PCI interface chip such as a PLX PCI-9656BA. It would not normally BE a PCI device on the Host side. The host CPU would communicate with it on a local bus.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.54 BogoMips). Warning : 98.36% of all statistics are fiction, book release in April. _


****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

#5 lore raw
Quoted 4 lines · linux-os \(Dick Johnson\)

Usually, option boards use a PCI interface chip such as a PLX PCI-9656BA. It would not normally BE a PCI device on the Host side. The host CPU would communicate with it on a local bus.

Correct but IPX4xx CPU has a PCI bus interface built in (in addition to interfaces for ROM, RAM, GPIO, Ethernet and serial).

Signature · 1 line
Krzysztof Halasa
#6 lore raw
In reply to linux-os (Dick Johnson)
Quoted 7 lines · linux-os (Dick Johnson)
Quoted 6 lines · Jon Ringle

When I have the IXDP465 in PCI Option mode, Linux still writes to pci configuration space which confuses the heck out of the PCI Host (Windows 2003). What do I need to do in order to have Linux work as a PCI option but still not mess with the pci configuration, and leave that task to the PCI Host?

Quoted 7 lines
Quoted 1 line

Jon

But anything on the PCI bus will get written, at least to find out the length of the address space. Please read about the PCI System Architecture. There are several writes that are mandatory. If somebody is attempting to configure the PCI devices, the following will occur.

Thanks for the book reference. I am reading it.

Quoted 9 lines

(1) The BIOS will find some available address-space and put it into any base-address register that has memory-space enabled in the command register.

(2) The BIOS will find some available I/O space and put it into a base-address register, too.

This all occurs long before any OS is booted. These writes will occur.

That's all well and fine. However, I think I need to explain a bit better the hardware topology here.

             +--------------------+
             |     POST/BIOS      |
             |    Windows 2003    |
             |       Pentium      |
             |      PCI Host      |
             |      PCI Arbiter   |
             +--------------------+
                       |
<-------- PCI bus ---------------------------------->
    |                       |
+---------+           +---------------------------+    
| Various |           | IXDP465 (PCI option mode) |
|  other  |           |         Redboot           |
|   PCI   |           |      Linux 2.6.16-rc5     |
| Devices |           +---------------------------+
+---------+

When power is applied to the system, several things are happening simultaneously, as I understand it:

1) The POST/BIOS code executes on the Pentium side that should do the actions you describe above.
2) The PCI devices are configuring themselves.
3) The IXDP465 is executing Redboot bootloader, which hands off control to Linux.

As it turns out, Linux completes it's bootup before Windows bootup even begins, and it seems that Linux changes the configuration of the various other PCI devices that happen to be on the system as well. I need to get Linux to leave the configuration of other PCI devices it finds alone. It should only mess with it's own configuration. Why should Linux need to change the configuration of other PCI devices when it is fulfilling the role of a PCI device itself?

Thanks,

Jon

#7 lore raw
Quoted 7 lines · Jon Ringle

As it turns out, Linux completes it's bootup before Windows bootup even begins, and it seems that Linux changes the configuration of the various other PCI devices that happen to be on the system as well. I need to get Linux to leave the configuration of other PCI devices it finds alone. It should only mess with it's own configuration. Why should Linux need to change the configuration of other PCI devices when it is fulfilling the role of a PCI device itself?

Have you disabled CONFIG_PCI?

CONFIG_PCI is the configuration option for a PCI host, just as CONFIG_USB is the configuration option for a USB host. Linux contains code for CONFIG_USB_GADGET, but what you need is the non-existent CONFIG_PCI_GADGET.

If you're running on a PCI option device (unless using a 21555 non-transparent bridge), you need to disable CONFIG_PCI and write your own driver for the PCI option device functionality.

Signature · 1 line
Adrian Cox <[email protected]>
#8 lore raw
Quoted 11 lines · Adrian Cox
Quoted 7 lines · Jon Ringle

As it turns out, Linux completes it's bootup before Windows bootup even begins, and it seems that Linux changes the configuration of the various other PCI devices that happen to be on the system as well. I need to get Linux to leave the configuration of other PCI devices it finds alone. It should only mess with it's own configuration. Why should Linux need to change the configuration of other PCI devices when it is fulfilling the role of a PCI device itself?

Have you disabled CONFIG_PCI?

Quoted 8 lines

CONFIG_PCI is the configuration option for a PCI host, just as CONFIG_USB is the configuration option for a USB host. Linux contains code for CONFIG_USB_GADGET, but what you need is the non-existent CONFIG_PCI_GADGET.

If you're running on a PCI option device (unless using a 21555 non-transparent bridge), you need to disable CONFIG_PCI and write your own driver for the PCI option device functionality.

Another requirement that I have that makes it difficult for me to disable CONFIG_PCI is that the hardware component that is running Windows (and therefore the PCI host) is optional hardware. If the Windows part is not present, then the IXP will be configured (via hardware means) as a PCI host. So, I need to detect at run time whether the IXP is in PCI option or PCI host mode. If it is in PCI host mode then the code encapuslated by CONFIG_PCI must be available.

I can see now that I also have a need to have a "CONFIG_PCI_GADGET".

Jon

#9 lore raw
Quoted 5 lines · Jon Ringle
Quoted 3 lines · Adrian Cox

If you're running on a PCI option device (unless using a 21555 non-transparent bridge), you need to disable CONFIG_PCI and write your own driver for the PCI option device functionality.

Quoted 7 lines

Another requirement that I have that makes it difficult for me to disable CONFIG_PCI is that the hardware component that is running Windows (and therefore the PCI host) is optional hardware. If the Windows part is not present, then the IXP will be configured (via hardware means) as a PCI host. So, I need to detect at run time whether the IXP is in PCI option or PCI host mode. If it is in PCI host mode then the code encapuslated by CONFIG_PCI must be available.

Based on only a quick look at the code: if the Windows host is present, don't call pci_common_init() in ixdp425_pci_init(). You then need to write the entire CONFIG_PCI_GADGET code, and ensure that you only enable it if the Windows host is present.

The greater problem is that these changes may work all the way up into your application. If the Windows host is present, then you need its cooperation to find other devices on the PCI bus.

Signature · 1 line
Adrian Cox <[email protected]>
#10 lore raw
Quoted 2 lines · Adrian Cox

Based on only a quick look at the code: if the Windows host is present, don't call pci_common_init() in ixdp425_pci_init().

Doing this will prevent the code in ixp4xx_pci_preinit() from executing which handles some initialization for both PCI host and option modes. Should I go ahead and explicitly call ixp4xx_pci_preinit() from ixdp425_pci_init() if in PCI option mode?

Jon

#11 lore raw
Quoted 8 lines · Jon Ringle
Quoted 2 lines · Adrian Cox

Based on only a quick look at the code: if the Windows host is present, don't call pci_common_init() in ixdp425_pci_init().

Doing this will prevent the code in ixp4xx_pci_preinit() from executing which handles some initialization for both PCI host and option modes. Should I go ahead and explicitly call ixp4xx_pci_preinit() from ixdp425_pci_init() if in PCI option mode?

At this point I have to give up - I'm not an ixp4xx expert. While I have developed a PCI option device under Linux, it was a PowerPC with a PLX bridge and some custom logic.

Signature · 1 line
Adrian Cox <[email protected]>
#12 lore raw
In reply to Jon Ringle

Hi Jon,

Quoted 7 lines · Jon Ringle
Quoted 2 lines · Adrian Cox

Based on only a quick look at the code: if the Windows host is present, don't call pci_common_init() in ixdp425_pci_init().

Doing this will prevent the code in ixp4xx_pci_preinit() from executing which handles some initialization for both PCI host and option modes. Should I go ahead and explicitly call ixp4xx_pci_preinit() from ixdp425_pci_init() if in PCI option mode?

The older 2.4 kernel code for the IXP425 support pretty much did it this way (which is what makes it look like it supports the device in option mode).

Code snippet from ixp425_pci_init() in arch/arm/mach-ixp425/ixp425-pci.c (from patches for 2.4 kernels to support ixp425):

       ...
       if (ixp425_pci_is_host())
        {
                local_write_config_word(PCI_COMMAND, PCI_COMMAND_MASTER |
                        PCI_COMMAND_MEMORY);

                DBG("allocating hose\n");
                hose = pcibios_alloc_controller();
                if (!hose)
                panic("Could not allocate PCI hose");

                hose->first_busno = 0;
                hose->last_busno = 0;
                hose->io_space.start = 0;
                hose->io_space.end = 0xffffffff;
                hose->mem_space.start = 0x48000000;
                hose->mem_space.end = 0x4bffffff;

                /* autoconfig the bus */ DBG("AUTOCONFIG\n");
                hose->last_busno = pciauto_bus_scan(hose, 0);

                /* scan the bus */
                DBG("SCANNING THE BUS\n");
                pci_scan_bus(0, &ixp425_ops, sysdata);
         }

Pretty much the rest of the PCI init is the same, it just doesn't do the bus scan.

Regards
Greg

------------------------------------------------------------------------

Greg Ungerer  --  Chief Software Dude       EMAIL:     [email protected]
SnapGear -- a Secure Computing Company      PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com