Boot hangs "Probing PCI hardware"

I've found RHL7.2 boot sometimes hangs with "Probing PCI hardware" message on our Itanium based server.

I also conclude it is a kernel problem in PCI device initialization.

There is the following architectural difference between IA-32 and IA-64. IA-64 has SAPIC for interrupts and its messages are generated by Memory write transaction to System reserved region on PCI Bus, instead of using APIC and APIC Bus in IA-32.

Therefore, from this point of view, BAR(Base Address Register) opertions in PCI device initialization must be changed between IA-32 and IA-64, but it has not actually revised yet.

I'll explain that in detail below and also provide the patch. If somebody reflect it in Kernel source tree, I would be appreciated so much. I also hope the next Kernel-2.4.19 includes following fixes.

Our machine has two Itaniums (733MHz 2MB cache) and 512MB RAM. It also has three PCI Buses and the problem occurs in case of putting a video card (ATI Rage 128 GL) on the first PCI Bus which also has SAPIC.

I've also found it occurs on RHL7.1(Kernel-2.4.3-12), RHL7.2(Kernel-2.4.9-18) and the latest Kernel-2.4.18.

Next, I'll describe the error sequence.

There are some BAR operations in pci_read_bases() function of pci.c to get memory and IO space ranges assigned for each PCI device by writing 0xffffffff on BAR and reading it again. If an interrupt from SAPIC occurs during BAR operations, the first PCI Bus hangs.

In this case, the video device gets 64MB memory space from 0xFC000000 to 0xFFFFFFFF improperly, and it conflicts with System reserve region (0xFCxxxxxx - 0xFExxxxxx) for SAPIC interrupt messages. After that the video device reacts to an SAPIC interrupt improperly.

According to PCI specifications, it is necessary to disable Memory IO target function during BAR operations by setting zero to bit0/1 of PCI_COMMAND, but current pci_read_bases() doesn't do that.

I'd like to provide the following patch, and I've already found it works well on both RHL7.2 and Kernel-2.4.18.

*** pci.c.orig  Tue Apr  2 19:56:06 2002
--- pci.c       Tue Apr  2 19:56:14 2002
***************
*** 939,946 ****
--- 939,950 ----
 {
       unsigned int pos, reg, next;
       u32 l, sz;

(quoted diff) +4 -0

+       u16 pcicmd_sv;
       struct resource *res;
 
+       pci_read_config_word(dev, PCI_COMMAND, &pcicmd_sv);
+       pci_write_config_word(dev, PCI_COMMAND, pcicmd_sv & 0xfffc);
+
       for(pos=0; pos<howmany; pos = next) {
               next = pos+1;
               res = &dev->resource[pos];

***************
*** 1004,1009 ****
--- 1008,1014 ----

               }
               res->name = dev->name;
       }

+ pci_write_config_word(dev, PCI_COMMAND, pcicmd_sv);
}

void __devinit pci_read_bridge_bases(struct pci_bus *child)

Finally, I think this problem may occur theoretically in case of having memory consuming devices (more than 32MB) like a video card and SAPIC on the same PCI Bus. Do you know the same problem on a Lion box ?

PS.
I've already sent the same mail to [email protected].

Best regards,
Tatsuya Tsurukawa
Mitsubishi Electric

----
Tatsuya [email protected]

In this case, the video device gets 64MB memory space from 0xFC000000 to 0xFFFFFFFF improperly, and it conflicts with System reserve region (0xFCxxxxxx - 0xFExxxxxx) for SAPIC interrupt messages. After that the video device reacts to an SAPIC interrupt improperly.