problems with pci bus on a pm520 board

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Folks,
I'm having trouble with the pci bus on a microsys pm520 (with mpc5200b) board the last days. There're a Ethernet controller (gd82551, supported by intel e100 driver) and an USB controller (philips isp1536bm philips, supported by hcd-ohci) on the pci bus. mpc52xx_pci is compiled in. I modified the interrupt map in the device tree, so that the pci bus got correctly scanned. If I don't activate any device driver except the one for the host bridge, the system starts correctly and the sysfs pci bus get the correct entries, which looks like this:

root@testboard:~ ls /sys/bus/pci/devices/
0000:00:10.0 0000:00:11.0 0000:00:11.1 0000:00:11.2 0000:00:12.0

while 10.0 ist the ethernet controller, 11.* is the usb controller and 12.0 is the hostbridge. Seemed to me that the mpc52xx pci driver is working properly. However, if I compile the usb driver in and try to boot. The system stucks somewhere at ohci_init or ohci_run without doing anything till I switch the power off. I attached a bdi to the system and found out that the system always stucks at the instruction lwbrx, which is called by ohci_readl(). Seemed that the usb driver failed to acces some mapped io memory address somehow. The funny thing is that the e100 driver also does some reading calls on iomem in the initialisation, such as readb() in e100_eeprom_read(). It however doesn't cause system freezing and the driver quits with an error message that the eeprom is corrupted. I suppose the problem is somehow still on the pci configuration in the device tree. I took the dts file of the lite5200 had to do quite some modifications to get the pci bus working at all, however I still don't get the idea of the mapping thing completely. Hence I suppose the file is still not OK. Anybody could help me with an idea where I still could look for or give me some detailed information on the format of device tree?

Along with this mail I attached my dts file. I am using the git kernel 2.6.21-rc1-dirty, compiled with gcc4.1.2

Thanks a lot for any suggestions.

Cheers
Luotao Fu
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFF5ez9iruQY+UyUmERAr3gAKCfJbVSi3WbP8oTyVUzYpCB3drANgCgqQKh ZVCJSd9FbaTGhdn1gQ9bhZA=
=MBxJ
-----END PGP SIGNATURE-----

Three OHCIs or is one an EHCI?

32-bit PCI read I suppose. Use the BDI to figure out what address is accessed, and what source code corresponds to this access.

Yeah.

Linux normally is perfectly happy if you leave out all of the PCI devices from your device tree (except the root bridges of course).

Sure, post it.

No you didn't.

Segher

Hi,

exactly, two ohci and one ehci.

right, I'll take a look as soon as I can grab the bdi.

I didn't make any extra entry for the pci devices.

Ooops. sorry. I'll attach it to this mail.

P.S
forgot the cc in my first reply mail, sorry for double posting Segher. It's getting late here.....
;-)

Thanx
Cheers
Luotao Fu

/*
*
* JohnDeere Vehicleserver (based on pm520 board by microsys) devicetree file
* Copyright 2006 Pengutronix
* Luotao Fu <[email protected]>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the License, or (at your
* option) any later version.
*/

/ {
	model = "testboard";
	compatible = "testboard\0mpc5200b\0mpc5200";
	#address-cells = <1>;
	#size-cells = <1>;

	cpus {
		#cpus = <1>;
		#address-cells = <1>;
		#size-cells = <0>;

		PowerPC,5200@0 {
			device_type = "cpu";
			reg = <0>;
			d-cache-line-size = <20>;
			i-cache-line-size = <20>;
			d-cache-size = <4000>;		// L1, 16K
			i-cache-size = <4000>;		// L1, 16K
			timebase-frequency = <1e84800>;	// 32MHz
			bus-frequency = <7de2900>;	// 132MHz
			clock-frequency = <179a7b00>;	// 396MHz
			32-bit;
		};
	};

	memory {
		device_type = "memory";
		reg = <00000000 08000000>;	// 128MB
	};

	soc5200@f0000000 {
		#interrupt-cells = <3>;
		device_type = "soc";
		ranges = <0 f0000000 f0010000>;
		reg = <f0000000 00010000>;
		bus-frequency = <7de2900>;	// 132MHz

		cdm@200 {
			compatible = "mpc5200b-cdm\0mpc5200-cdm";
			reg = <200 38>;
		};

		pic@500 {
			// 5200 interrupts are encoded into two levels;
			linux,phandle = <500>;
			interrupt-controller;
			#interrupt-cells = <3>;
			device_type = "interrupt-controller";
			compatible = "mpc5200b-pic\0mpc5200-pic";
			reg = <500 80>;
			built-in;
		};

		gpt@600 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <600 10>;
			interrupts = <1 9 0>;
			interrupt-parent = <500>;
		};

		gpt@610 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <610 10>;
			interrupts = <1 a 0>;
			interrupt-parent = <500>;
		};

		gpt@620 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <620 10>;
			interrupts = <1 b 0>;
			interrupt-parent = <500>;
		};

		gpt@630 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <630 10>;
			interrupts = <1 c 0>;
			interrupt-parent = <500>;
		};

		gpt@640 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <640 10>;
			interrupts = <1 d 0>;
			interrupt-parent = <500>;
		};

		gpt@650 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <650 10>;
			interrupts = <1 e 0>;
			interrupt-parent = <500>;
		};

		gpt@660 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <660 10>;
			interrupts = <1 f 0>;
			interrupt-parent = <500>;
		};

		gpt@670 {	// General Purpose Timer
			compatible = "mpc5200b-gpt\0mpc5200-gpt";
			device_type = "gpt";
			reg = <670 10>;
			interrupts = <1 10 0>;
			interrupt-parent = <500>;
		};

		rtc@800 {	// Real time clock
			compatible = "mpc5200b-rtc\0mpc5200-rtc";
			device_type = "rtc";
			reg = <800 100>;
			interrupts = <1 5 0 1 6 0>;
			interrupt-parent = <500>;
		};

		mscan@900 {
			device_type = "mscan";
			compatible = "mpc5200b-mscan\0mpc5200-mscan";
			interrupts = <2 11 0>;
			interrupt-parent = <500>;
			reg = <900 80>;
		};

		mscan@980 {
			device_type = "mscan";
			compatible = "mpc5200b-mscan\0mpc5200-mscan";
			interrupts = <2 12 0>;
			interrupt-parent = <500>;
			reg = <980 80>;
		};

		gpio@b00 {
			compatible = "mpc5200b-gpio\0mpc5200-gpio";
			reg = <b00 40>;
			interrupts = <1 7 0>;
			interrupt-parent = <500>;
		};

		gpio-wkup@b00 {
			compatible = "mpc5200b-gpio-wkup\0mpc5200-gpio-wkup";
			reg = <c00 40>;
			interrupts = <1 8 0 0 3 0>;
			interrupt-parent = <500>;
		};

		pci@0d00 {
			#interrupt-cells = <1>;
			#size-cells = <2>;
			#address-cells = <3>;
			device_type = "pci";
			compatible = "mpc5200b-pci\0mpc5200-pci";
			reg = <d00 100>;
			interrupt-map-mask = <f800 0 0 7>;
			interrupt-map = <8000 0 0 1 500 1 1 3 // e100, 10.0
			                 8000 0 0 2 500 1 1 3
			                 8000 0 0 3 500 1 1 3
			                 8000 0 0 4 500 1 1 3

			                 8800 0 0 1 500 1 0 3 // usb 11.0, 11.1, 11.2
			                 8800 0 0 2 500 1 2 3
			                 8800 0 0 3 500 1 3 3
			                 8800 0 0 4 500 0 0 3>;
			clock-frequency = <0>; // From boot loader
			interrupts = <2 8 0 2 9 0 2 a 0>;
			interrupt-parent = <500>;
			bus-range = <0 0>;
			ranges = <42000000 0 80000000 80000000 0 20000000
			          02000000 0 a0000000 a0000000 0 10000000
			          01000000 0 00000000 b0000000 0 01000000>;
		};

		spi@f00 {
			device_type = "spi";
			compatible = "mpc5200b-spi\0mpc5200-spi";
			reg = <f00 20>;
			interrupts = <2 d 0 2 e 0>;
			interrupt-parent = <500>;
		};

		bestcomm@1200 {
			device_type = "dma-controller";
			compatible = "mpc5200b-bestcomm\0mpc5200-bestcomm";
			reg = <1200 80>;
			interrupts = <3 0 0  3 1 0  3 2 0  3 3 0
			              3 4 0  3 5 0  3 6 0  3 7 0
			              3 8 0  3 9 0  3 a 0  3 b 0
			              3 c 0  3 d 0  3 e 0  3 f 0>;
			interrupt-parent = <500>;
		};

		xlb@1f00 {
			compatible = "mpc5200b-xlb\0mpc5200-xlb";
			reg = <1f00 100>;
		};

		// PSC1 is uart
		serial@2000 {		// PSC1
			device_type = "serial";
			compatible = "mpc5200b-psc-uart\0mpc5200-psc-uart";
			reg = <2000 100>;
			interrupts = <2 1 0>;
			interrupt-parent = <500>;
		};

		// PSC6 is uart
		serial@2c00 {		// PSC6
			device_type = "serial";
			compatible = "mpc5200b-psc-uart\0mpc5200-psc-uart";
			reg = <2c00 100>;
			interrupts = <2 4 0>;
			interrupt-parent = <500>;
		};

		ethernet@3000 {
			device_type = "network";
			compatible = "mpc52xx-fec\0mpc5200-ethernet";
			reg = <3000 800>;
			mac-address = [ 02 03 04 05 06 07 ]; // Bad!
			interrupts = <2 5 0>;
			interrupt-parent = <500>;
		};

		i2c@3d00 {
			device_type = "i2c";
			compatible = "mpc5200b-i2c\0mpc5200-i2c";
			reg = <3d00 40>;
			interrupts = <2 f 0>;
			interrupt-parent = <500>;
		};

		i2c@3d40 {
			device_type = "i2c";
			compatible = "mpc5200b-i2c\0mpc5200-i2c";
			reg = <3d40 40>;
			interrupts = <2 10 0>;
			interrupt-parent = <500>;
		};
		sram@8000 {
			device_type = "sram";
			compatible = "mpc5200b-sram\0mpc5200-sram\0sram";
			reg = <8000 4000>;
		};
	};
};

You removed the original copyright notice, please don't; it might even be illegal to do so, depending on where you live/work. Just add your own, don't remove the original.

8000 is device 16, not 10

and 8800 is device 17

Your tree looks reasonable otherwise, I didn't look at the details though.

Hope that helps,

Segher

Hi,

OK, thanx for the hint.

Hmm, this is strange. I put some debug printks in the of parser prom_parser.c in the function of_irq_map_raw somewhere in the machting loop, which prints the imap and address being parsed. So that the lines at about 834 now looks like this:
match = 1;
for (i = 0; i < addrsize && match; ++i) {
<snip>

	printk(" -> imap %x addr %x mask %x\n", imap[i], addr[i], mask);
}
for (; i < (addrsize + intsize) && match; ++i) {
<snip>
	printk(" -> imap %x intspec %x mask %x\n", imap[i],

intspec[i-addrsize], mask);
}
<snip>
printk(" -> match=%d (imaplen=%d), addr %d \n", match, imaplen, addr[i]);

With these printks I got for example for device 10.0 the output

Try to map irq for 0000:00:10.0...
PCI: pdev bus number: 0, devfn: 128
of_irq_map_raw: par=/soc5200@f0000000/pci@0d00,intspec=[0x00000001 0x00008000...],ointsize=1
of_irq_map_raw: ipar=/soc5200@f0000000/pci@0d00, size=1
-> imap 8000 addr 8000 mask f800
-> imap 0 addr 0 mask 0
-> imap 0 addr 0 mask 0
-> imap 1 intspec 1 mask 7
-> match=1 (imaplen=60), addr 1
-> newintsize=3, newaddrsize=0
-> imaplen=56
-> new parent: /soc5200@f0000000/pic@500
-> got it !
-> got one, spec 3 cells (0x00000001...) on /soc5200@f0000000/pic@500 -> virq :65

As I have noticed, the array addr is filled in of_irq_map_pci() by laddr[0] = (pdev->bus->number << 16) | (pdev->devfn << 8); Since my pdev->bus->number is 0. I got the address 8000 here. Is the pdev->bus->number actually supposed to be 10 for device 10? I had some trouble tracking down how the bus number is initialized. It might be nice if you or some one could give me a hint on this.

Thanx again
cheers
Luotao Fu

<snip>

8000 is device 16 is 0x10 hex. PCI device numbers are usually written in decimal. Is your device 10 == 0x0a or 16 == 0x10?

Segher

I took the number 10 out of the sysfs bus entry. As they are printed as %x there. The 10 is a 0x10 = 16. Hence 8000 seemed to be OK then. How is this actually supposed to be calculated? (As you can see, I know only little about pci stuffs) As I wrote in the last mail. The addr is calculated by the parser with
laddr[0] = (pdev->bus->number << 16) | (pdev->devfn << 8) Since we have only one pci bus here, I suppose the bus number should be
0. However I'm wondering about what does this devfn stands for?

I now finally made some processes. I made a really stupid failure and put the pci bus in 66 MhZ, the USB controller doesn't like it at all and went dow the hill.... Now I run the pci bus with 33MhZ and the USB controller is working all fine now. However I still get troubles with the intel gd82551t ethernet controller. I connected it to the network and generated some activities, I even saw the phy blinking. The watchdog also works properly and reports link status. However the controller just won't generate or receive any interrupts. Both the Becker Driver and the intel driver just won't work.

I tried the irqpoll option in the kernel command line, which also didn't help. Seemed that the no interrupts is coming through at all, which is really strange since the controller seemed to be corretly configured.

I noticed that the L1 information are different for the ethernet and usb controllers according to /proc/interrupts. for eth1 it says Main and for usb it says IRQ[0-3], might this be the reason of my problem?

root@testboard:~ cat /proc/interrupts
CPU0
64: 0 MPC52xx Main Edge eth1
65: 2 MPC52xx IRQ[0-3] Edge ehci_hcd:usb1,
ohci_hcd:usb2, ohci_hcd:usb3

129:        903  MPC52xx Peripherals Edge      mpc52xx_psc_uart
133:          7  MPC52xx Peripherals Edge      mpc52xx-fec_ctrl
145:          0  MPC52xx Peripherals Edge      can0
146:          0  MPC52xx Peripherals Edge      can1
192:       4004  MPC52xx SDMA Edge      mpc52xx-fec_rx
193:       3343  MPC52xx SDMA Edge      mpc52xx-fec_tx

BAD: 0

The interrupt pin of the ethernet controller is connected to irq0 pin of the MPC5200B while the interrupt pin of the usb controller is connected to irq1 of the MPC5200B. Since the usb has 65 and is working, I suppose strongly that 64 for the ethernet controller should also be correct.

The pci node of my the device tree now looks like this:

pci@0d00 {
	#interrupt-cells = <1>;
	#size-cells = <2>;
	#address-cells = <3>;
	device_type = "pci";
	compatible = "mpc5200b-pci\0mpc5200-pci";
	reg = <d00 100>;
	interrupt-map-mask = <f800 0 0 7>;
	interrupt-map = <8000 0 0 1 500 1 0 3 // e100, 10.0               			
8000 0 0 2 500 1 1 3
			 8000 0 0 3 500 1 2 3
			 8000 0 0 4 500 1 3 3

			 8800 0 0 1 500 1 1 3 // usb 11.*
			 8800 0 0 2 500 1 2 3
			 8800 0 0 3 500 1 3 3
			 8800 0 0 4 500 0 0 3>;
	clock-frequency = <0>; // From boot loader
	interrupts = <2 8 0 2 9 0 2 a 0>;
	interrupt-parent = <500>;
	bus-range = <0 0>;
	ranges = <42000000 0 80000000 80000000 0 20000000
		  02000000 0 a0000000 a0000000 0 10000000
		  01000000 0 00000000 b0000000 0 01000000>;
};

Like I mentioned before. I quite did not get the idea of all the open firmware things. I'd be very appreciate if you could take a near look on this and tell me if you see anything wrong with this node.

Thanx a lot
cheers
Luotao Fu

Oh okay.

bus-dev-fn-offset is a 24-bit number; the high 8 bits are bus number, then 5 bits device number, 3 bits function number, and 8 bits offset. devfn is simply device and function number combined; so your 0x8000 is bus 0, device 16, function 0 (and offset 0).

Progress, good :-)

Something is wrong with either the interrupts in your device tree, or with the board wiring then.

I have no idea what L1 is?

Yeah, since...

...you say that the ethernet controller is on IRQ0, which would be Linux IRQ 65 as well.

Sounds like both should be 65 to me? Maybe someone who has experience with 5200 can chime in, I have never used it.

That says: device 16.0, PIRQA, goes to phandle 500 IRQ 1 (so far so good), priority 0? I believe priority should be one from 1..3 (but I could very well be mistaken).

That doesn't explain why Linux throws your interrupt at 64 instead of 65 though.

Oh and if you have an interrupt-map, you shouldn't have an interrupt-parent. Not that that causes your problem though.

It looks pretty much sane.

Enable DEBUG in arch/powerpc/kernel/prom_parse.c , maybe it'll tell you a bit more. Oh and read the 5200 datasheet again, make double sure which interrupt on the interrupt controller the four PCI interrupts connect to.

Good luck and have fun,

Segher