750 and Discovery II coherency with PCI....

Greetings.

I am writing a device driver for a PMC card that is nothing fancy. It is not a PCI-X card though. My Linux driver properly detects the device, 'ioremaps' the PCI IO and MEM regions then attempts to start writing the devices registers for setup and configuration. I am not getting valid reads and there appears to be some coherency issues. I have verified that no other drivers or resources in the system are utilizing the addresses that are returned to me from 'ioremap'. I have the following code snippet:

#define shunt_read_8(offset)		\
	*((volatile u8 *) (shunt_dev->ctrl_base + offset))
[SNIP]
		svalue[0] = shunt_read_8(SHUNT_ADDRESS + 0x220);
		svalue[1] = shunt_read_8(SHUNT_ADDRESS + 0x221);
		svalue[2] = shunt_read_8(SHUNT_ADDRESS + 0x222);
		svalue[3] = shunt_read_8(SHUNT_ADDRESS + 0x223);

		printk("0x%02x\n", svalue[0]);
		printk("0x%02x\n", svalue[1]);
		printk("0x%02x\n", svalue[2]);
		printk("0x%02x\n", svalue[3]);

If I read the byte addresses above, which are in the remapped PCI I/O address space and I get the following output:

0x00
0x00
0x00
0x00

which is of course, incorrect. I even tried using the 'in_8' macro functions in 'asm-ppc/io.h'. I get the same results. Now, if I instead do the following code snippet:

		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x221));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x222));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x223));

I now get the output of:

0x00
0x81
0x81
0x02
0x0e
0x0e

The first time I read 0x220, it is incorrect, but the following two times and the other remaining values are now correct. Unfortunately, this device is the only PMC device that I have to test. I do not have any other PMC cards. This behavior exists in both the 2.4 kernel and in my 2.6 kernel. The virtual addresses received from the 'ioremap' call are identical on both 2.4 and 2.6, just for reference. I tried adding in memory barriers and 'eieio' calls, but I did not expect those to make a difference...and it did not. Has anyone ever seen similar behavior? Preferably I would like to hear from people working with Discovery II parts. I have documentation under NDA, but I am still working my way through it. Thanks in advance.

-Steve

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Steve,
I have been using the disco II now for quite a while I have been able to make 4 different PCI cards work without many problems. Can you look at your memory windows and make sure the mappings are set up correctly? Is this a single or a dual processor? I would start by decoding the PCI memory windows that sounds like where you might be getting into trouble. Do you have a PCI bus analyzer handy to make sure you are putting the right addresses on the bus?
Thanks
Brian

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Steve,

You leave out a lot of pertinent information that would help us help you. For example, what board are you using? Is it a board with a known good Linux board port? What kernel base are you using (the *exact* tree and version of that tree; there are several 2.4 trees)? Are you sure that the PCI I/O & MEM space allocations are correct (e.g., did you run pci_auto)?

Somc comments follow...

PCI I/O space is already "ioremap'd" for you, don't do it again. Assuming your board code in arch/ppc/platforms is correct and you are using the correct PCI I/O space access routines (e.g., inb/outb and friends), it will just work. Make sure those assumptions are valid.

There are many, many linux drivers available for you to look at, please use them for examples on how to access PCI I/O and MEM space. Also, there is lots of good info in .../Documentation/*. It will be well worth your time to read through it.

What makes you think there are coherency issues?

Of course not, ioremap returns the virtual address from the virt->CPU phys mapping it just made. That's always going to look fine. Its the CPU physical, PCI I/O, and PCI MEM addresses that you need to make sure don't overlap.

Don't do that. Use the proper access routines that are already available to you.

Also, I see nothing remotely related to cache coherency here. You only talk about accessing registers in ioremap'd areas. Caches are not involved nor are you looking at areas in memory where your device DMA'd data into or out of.

Mark

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

There is no coherency issues with non cacheable (ioremap'ed) access of that kind... Note that if you are dealing with IO space, then you should use inX/outX accessors and not ioremap.

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/