Problem mapping small PCI memory space.
Hi all.
I'm writing a driver for a PCI board that exposes two memory spaces (out of the 6 IO address regions).
One of them is 1MB, and I can map it to user level without problems. The other one is only 512 bytes.
If I try to open it with /dev/mem, it returns EINVAL (the 1MB memory space is opened without any problem). If I try to expose it through mmap, mmap succeeds, but I only see garbage at user level. At kernel level, I can access that 512 bytes memory by using ioremap() on the physical address returned by pci_resource_start().
Are there any lower limits on the size of a PCI memory region?
Have a nice day
GV
Quoted 13 lines · Gianluca Varenni
Hi all.
I'm writing a driver for a PCI board that exposes two memory spaces (out of the 6 IO address regions).
One of them is 1MB, and I can map it to user level without problems. The other one is only 512 bytes.
If I try to open it with/dev/mem, it returns EINVAL (the 1MB memory space is opened without any problem). If I try to expose it through mmap, mmap succeeds, but I only see garbage at user level. At kernel level, I can access that 512 bytes memory by usingioremap()on the physical address returned bypci_resource_start().
Quoted 5 lines
Are there any lower limits on the size of a PCI memory region?
Have a nice day
GV
You impliment mmap() in your driver. It accesses the first megabyte as 256 pages. Then you tack on the additional page that your 512 bytes resides at. mmap() only works with pages. The pages must be ioremap_nocache and they must be reserved. The reserved part is important to have them visible from user-space using mmap.
That's IFF you really need to see the stuff in user-mode. Normally, you write a driver that accesses everything using the PCI primatives provided in the kernel, for the kernel.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.11.9 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.
----- Original Message ----- From: "Richard B. Johnson" <[email protected]>
To: "Gianluca Varenni" <[email protected]>
Cc: "Linux kernel" <[email protected]>
Sent: Thursday, May 19, 2005 12:43 PM
Subject: Re: Problem mapping small PCI memory space.
Quoted 27 lines · Gianluca Varenni
Quoted 14 lines
Hi all.
I'm writing a driver for a PCI board that exposes two memory spaces (out of the 6 IO address regions).
One of them is 1MB, and I can map it to user level without problems. The other one is only 512 bytes.
If I try to open it with/dev/mem, it returns EINVAL (the 1MB memory space is opened without any problem). If I try to expose it through mmap, mmap succeeds, but I only see garbage at user level. At kernel level, I can access that 512 bytes memory by usingioremap()on the physical address returned bypci_resource_start().Quoted 5 lines
Are there any lower limits on the size of a PCI memory region?
Have a nice day
GVYou impliment
mmap()in your driver. It accesses the first megabyte as 256 pages. Then you tack on the additional page that your 512 bytes resides at.mmap()only works with pages. The pages must beioremap_nocacheand they must be reserved. The reserved part is important to have them visible from user-space using mmap.
Quoted 3 lines
That's IFF you really need to see the stuff in user-mode. Normally, you write a driver that accesses everything using the PCI primatives provided in the kernel, for the kernel.
Do you have any link to a linux driver working exactly like this?
Have a nice day
GV
Quoted 7 lines
Cheers,
Dick Johnson
Penguin : Linux version 2.6.11.9 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.
Quoted 21 lines · Richard B. Johnson
Quoted 13 lines · Gianluca Varenni
Hi all.
I'm writing a driver for a PCI board that exposes two memory spaces (out of the 6 IO address regions).
One of them is 1MB, and I can map it to user level without problems. The other one is only 512 bytes.
If I try to open it with/dev/mem, it returns EINVAL (the 1MB memory space is opened without any problem). If I try to expose it through mmap, mmap succeeds, but I only see garbage at user level. At kernel level, I can access that 512 bytes memory by usingioremap()on the physical address returned bypci_resource_start().Quoted 5 lines
Are there any lower limits on the size of a PCI memory region?
Have a nice day
GV
Quoted 5 lines
You impliment
mmap()in your driver. It accesses the first megabyte as 256 pages. Then you tack on the additional page that your 512 bytes resides at.mmap()only works with pages. The pages must beioremap_nocacheand they must be reserved. The reserved part is important to have them visible from user-space using mmap.
There are a number of totally bogus statements above.
First of all, you don't have to implement mmap in your driver (though it would probably better to do so). You can perfectly map your PCI memory space from userland without help of a specific driver (see below).
Then, ioremap has absolutely nothing to do with the mapping done to userland (unless there is some gory x86 in there, but I really doubt it). ioremap_* has to do with mapping the memory in kernel space, not user space. Thus your statement "must be ioremap_nocache" is wrong.
I don't know why you think the pages should be reserved as well, damn, he's not mapping RAM, he's mapping PCI MMIO space, do you know what you are talking about at all ? There is usually no struct page for PCI MMIO space.
Quoted 3 lines
That's IFF you really need to see the stuff in user-mode. Normally, you write a driver that accesses everything using the PCI primatives provided in the kernel, for the kernel.
Oh well, Ginancula, here is what you can do:
- If you want to stick to /dev/mem, which is not recommended and may
not work on all platforms but is fine for a quick hack, just make sure you are mmap'ing a full page. That is, your 512 bytes will be 512-bytes-aligned somewhere in a page of 4k on most platforms, maybe more. You have to map this entire page, and then take an offset in the resulting mapping to the 512 bytes.
Basically, you use getpagesize() to get the page size, then you mmap your base_addr & ~(page_size - 1), and you access your data at the resulting address + base_addr & (page_size - 1)
- Best is to use the PCI mmap facility. The "old" one via /proc/bus/pci
or the "new one" via /sys/...path to your device.../resourceN. You directly mmap the resource file you want. HOWEVER, the alignement restriction will still be there, thus you should also look at the "resource" file to get the actual base address, and deduce the alignement, and then map a whole page and add the alignement offset to the result as for /dev/mem.
Ben.
Hi Benjamin.
The problem was exactly the one you described at the end of the mail: my PCI IO memory is not aligned at the beginning of a page, it's right in the middle of the page. So my mmap was working perfectly, but I was looking at the wrong place at user level (I looked at the beginning of the page, while my PCI Io memory was 2k away)....
Thank you for you help!!
Have a nice day
GV
----- Original Message ----- From: "Benjamin Herrenschmidt" <[email protected]>
To: <[email protected]>
Cc: "Gianluca Varenni" <[email protected]>; "Linux kernel" <[email protected]>
Sent: Friday, May 20, 2005 5:17 PM
Subject: Re: Problem mapping small PCI memory space.
Quoted 71 lines · Richard B. Johnson
Quoted 21 lines · Gianluca Varenni
Quoted 14 lines
Hi all.
I'm writing a driver for a PCI board that exposes two memory spaces (out of the 6 IO address regions).
One of them is 1MB, and I can map it to user level without problems. The other one is only 512 bytes.
If I try to open it with/dev/mem, it returns EINVAL (the 1MB memory space is opened without any problem). If I try to expose it through mmap, mmap succeeds, but I only see garbage at user level. At kernel level, I can access that 512 bytes memory by usingioremap()on the physical address returned bypci_resource_start().Quoted 5 lines
Are there any lower limits on the size of a PCI memory region?
Have a nice day
GVQuoted 5 lines
You impliment
mmap()in your driver. It accesses the first megabyte as 256 pages. Then you tack on the additional page that your 512 bytes resides at.mmap()only works with pages. The pages must beioremap_nocacheand they must be reserved. The reserved part is important to have them visible from user-space using mmap.There are a number of totally bogus statements above.
First of all, you don't have to implement mmap in your driver (though it would probably better to do so). You can perfectly map your PCI memory space from userland without help of a specific driver (see below).
Then, ioremap has absolutely nothing to do with the mapping done to userland (unless there is some gory x86 in there, but I really doubt it). ioremap_* has to do with mapping the memory in kernel space, not user space. Thus your statement "must be
ioremap_nocache" is wrong.I don't know why you think the pages should be reserved as well, damn, he's not mapping RAM, he's mapping PCI MMIO space, do you know what you are talking about at all ? There is usually no struct page for PCI MMIO space.
Quoted 3 lines
That's IFF you really need to see the stuff in user-mode. Normally, you write a driver that accesses everything using the PCI primatives provided in the kernel, for the kernel.
Oh well, Ginancula, here is what you can do:
- If you want to stick to
/dev/mem, which is not recommended and may not work on all platforms but is fine for a quick hack, just make sure you are mmap'ing a full page. That is, your 512 bytes will be 512-bytes-aligned somewhere in a page of 4k on most platforms, maybe more. You have to map this entire page, and then take an offset in the resulting mapping to the 512 bytes.
Basically, you usegetpagesize()to get the page size, then you mmap yourbase_addr& ~(page_size - 1), and you access your data at the resulting address +base_addr& (page_size- 1)- Best is to use the PCI mmap facility. The "old" one via
/proc/bus/pcior the "new one" via/sys/...pathto your device.../resourceN. You directly mmap the resource file you want. HOWEVER, the alignement restriction will still be there, thus you should also look at the "resource" file to get the actual base address, and deduce the alignement, and then map a whole page and add the alignement offset to the result as for/dev/mem.Ben.