Special handling of sysfs device resource files?
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'm in the process of modifying X to be civilized in it's handling of PCI devices on Linux. As part of that, I've modified it to map the /sys/bus/pci/device/*/resource[0-6] files instead of mucking about with /dev/mem.
This seems to mostly work, but I am having one problem. I map the region by opening the file with O_RDWR, then mmap with
(PROT_READ|PROT_WRITE) and MAP_SHARED. In all cases, the open and mmap succeed. However, for I/O BARs, the resulting pointer from mmap is invalid. Any access to it results in a segfault and GDB says it's "out
of range".
The base address of the BAR is page aligned, so its not a problem with the alignment of mmap vs. the alignment of the BAR. What else could it be? I'm pretty stumped.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (GNU/Linux)
iD8DBQFEPB7KX1gOwKyEAw8RAhG/AJ4x+Vjl8V9SNeyMhYe2txeAeKALKACePCwL 6s0kj4YhDY3/thVh6mvO5X4=
=g9eu
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 11 lines · Ian Romanick
I'm in the process of modifying X to be civilized in it's handling of PCI devices on Linux. As part of that, I've modified it to map the /sys/bus/pci/device/*/resource[0-6] files instead of mucking about with
/dev/mem.This seems to mostly work, but I am having one problem. I map the region by opening the file with
O_RDWR, then mmap with(PROT_READ|PROT_WRITE) and MAP_SHARED. In all cases, the open and mmap succeed. However, for I/O BARs, the resulting pointer from mmap is invalid. Any access to it results in a segfault and GDB says it's "outof range".
I was a little mistaken about this. The BAR that causes the problem is not I/O. It *is* memory.
01:00.0 VGA compatible controller: Matrox Graphics, Inc. G400/G450 (rev
03) (prog-if 00 [VGA])
Subsystem: Matrox Graphics, Inc. Millennium G400 16Mb SGRAM
Flags: bus master, medium devsel, latency 64, IRQ 11 Memory at cc000000 (32-bit, prefetchable) [size=32M] Memory at cfefc000 (32-bit, non-prefetchable) [size=16K] Memory at cf000000 (32-bit, non-prefetchable) [size=8M] Expansion ROM at cfee0000 [disabled] [size=64K]
When I open and mmap resource0 (the framebuffer) I get 0x2b9aa48ea000. When I open and mmap resource1 (the card's registers) I get 0x2b9aa68ea000. I can access the resource0 pointer all day long without problems. The firs access to the resource1 pointer results in a segfault.
Quoted 3 lines
The base address of the BAR is page aligned, so its not a problem with the alignment of mmap vs. the alignment of the BAR. What else could it be? I'm pretty stumped.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (GNU/Linux)
iD8DBQFEPELpX1gOwKyEAw8RAjT+AJ0ZzDb49tr5WwKWE7eyKWdT7hRLUQCgkOSS twDrsx8VrWG5xEf+hbbkFvg=
=im+D
-----END PGP SIGNATURE-----
Quoted 6 lines · Ian Romanick
This seems to mostly work, but I am having one problem. I map the region by opening the file with
O_RDWR, then mmap with(PROT_READ|PROT_WRITE) and MAP_SHARED. In all cases, the open and mmap succeed. However, for I/O BARs, the resulting pointer from mmap is invalid. Any access to it results in a segfault and GDB says it's "outof range".
On which architecture(s)?
Paul.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 10 lines · Paul Mackerras
Quoted 6 lines · Ian Romanick
This seems to mostly work, but I am having one problem. I map the region by opening the file with
O_RDWR, then mmap with(PROT_READ|PROT_WRITE) and MAP_SHARED. In all cases, the open and mmap succeed. However, for I/O BARs, the resulting pointer from mmap is invalid. Any access to it results in a segfault and GDB says it's "outof range".
On which architecture(s)?
I've only tried on x86-64 (an Athlon64 3000+ to be exact) so far. -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (GNU/Linux)
iD8DBQFEPRd4X1gOwKyEAw8RAhdxAKCX9Y/ju2jK7q5Wdzmb9kfwt9T3lwCbBwrq BIRJkEIGqSOrXtkDC3/2Fts=
=mePG
-----END PGP SIGNATURE-----
Quoted 25 lines · Ian Romanick
Quoted 11 lines · Ian Romanick
I'm in the process of modifying X to be civilized in it's handling of PCI devices on Linux. As part of that, I've modified it to map the /sys/bus/pci/device/*/resource[0-6] files instead of mucking about with
/dev/mem.This seems to mostly work, but I am having one problem. I map the region by opening the file with
O_RDWR, then mmap with (PROT_READ|PROT_WRITE) andMAP_SHARED. In all cases, the open and mmap succeed. However, for I/O BARs, the resulting pointer from mmap is invalid. Any access to it results in a segfault and GDB says it's "out of range".I was a little mistaken about this. The BAR that causes the problem is not I/O. It *is* memory.
01:00.0 VGA compatible controller: Matrox Graphics, Inc. G400/G450 (rev 03) (prog-if 00 [VGA])
Subsystem: Matrox Graphics, Inc. Millennium G400 16Mb SGRAM
Flags: bus master, medium devsel, latency 64, IRQ 11 Memory atcc000000(32-bit, prefetchable) [size=32M] Memory atcfefc000(32-bit, non-prefetchable) [size=16K] Memory atcf000000(32-bit, non-prefetchable) [size=8M] Expansion ROM atcfee0000[disabled] [size=64K]
Quoted 5 lines
When I open and mmap resource0 (the framebuffer) I get
0x2b9aa48ea000. When I open and mmap resource1 (the card's registers) I get0x2b9aa68ea000. I can access the resource0 pointer all day long without problems. The firs access to the resource1 pointer results in a segfault.
Hm, that's strange. On my via machine I can access all my VGA resources correctly, but that's not x86-64. Maybe the x86-64 implementation of pci_mmap_page_range is doing the wrong thing for uncacheable, non-WC regions somehow?
Jesse
Quoted 12 lines · Ian Romanick
I was a little mistaken about this. The BAR that causes the problem is not I/O. It *is* memory.
01:00.0 VGA compatible controller: Matrox Graphics, Inc. G400/G450 (rev 03) (prog-if 00 [VGA])
Subsystem: Matrox Graphics, Inc. Millennium G400 16Mb SGRAM
Flags: bus master, medium devsel, latency 64, IRQ 11 Memory atcc000000(32-bit, prefetchable) [size=32M] Memory atcfefc000(32-bit, non-prefetchable) [size=16K] Memory atcf000000(32-bit, non-prefetchable) [size=8M] Expansion ROM atcfee0000[disabled] [size=64K]
Quoted 5 lines
When I open and mmap resource0 (the framebuffer) I get
0x2b9aa48ea000. When I open and mmap resource1 (the card's registers) I get0x2b9aa68ea000. I can access the resource0 pointer all day long without problems. The firs access to the resource1 pointer results in a segfault.
Just tested on my x86-64 machine with this dumb little test program. It seems to work ok, though I haven't tried writing data to the resource.
Jesse
#include <errno.h>
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
int main(int argc, char *argv[])
{
size_t len;
int fd, i;
void *ptr;
uint32_t *val;
if (argc != 3) {
fprintf(stderr, "usage: %s <file> <mapsize>\n",
argv[0]);
return -1;
}
len = atoi(argv[2]);
fd = open(argv[1], O_RDONLY);
if (fd == -1) {
fprintf(stderr, "open failed: %s\n", strerror(errno));
return errno;
}
ptr = mmap(NULL, len, PROT_READ, MAP_SHARED, fd, 0);
if (ptr == MAP_FAILED) {
fprintf(stderr, "mmap failed: %s\n", strerror(errno));
return errno;
}
val = ptr;
len = len / sizeof(uint32_t);
for (i = 0; i < len; i += 4) {
printf("%08x: 0x%08x 0x%08x 0x%08x 0x%08x\n", i * 4, val[i],
val[i+1], val[i+2], val[i+3]);
}
munmap(ptr, len); /* ignore any errors, we don't care */
close(fd);
return 0;
}