Skip to contentSkip to conversation

PCI userland access non-mmap APIs, kernel access to legacy space

DISCUSSION Benjamin Herrenschmidt 2 participants 3 messages Jun 18, 2007 – Jun 19, 2007

lore
Share & tools
lore
#0 lore raw

Hi folks !

Before I start actually writing code, I want to do a quick brain dump here to see if there are any objections.

So a problem we have is that on non-x86, we expect to be able to access PCI IO space via mmap'ing of sysfs based "resource" files.

However, there are various PCI host bridges out there that don't make that a practical solution. So far, I have two cases on ppc of bridges that either use indirect access for the IO space or require specific HW workarounds on accesses that involve spinlocks (to properly deal with the non-posted aspect of IO transactions). In addition, Lennert tells me that similar issues exist on ARM.

Thus, I would like to introduce an additional read/write interface to sysfs "resource" files (and then get Ian's libpciaccess use that when mmap is unavailable).

In fact, it's funny in a way becauase the "legacy" files interface is already read/write based (exclusively even, mmap isn't supported) for legacy IOs.

I beleive that doing so is better than trying to extend /dev/port to understand multiple domains, >64K IO space, etc... etc...., because you don't have to understand all the remapping tricks that the operating system might be doing behind the scene, just open the resource file (or the legacy file of that bus for legacy accesses) and read/write to it.

Any objection ?

Also, while at it, I would like to introduce a pair of in-kernel interfaces:

int pci_translate_legacy_resource(struct pci_bus *bus, struct resource *res);

Which takes a resource containing an absolute legacy range on that bus (for example, an IO resource with the VGA port numbers, or a memory resource with the VGA text mode aperture). That resource is then "converted" for the given PCI bus. For x86, that function would boil down to "return 0;".

The reason is that on multiple domains machines, it's actually not trivial to figure out where the legacy space of a given domain is mapped, if available at all. The above function is a way to solve that problem, allowing things like vgacon to work in multi-domains setup without half of the current arch specific hacks in there.

That would be also allowed to fail (some platforms can't give access to legacy memory or legacy IOs at all).

We could probably even implement the pci sysfs "legacy" interface on top of these as well, thus making it generic to all platforms.

Any objection there too ?

Cheers,
Ben.

#1 lore raw
Quoted 23 lines · On Sunday, June 17, 2007 7:16:44 Benjamin Herrenschmidt wrote:

Hi folks !

Before I start actually writing code, I want to do a quick brain dump here to see if there are any objections.

So a problem we have is that on non-x86, we expect to be able to access PCI IO space via mmap'ing of sysfs based "resource" files.

However, there are various PCI host bridges out there that don't make that a practical solution. So far, I have two cases on ppc of bridges that either use indirect access for the IO space or require specific HW workarounds on accesses that involve spinlocks (to properly deal with the non-posted aspect of IO transactions). In addition, Lennert tells me that similar issues exist on ARM.

Thus, I would like to introduce an additional read/write interface to sysfs "resource" files (and then get Ian's libpciaccess use that when mmap is unavailable).

In fact, it's funny in a way becauase the "legacy" files interface is already read/write based (exclusively even, mmap isn't supported) for legacy IOs.

Quoted 8 lines

I beleive that doing so is better than trying to extend /dev/port to understand multiple domains, >64K IO space, etc... etc...., because you don't have to understand all the remapping tricks that the operating system might be doing behind the scene, just open the resource file (or the legacy file of that bus for legacy accesses) and read/write to it.

Any objection ?

Sounds like a good idea. Originally, the "legacy_io" file was implemented this way because I knew of machines that couldn't properly handle it as a mmappable interface. But when I pinged people about the resource files, I didn't hear anything, so they were just done with mmap.

But adding read/write is fine with me. In fact, having read/write hooks for the memory backed resource files is also nice, since it makes dealing with them on the command line a bit easier.

Quoted 2 lines

Also, while at it, I would like to introduce a pair of in-kernel interfaces:

What's the other half of the pair?

Quoted 15 lines

int pci_translate_legacy_resource(struct pci_bus *bus, struct resource *res);

Which takes a resource containing an absolute legacy range on that bus (for example, an IO resource with the VGA port numbers, or a memory resource with the VGA text mode aperture). That resource is then "converted" for the given PCI bus. For x86, that function would boil down to "return 0;".

The reason is that on multiple domains machines, it's actually not trivial to figure out where the legacy space of a given domain is mapped, if available at all. The above function is a way to solve that problem, allowing things like vgacon to work in multi-domains setup without half of the current arch specific hacks in there.

Quoted 7 lines

That would be also allowed to fail (some platforms can't give access to legacy memory or legacy IOs at all).

We could probably even implement the pci sysfs "legacy" interface on top of these as well, thus making it generic to all platforms.

Any objection there too ?

Yeah, sounds like a good additional abstraction for that interface. Should make porting it to other arches easier (I talked with kyle about this last week, so maybe this'll give him motivation to do it for parisc).

Jesse

#2 lore raw
Quoted 4 lines

But adding read/write is fine with me. In fact, having read/write hooks for the memory backed resource files is also nice, since it makes dealing with them on the command line a bit easier.

Plan is to use iomap so I get both IO and MMIO for (almost) free :-)

Quoted 4 lines
Quoted 2 lines

Also, while at it, I would like to introduce a pair of in-kernel interfaces:

What's the other half of the pair?

Heh, indeed, there is none, I initially had a separate function for mem and for io and at the last minute though "ugh, let's just use a struct resource !" :-)

Quoted 4 lines

Yeah, sounds like a good additional abstraction for that interface. Should make porting it to other arches easier (I talked with kyle about this last week, so maybe this'll give him motivation to do it for parisc).

Ok, good. I'll do some code asap.

Ben.