Skip to contentSkip to conversation

ioremap_cached()

PATCH In review Matthew Wilcox 4 participants 9 messages Mar 30, 2006 – Mar 31, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ioremap_cached()

We currently have three ways for getting access to device memory -- ioremap(), ioremap_nocache() and pci_iomap(). 99% of the callers of ioremap() are doing it to access device registers, and really, really want to use ioremap_nocache() instead. I presume nobody notices on PCs because they have write-through caches, but it ought to trip up people trying to flush writes.

pci_iomap() gets this right -- it calls ioremap() if the resource is tagged with IORESOURCE_CACHEABLE and ioremap_nocache() if it isn't. Unfortunately, the only resources tagged with IORESOURCE_CACHEABLE are PCI ROM resources, so even frame buffers and similar that could be cached aren't.

It seems clear to me that ioremap() and ioremap_nocache() are the wrong way around. The default needs to be uncached -- and the obvious (rare) alternative becomes ioremap_cached().

So here's a patch for i386 to add ioremap_cached() and make ioremap() uncached. ioremap_nocache() remains as an alias for ioremap() so we don't needlessly break old drivers. Architecture maintainers will need to fix up their ports if this patch is accepted.

Index: include/asm-i386/io.h
===================================================================
RCS file: /var/cvs/linux-2.6/include/asm-i386/io.h,v retrieving revision 1.10

include/asm-i386/io.h +3 -2
diff -u -p -r1.10 io.h
@@ -115,12 +115,13 @@ extern void __iomem * __ioremap(unsigned
  * address. 
  */
 
-static inline void __iomem * ioremap(unsigned long offset, unsigned long size)
+static inline void __iomem * ioremap_cached(unsigned long offset, unsigned long size)
 {
 	return __ioremap(offset, size, 0);
 }
 
-extern void __iomem * ioremap_nocache(unsigned long offset, unsigned long size);
+#define ioremap_nocache(a, b) ioremap(a, b)
+extern void __iomem * ioremap(unsigned long offset, unsigned long size);
 extern void iounmap(volatile void __iomem *addr);
 
 /*

Index: arch/i386/mm/ioremap.c
===================================================================
RCS file: /var/cvs/linux-2.6/arch/i386/mm/ioremap.c,v retrieving revision 1.13

arch/i386/mm/ioremap.c +4 -4
diff -u -p -r1.13 ioremap.c
@@ -167,11 +167,11 @@ void __iomem * __ioremap(unsigned long p
 EXPORT_SYMBOL(__ioremap);
 
 /**
- * ioremap_nocache     -   map bus memory into CPU space
+ * ioremap     -   map bus memory into CPU space
  * @offset:    bus address of the memory
  * @size:      size of the resource to map
  *
- * ioremap_nocache performs a platform specific sequence of operations to
+ * ioremap performs a platform specific sequence of operations to
  * make bus memory CPU accessible via the readb/readw/readl/writeb/
  * writew/writel functions and the other mmio helpers. The returned
  * address is not guaranteed to be usable directly as a virtual
@@ -188,7 +188,7 @@ EXPORT_SYMBOL(__ioremap);
  * Must be freed with iounmap.
  */
 
-void __iomem *ioremap_nocache (unsigned long phys_addr, unsigned long size)
+void __iomem *ioremap(unsigned long phys_addr, unsigned long size)
 {
 	unsigned long last_addr;
 	void __iomem *p = __ioremap(phys_addr, size, _PAGE_PCD);
@@ -221,7 +221,7 @@ void __iomem *ioremap_nocache (unsigned 
 
 	return p;					
 }
-EXPORT_SYMBOL(ioremap_nocache);
+EXPORT_SYMBOL(ioremap);
 
 /**
  * iounmap - Free a IO remapping
#1 lore raw
Quoted 6 lines · Matthew Wilcox

We currently have three ways for getting access to device memory -- ioremap(), ioremap_nocache() and pci_iomap(). 99% of the callers of ioremap() are doing it to access device registers, and really, really want to use ioremap_nocache() instead. I presume nobody notices on PCs because they have write-through caches, but it ought to trip up people trying to flush writes.

Actually MTRRs take care of that on x86.
So essentially on x86 ioremap() for devices is already ioremap_uncached() And ioremap on memory is cached.

That's nice and simple semantics that other platforms can emulate too. Doing things differently will just cause pain for the other platforms when they have to fix up drivers all the time.

It all works fine until someone wants WC too. I would rather add a ioremap_wc(), that would be more useful.

-Andi

#2 lore raw
Quoted 9 lines · Andi Kleen
Quoted 6 lines · Matthew Wilcox

We currently have three ways for getting access to device memory -- ioremap(), ioremap_nocache() and pci_iomap(). 99% of the callers of ioremap() are doing it to access device registers, and really, really want to use ioremap_nocache() instead. I presume nobody notices on PCs because they have write-through caches, but it ought to trip up people trying to flush writes.

Quoted 7 lines

Actually MTRRs take care of that on x86.
So essentially on x86 ioremap() for devices is already ioremap_uncached() And ioremap on memory is cached.

That's nice and simple semantics that other platforms can emulate too. Doing things differently will just cause pain for the other platforms when they have to fix up drivers all the time.

That doesn't make any sense. What's the point of ioremap_nocache() if ioremap() does magic things that make things uncached? And who says you're allowed to ioremap() memory anyway?

Quoted 2 lines

It all works fine until someone wants WC too. I would rather add a ioremap_wc(), that would be more useful.

ioremap_wc() sounds like a good idea.

#3 lore raw
Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andi Kleen
Quoted 6 lines · Matthew Wilcox

We currently have three ways for getting access to device memory -- ioremap(), ioremap_nocache() and pci_iomap(). 99% of the callers of ioremap() are doing it to access device registers, and really, really want to use ioremap_nocache() instead. I presume nobody notices on PCs because they have write-through caches, but it ought to trip up people trying to flush writes.

Actually MTRRs take care of that on x86.
So essentially on x86 ioremap() for devices is already ioremap_uncached() And ioremap on memory is cached.

Quoted 6 lines
Quoted 3 lines

That's nice and simple semantics that other platforms can emulate too. Doing things differently will just cause pain for the other platforms when they have to fix up drivers all the time.

That doesn't make any sense. What's the point of ioremap_nocache() if ioremap() does magic things that make things uncached?

In theory ioremap_nocache would force uncached even if there is no MTRR and is better/clearer.

But on x86 there normally is, so lots of code gets it wrong.

My point is just that forcing a semantics that's not enforced on x86 would be a uphill battle for everybody else. Probably not a good idea. Better fake x86.

Quoted 2 lines

And who says
you're allowed to ioremap() memory anyway?

Why not? There are cases when it's needed.

Quoted 4 lines
Quoted 2 lines

It all works fine until someone wants WC too. I would rather add a ioremap_wc(), that would be more useful.

ioremap_wc() sounds like a good idea.

It's unfortunately tricky to get it fully right on x86. The issue is to have a good way avoid illegal cache aliases. There were some attempts, but so far they never were polished up from the initial prototypes.

-Andi

#4 lore raw
Quoted 11 lines · Andi Kleen
Quoted 2 lines

That doesn't make any sense. What's the point of ioremap_nocache() if ioremap() does magic things that make things uncached?

In theory ioremap_nocache would force uncached even if there is no MTRR and is better/clearer.

But on x86 there normally is, so lots of code gets it wrong.

My point is just that forcing a semantics that's not enforced on x86 would be a uphill battle for everybody else. Probably not a good idea. Better fake x86.

I think you misunderstood. The right interface to call, that should work everywhere, should be the simple, obvious one. ioremap(). That effectively is what everyone gets anyway (since they test on x86). So change the *definition* of ioremap() to be uncached. Then we can add ioremap_wc() and ioremap_cached() for these special purpose mappings.

Quoted 4 lines

It's unfortunately tricky to get it fully right on x86. The issue is to have a good way avoid illegal cache aliases. There were some attempts, but so far they never were polished up from the initial prototypes.

I know there's similar issues with Itanium. IIRC, the EFI memory map helps here by saying how various different types of memory should be mapped.

#5 lore raw
Quoted 5 lines · Matthew Wilcox

I think you misunderstood. The right interface to call, that should work everywhere, should be the simple, obvious one. ioremap(). That effectively is what everyone gets anyway (since they test on x86). So change the *definition* of ioremap() to be uncached. Then we can add ioremap_wc() and ioremap_cached() for these special purpose mappings.

That would break all the current users who do ioremap on memory and want it cached.

-Andi

#6 lore raw
Quoted 10 lines · Andi Kleen
Quoted 5 lines · Matthew Wilcox

I think you misunderstood. The right interface to call, that should work everywhere, should be the simple, obvious one. ioremap(). That effectively is what everyone gets anyway (since they test on x86). So change the *definition* of ioremap() to be uncached. Then we can add ioremap_wc() and ioremap_cached() for these special purpose mappings.

That would break all the current users who do ioremap on memory and want it cached.

What's an example of this? I ask since on powerpc ioremap() is always _PAGE_NO_CACHE.

- kumar

#7 lore raw
Quoted 12 lines · Kumar Gala
Quoted 9 lines · Andi Kleen
Quoted 6 lines · Matthew Wilcox

I think you misunderstood. The right interface to call, that should work everywhere, should be the simple, obvious one. ioremap(). That effectively is what everyone gets anyway (since they test on x86). So change the *definition* of ioremap() to be uncached. Then we can add
ioremap_wc() and ioremap_cached() for these special purpose mappings.

Quoted 5 lines
Quoted 2 lines

That would break all the current users who do ioremap on memory and want it cached.

What's an example of this? I ask since on powerpc ioremap() is always _PAGE_NO_CACHE.

ACPI, IPMI, DMI, ... I bet there are more. It's needed every time a driver needs to look for some firmware table because it might not be mapped.

-Andi

#8 lore raw
In reply to Matthew Wilcox
Quoted 10 lines

It seems clear to me that ioremap() and ioremap_nocache() are the wrong way around. The default needs to be uncached -- and the obvious (rare) alternative becomes ioremap_cached().

So here's a patch for i386 to add ioremap_cached() and make ioremap() uncached. ioremap_nocache() remains as an alias for ioremap() so we don't needlessly break old drivers. Architecture maintainers will need to fix up their ports if this patch is accepted.

I'd actually suggest deprecating ioremap() and moving away from it to make sure all users think of which variant they want. Explicit naming is always better than "some" unknown behavior (especially for old drivers that work on multiple kernels... while you allow them to keep compiling, you change the world under them so I think eventually break them compiling is actually better than having them limp along broken)