Cross-architecture definitions of relaxed MMIO accessors
Subsystems (13)
v2 cover Cross-architecture definitions of relaxed MMIO accessors
Hi all,
This is version 2 of the series I originally posted here:
https://lkml.org/lkml/2014/4/17/269
Changes since v1 include:
- Added relevant acks from arch maintainers
- Fixed potential compiler re-ordering issue for x86 definitions
I'd *really* appreciate some feedback on the proposed semantics here, but acks are still good :)
The original cover letter is duplicated below.
Cheers,
Will
--->8
This RFC series attempts to define a portable (i.e. cross-architecture) definition of the {readX,writeX}_relaxed MMIO accessor functions. These functions are already in widespread use amongst drivers (mainly those supporting devices embedded in ARM SoCs), but lack any well-defined semantics and, subsequently, any portable definitions to allow these drivers to be compiled for other architectures.
The two main motivations for this series are:
(1) To promote use of the _relaxed MMIO accessors on weakly-ordered architectures, where they can bring significant performance improvements over their non-relaxed counterparts.
(2) To allow COMPILE_TEST to build drivers using the relaxed accessors across all architectures.
The proposed semantics largely match exactly those provided by the ARM implementation (i.e. no weaker), with one exception (see below).
Informally:
- Relaxed accesses to the same device are ordered with respect to each other.
- Relaxed accesses are *not* guaranteed to be ordered with respect to normal memory accesses (e.g. DMA buffers -- this is what gives us the performance boost over the non-relaxed versions).
- Relaxed accesses are not guaranteed to be ordered with respect to LOCK/UNLOCK operations.
In actual fact, the relaxed accessors *are* ordered with respect to LOCK/UNLOCK operations on ARM[64], but I have added this constraint for the benefit of PowerPC, which has expensive I/O barriers in the spin_unlock path for the non-relaxed accessors.
A corollary to this is that mmiowb() probably needs rethinking. As it currently stands, an mmiowb() is required to order MMIO writes to a device from multiple CPUs, even if that device is protected by a lock. However, this isn't often used in practice, leading to PowerPC implementing both mmiowb() *and* synchronising I/O in spin_unlock.
I would propose making the non-relaxed I/O accessors ordered with respect to LOCK/UNLOCK, leaving mmiowb() to be used with the relaxed accessors, if required, but would welcome thoughts/suggestions on this topic.
Will Deacon (18):
asm-generic: io: implement relaxed accessor macros as conditional
wrappers
microblaze: io: remove dummy relaxed accessor macros
s390: io: remove dummy relaxed accessor macros for reads
xtensa: io: remove dummy relaxed accessor macros for reads
alpha: io: implement relaxed accessor macros for writes
frv: io: implement dummy relaxed accessor macros for writes
cris: io: implement dummy relaxed accessor macros for writes
ia64: io: implement dummy relaxed accessor macros for writes
m32r: io: implement dummy relaxed accessor macros for writes
m68k: io: implement dummy relaxed accessor macros for writes
mn10300: io: implement dummy relaxed accessor macros for writes
parisc: io: implement dummy relaxed accessor macros for writes
powerpc: io: implement dummy relaxed accessor macros for writes
sparc: io: implement dummy relaxed accessor macros for writes
tile: io: implement dummy relaxed accessor macros for writes
x86: io: implement dummy relaxed accessor macros for writes
documentation: memory-barriers: clarify relaxed io accessor semantics
asm-generic: io: define relaxed accessor macros unconditionally
Documentation/memory-barriers.txt | 13 +++++++++---- arch/alpha/include/asm/io.h | 12 ++++++++---- arch/cris/include/asm/io.h | 3 +++ arch/frv/include/asm/io.h | 3 +++ arch/ia64/include/asm/io.h | 4 ++++ arch/m32r/include/asm/io.h | 3 +++ arch/m68k/include/asm/io.h | 8 ++++++++ arch/m68k/include/asm/io_no.h | 4 ---- arch/microblaze/include/asm/io.h | 8 -------- arch/mn10300/include/asm/io.h | 4 ++++ arch/parisc/include/asm/io.h | 12 ++++++++---- arch/powerpc/include/asm/io.h | 12 ++++++++---- arch/s390/include/asm/io.h | 5 ----- arch/sparc/include/asm/io.h | 9 +++++++++ arch/sparc/include/asm/io_32.h | 3 --- arch/sparc/include/asm/io_64.h | 22 ++++++++++------------ arch/tile/include/asm/io.h | 4 ++++ arch/x86/include/asm/io.h | 10 +++++++--- arch/xtensa/include/asm/io.h | 7 ------- include/asm-generic/io.h | 10 ++++++++++ 20 files changed, 98 insertions(+), 58 deletions(-)
Signature · 1 line
1.9.2
PATCH v2 1/18 asm-generic: io: implement relaxed accessor macros as conditional wrappers
{read,write}{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds wrappers to asm-generic so that drivers can rely on the relaxed accessors being available, even if they don't always provide weaker ordering guarantees. Since some architectures both include asm-generic/io.h and define some relaxed accessors, the definitions here are conditional for the time being.
--- include/asm-generic/io.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
include/asm-generic/io.h +26 -0
@@ -53,18 +53,27 @@ static inline u32 __raw_readl(const volatile void __iomem *addr)#endif #define readb __raw_readb +#ifndef readb_relaxed +#define readb_relaxed readb +#endif #define readw readwstatic inline u16 readw (const volatile void __iomem *addr) {return __le16_to_cpu (__raw_readw (addr)); } +#ifndef readw_relaxed +#define readw_relaxed readw +#endif #define readl readlstatic inline u32 readl (const volatile void __iomem *addr) {return __le32_to_cpu (__raw_readl (addr)); } +#ifndef readl_relaxed +#define readl_relaxed readl +#endif #ifndef __raw_writebstatic inline void __raw_writeb (u8 b,volatile void __iomem *addr)
@@ -88,8 +97,19 @@ static inline void __raw_writel(u32 b, volatile void __iomem *addr)#endif #define writeb __raw_writeb +#ifndef writeb_relaxed +#define writeb_relaxed writeb +#endif +#define writew (b,addr) __raw_writew(__cpu_to_le16(b),addr) +#ifndef writew_relaxed +#define writew_relaxed writew +#endif +#define writel (b,addr) __raw_writel(__cpu_to_le32(b),addr) +#ifndef writel_relaxed +#define writel_relaxed writel +#endif #ifdef CONFIG_64BIT #ifndef __raw_readq
@@ -104,6 +124,9 @@ static inline u64 readq(const volatile void __iomem *addr) {return __le64_to_cpu (__raw_readq (addr)); } +#ifndef readq_relaxed +#define readq_relaxed readq +#endif #ifndef __raw_writeqstatic inline void __raw_writeq (u64 b,volatile void __iomem *addr)
@@ -113,6 +136,9 @@ static inline void __raw_writeq(u64 b, volatile void __iomem *addr)#endif #define writeq (b, addr) __raw_writeq(__cpu_to_le64(b), addr) +#ifndef writeq_relaxed +#define writeq_relaxed writeq +#endif #endif /* CONFIG_64BIT */ #ifndef PCI_IOBASE
Signature · 1 line
1.9.2
PATCH v2 2/18 microblaze: io: remove dummy relaxed accessor macros
In reply to Will DeaconThese are now defined by asm-generic/io.h, so we don't need the private definitions anymore.
--- arch/microblaze/include/asm/io.h | 8 -------- 1 file changed, 8 deletions(-)
arch/microblaze/include/asm/io.h +0 -8
@@ -72,12 +72,4 @@ extern void __iomem *ioremap(phys_addr_t address, unsigned long size);#include <asm-generic/io.h> -#define readb_relaxed readb -#define readw_relaxed readw -#define readl_relaxed readl - -#define writeb_relaxed writeb -#define writew_relaxed writew -#define writel_relaxed writel -#endif /* _ASM_MICROBLAZE_IO_H */
Signature · 1 line
1.9.2
PATCH v2 3/18 s390: io: remove dummy relaxed accessor macros for reads
In reply to Will DeaconThese are now defined by asm-generic/io.h, so we don't need the private definitions anymore.
--- arch/s390/include/asm/io.h | 5 ----- 1 file changed, 5 deletions(-)
arch/s390/include/asm/io.h +0 -5
@@ -60,11 +60,6 @@ static inline void iounmap(volatile void __iomem *addr)#define __raw_writel zpci_write_u32#define __raw_writeq zpci_write_u64 -#define readb_relaxed readb -#define readw_relaxed readw -#define readl_relaxed readl -#define readq_relaxed readq -#endif /* CONFIG_PCI */ #include <asm-generic/io.h>
Signature · 1 line
1.9.2
PATCH v2 4/18 xtensa: io: remove dummy relaxed accessor macros for reads
In reply to Will DeaconThese are now defined by asm-generic/io.h, so we don't need the private definitions anymore.
--- arch/xtensa/include/asm/io.h | 7 ------- 1 file changed, 7 deletions(-)
arch/xtensa/include/asm/io.h +0 -7
@@ -74,13 +74,6 @@ static inline void iounmap(volatile void __iomem *addr)#endif /* CONFIG_MMU */ -/* -* Generic I/O -*/ -#define readb_relaxed readb -#define readw_relaxed readw -#define readl_relaxed readl -#endif /* __KERNEL__ */ #include <asm-generic/io.h>
Signature · 1 line
1.9.2
PATCH v2 5/18 alpha: io: implement relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O writes with weaker barrier semantics than the non-relaxed variants.
This patch implements these write macros for Alpha, in the same vein as the relaxed read macros, which are already implemented.
--- arch/alpha/include/asm/io.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/alpha/include/asm/io.h +8 -4
@@ -500,10 +500,14 @@ extern inline void writeq(u64 b, volatile void __iomem *addr)#define outb_p outb#define outw_p outw#define outl_p outl -#define readb_relaxed (addr) __raw_readb(addr) -#define readw_relaxed (addr) __raw_readw(addr) -#define readl_relaxed (addr) __raw_readl(addr) -#define readq_relaxed (addr) __raw_readq(addr) +#define readb_relaxed (addr) __raw_readb(addr) +#define readw_relaxed (addr) __raw_readw(addr) +#define readl_relaxed (addr) __raw_readl(addr) +#define readq_relaxed (addr) __raw_readq(addr) +#define writeb_relaxed (b, addr) __raw_writeb(b, addr) +#define writew_relaxed (b, addr) __raw_writew(b, addr) +#define writel_relaxed (b, addr) __raw_writel(b, addr) +#define writeq_relaxed (b, addr) __raw_writeq(b, addr)#define mmiowb ()
Signature · 1 line
1.9.2
Re: [PATCH v2 05/18] alpha: io: implement relaxed accessor macros for writes
Quoted 7 lines · Will Deacon
write{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O writes with weaker barrier semantics than the non-relaxed variants.
This patch implements these write macros for Alpha, in the same vein as the relaxed read macros, which are already implemented.
Quoted 7 lines
Cc Richard Henderson <[email protected]>Cc Ivan Kokshaysky <[email protected]>Cc Matt Turner <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/alpha/include/asm/io.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
r~
PATCH v2 6/18 frv: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to frv, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/frv/include/asm/io.h | 3 +++ 1 file changed, 3 insertions(+)
arch/frv/include/asm/io.h +3 -0
@@ -243,6 +243,9 @@ static inline void writel(uint32_t datum, volatile void __iomem *addr)__flush_PCI_writes (); } +#define writeb_relaxed writeb +#define writew_relaxed writew +#define writel_relaxed writel/* Values for nocacheflag and cmode */ #define IOMAP_FULL_CACHING 0
Signature · 1 line
1.9.2
PATCH v2 7/18 cris: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to Cris, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/cris/include/asm/io.h | 3 +++ 1 file changed, 3 insertions(+)
arch/cris/include/asm/io.h +3 -0
@@ -112,6 +112,9 @@ static inline void writel(unsigned int b, volatile void __iomem *addr)else *(volatile unsigned int __force *) addr = b; } +#define writeb_relaxed (b, addr) writeb(b, addr) +#define writew_relaxed (b, addr) writew(b, addr) +#define writel_relaxed (b, addr) writel(b, addr)#define __raw_writeb writeb#define __raw_writew writew#define __raw_writel writel
Signature · 1 line
1.9.2
PATCH v2 8/18 ia64: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to ia64, which may be able to be optimised in a similar manner to the relaxed read accessors at a later date.
--- arch/ia64/include/asm/io.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/ia64/include/asm/io.h +4 -0
@@ -393,6 +393,10 @@ __writeq (unsigned long val, volatile void __iomem *addr)#define writew (v,a) __writew((v), (a))#define writel (v,a) __writel((v), (a))#define writeq (v,a) __writeq((v), (a)) +#define writeb_relaxed (v,a) __writeb((v), (a)) +#define writew_relaxed (v,a) __writew((v), (a)) +#define writel_relaxed (v,a) __writel((v), (a)) +#define writeq_relaxed (v,a) __writeq((v), (a))#define __raw_writeb writeb#define __raw_writew writew#define __raw_writel writel
Signature · 1 line
1.9.2
PATCH v2 9/18 m32r: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to m32r, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/m32r/include/asm/io.h | 3 +++ 1 file changed, 3 insertions(+)
arch/m32r/include/asm/io.h +3 -0
@@ -161,6 +161,9 @@ static inline void _writel(unsigned long l, unsigned long addr)#define __raw_writeb writeb#define __raw_writew writew#define __raw_writel writel +#define writeb_relaxed writeb +#define writew_relaxed writew +#define writel_relaxed writel#define ioread8 read#define ioread16 readw
Signature · 1 line
1.9.2
PATCH v2 10/18 m68k: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to m68k, in the same vein as the dummy definitions for the relaxed read accessors. Additionally, the existing relaxed read accessors are moved into asm/io.h, so that they can be used by m68k targets with an MMU.
--- arch/m68k/include/asm/io.h | 8 ++++++++ arch/m68k/include/asm/io_no.h | 4 ---- 2 files changed, 8 insertions(+), 4 deletions(-)
arch/m68k/include/asm/io.h +8 -0
@@ -3,3 +3,11 @@#else #include <asm/io_mm.h> #endif + +#define readb_relaxed (addr) readb(addr) +#define readw_relaxed (addr) readw(addr) +#define readl_relaxed (addr) readl(addr) + +#define writeb_relaxed (b, addr) writeb(b, addr) +#define writew_relaxed (b, addr) writew(b, addr) +#define writel_relaxed (b, addr) writel(b, addr)
arch/m68k/include/asm/io_no.h +0 -4
@@ -40,10 +40,6 @@ static inline unsigned int _swapl(volatile unsigned long v)#define readl (addr) \ ({ unsigned int __v = (*(volatile unsigned int *) (addr)); __v; }) -#define readb_relaxed (addr) readb(addr) -#define readw_relaxed (addr) readw(addr) -#define readl_relaxed (addr) readl(addr) -#define writeb (b,addr) (void)((*(volatile unsigned char *) (addr)) = (b))#define writew (b,addr) (void)((*(volatile unsigned short *) (addr)) = (b))#define writel (b,addr) (void)((*(volatile unsigned int *) (addr)) = (b))
Signature · 1 line
1.9.2
PATCH v2 11/18 mn10300: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to mn10300, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/mn10300/include/asm/io.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/mn10300/include/asm/io.h +4 -0
@@ -67,6 +67,10 @@ static inline void writel(u32 b, volatile void __iomem *addr)#define __raw_writew writew#define __raw_writel writel +#define writeb_relaxed writeb +#define writew_relaxed writew +#define writel_relaxed writel +/*****************************************************************************/ /* * traditional input/output functions
Signature · 1 line
1.9.2
PATCH v2 12/18 parisc: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to parisc, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/parisc/include/asm/io.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/parisc/include/asm/io.h +8 -4
@@ -217,10 +217,14 @@ static inline void writeq(unsigned long long q, volatile void __iomem *addr)#define writel writel#define writeq writeq -#define readb_relaxed (addr) readb(addr) -#define readw_relaxed (addr) readw(addr) -#define readl_relaxed (addr) readl(addr) -#define readq_relaxed (addr) readq(addr) +#define readb_relaxed (addr) readb(addr) +#define readw_relaxed (addr) readw(addr) +#define readl_relaxed (addr) readl(addr) +#define readq_relaxed (addr) readq(addr) +#define writeb_relaxed (b, addr) writeb(b, addr) +#define writew_relaxed (w, addr) writew(w, addr) +#define writel_relaxed (l, addr) writel(l, addr) +#define writeq_relaxed (q, addr) writeq(q, addr)#define mmiowb () do { } while (0)
Signature · 1 line
1.9.2
PATCH v2 13/18 powerpc: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to powerpc, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/powerpc/include/asm/io.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/powerpc/include/asm/io.h +8 -4
@@ -617,10 +617,14 @@ static inline void name at \/* * We don't do relaxed operations yet, at least not with this semantic */ -#define readb_relaxed (addr) readb(addr) -#define readw_relaxed (addr) readw(addr) -#define readl_relaxed (addr) readl(addr) -#define readq_relaxed (addr) readq(addr) +#define readb_relaxed (addr) readb(addr) +#define readw_relaxed (addr) readw(addr) +#define readl_relaxed (addr) readl(addr) +#define readq_relaxed (addr) readq(addr) +#define writeb_relaxed (v, addr) writeb(v, addr) +#define writew_relaxed (v, addr) writew(v, addr) +#define writel_relaxed (v, addr) writel(v, addr) +#define writeq_relaxed (v, addr) writeq(v, addr)#ifdef CONFIG_PPC32 #define mmiowb ()
Signature · 1 line
1.9.2
PATCH v2 14/18 sparc: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to sparc, in the same vein as the dummy definitions for the relaxed read accessors. The existing relaxed read{b,w,l} accessors are moved into asm/io.h, since they are identical between 32-bit and 64-bit machines.
--- arch/sparc/include/asm/io.h | 9 +++++++++ arch/sparc/include/asm/io_32.h | 3 --- arch/sparc/include/asm/io_64.h | 22 ++++++++++------------ 3 files changed, 19 insertions(+), 15 deletions(-)
arch/sparc/include/asm/io.h +9 -0
@@ -10,6 +10,15 @@* Defines used for both SPARC32 and SPARC64 */ +/* Relaxed accessors for MMIO */ +#define readb_relaxed (__addr) readb(__addr) +#define readw_relaxed (__addr) readw(__addr) +#define readl_relaxed (__addr) readl(__addr) + +#define writeb_relaxed (__b, __addr) writeb(__b, __addr) +#define writew_relaxed (__w, __addr) writew(__w, __addr) +#define writel_relaxed (__l, __addr) writel(__l, __addr) +/* Big endian versions of memory read/write routines */ #define readb_be (__addr) __raw_readb(__addr)#define readw_be (__addr) __raw_readw(__addr)
arch/sparc/include/asm/io_32.h +0 -3
@@ -89,9 +89,6 @@ static inline void __writel(u32 l, volatile void __iomem *addr)#define readb (__addr) __readb(__addr)#define readw (__addr) __readw(__addr)#define readl (__addr) __readl(__addr) -#define readb_relaxed (__addr) readb(__addr) -#define readw_relaxed (__addr) readw(__addr) -#define readl_relaxed (__addr) readl(__addr)#define writeb (__b, __addr) __writeb((__b),(__addr))#define writew (__w, __addr) __writew((__w),(__addr))
arch/sparc/include/asm/io_64.h +10 -12
@@ -203,18 +203,16 @@ static inline void _writeq(u64 q, volatile void __iomem *addr) : "memory"); } -#define readb (__addr) _readb(__addr) -#define readw (__addr) _readw(__addr) -#define readl (__addr) _readl(__addr) -#define readq (__addr) _readq(__addr) -#define readb_relaxed (__addr) _readb(__addr) -#define readw_relaxed (__addr) _readw(__addr) -#define readl_relaxed (__addr) _readl(__addr) -#define readq_relaxed (__addr) _readq(__addr) -#define writeb (__b, __addr) _writeb(__b, __addr) -#define writew (__w, __addr) _writew(__w, __addr) -#define writel (__l, __addr) _writel(__l, __addr) -#define writeq (__q, __addr) _writeq(__q, __addr) +#define readb (__addr) _readb(__addr) +#define readw (__addr) _readw(__addr) +#define readl (__addr) _readl(__addr) +#define readq (__addr) _readq(__addr) +#define readq_relaxed (__addr) _readq(__addr) +#define writeb (__b, __addr) _writeb(__b, __addr) +#define writew (__w, __addr) _writew(__w, __addr) +#define writel (__l, __addr) _writel(__l, __addr) +#define writeq (__q, __addr) _writeq(__q, __addr) +#define writeq_relaxed (__q, __addr) _writeq(__q, __addr)/* Now versions without byte-swapping. */ static inline u8 _raw_readb (unsigned long addr)
Signature · 1 line
1.9.2
Re: [PATCH v2 14/18] sparc: io: implement dummy relaxed accessor macros for writes
Quoted 11 lines · Will Deacon
write{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to sparc, in the same vein as the dummy definitions for the relaxed read accessors. The existing relaxed read{b,w,l} accessors are moved into
asm/io.h, since they are identical between 32-bit and 64-bit machines.Cc "David S. Miller" <[email protected]>Signed-off-by Will Deacon <[email protected]>
Look good:
But you should wait for David's ack too.
Sam
Re: [PATCH v2 14/18] sparc: io: implement dummy relaxed accessor macros for writes
Quoted 10 lines · Sam Ravnborg
Quoted 9 lines · Will Deacon
write{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to sparc, in the same vein as the dummy definitions for the relaxed read accessors. The existing relaxed read{b,w,l} accessors are moved into
asm/io.h, since they are identical between 32-bit and 64-bit machines.
Quoted 4 lines
Quoted 2 lines
Cc "David S. Miller" <[email protected]>Signed-off-by Will Deacon <[email protected]>Look good:
Acked-by Sam Ravnborg <[email protected]>
Thanks, Sam.
Quoted 1 line
But you should wait for David's ack too.
Yeah, I still need to get buy-in on the semantics from the PPC folks anyway.
Will
Re: [PATCH v2 14/18] sparc: io: implement dummy relaxed accessor macros for writes
From: Will Deacon <[email protected]>
Date: Fri, 23 May 2014 15:38:10 +0100
Quoted 16 lines · Sam Ravnborg
Quoted 10 lines · Will Deacon
Quoted 9 lines
write{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to sparc, in the same vein as the dummy definitions for the relaxed read accessors. The existing relaxed read{b,w,l} accessors are moved into
asm/io.h, since they are identical between 32-bit and 64-bit machines.Quoted 4 lines
Quoted 2 lines
Cc "David S. Miller" <[email protected]>Signed-off-by Will Deacon <[email protected]>Look good:
Acked-by Sam Ravnborg <[email protected]>
Quoted 6 lines
Thanks, Sam.
Quoted 1 line
But you should wait for David's ack too.
Yeah, I still need to get buy-in on the semantics from the PPC folks anyway.
I'm fine with these changes so:
Unfortunately, whilst sparc64 could support the relaxed variants, there is no easy way to implement this.
I/O addrs are simply physical addresses on sparc64, and we therefore do loads and stores via the ASY_PHYS_BYPASS_EC_E* address spaces. What this address space means is "physical address", "bypass caches", "side effect".
To do a relaxed variant we'd need something without the "side effect" part, but no such ASI exists.
These are all page protection bits, so we could move to using virtual mappings on I/O things, but that's so much overkill just for this I think.
Besides there are bigger fish to fry on sparc64 :-)
PATCH v2 15/18 tile: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the write accessors to tile, in the same vein as the dummy definitions for the relaxed read accessors.
--- arch/tile/include/asm/io.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/tile/include/asm/io.h +4 -0
@@ -241,6 +241,10 @@ static inline void writeq(u64 val, unsigned long addr)#define readw_relaxed readw#define readl_relaxed readl#define readq_relaxed readq +#define writeb_relaxed writeb +#define writew_relaxed writew +#define writel_relaxed writel +#define writeq_relaxed writeq#define ioread8 readb#define ioread16 readw
Signature · 1 line
1.9.2
PATCH v2 16/18 x86: io: implement dummy relaxed accessor macros for writes
In reply to Will Deaconwrite{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the read and write accessors to x86, which simply expand to the non-relaxed variants. Note that this strengthens the relaxed read accessors, since they are now ordered with respect to each other by way of a compiler barrier.
--- arch/x86/include/asm/io.h | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/include/asm/io.h +7 -3
@@ -67,13 +67,16 @@ build_mmio_write(__writeb, "b", unsigned char, "q", ) build_mmio_write(__writew, "w", unsigned short, "r", ) build_mmio_write(__writel, "l", unsigned int, "r", ) -#define readb_relaxed(a) __readb(a) -#define readw_relaxed(a) __readw(a) -#define readl_relaxed(a) __readl(a) +#define readb_relaxed(a) readb(a) +#define readw_relaxed(a) readw(a) +#define readl_relaxed(a) readl(a) #define __raw_readb __readb #define __raw_readw __readw #define __raw_readl __readl +#define writeb_relaxed(v, a) writeb(v, a) +#define writew_relaxed(v, a) writew(v, a) +#define writel_relaxed(v, a) writel(v, a) #define __raw_writeb __writeb #define __raw_writew __writew #define __raw_writel __writel
@@ -86,6 +89,7 @@ build_mmio_read(readq, "q", unsigned long, "=r", :"memory") build_mmio_write(writeq, "q", unsigned long, "r", :"memory")#define readq_relaxed (a) readq(a) +#define writeq_relaxed (v, a) writeq(v, a)#define __raw_readq (a) readq(a)#define __raw_writeq (val, addr) writeq(val, addr)
Signature · 1 line
1.9.2
H. Peter Anvin
reviewerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 8 lines · Will Deacon
write{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
This patch adds dummy macros for the read and write accessors to x86, which simply expand to the non-relaxed variants. Note that this strengthens the relaxed read accessors, since they are now ordered with respect to each other by way of a compiler barrier.
OK, do we want/need that compiler barrier? And you say "strengthens" - strengthens with respect to what if we didn't have them before?
-hpa
Re: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Hi Peter,
Quoted 5 lines · H. Peter Anvin
Quoted 4 lines · Will Deacon
write{b,w,l,q}_relaxed are implemented by some architectures in order to permit memory-mapped I/O accesses with weaker barrier semantics than the non-relaxed variants.
Quoted 7 lines
Quoted 4 lines
This patch adds dummy macros for the read and write accessors to x86, which simply expand to the non-relaxed variants. Note that this strengthens the relaxed read accessors, since they are now ordered with respect to each other by way of a compiler barrier.
OK, do we want/need that compiler barrier? And you say "strengthens" - strengthens with respect to what if we didn't have them before?
Actually, x86 does already implement the relaxed read accessors:
#define readb_relaxed(a) __readb(a) #define readw_relaxed(a) __readw(a) #define readl_relaxed(a) __readl(a)
where __read* don't have "memory" clobbers, unlike the read* implementations.
I would like the relaxed accessors to be ordered with respect to each other but, looking again, that is already achieved through the use of volatile asm so I've come full circle and decided that we don't need the clobbers after all.
What do you think?
Will
H. Peter Anvin
reviewerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 5 lines · Will Deacon
I would like the relaxed accessors to be ordered with respect to each other...
What do you think?
I think "I would like" isn't a very good motivation. What are the semantics of these things supposed to be? It seems more than a bit odd to require them to be ordered with respect to each other and everything else (which is what a memory clobber does) and then call them "relaxed".
-hpa
Re: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 11 lines · H. Peter Anvin
Quoted 5 lines · Will Deacon
I would like the relaxed accessors to be ordered with respect to each other...
What do you think?
I think "I would like" isn't a very good motivation. What are the semantics of these things supposed to be? It seems more than a bit odd to require them to be ordered with respect to each other and everything else (which is what a memory clobber does) and then call them "relaxed".
I suggested some informal semantics in the cover letter:
https://lkml.org/lkml/2014/4/17/269
Basically, if we define relaxed accesses not to be ordered against anything apart from other accesses (relaxed or otherwise) to the same device, then they become a tonne cheaper on arm/arm64/powerpc. Currently we have to include expensive memory barriers in order to synchronise with accesses to DMA buffers which is rarely needed.
For those requirements, I don't think we need the "memory" clobber for x86, but would appreciate your views on this.
Will
H. Peter Anvin
reviewerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 17 lines · Will Deacon
Quoted 11 lines · H. Peter Anvin
Quoted 5 lines · Will Deacon
I would like the relaxed accessors to be ordered with respect to each other...
What do you think?
I think "I would like" isn't a very good motivation. What are the semantics of these things supposed to be? It seems more than a bit odd to require them to be ordered with respect to each other and everything else (which is what a memory clobber does) and then call them "relaxed".
I suggested some informal semantics in the cover letter:
Quoted 9 lines
Basically, if we define relaxed accesses not to be ordered against anything apart from other accesses (relaxed or otherwise) to the same device, then they become a tonne cheaper on arm/arm64/powerpc. Currently we have to include expensive memory barriers in order to synchronise with accesses to DMA buffers which is rarely needed.
For those requirements, I don't think we need the "memory" clobber for x86, but would appreciate your views on this.
OK... first of all you didn't send the cover letter to the union of all the people you sent patches to, but second, documenting semantics in the one piece of the patchset that wouldn't make it into git is just about the worst possible place to put it.
This documentation is absolutely critical if we expect people to be able to use these correctly, including when additional barriers may be required.
As far as x86 is concerned, in gcc volatiles are ordered with respect to each other, so as you say I don't think we need a memory clobber here.
-hpa
Re: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 27 lines · H. Peter Anvin
Quoted 16 lines · Will Deacon
Quoted 10 lines · H. Peter Anvin
Quoted 4 lines · Will Deacon
I would like the relaxed accessors to be ordered with respect to each other...
What do you think?
I think "I would like" isn't a very good motivation. What are the semantics of these things supposed to be? It seems more than a bit odd to require them to be ordered with respect to each other and everything else (which is what a memory clobber does) and then call them "relaxed".
I suggested some informal semantics in the cover letter:
Quoted 9 lines
Basically, if we define relaxed accesses not to be ordered against anything apart from other accesses (relaxed or otherwise) to the same device, then they become a tonne cheaper on arm/arm64/powerpc. Currently we have to include expensive memory barriers in order to synchronise with accesses to DMA buffers which is rarely needed.
For those requirements, I don't think we need the "memory" clobber for x86, but would appreciate your views on this.
Quoted 7 lines
OK... first of all you didn't send the cover letter to the union of all the people you sent patches to, but second, documenting semantics in the one piece of the patchset that wouldn't make it into git is just about the worst possible place to put it.
This documentation is absolutely critical if we expect people to be able to use these correctly, including when additional barriers may be required.
There is also a documentation patch [1] in this series but, again, I didn't CC everybody on it. Sorry, but the level of interest this sort of stuff generates amongst kernel developers is close to zero so I only included people I thought cared on CC for the entire series. I'm stuck between a rock and a hard place trying to CC interested people whilst at the same time trying to avoid spamming all the arch maintainers.
I'll add you to CC if/when I post a third version. In the meantime, it's all archived on lkml and linux-arch.
Quoted 2 lines
As far as x86 is concerned, in gcc volatiles are ordered with respect to each other, so as you say I don't think we need a memory clobber here.
Thanks for the confirmation, I'll put that patch back like it was originally.
Will
H. Peter Anvin
reviewerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 8 lines · Will Deacon
There is also a documentation patch [1] in this series but, again, I didn't CC everybody on it. Sorry, but the level of interest this sort of stuff generates amongst kernel developers is close to zero so I only included people I thought cared on CC for the entire series. I'm stuck between a rock and a hard place trying to CC interested people whilst at the same time trying to avoid spamming all the arch maintainers.
If you are sending me a patch, please include me on the cover letter for the patch series. You don't have to send me the entire patch series (although for something like a Documentation patch which affects x86 I would consider including the union list as well.)
I think regardless of level of interest, the definition of cross-architectural operations is exactly the arch maintainers job, so it isn't really out of place to "spam" us...
-hpa
Peter Zijlstra
maintainerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 10 lines · H. Peter Anvin
Quoted 8 lines · Will Deacon
There is also a documentation patch [1] in this series but, again, I didn't CC everybody on it. Sorry, but the level of interest this sort of stuff generates amongst kernel developers is close to zero so I only included people I thought cared on CC for the entire series. I'm stuck between a rock and a hard place trying to CC interested people whilst at the same time trying to avoid spamming all the arch maintainers.
Quoted 8 lines
If you are sending me a patch, please include me on the cover letter for the patch series. You don't have to send me the entire patch series (although for something like a Documentation patch which affects x86 I would consider including the union list as well.)
I think regardless of level of interest, the definition of cross-architectural operations is exactly the arch maintainers job, so it isn't really out of place to "spam" us...
So the one issue I had with that, is that if one tries to send an email to all arch maintainers + linux-arch + linux-kernel, the header gets too big and vger chokes and davem slaps you.
So while its possibly desirable to do big unions with 0/xx and the like, its practically infeasible.
H. Peter Anvin
reviewerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 22 lines · Peter Zijlstra
Quoted 17 lines · H. Peter Anvin
Quoted 9 lines · Will Deacon
There is also a documentation patch [1] in this series but, again, I didn't CC everybody on it. Sorry, but the level of interest this sort of stuff generates amongst kernel developers is close to zero so I only included people I thought cared on CC for the entire series. I'm stuck between a rock and a hard place trying to CC interested people whilst at the same time trying to avoid spamming all the arch maintainers.
If you are sending me a patch, please include me on the cover letter for the patch series. You don't have to send me the entire patch series (although for something like a Documentation patch which affects x86 I would consider including the union list as well.)
Quoted 3 lines
I think regardless of level of interest, the definition of cross-architectural operations is exactly the arch maintainers job, so it isn't really out of place to "spam" us...
Quoted 7 lines
So the one issue I had with that, is that if one tries to send an email to all arch maintainers + linux-arch + linux-kernel, the header gets too big and vger chokes and davem slaps you.
So while its possibly desirable to do big unions with 0/xx and the like, its practically infeasible.
... or at least requires the use of a particular advanced technology called Bcc:. Not that I like Bcc:s, mind you, but it is probably better in this case.
I have no idea how smart vger is, but it would be interesting if it could ignore addresses from the MAINTAINERS list...
-hpa
Peter Zijlstra
maintainerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 11 lines · H. Peter Anvin
Quoted 7 lines
So the one issue I had with that, is that if one tries to send an email to all arch maintainers + linux-arch + linux-kernel, the header gets too big and vger chokes and davem slaps you.
So while its possibly desirable to do big unions with 0/xx and the like, its practically infeasible.
... or at least requires the use of a particular advanced technology called Bcc:. Not that I like Bcc:s, mind you, but it is probably better in this case.
That's actually a good idea, I'll use Bcc next time.
Quoted 2 lines
I have no idea how smart vger is, but it would be interesting if it could ignore addresses from the
MAINTAINERSlist...
I suppose we should ask davem, although if such a particular loop hole were known it might get abused and we'd all end up with more email :/
Geert Uytterhoeven
maintainerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
In reply to Peter ZijlstraQuoted 3 lines · Peter Zijlstra
So the one issue I had with that, is that if one tries to send an email to all arch maintainers + linux-arch + linux-kernel, the header gets too big and vger chokes and davem slaps you.
The arch maintainers are (supposed to be) on linux-arch.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
H. Peter Anvin
reviewerRe: [PATCH v2 16/18] x86: io: implement dummy relaxed accessor macros for writes
Quoted 7 lines · Geert Uytterhoeven
Quoted 3 lines · Peter Zijlstra
So the one issue I had with that, is that if one tries to send an email to all arch maintainers + linux-arch + linux-kernel, the header gets too big and vger chokes and davem slaps you.
The arch maintainers are (supposed to be) on linux-arch.
That doesn't mean we read it in real time like something that comes into our main inbox. The messed-up context is annoying.
-hpa
PATCH v2 17/18 documentation: memory-barriers: clarify relaxed io accessor semantics
In reply to Will DeaconThis patch extends the paragraph describing the relaxed read io accessors so that the relaxed accessors are defined to be:
- Ordered with respect to each other if accessing the same peripheral
- Unordered with respect to normal memory accesses
- Unordered with respect to LOCK/UNLOCK operations
Whilst many architectures will provide stricter semantics, ARM, Alpha and PPC can achieve significant performance gains by taking advantage of some or all of the above relaxations.
--- Documentation/memory-barriers.txt | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
Documentation/memory-barriers.txt +9 -4
@@ -2462,10 +2462,15 @@ functions: Please refer to the PCI specification for more information on interactions between PCI transactions. - (*) readX_relaxed() - - These are similar to readX(), but are not guaranteed to be ordered in any - way. Be aware that there is no I/O read barrier available. + (*) readX_relaxed(), writeX_relaxed() + + These are similar to readX() and writeX(), but provide weaker memory + ordering guarantees. Specifically, they do not guarantee ordering with + respect to normal memory accesses (e.g. DMA buffers) nor do they guarantee + ordering with respect to LOCK or UNLOCK operations. If the latter is + required, an mmiowb() barrier can be used. Note that relaxed accesses to + the same peripheral are guaranteed to be ordered with respect to each + other. (*) ioreadX(), iowriteX()
Signature · 1 line
1.9.2
PATCH v2 18/18 asm-generic: io: define relaxed accessor macros unconditionally
In reply to Will DeaconNow that no architectures using asm-generic/io.h define their own relaxed accessors, the dummy definitions can be used unconditionally.
--- include/asm-generic/io.h | 16 ---------------- 1 file changed, 16 deletions(-)
include/asm-generic/io.h +0 -16
@@ -53,27 +53,21 @@ static inline u32 __raw_readl(const volatile void __iomem *addr)#endif #define readb __raw_readb -#ifndef readb_relaxed#define readb_relaxed readb -#endif #define readw readwstatic inline u16 readw (const volatile void __iomem *addr) {return __le16_to_cpu (__raw_readw (addr)); } -#ifndef readw_relaxed#define readw_relaxed readw -#endif #define readl readlstatic inline u32 readl (const volatile void __iomem *addr) {return __le32_to_cpu (__raw_readl (addr)); } -#ifndef readl_relaxed#define readl_relaxed readl -#endif #ifndef __raw_writebstatic inline void __raw_writeb (u8 b,volatile void __iomem *addr)
@@ -97,19 +91,13 @@ static inline void __raw_writel(u32 b, volatile void __iomem *addr)#endif #define writeb __raw_writeb -#ifndef writeb_relaxed#define writeb_relaxed writeb -#endif #define writew (b,addr) __raw_writew(__cpu_to_le16(b),addr) -#ifndef writew_relaxed#define writew_relaxed writew -#endif #define writel (b,addr) __raw_writel(__cpu_to_le32(b),addr) -#ifndef writel_relaxed#define writel_relaxed writel -#endif #ifdef CONFIG_64BIT #ifndef __raw_readq
@@ -124,9 +112,7 @@ static inline u64 readq(const volatile void __iomem *addr) {return __le64_to_cpu (__raw_readq (addr)); } -#ifndef readq_relaxed#define readq_relaxed readq -#endif #ifndef __raw_writeqstatic inline void __raw_writeq (u64 b,volatile void __iomem *addr)
@@ -136,9 +122,7 @@ static inline void __raw_writeq(u64 b, volatile void __iomem *addr)#endif #define writeq (b, addr) __raw_writeq(__cpu_to_le64(b), addr) -#ifndef writeq_relaxed#define writeq_relaxed writeq -#endif #endif /* CONFIG_64BIT */ #ifndef PCI_IOBASE
Signature · 1 line
1.9.2
Quoted 9 lines · Will Deacon
A corollary to this is that
mmiowb()probably needs rethinking. As it currently stands, anmmiowb()is required to order MMIO writes to a device from multiple CPUs, even if that device is protected by a lock. However, this isn't often used in practice, leading to PowerPC implementing bothmmiowb()*and* synchronising I/O inspin_unlock.I would propose making the non-relaxed I/O accessors ordered with respect to LOCK/UNLOCK, leaving
mmiowb()to be used with the relaxed accessors, if required, but would welcome thoughts/suggestions on this topic.
I agree on the proposed semantics, though for us that does mean we still need that per-cpu flag tracking non-relaxed MMIO stores and corresponding added barrier in unlock. Eventually, if the use of the relaxed accessors becomes pervasive enough I suppose I can just make the ordered ones unconditionally do 2 barriers.
Cheers,
Ben.
Hi Ben,
Quoted 7 lines · Benjamin Herrenschmidt
Quoted 6 lines · Will Deacon
A corollary to this is that
mmiowb()probably needs rethinking. As it currently stands, anmmiowb()is required to order MMIO writes to a device from multiple CPUs, even if that device is protected by a lock. However, this isn't often used in practice, leading to PowerPC implementing bothmmiowb()*and* synchronising I/O inspin_unlock.
Quoted 8 lines
Quoted 3 lines
I would propose making the non-relaxed I/O accessors ordered with respect to LOCK/UNLOCK, leaving
mmiowb()to be used with the relaxed accessors, if required, but would welcome thoughts/suggestions on this topic.I agree on the proposed semantics, though for us that does mean we still need that per-cpu flag tracking non-relaxed MMIO stores and corresponding added barrier in unlock. Eventually, if the use of the relaxed accessors becomes pervasive enough I suppose I can just make the ordered ones unconditionally do 2 barriers.
Why would you need two barriers? I would have though an mmiowb() inlined into writel after the store operation would be sufficient. Or is this to ensure a non-relaxed write is ordered with respect to a relaxed write?
Anyway, we may need something similar for other architectures with mmiowb implementations:
blackfin
frv
ia64
mips
sh
so I'm anticipating some more discussion when I try to push that patch :)
Cheers,
Will
Quoted 3 lines · Will Deacon
Why would you need two barriers? I would have though an
mmiowb()inlined into writel after the store operation would be sufficient. Or is this to ensure a non-relaxed write is ordered with respect to a relaxed write?
Well, so the non-relaxed writel would have to do:
sync
store
sync
The first sync is to synchronize with DMAs, so that a sequence of
store to mem
writel
Remains ordered vs. the device (ie, when the writel causes the device to do a DMA, it will see the previous store to mem).
The second sync is needed as mmiowb, to order with unlocks.
At this point, I'm keen on keeping my per-cpu trick to avoid that second one in most cases.
Quoted 14 lines
Anyway, we may need something similar for other architectures with mmiowb implementations:
blackfin
frv
ia64
mips
shso I'm anticipating some more discussion when I try to push that patch :)
Cheers,
Will
Quoted 14 lines · Benjamin Herrenschmidt
Quoted 3 lines · Will Deacon
Why would you need two barriers? I would have though an
mmiowb()inlined into writel after the store operation would be sufficient. Or is this to ensure a non-relaxed write is ordered with respect to a relaxed write?Well, so the non-relaxed writel would have to do:
sync
store
syncThe first sync is to synchronize with DMAs, so that a sequence of
Quoted 7 lines
store to mem
writelRemains ordered vs. the device (ie, when the writel causes the device to do a DMA, it will see the previous store to mem).
The second sync is needed as mmiowb, to order with unlocks.
Ah yeah, thanks. I was so hung up on the ordering against locks that I completely forgot about DMA!
Quoted 2 lines
At this point, I'm keen on keeping my per-cpu trick to avoid that second one in most cases.
Makes sense. The alternative is dropping that requirement and instead relying on drivers to use mmiowb() even with the non-relaxed accessors, but I think that's going to be fairly painful (and hence why you have the trick to start with).
Will
Quoted 8 lines · Will Deacon
Hi all,
This is version 2 of the series I originally posted here:
https://lkml.org/lkml/2014/4/17/269
Changes since v1 include:
Quoted 7 lines
- Added relevant acks from arch maintainers
- Fixed potential compiler re-ordering issue for x86 definitionsI'd *really* appreciate some feedback on the proposed semantics here, but acks are still good :)
The original cover letter is duplicated below.
Question (sorry if I missed an existing explanation...), do we have an equivalent bunch for iomap ?
Cheers,
Ben.
Quoted 95 lines
Cheers,
Will
--->8
This RFC series attempts to define a portable (i.e. cross-architecture) definition of the {readX,writeX}_relaxed MMIO accessor functions. These functions are already in widespread use amongst drivers (mainly those supporting devices embedded in ARM SoCs), but lack any well-defined semantics and, subsequently, any portable definitions to allow these drivers to be compiled for other architectures.
The two main motivations for this series are:
(1) To promote use of the
_relaxedMMIO accessors on weakly-ordered architectures, where they can bring significant performance improvements over their non-relaxed counterparts.(2) To allow
COMPILE_TESTto build drivers using the relaxed accessors across all architectures.The proposed semantics largely match exactly those provided by the ARM implementation (i.e. no weaker), with one exception (see below).
Informally:
- Relaxed accesses to the same device are ordered with respect to each other.
- Relaxed accesses are *not* guaranteed to be ordered with respect to normal memory accesses (e.g. DMA buffers -- this is what gives us the performance boost over the non-relaxed versions).
- Relaxed accesses are not guaranteed to be ordered with respect to LOCK/UNLOCK operations.
In actual fact, the relaxed accessors *are* ordered with respect to LOCK/UNLOCK operations on ARM[64], but I have added this constraint for the benefit of PowerPC, which has expensive I/O barriers in the
spin_unlockpath for the non-relaxed accessors.A corollary to this is that
mmiowb()probably needs rethinking. As it currently stands, anmmiowb()is required to order MMIO writes to a device from multiple CPUs, even if that device is protected by a lock. However, this isn't often used in practice, leading to PowerPC implementing bothmmiowb()*and* synchronising I/O inspin_unlock.I would propose making the non-relaxed I/O accessors ordered with respect to LOCK/UNLOCK, leaving
mmiowb()to be used with the relaxed accessors, if required, but would welcome thoughts/suggestions on this topic.Will Deacon (18):
asm-generic: io: implement relaxed accessor macros as conditional
wrappers
microblaze: io: remove dummy relaxed accessor macros
s390: io: remove dummy relaxed accessor macros for reads
xtensa: io: remove dummy relaxed accessor macros for reads
alpha: io: implement relaxed accessor macros for writes
frv: io: implement dummy relaxed accessor macros for writes
cris: io: implement dummy relaxed accessor macros for writes
ia64: io: implement dummy relaxed accessor macros for writes
m32r: io: implement dummy relaxed accessor macros for writes
m68k: io: implement dummy relaxed accessor macros for writes
mn10300: io: implement dummy relaxed accessor macros for writes
parisc: io: implement dummy relaxed accessor macros for writes
powerpc: io: implement dummy relaxed accessor macros for writes
sparc: io: implement dummy relaxed accessor macros for writes
tile: io: implement dummy relaxed accessor macros for writes
x86: io: implement dummy relaxed accessor macros for writes
documentation: memory-barriers: clarify relaxed io accessor semantics
asm-generic: io: define relaxed accessor macros unconditionallyDocumentation/memory-barriers.txt | 13 +++++++++---- arch/alpha/include/asm/io.h | 12 ++++++++---- arch/cris/include/asm/io.h | 3 +++ arch/frv/include/asm/io.h | 3 +++ arch/ia64/include/asm/io.h | 4 ++++ arch/m32r/include/asm/io.h | 3 +++ arch/m68k/include/asm/io.h | 8 ++++++++ arch/m68k/include/asm/io_no.h | 4 ---- arch/microblaze/include/asm/io.h | 8 -------- arch/mn10300/include/asm/io.h | 4 ++++ arch/parisc/include/asm/io.h | 12 ++++++++---- arch/powerpc/include/asm/io.h | 12 ++++++++---- arch/s390/include/asm/io.h | 5 ----- arch/sparc/include/asm/io.h | 9 +++++++++ arch/sparc/include/asm/io_32.h | 3 --- arch/sparc/include/asm/io_64.h | 22 ++++++++++------------ arch/tile/include/asm/io.h | 4 ++++ arch/x86/include/asm/io.h | 10 +++++++--- arch/xtensa/include/asm/io.h | 7 ------- include/asm-generic/io.h | 10 ++++++++++ 20 files changed, 98 insertions(+), 58 deletions(-)
Quoted 12 lines · Benjamin Herrenschmidt
Quoted 11 lines · Will Deacon
Hi all,
This is version 2 of the series I originally posted here:
https://lkml.org/lkml/2014/4/17/269
Changes since v1 include:
- Added relevant acks from arch maintainers
- Fixed potential compiler re-ordering issue for x86 definitions
Quoted 7 lines
Quoted 4 lines
I'd *really* appreciate some feedback on the proposed semantics here, but acks are still good :)
The original cover letter is duplicated below.
Question (sorry if I missed an existing explanation...), do we have an equivalent bunch for iomap ?
Do you mean the io{read,write} functions? Funnily enough, they're already relaxed on ARM if you go by the semantics I've proposed. That implies we at least need some Documentation to that effect...
What do you do on ppc?
Will
Quoted 5 lines · Will Deacon
Do you mean the io{read,write} functions? Funnily enough, they're already relaxed on ARM if you go by the semantics I've proposed. That implies we at least need some Documentation to that effect...
What do you do on ppc?
They are not supposed to be relaxed. If they are, you probably have a whole lot of busted drivers :-)
They have the same semantics as readl/writel for memory and as inb/outb for IO space, they just allow to hide the "type" (memory vs. IO) from most of the driver code.
We probably need to create a set of _relaxed variants.
Cheers,
Ben.
Quoted 10 lines · Benjamin Herrenschmidt
Quoted 5 lines · Will Deacon
Do you mean the io{read,write} functions? Funnily enough, they're already relaxed on ARM if you go by the semantics I've proposed. That implies we at least need some Documentation to that effect...
What do you do on ppc?
They are not supposed to be relaxed. If they are, you probably have a whole lot of busted drivers :-)
Lucky me!
Quoted 5 lines
They have the same semantics as readl/writel for memory and as inb/outb for IO space, they just allow to hide the "type" (memory vs. IO) from most of the driver code.
We probably need to create a set of
_relaxedvariants.
Ok. I'll try putting together a v3 including this and the mmiowb work.
Thanks for the feedback,
Will