Use Normal uncached memory rather than Strongly Ordered on ARMv6+
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
ARMv6 onwards requires that there are no aliases to the same physical location using different memory types (i.e. Normal vs Strongly Ordered). Access to SO mappings when the unaligned accesses are handled in hardware is also Unpredictable (pgprot_noncached() mappings in user space).
The patch modifies the pgprot_noncached() for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to use dmb() rather than being simple compiler barriers.
--- arch/arm/include/asm/pgtable.h | 12 ++++++++++++ arch/arm/include/asm/system.h | 10 ++++++++++ 2 files changed, 22 insertions(+), 0 deletions(-)
arch/arm/include/asm/pgtable.h +12 -0
@@ -372,11 +372,23 @@ PTE_BIT_FUNC(mkyoung, |= L_PTE_YOUNG);static inline pte_t pte_mkspecial (pte_t pte) {return pte; } +#if __LINUX_ARM_ARCH__ <6 /* * Mark the prot value as uncacheable and unbufferable. */ #define pgprot_noncached (prot) \ __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_UNCACHED) +#else +/* +* Mark the prot value as uncacheable, bufferable (Normal uncached memory). +* ARMv6 onwards requires that there are no aliases to the same physical +* location using different memory types (i.e. Normal vs Strongly Ordered). +* Access to SO mappings when the unaligned accesses are handled in hardware +* is also Unpredictable (pgprot_noncached() mappings in user space). +*/ +#define pgprot_noncached (prot) \ + __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_BUFFERABLE) +#endif #define pgprot_writecombine (prot) \ __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_BUFFERABLE)
arch/arm/include/asm/system.h +10 -0
@@ -139,9 +139,19 @@ extern unsigned int user_debug;#endif #ifndef CONFIG_SMP +#if __LINUX_ARM_ARCH__ <6 #define mb () do { if (arch_is_coherent()) dmb(); else barrier(); } while (0)#define rmb () do { if (arch_is_coherent()) dmb(); else barrier(); } while (0)#define wmb () do { if (arch_is_coherent()) dmb(); else barrier(); } while (0) +#else +/* +* pgprot_noncached() creates Normal uncached mappings, therefore mandatory +* barriers are needed. +*/ +#define mb () dmb() +#define rmb () dmb() +#define wmb () dmb() +#endif #define smp_mb () barrier()#define smp_rmb () barrier()#define smp_wmb () barrier()
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 10 lines · Catalin Marinas
ARMv6 onwards requires that there are no aliases to the same physical location using different memory types (i.e. Normal vs Strongly Ordered). Access to SO mappings when the unaligned accesses are handled in hardware is also Unpredictable (
pgprot_noncached()mappings in user space).The patch modifies the
pgprot_noncached()for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to usedmb()rather than being simple compiler barriers.
It's not this simple - pgprot_noncached() is used for /dev/mem O_SYNC mappings, which are used for device mappings. This means you still end up with the unpredictable issue.
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 7 lines · Russell King - ARM Linux
Quoted 6 lines · Catalin Marinas
ARMv6 onwards requires that there are no aliases to the same physical location using different memory types (i.e. Normal vs Strongly Ordered). Access to SO mappings when the unaligned accesses are handled in hardware is also Unpredictable (
pgprot_noncached()mappings in user space).
Quoted 8 lines
Quoted 4 lines
The patch modifies the
pgprot_noncached()for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to usedmb()rather than being simple compiler barriers.It's not this simple -
pgprot_noncached()is used for/dev/memO_SYNCmappings, which are used for device mappings. This means you still end up with the unpredictable issue.
If it is used for accessing device memory, why not create a pgprot_device() and use this instead (maybe after checking for pfn_valid to make sure we don't map the RAM as device memory)? Of course, it needs some modifications to the mem.c driver which may be ARM specific, though some other architecture might benefit as well.
Quoted 1 line
From
drivers/char/mem.c:
/* * On ia64, we ignore O_SYNC because we cannot tolerate memory attribute aliases. */
So ARM it's not the only architecture with this issue (I know you don't like this argument :-)).
I can see MIPS has its own #ifdef in the mem.c driver as well.
Signature · 1 line
Catalin
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 17 lines · Catalin Marinas
Quoted 7 lines · Russell King - ARM Linux
Quoted 6 lines · Catalin Marinas
ARMv6 onwards requires that there are no aliases to the same physical location using different memory types (i.e. Normal vs Strongly Ordered). Access to SO mappings when the unaligned accesses are handled in hardware is also Unpredictable (
pgprot_noncached()mappings in user space).Quoted 8 lines
Quoted 4 lines
The patch modifies the
pgprot_noncached()for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to usedmb()rather than being simple compiler barriers.It's not this simple -
pgprot_noncached()is used for/dev/memO_SYNCmappings, which are used for device mappings. This means you still end up with the unpredictable issue.
Quoted 5 lines
If it is used for accessing device memory, why not create a
pgprot_device()and use this instead (maybe after checking forpfn_validto make sure we don't map the RAM as device memory)? Of course, it needs some modifications to themem.cdriver which may be ARM specific, though some other architecture might benefit as well.
People keep fiddling with /dev/mem, and it keeps having these problems. I suggest we leave that area well alone - we can not properly solve the /dev/mem problems. Fixing it to use "memory" mappings will break attempts to use it to access devices, and fixing it for "device" mappings will break attempts to access memory.
No amount of fiddling with uncached_access() can fix that.
When it comes to the writecombine and DMA coherent mappings, these are architecture private stuff. I guess on ARMv6 and v7, there is no difference between these two (since writecombine is already uncacheable normal memory).
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 18 lines · Russell King - ARM Linux
Quoted 11 lines · Catalin Marinas
Quoted 9 lines · Russell King - ARM Linux
Quoted 4 lines · Catalin Marinas
The patch modifies the
pgprot_noncached()for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to usedmb()rather than being simple compiler barriers.It's not this simple -
pgprot_noncached()is used for/dev/memO_SYNCmappings, which are used for device mappings. This means you still end up with the unpredictable issue.Quoted 5 lines
If it is used for accessing device memory, why not create a
pgprot_device()and use this instead (maybe after checking forpfn_validto make sure we don't map the RAM as device memory)? Of course, it needs some modifications to themem.cdriver which may be ARM specific, though some other architecture might benefit as well.
Quoted 7 lines
People keep fiddling with
/dev/mem, and it keeps having these problems. I suggest we leave that area well alone - we can not properly solve the/dev/memproblems. Fixing it to use "memory" mappings will break attempts to use it to access devices, and fixing it for "device" mappings will break attempts to access memory.No amount of fiddling with
uncached_access()can fix that.
You can also fix phys_mem_access_prot() to differentiate between standard and device memory. You can simply define __HAVE_PHYS_MEM_ACCESS_PROT and implement an ARM-specific function (we probably don't even need to modify uncached_access()).
I can send you a patch on this (or update the one I posted) which doesn't touch the mem.c file.
We really need to sort out this issue before we start to see people complaining of random failures as a result aliased memory (no specific case yet but I don't think it would be long).
Quoted 4 lines
When it comes to the writecombine and DMA coherent mappings, these are architecture private stuff. I guess on ARMv6 and v7, there is no difference between these two (since writecombine is already uncacheable normal memory).
Yes, the writecombine is fine.
But my patch is intended for dma_alloc_coherent which uses pgprot_noncached(). There are also several drivers that make use of pgprot_noncached(), especially framebuffer drivers which are not ARM-specific.
Signature · 1 line
Catalin
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 25 lines · Catalin Marinas
Quoted 24 lines · Russell King - ARM Linux
Quoted 11 lines · Catalin Marinas
Quoted 9 lines · Russell King - ARM Linux
Quoted 4 lines · Catalin Marinas
The patch modifies the
pgprot_noncached()for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to usedmb()rather than being simple compiler barriers.It's not this simple -
pgprot_noncached()is used for/dev/memO_SYNCmappings, which are used for device mappings. This means you still end up with the unpredictable issue.Quoted 5 lines
If it is used for accessing device memory, why not create a
pgprot_device()and use this instead (maybe after checking forpfn_validto make sure we don't map the RAM as device memory)? Of course, it needs some modifications to themem.cdriver which may be ARM specific, though some other architecture might benefit as well.People keep fiddling with
/dev/mem, and it keeps having these problems. I suggest we leave that area well alone - we can not properly solve the/dev/memproblems. Fixing it to use "memory" mappings will break attempts to use it to access devices, and fixing it for "device" mappings will break attempts to access memory.
Quoted 6 lines
Quoted 1 line
No amount of fiddling with
uncached_access()can fix that.You can also fix
phys_mem_access_prot()to differentiate between standard and device memory. You can simply define__HAVE_PHYS_MEM_ACCESS_PROTand implement an ARM-specific function (we probably don't even need to modifyuncached_access()).
I don't think you understand the problem. /dev/mem can map *both* memory and devices. Fixing the mapping type for one doesn't solve the problem. The only real way to fix it "properly" is to make /dev/mem lookup in a table to find the proper mapping type for a specific physical address, and get platforms to register these physical regions...
That said, /dev/mem is more or less legacy when it comes to frame buffers (which are supposed to use the frame buffer stuff - but iirc X11 stuff either took ages or just doesn't use the frame buffer stuff, especially with PCI cards.)
Quoted 3 lines
We really need to sort out this issue before we start to see people complaining of random failures as a result aliased memory (no specific case yet but I don't think it would be long).
That's a pipedream. Changing the memory type of dma_alloc_coherent() is going to produce random failures by itself, since device drivers as currently written for ARM assume that writes to this memory are strongly ordered.
The only way to avoid this is to audit all the ARM drivers first, before changing this. Not doing this will be a recipe for an unstable system - which could be really difficult to debug - eg, if DMA ends up scribbling over some random part of the kernel because the DMA address hasn't been written to the descriptor.
Quoted 6 lines
Quoted 4 lines
When it comes to the writecombine and DMA coherent mappings, these are architecture private stuff. I guess on ARMv6 and v7, there is no difference between these two (since writecombine is already uncacheable normal memory).
Yes, the writecombine is fine.
I don't think you read what I said. What I was implying is that on v6 and v7, there is (and can be) no difference between dma_alloc_coherent() and dma_alloc_writecombine() - see what I said in parens where I detail the type of memory _writecombine() provides you with, which is precisely what you're proposing the _coherent() interface returns.
Quoted 4 lines
But my patch is intended for
dma_alloc_coherentwhich usespgprot_noncached(). There are also several drivers that make use ofpgprot_noncached(), especially framebuffer drivers which are not ARM-specific.
These other drivers are a separate problem, and need auditing to see what type of mapping is actually required. pgprot_noncached() has been used both for things like frame buffers and for things like device mappings. To change it to suit one case is likely to break the other case.
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 33 lines · Russell King - ARM Linux
Quoted 25 lines · Catalin Marinas
Quoted 24 lines · Russell King - ARM Linux
Quoted 11 lines · Catalin Marinas
Quoted 9 lines · Russell King - ARM Linux
Quoted 4 lines · Catalin Marinas
The patch modifies the
pgprot_noncached()for ARMv6+ architecture versions to generate Normal uncached memory attributes rather than Strongly Ordered. The patch also modifies the mandatory barriers to usedmb()rather than being simple compiler barriers.It's not this simple -
pgprot_noncached()is used for/dev/memO_SYNCmappings, which are used for device mappings. This means you still end up with the unpredictable issue.Quoted 5 lines
If it is used for accessing device memory, why not create a
pgprot_device()and use this instead (maybe after checking forpfn_validto make sure we don't map the RAM as device memory)? Of course, it needs some modifications to themem.cdriver which may be ARM specific, though some other architecture might benefit as well.People keep fiddling with
/dev/mem, and it keeps having these problems. I suggest we leave that area well alone - we can not properly solve the/dev/memproblems. Fixing it to use "memory" mappings will break attempts to use it to access devices, and fixing it for "device" mappings will break attempts to access memory.Quoted 6 lines
Quoted 1 line
No amount of fiddling with
uncached_access()can fix that.You can also fix
phys_mem_access_prot()to differentiate between standard and device memory. You can simply define__HAVE_PHYS_MEM_ACCESS_PROTand implement an ARM-specific function (we probably don't even need to modifyuncached_access()).
Quoted 5 lines
I don't think you understand the problem.
/dev/memcan map *both* memory and devices. Fixing the mapping type for one doesn't solve the problem. The only real way to fix it "properly" is to make/dev/memlookup in a table to find the proper mapping type for a specific physical address, and get platforms to register these physical regions...
Something like below defined under arch/arm (if it's not RAM, we probably don't want Normal cached memory anyway, so no check for O_SYNC):
#if __LINUX_ARM_ARCH__ >= 6pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t vma_prot)
{
if (!pfn_valid(pfn))
return pgprot_device(vma_prot);
else if (file->f_flags & O_SYNC)
return pgprot_noncached(vma_prot);
return vma_prot;
}EXPORT_SYMBOL(phys_mem_access_prot);
#endif
Quoted 11 lines
Quoted 3 lines
We really need to sort out this issue before we start to see people complaining of random failures as a result aliased memory (no specific case yet but I don't think it would be long).
That's a pipedream. Changing the memory type of
dma_alloc_coherent()is going to produce random failures by itself, since device drivers as currently written for ARM assume that writes to this memory are strongly ordered.The only way to avoid this is to audit all the ARM drivers first, before changing this.
Yes, I don't say that's not needed (together with testing). If you want, we can restrict this requirement to ARMv7 onwards only (that's where it actually matters most) and there aren't many platforms around. The driver fix is simple enough - adding mb() before starting DMA transfers (I don't think this contradicts Documentation/memory-barriers.txt). I can see some well written drivers already use wmb() in functions like start_xmit().
Alternatively, we can place a DMB in readl/writel but I'm not sure about the performance impact.
Of course, you can use a more complicated workaround and ensure that coherent mappings first unmap the pages from the linear kernel mapping but we end up using pages instead of sections.
Quoted 4 lines
Not doing this will be a recipe for an unstable
system - which could be really difficult to debug - eg, if DMA ends up scribbling over some random part of the kernel because the DMA address hasn't been written to the descriptor.
The current configuration will lead to unstable systems in the future anyway (especially with processors doing aggressive prefetching via the cached alias). That would be even more difficult to debug.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
When it comes to the writecombine and DMA coherent mappings, these are architecture private stuff. I guess on ARMv6 and v7, there is no difference between these two (since writecombine is already uncacheable normal memory).
Quoted 8 lines
Quoted 1 line
Yes, the writecombine is fine.
I don't think you read what I said. What I was implying is that on v6 and v7, there is (and can be) no difference between
dma_alloc_coherent()anddma_alloc_writecombine()- see what I said in parens where I detail the type of memory_writecombine()provides you with, which is precisely what you're proposing the_coherent()interface returns.
I agree, with my patch there is no difference between coherent and writecombine. Why does it need to be one?
Signature · 1 line
Catalin
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 21 lines · Catalin Marinas
Quoted 5 lines · Russell King - ARM Linux
I don't think you understand the problem.
/dev/memcan map *both* memory and devices. Fixing the mapping type for one doesn't solve the problem. The only real way to fix it "properly" is to make/dev/memlookup in a table to find the proper mapping type for a specific physical address, and get platforms to register these physical regions...Something like below defined under
arch/arm(if it's not RAM, we probably don't want Normal cached memory anyway, so no check forO_SYNC):#if
__LINUX_ARM_ARCH__>= 6pgprot_tphys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size,pgprot_tvma_prot){ if (!pfn_valid(pfn)) return pgprot_device(vma_prot); else if (file->f_flags & O_SYNC) return pgprot_noncached(vma_prot); return vma_prot; }
Quoted 2 lines
EXPORT_SYMBOL(phys_mem_access_prot);
#endif
This WILL NOT WORK, as I've tried to explain.
For example, what about framebuffers which aren't part of system RAM? With the above, you end up mapping them as device, but the kernel will map them as normal memory. So you still end up with the same physical address mapped using two different types.
It is not as simple as you seem to think, which is the point I've been *painfully* trying to get across to you, and apparantly keeps being ignored.
Quoted 8 lines
Quoted 7 lines
That's a pipedream. Changing the memory type of
dma_alloc_coherent()is going to produce random failures by itself, since device drivers as currently written for ARM assume that writes to this memory are strongly ordered.The only way to avoid this is to audit all the ARM drivers first, before changing this.
Quoted 7 lines
Yes, I don't say that's not needed (together with testing). If you want, we can restrict this requirement to ARMv7 onwards only (that's where it actually matters most) and there aren't many platforms around. The driver fix is simple enough - adding
mb()before starting DMA transfers (I don't think this contradictsDocumentation/memory-barriers.txt). I can see some well written drivers already usewmb()in functions likestart_xmit().
Indeed, and as I say this is what needs to happen first. But there is no option (as you seem to want) to do this without causing instability. One way or another, we're going to get unstable systems - the choice is whether we get them *now* by changing the memory returned by dma_alloc_coherent() or *later* when we actually have platforms which suffer a problem.
Quoted 2 lines
Alternatively, we can place a DMB in readl/writel but I'm not sure about the performance impact.
If you do that, you might as well use strongly ordered memory for device mappings - since you'll be causing the CPU to stall after each access.
Quoted 3 lines
Of course, you can use a more complicated workaround and ensure that coherent mappings first unmap the pages from the linear kernel mapping but we end up using pages instead of sections.
Please put some thought into your statements before you make them. This is impossible without getting rid of the section mappings for the kernel's direct-mapped pages, and converting them to 4K page mappings. That means a lot of additional TLB pressure which we _already_ know people will not be happy with.
There is no way in hell that this would be a sane solution.
Quoted 9 lines
Quoted 6 lines
I don't think you read what I said. What I was implying is that on v6 and v7, there is (and can be) no difference between
dma_alloc_coherent()anddma_alloc_writecombine()- see what I said in parens where I detail the type of memory_writecombine()provides you with, which is precisely what you're proposing the_coherent()interface returns.I agree, with my patch there is no difference between coherent and writecombine. Why does it need to be one?
Because writecombine is unsafe for older architectures to use for coherent DMA - it isn't coherent. Especially when you realise that older architecture write buffers are buggy as hell by allowing writes to buffered memory to be randomly reordered, in spite of documentation to the contary.
If you have a shared ownership ring buffer, you need the writes for updating the DMA pointers to hit it before the write to change the ownership of the descriptor to device. Reverse those two writes and you end up transmitting junk.
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 18 lines · Russell King - ARM Linux
Quoted 14 lines · Catalin Marinas
Something like below defined under
arch/arm(if it's not RAM, we probably don't want Normal cached memory anyway, so no check forO_SYNC):#if
__LINUX_ARM_ARCH__>= 6pgprot_tphys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size,pgprot_tvma_prot){ if (!pfn_valid(pfn)) return pgprot_device(vma_prot); else if (file->f_flags & O_SYNC) return pgprot_noncached(vma_prot); return vma_prot; }Quoted 2 lines
EXPORT_SYMBOL(phys_mem_access_prot);
#endif
Quoted 6 lines
This WILL NOT WORK, as I've tried to explain.
For example, what about framebuffers which aren't part of system RAM? With the above, you end up mapping them as device, but the kernel will map them as normal memory. So you still end up with the same physical address mapped using two different types.
In this case we would need other ways of checking whether a page is RAM, maybe having a per-platform phys_mem_access_prot function (similar to what powerpc seems to do) which is aware of other RAM-like areas and what drivers are used.
It's not simple but IMHO not impossible either.
Quoted 5 lines
Quoted 2 lines
Alternatively, we can place a DMB in readl/writel but I'm not sure about the performance impact.
If you do that, you might as well use strongly ordered memory for device mappings - since you'll be causing the CPU to stall after each access.
I'm not sure that's guaranteed. The ARM ARM doesn't state any ordering requirements between normal memory (cached or uncached) and strongly ordered memory.
Quoted 9 lines
Quoted 3 lines
Of course, you can use a more complicated workaround and ensure that coherent mappings first unmap the pages from the linear kernel mapping but we end up using pages instead of sections.
Please put some thought into your statements before you make them. This is impossible without getting rid of the section mappings for the kernel's direct-mapped pages, and converting them to 4K page mappings. That means a lot of additional TLB pressure which we _already_ know people will not be happy with.
I'm aware of this and I don't think its a viable solution. An alternative is to allocate already unmapped pages (highmem) in dma_alloc_coherent but I haven't explored this variant.
Signature · 1 line
Catalin
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
In reply to Russell King - ARM LinuxQuoted 4 lines · Russell King - ARM Linux
If you have a shared ownership ring buffer, you need the writes for updating the DMA pointers to hit it before the write to change the ownership of the descriptor to device. Reverse those two writes and you end up transmitting junk.
But if we have arch_is_coherent() enabled (possibly with hardware support like ARM's ACP), we need the drivers to have barriers anyway since we would use normal cached memory (and ordering isn't guaranteed). Of course, one has to go through the drivers used with a platform and check them but we already have a precedent.
BTW, there are two other architectures (x86 and PowerPC) with a similar restriction. Do they do anything about this in Linux?
PowerPC, 4.8.1 (page 36):
x86, 10.12.4 (page 10-44):
ftp://download.intel.com/support/processors/celeron/sb/25366820.pdf
Signature · 1 line
Catalin
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 11 lines · Catalin Marinas
Quoted 4 lines · Russell King - ARM Linux
If you have a shared ownership ring buffer, you need the writes for updating the DMA pointers to hit it before the write to change the ownership of the descriptor to device. Reverse those two writes and you end up transmitting junk.
But if we have
arch_is_coherent()enabled (possibly with hardware support like ARM's ACP), we need the drivers to have barriers anyway since we would use normal cached memory (and ordering isn't guaranteed). Of course, one has to go through the drivers used with a platform and check them but we already have a precedent.
Actually, we do not. arch_is_coherent() is for one ARM platform (Intel IXP) which has their own DMA coherent support for PCI devices - devices which quite possibly already have the necessary barriers.
Quoted 2 lines
BTW, there are two other architectures (x86 and PowerPC) with a similar restriction. Do they do anything about this in Linux?
PPC doesn't appear to - they seem to live with multiple mappings for the same space with different attributes.
x86 is fully cache coherent, and so doesn't have the DMA issue to worry about - there's no need for dma_alloc_coherent() to be anything but a call to get_free_pages().
ARM is more complex because we don't have any coherency hardware, and the only way to safely get the required semantics for this memory is by returning "strongly ordered" on ARMv5 and below.
In summary, the concern is:
1. The ARM support is currently setup to return strongly ordered memory.
2. Drivers are _currently_ written with this expectation in mind no matter what CPU architecture is in use.
3. Relaxing the returned memory type will destabilise systems, especially on ARMv6 and above.
As far as I'm concerned, we've spent far too long discussing this issue - there's nothing really to discuss. We need dma_alloc_coherent() to use its own page protection modifier, which causes it to behave the same as dma_alloc_writecombine() on ARMv7, and with the existing behaviour (for the time being) on ARMv6 and below. We should leave pgprot_noncached() well alone until we know that its other places need to be changed.
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 10 lines
From: Russell King - ARM Linux [mailto:linux at arm.linux.org.uk]
Sent: Friday, October 23, 2009 12:12 PMAs far as I'm concerned, we've spent far too long discussing this issue - there's nothing really to discuss. We need
dma_alloc_coherent()to use its own page protection modifier, which causes it to behave the same asdma_alloc_writecombine()on ARMv7, and with the existing behaviour (for the time being) on ARMv6 and below. We should leavepgprot_noncached()well alone until we know that its other places need to be changed.
We do have a few ARMv7 versions in the pipeline at various stages. ARM has projected speculative access is going to be an issue and we do believe them. Any tools which will help here and finding the problems will be useful. Some of these probably will be internal hack patches. However, once hardware is fully out there having some conditionals out there would help everyone out.
This week there was some odd coherency issue yet to be explained. That eventual debug will look at this as one of the angles.
On CortexA8-OMAP3430 in the field, if L2-Speculative access is enabled using AuxCR, defiantly we did run into some issues. In instance I know about the L2 speculative fetch had some collision with secure protection. This happened across a SMI to monitor mode. Perhaps it could have as easily been elsewhere like what will be up coming.
Regards,
Richard W.
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
In reply to Russell King - ARM LinuxQuoted 6 lines · Russell King - ARM Linux
As far as I'm concerned, we've spent far too long discussing this issue - there's nothing really to discuss. We need
dma_alloc_coherent()to use its own page protection modifier, which causes it to behave the same asdma_alloc_writecombine()on ARMv7, and with the existing behaviour (for the time being) on ARMv6 and below. We should leavepgprot_noncached()well alone until we know that its other places need to be changed.
And here is the patch which does this. Note that it also ensures that NX is set for these pages on both ARMv6 and later.
arch/arm/include/asm/pgtable.h +12 -2
@@ -304,13 +304,23 @@ PTE_BIT_FUNC(mkyoung, |= L_PTE_YOUNG);static inline pte_t pte_mkspecial (pte_t pte) {return pte; } +#define __pgprot_modify (prot,mask,bits) \ + __pgprot((pgprot_val(prot) & ~(mask)) | (bits)) +/* * Mark the prot value as uncacheable and unbufferable. */ #define pgprot_noncached (prot) \ - __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_UNCACHED) + __pgprot_modify(prot, L_PTE_MT_MASK, L_PTE_MT_UNCACHED)#define pgprot_writecombine (prot) \ - __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_BUFFERABLE) + __pgprot_modify(prot, L_PTE_MT_MASK, L_PTE_MT_BUFFERABLE) +#if __LINUX_ARM_ARCH__ >=7 +#define pgprot_dmacoherent (prot) \ + __pgprot_modify(prot, L_PTE_MT_MASK|L_PTE_EXEC, L_PTE_MT_BUFFERABLE) +#else +#define pgprot_dmacoherent (prot) \ + __pgprot_modify(prot, L_PTE_MT_MASK|L_PTE_EXEC, L_PTE_MT_UNCACHED) +#endif #define pmd_none (pmd) (!pmd_val(pmd))#define pmd_present (pmd) (pmd_val(pmd))
arch/arm/mm/dma-mapping.c +2 -2
@@ -328,7 +328,7 @@ dma_alloc_coherent(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gf }return __dma_alloc (dev, size, handle, gfp, -pgprot_noncached (pgprot_kernel)); +pgprot_dmacoherent (pgprot_kernel)); }EXPORT_SYMBOL (dma_alloc_coherent);
@@ -379,7 +379,7 @@ static int dma_mmap(struct device *dev, struct vm_area_struct *vma,int dma_mmap_coherent (struct device *dev,struct vm_area_struct *vma,void *cpu_addr,dma_addr_t dma_addr,size_t size) { - vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot); + vma->vm_page_prot =pgprot_dmacoherent (vma->vm_page_prot);return dma_mmap (dev, vma, cpu_addr, dma_addr, size); }EXPORT_SYMBOL (dma_mmap_coherent);
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
In addition we would need to change mb() rmb() wmb() to expand to dmb() for arch = v7
Abhijeet Dharmapurikar
Quoted 73 lines · Russell King - ARM Linux
Quoted 7 lines
As far as I'm concerned, we've spent far too long discussing this issue -
there's nothing really to discuss. We needdma_alloc_coherent()to use its own page protection modifier, which causes it to behave the same asdma_alloc_writecombine()on ARMv7, and with the existing behaviour (for the time being) on ARMv6 and below. We should leavepgprot_noncached()well alone until we know that its other places need to be changed.And here is the patch which does this. Note that it also ensures that NX is set for these pages on both ARMv6 and later.
arch/arm/include/asm/pgtable.h +12
-2@@ -304,13 +304,23 @@ PTE_BIT_FUNC(mkyoung, |= L_PTE_YOUNG);static inline pte_t pte_mkspecial (pte_t pte) {return pte; } +#define __pgprot_modify (prot,mask,bits) \ + __pgprot((pgprot_val(prot) & ~(mask)) | (bits)) +/* * Mark the prot value as uncacheable and unbufferable. */ #define pgprot_noncached (prot) \ - __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_UNCACHED) + __pgprot_modify(prot, L_PTE_MT_MASK, L_PTE_MT_UNCACHED)#define pgprot_writecombine (prot) \ - __pgprot((pgprot_val(prot) & ~L_PTE_MT_MASK) | L_PTE_MT_BUFFERABLE) + __pgprot_modify(prot, L_PTE_MT_MASK, L_PTE_MT_BUFFERABLE) +#if __LINUX_ARM_ARCH__ >=7 +#define pgprot_dmacoherent (prot) \ + __pgprot_modify(prot, L_PTE_MT_MASK|L_PTE_EXEC, L_PTE_MT_BUFFERABLE) +#else +#define pgprot_dmacoherent (prot) \ + __pgprot_modify(prot, L_PTE_MT_MASK|L_PTE_EXEC, L_PTE_MT_UNCACHED) +#endif #define pmd_none (pmd) (!pmd_val(pmd))#define pmd_present (pmd) (pmd_val(pmd))arch/arm/mm/dma-mapping.c +0
-0@@ -328,7 +328,7 @@ dma_alloc_coherent(struct device *dev, size_t size,
dma_addr_t*handle,gfp_tgfarch/arm/mm/dma-mapping.c +2
-2}return __dma_alloc (dev, size, handle, gfp, -pgprot_noncached (pgprot_kernel)); +pgprot_dmacoherent (pgprot_kernel)); }EXPORT_SYMBOL (dma_alloc_coherent);@@ -379,7 +379,7 @@ static int dma_mmap(struct device *dev, structvm_area_struct *vma, int dma_mmap_coherent(struct device *dev, struct vm_area_struct *vma, void *cpu_addr, dma_addr_t dma_addr, size_t size) { - vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + vma->vm_page_prot = pgprot_dmacoherent(vma->vm_page_prot);return dma_mmap (dev, vma, cpu_addr, dma_addr, size); }
EXPORT_SYMBOL(dma_mmap_coherent);_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
In reply to Russell King - ARM LinuxQuoted 10 lines · Russell King - ARM Linux
Quoted 6 lines · Russell King - ARM Linux
As far as I'm concerned, we've spent far too long discussing this issue - there's nothing really to discuss. We need
dma_alloc_coherent()to use its own page protection modifier, which causes it to behave the same asdma_alloc_writecombine()on ARMv7, and with the existing behaviour (for the time being) on ARMv6 and below. We should leavepgprot_noncached()well alone until we know that its other places need to be changed.And here is the patch which does this. Note that it also ensures that NX is set for these pages on both ARMv6 and later.
With Abhijeet's comment for mb(), I'm ok with the patch.
There are a few other explicit uses for pgprot_noncached() in the kernel but I don't think they affect us yet. Some of them could probably be changed to writecombine.
For /dev/mem, I still think we should do what IA-64 or PPC do - using cached or writecombine if it is known to be RAM (yes, I know what you said in the previous e-mails, it's not simple but I'll try to post another patch for that).
Signature · 1 line
Catalin
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
In reply to Catalin MarinasQuoted 5 lines · Catalin Marinas
But if we have
arch_is_coherent()enabled (possibly with hardware support like ARM's ACP), we need the drivers to have barriers anyway since we would use normal cached memory (and ordering isn't guaranteed). Of course, one has to go through the drivers used with a platform and check them but we already have a precedent.
Our Scorpion target definitely needs barriers in the drivers. There aren't any devices available externally, yet, but we can test things internally.
David
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
In reply to Russell King - ARM LinuxQuoted 5 lines · Russell King - ARM Linux
Quoted 2 lines
Alternatively, we can place a DMB in readl/writel but I'm not sure about the performance impact.
If you do that, you might as well use strongly ordered memory for device mappings - since you'll be causing the CPU to stall after each access.
Actually, the DMB is stronger than strongly ordered. Strongly ordered only guarantees ordering within somewhat arbitrarily defined (and not completely settled) 1KB boundaries, for memory mapped peripherals.
David
PATCH 1/1 Use Normal uncached memory rather than Strongly Ordered on ARMv6+
Quoted 5 lines · David Brown
Quoted 3 lines · Russell King - ARM Linux
Quoted 2 lines
Alternatively, we can place a DMB in readl/writel but I'm not sure about the performance impact.
Quoted 7 lines
Quoted 2 lines
If you do that, you might as well use strongly ordered memory for device mappings - since you'll be causing the CPU to stall after each access.
Actually, the DMB is stronger than strongly ordered. Strongly ordered only guarantees ordering within somewhat arbitrarily defined (and not completely settled) 1KB boundaries, for memory mapped peripherals.
I don't believe so for ARMv6 and below. The ARMv6 ARM (DDI0100I) is quite explicit about the ordering there and contains no such exception (I checked tonight). See B2.5, figure B2-1 and its associated text.
ARMv7 is a different kettle of fish, where memory accesses are completely unordered with respect to any other kind of access.