Add memcpy_nc, a copy routine that tries to keep cache pressure down
Subsystems (2)
PATCH 1/1 Add memcpy_nc, a copy routine that tries to keep cache pressure down
This copy routine is memcpy-compatible, but on some architectures will use cache-bypassing loads to avoid bringing the source data into the cache.
One case where this is useful is when a device issues a DMA to a memory region, and the CPU must copy the DMAed data elsewhere before doing any work with it. Since the source data is read-once, write-never from the CPU's perspective, caching those addresses can only evict potentially useful data.
We provide an x86_64 implementation that uses SSE non-temporal loads, and a generic version that falls back to plain memcpy.
Implementors for other arches should not use cache-bypassing stores to the destination, as in most cases, the destination is accessed almost immediately after a copy finishes.
include/linux/string.h +1 -0
@@ -85,6 +85,7 @@ extern void * memset(void *,int,__kernel#ifndef __HAVE_ARCH_MEMCPYextern void *memcpy (void *,const void *,__kernel_size_t );#endif +extern void *memcpy_nc (void *,const void *,__kernel_size_t );#ifndef __HAVE_ARCH_MEMMOVEextern void *memmove (void *,const void *,__kernel_size_t );#endif
lib/string.c +32 -0
@@ -509,6 +509,38 @@ EXPORT_SYMBOL(memcpy);EXPORT_SYMBOL (memcpy);#endif +void *memcpy_nc (void *dest,const void *src,size_t count) + __attribute__((weak)); + +/** +* memcpy_nc - Copy one area of memory to another, if possible +* bypassing the CPU's cache when loading the copied-from data +* @dest: Where to copy to +* @src: Where to copy from (bypassing the CPU's cache, if possible) +* @count: The size of the area. +* +* This memcpy-compatible routine is intended for use when the CPU +* only reads the source data once. It is useful when, for example, a +* hardware device writes to a memory region, and the CPU needs to +* copy this data somewhere else before working on it. In such a +* case, caching the source addresses only serves to evict possibly +* useful data that will probably have to be reloaded. +* +* An arch-specific implementation should not attempt to bypass the +* cache when storing to the destination, as copied data is usually +* accessed almost immediately after a copy finishes. +* +* This routine does not *guarantee* that the source addresses won't +* be cached; a user of this code must not rely on this behaviour for +* correctness. It should only be used in cases where it provides a +* measurable performance improvement. +*/ +void *memcpy_nc (void *dest,const void *src,size_t count) +{ +return memcpy (dest, src, count); +} +EXPORT_SYMBOL_GPL (memcpy_nc); +#ifndef __HAVE_ARCH_MEMMOVE/** * memmove - Copy one area of memory to another
arch/x86_64/lib/memcpy_nc.S new file +142 -0
142 lines not on this page · raw message
Quoted 11 lines · Bryan O'Sullivan
This copy routine is memcpy-compatible, but on some architectures will use cache-bypassing loads to avoid bringing the source data into the cache.
One case where this is useful is when a device issues a DMA to a memory region, and the CPU must copy the DMAed data elsewhere before doing any work with it. Since the source data is read-once, write-never from the CPU's perspective, caching those addresses can only evict potentially useful data.
We provide an
x86_64implementation that uses SSE non-temporal loads, and a generic version that falls back to plain memcpy.
Hi,
can we give this a more descriptive name please? Say memcpy_cachebypass() or something? "_nc" doesn't say a lot of things to many people... which means it'll either not get used when it should, or it gets used when it shouldn't....
Greetings,
Arjan van de Ven
Quoted 2 lines · Arjan van de Ven
can we give this a more descriptive name please? Say
memcpy_cachebypass()or something?
Sure. I figured there was no way to coax a better name out of people short of posting a patch with a bad name :-)
<b