Skip to contentSkip to conversation

Add memcpy_nc, a copy routine that tries to keep cache pressure down

PATCH In review Bryan O'Sullivan 2 participants 3 messages Jul 1, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add memcpy_nc, a copy routine that tries to keep cache pressure down

This copy routine is memcpy-compatible, but on some architectures will use cache-bypassing loads to avoid bringing the source data into the cache.

One case where this is useful is when a device issues a DMA to a memory region, and the CPU must copy the DMAed data elsewhere before doing any work with it. Since the source data is read-once, write-never from the CPU's perspective, caching those addresses can only evict potentially useful data.

We provide an x86_64 implementation that uses SSE non-temporal loads, and a generic version that falls back to plain memcpy.

Implementors for other arches should not use cache-bypassing stores to the destination, as in most cases, the destination is accessed almost immediately after a copy finishes.

include/linux/string.h +1 -0
diff -r 8bf6291faa35 -r c37555581c77 include/linux/string.h
@@ -85,6 +85,7 @@ extern void * memset(void *,int,__kernel
 #ifndef __HAVE_ARCH_MEMCPY
 extern void * memcpy(void *,const void *,__kernel_size_t);
 #endif
+extern void * memcpy_nc(void *,const void *,__kernel_size_t);
 #ifndef __HAVE_ARCH_MEMMOVE
 extern void * memmove(void *,const void *,__kernel_size_t);
 #endif
lib/string.c +32 -0
diff -r 8bf6291faa35 -r c37555581c77 lib/string.c
@@ -509,6 +509,38 @@ EXPORT_SYMBOL(memcpy);
 EXPORT_SYMBOL(memcpy);
 #endif
 
+void *memcpy_nc(void *dest, const void *src, size_t count)
+	__attribute__((weak));
+
+/**
+ * memcpy_nc - Copy one area of memory to another, if possible
+ * bypassing the CPU's cache when loading the copied-from data
+ * @dest: Where to copy to
+ * @src: Where to copy from (bypassing the CPU's cache, if possible)
+ * @count: The size of the area.
+ *
+ * This memcpy-compatible routine is intended for use when the CPU
+ * only reads the source data once.  It is useful when, for example, a
+ * hardware device writes to a memory region, and the CPU needs to
+ * copy this data somewhere else before working on it.  In such a
+ * case, caching the source addresses only serves to evict possibly
+ * useful data that will probably have to be reloaded.
+ *
+ * An arch-specific implementation should not attempt to bypass the
+ * cache when storing to the destination, as copied data is usually
+ * accessed almost immediately after a copy finishes.
+ *
+ * This routine does not *guarantee* that the source addresses won't
+ * be cached; a user of this code must not rely on this behaviour for
+ * correctness.  It should only be used in cases where it provides a
+ * measurable performance improvement.
+ */
+void *memcpy_nc(void *dest, const void *src, size_t count)
+{
+	return memcpy(dest, src, count);
+}
+EXPORT_SYMBOL_GPL(memcpy_nc);
+
 #ifndef __HAVE_ARCH_MEMMOVE
 /**
  * memmove - Copy one area of memory to another
arch/x86_64/lib/memcpy_nc.S new file +142 -0

142 lines not on this page · raw message

#1 lore raw
Quoted 11 lines · Bryan O'Sullivan

This copy routine is memcpy-compatible, but on some architectures will use cache-bypassing loads to avoid bringing the source data into the cache.

One case where this is useful is when a device issues a DMA to a memory region, and the CPU must copy the DMAed data elsewhere before doing any work with it. Since the source data is read-once, write-never from the CPU's perspective, caching those addresses can only evict potentially useful data.

We provide an x86_64 implementation that uses SSE non-temporal loads, and a generic version that falls back to plain memcpy.

Hi,

can we give this a more descriptive name please? Say memcpy_cachebypass() or something? "_nc" doesn't say a lot of things to many people... which means it'll either not get used when it should, or it gets used when it shouldn't....

Greetings,
Arjan van de Ven

#2 lore raw
Quoted 2 lines · Arjan van de Ven

can we give this a more descriptive name please? Say memcpy_cachebypass() or something?

Sure. I figured there was no way to coax a better name out of people short of posting a patch with a bad name :-)

	<b