Skip to contentSkip to conversation

uaccess: unify inline vs outline copy_{from,to}_user() selection

PATCH Applied Yury Norov 4 participants 7 messages Apr 25 – Apr 27

Subsystems (19)
Arm (32-bit) Library code MIPS arm64 +15 more subsystems
lore
Share & tools
lore

Yury Norov

author
#0 lore raw

v2 cover uaccess: unify inline vs outline copy_{from,to}_user() selection

The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.

Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.

The first patch of the series fixes rust/uaccess coppy_to_user() wrapper guarded with INLINE_COPY_FROM_USER. The 2nd patch switches codebase to the unified INLINE_COPY_USER. And the last patch cleans up ifdefery in the include/linux/uaccess.h

---
v1: https://lore.kernel.org/all/[email protected]/
v2:
- reword cover letter (Andrew);
- add v2#1 to fix rust/uaccess explicitly (Alice);

Yury Norov (3):
rust: uaccess: use INLINE_COPY_TO_USER to guard copy_to_user()
uaccess: unify inline vs outline copy_{from,to}_user() selection
uaccess: minimize INLINE_COPY_USER-related ifdefery

 arch/arc/include/asm/uaccess.h        |  3 +--
 arch/arm/include/asm/uaccess.h        |  3 +--
 arch/arm64/include/asm/uaccess.h      |  3 +--
 arch/hexagon/include/asm/uaccess.h    |  3 +--
 arch/loongarch/include/asm/uaccess.h  |  3 +--
 arch/m68k/include/asm/uaccess.h       |  3 +--
 arch/microblaze/include/asm/uaccess.h |  3 +--
 arch/mips/include/asm/uaccess.h       |  3 +--
 arch/nios2/include/asm/uaccess.h      |  3 +--
 arch/openrisc/include/asm/uaccess.h   |  3 +--
 arch/parisc/include/asm/uaccess.h     |  3 +--
 arch/s390/include/asm/uaccess.h       |  3 +--
 arch/sh/include/asm/uaccess.h         |  3 +--
 arch/sparc/include/asm/uaccess_32.h   |  3 +--
 arch/sparc/include/asm/uaccess_64.h   |  3 +--
 arch/um/include/asm/uaccess.h         |  3 +--
 arch/xtensa/include/asm/uaccess.h     |  3 +--
 include/asm-generic/uaccess.h         |  3 +--
 include/linux/uaccess.h               | 25 +++++++++----------------
 lib/usercopy.c                        |  4 +---
 rust/helpers/uaccess.c                |  2 +-
 21 files changed, 29 insertions(+), 56 deletions(-)
Signature · 1 line
2.51.0
author
#1 lore raw

PATCH v2 1/3 rust: uaccess: use INLINE_COPY_TO_USER to guard copy_to_user()

The copy_to_user() rust helper is only needed when the main kernel inlines the function. It is controlled by INLINE_COPY_TO_USER, but the rust helper is protected with INLINE_COPY_FROM_USER.

Fix that.

Fixes d99dc586ca7c7 ("uaccess: decouple INLINE_COPY_FROM_USER and CONFIG_RUST")
---
 rust/helpers/uaccess.c | 2 ++
 1 file changed, 2 insertions(+)
rust/helpers/uaccess.c +2 -0
index d9625b9ee046..aff22f16ab38 100644
@@ -20,7 +20,9 @@ unsigned long rust_helper__copy_from_user(void *to, const void __user *from, uns
 {
 	return _inline_copy_from_user(to, from, n);
 }
+#endif
 
+#ifdef INLINE_COPY_TO_USER
 __rust_helper
 unsigned long rust_helper__copy_to_user(void __user *to, const void *from, unsigned long n)
 {
Signature · 1 line
2.51.0
reviewer Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 1/3] rust: uaccess: use INLINE_COPY_TO_USER to guard copy_to_user()\

Quoted 10 lines · Yury Norov

The copy_to_user() rust helper is only needed when the main kernel inlines the function. It is controlled by INLINE_COPY_TO_USER, but the rust helper is protected with INLINE_COPY_FROM_USER.

Fix that.

Fixes d99dc586ca7c7 ("uaccess: decouple INLINE_COPY_FROM_USER and CONFIG_RUST")
author
#3 lore raw

PATCH v2 2/3 uaccess: unify inline vs outline copy_{from,to}_user() selection

In reply to Yury Norov

The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.

Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.

Switch the codebase to the single unified INLINE_COPY_USER control.

---
 arch/arc/include/asm/uaccess.h        |  3 +--
 arch/arm/include/asm/uaccess.h        |  3 +--
 arch/arm64/include/asm/uaccess.h      |  3 +--
 arch/hexagon/include/asm/uaccess.h    |  3 +--
 arch/loongarch/include/asm/uaccess.h  |  3 +--
 arch/m68k/include/asm/uaccess.h       |  3 +--
 arch/microblaze/include/asm/uaccess.h |  3 +--
 arch/mips/include/asm/uaccess.h       |  3 +--
 arch/nios2/include/asm/uaccess.h      |  3 +--
 arch/openrisc/include/asm/uaccess.h   |  3 +--
 arch/parisc/include/asm/uaccess.h     |  3 +--
 arch/s390/include/asm/uaccess.h       |  3 +--
 arch/sh/include/asm/uaccess.h         |  3 +--
 arch/sparc/include/asm/uaccess_32.h   |  3 +--
 arch/sparc/include/asm/uaccess_64.h   |  3 +--
 arch/um/include/asm/uaccess.h         |  3 +--
 arch/xtensa/include/asm/uaccess.h     |  3 +--
 include/asm-generic/uaccess.h         |  3 +--
 include/linux/uaccess.h               | 12 ++++++------
 lib/usercopy.c                        |  4 +---
 rust/helpers/uaccess.c                |  4 +---
 21 files changed, 26 insertions(+), 48 deletions(-)
arch/arc/include/asm/uaccess.h +1 -2
index 1e8809ea000a..6df2209541ac 100644
@@ -628,8 +628,7 @@ static inline unsigned long __clear_user(void __user *to, unsigned long n)
 	return res;
 }
 
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
+#define INLINE_COPY_USER
 
 #define __clear_user			__clear_user
 
arch/arm/include/asm/uaccess.h +1 -2
index d6ae80b5df36..1593cf3b9800 100644
@@ -616,8 +616,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 }
 #define __clear_user(addr, n)		(memset((void __force *)addr, 0, n), 0)
 #endif
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
+#define INLINE_COPY_USER
 
 static inline unsigned long __must_check clear_user(void __user *to, unsigned long n)
 {
arch/arm64/include/asm/uaccess.h +1 -2
index b0c83a08dda9..9f5bd9c69c24 100644
@@ -456,8 +456,7 @@ do {									\
 	unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label);	\
 } while (0)
 
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
+#define INLINE_COPY_USER
 
 extern unsigned long __must_check __arch_clear_user(void __user *to, unsigned long n);
 static inline unsigned long __must_check __clear_user(void __user *to, unsigned long n)
arch/hexagon/include/asm/uaccess.h +1 -2
index bff77efc0d9a..1aecf60ec4f5 100644
@@ -26,8 +26,7 @@ unsigned long raw_copy_from_user(void *to, const void __user *from,
 				     unsigned long n);
 unsigned long raw_copy_to_user(void __user *to, const void *from,
 				   unsigned long n);
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 __kernel_size_t __clear_user_hexagon(void __user *dest, unsigned long count);
 #define __clear_user(a, s) __clear_user_hexagon((a), (s))
arch/loongarch/include/asm/uaccess.h +1 -2
index 438269313e78..428f373feabf 100644
@@ -292,8 +292,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 	return __copy_user((__force void *)to, from, n);
 }
 
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 /*
  * __clear_user: - Zero a block of memory in user space, with less checking.
arch/m68k/include/asm/uaccess.h +1 -2
index 64914872a5c9..31d133faa45e 100644
@@ -377,8 +377,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 		return __constant_copy_to_user(to, from, n);
 	return __generic_copy_to_user(to, from, n);
 }
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 #define __get_kernel_nofault(dst, src, type, err_label)			\
 do {									\
arch/microblaze/include/asm/uaccess.h +1 -2
index 3aab2f17e046..afa0dd8d013f 100644
@@ -250,8 +250,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 {
 	return __copy_tofrom_user(to, (__force const void __user *)from, n);
 }
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 /*
  * Copy a null terminated string from userspace.
arch/mips/include/asm/uaccess.h +1 -2
index c0cede273c7c..f00c36676b73 100644
@@ -433,8 +433,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 	return __cu_len_r;
 }
 
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 extern __kernel_size_t __bzero(void __user *addr, __kernel_size_t size);
 
arch/nios2/include/asm/uaccess.h +1 -2
index 6ccc9a232c23..5e6e05cc6efc 100644
@@ -57,8 +57,7 @@ extern unsigned long
 raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long
 raw_copy_to_user(void __user *to, const void *from, unsigned long n);
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 extern long strncpy_from_user(char *__to, const char __user *__from,
 			      long __len);
arch/openrisc/include/asm/uaccess.h +1 -2
index d6500a374e18..db934ebc0069 100644
@@ -218,8 +218,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long size)
 {
 	return __copy_tofrom_user((__force void *)to, from, size);
 }
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 extern unsigned long __clear_user(void __user *addr, unsigned long size);
 
arch/parisc/include/asm/uaccess.h +1 -2
index 6c531d2c847e..0d17f81c8b27 100644
@@ -197,7 +197,6 @@ unsigned long __must_check raw_copy_to_user(void __user *dst, const void *src,
 					    unsigned long len);
 unsigned long __must_check raw_copy_from_user(void *dst, const void __user *src,
 					    unsigned long len);
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
+#define INLINE_COPY_USER
 
 #endif /* __PARISC_UACCESS_H */
arch/s390/include/asm/uaccess.h +1 -2
index dff035372601..a9f32c53f699 100644
@@ -30,8 +30,7 @@ void debug_user_asce(int exit);
 #define uaccess_kmsan_or_inline __always_inline
 #endif
 
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 static uaccess_kmsan_or_inline __must_check unsigned long
 raw_copy_from_user(void *to, const void __user *from, unsigned long size)
arch/sh/include/asm/uaccess.h +1 -2
index a79609eb14be..02e7a066538e 100644
@@ -95,8 +95,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 {
 	return __copy_user((__force void *)to, from, n);
 }
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 /*
  * Clear the area and return remaining number of bytes
arch/sparc/include/asm/uaccess_32.h +1 -2
index 43284b6ec46a..5542d5b32994 100644
@@ -190,8 +190,7 @@ static inline unsigned long raw_copy_from_user(void *to, const void __user *from
 	return __copy_user((__force void __user *) to, from, n);
 }
 
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 static inline unsigned long __clear_user(void __user *addr, unsigned long size)
 {
arch/sparc/include/asm/uaccess_64.h +1 -2
index b825a5dd0210..e2989cfba626 100644
@@ -231,8 +231,7 @@ unsigned long __must_check raw_copy_from_user(void *to,
 unsigned long __must_check raw_copy_to_user(void __user *to,
 					   const void *from,
 					   unsigned long size);
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 unsigned long __must_check raw_copy_in_user(void __user *to,
 					   const void __user *from,
arch/um/include/asm/uaccess.h +1 -2
index 0df9ea4abda8..4417c8b1d37a 100644
@@ -27,8 +27,7 @@ static inline int __access_ok(const void __user *ptr, unsigned long size);
 #define __access_ok __access_ok
 #define __clear_user __clear_user
 
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 #include <asm-generic/uaccess.h>
 
arch/xtensa/include/asm/uaccess.h +1 -2
index 56aec6d504fe..6538a29a2bbd 100644
@@ -237,8 +237,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 	prefetch(from);
 	return __xtensa_copy_user((__force void *)to, from, n);
 }
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 
 /*
  * We need to return the number of bytes not cleared.  Our memset()
include/asm-generic/uaccess.h +1 -2
index b276f783494c..4569045e7139 100644
@@ -91,8 +91,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
 	memcpy((void __force *)to, from, n);
 	return 0;
 }
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
+#define INLINE_COPY_USER
 #endif /* CONFIG_UACCESS_MEMCPY */
 
 /*
include/linux/uaccess.h +6 -6
index 56328601218c..6100f1046546 100644
@@ -84,7 +84,7 @@
  * the 6 functions (copy_{to,from}_user(), __copy_{to,from}_user_inatomic())
  * that are used instead.  Out of those, __... ones are inlined.  Plain
  * copy_{to,from}_user() might or might not be inlined.  If you want them
- * inlined, have asm/uaccess.h define INLINE_COPY_{TO,FROM}_USER.
+ * inlined, have asm/uaccess.h define INLINE_COPY_USER.
  *
  * NOTE: only copy_from_user() zero-pads the destination in case of short copy.
  * Neither __copy_from_user() nor __copy_from_user_inatomic() zero anything
@@ -157,7 +157,7 @@ __copy_to_user(void __user *to, const void *from, unsigned long n)
 }
 
 /*
- * Architectures that #define INLINE_COPY_TO_USER use this function
+ * Architectures that #define INLINE_COPY_USER use this function
  * directly in the normal copy_to/from_user(), the other ones go
  * through an extern _copy_to/from_user(), which expands the same code
  * here.
@@ -190,7 +190,7 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)
 	memset(to + (n - res), 0, res);
 	return res;
 }
-#ifndef INLINE_COPY_FROM_USER
+#ifndef INLINE_COPY_USER
 extern __must_check unsigned long
 _copy_from_user(void *, const void __user *, unsigned long);
 #endif
@@ -207,7 +207,7 @@ _inline_copy_to_user(void __user *to, const void *from, unsigned long n)
 	}
 	return n;
 }
-#ifndef INLINE_COPY_TO_USER
+#ifndef INLINE_COPY_USER
 extern __must_check unsigned long
 _copy_to_user(void __user *, const void *, unsigned long);
 #endif
@@ -217,7 +217,7 @@ copy_from_user(void *to, const void __user *from, unsigned long n)
 {
 	if (!check_copy_size(to, n, false))
 		return n;
-#ifdef INLINE_COPY_FROM_USER
+#ifdef INLINE_COPY_USER
 	return _inline_copy_from_user(to, from, n);
 #else
 	return _copy_from_user(to, from, n);
@@ -230,7 +230,7 @@ copy_to_user(void __user *to, const void *from, unsigned long n)
 	if (!check_copy_size(from, n, true))
 		return n;
 
-#ifdef INLINE_COPY_TO_USER
+#ifdef INLINE_COPY_USER
 	return _inline_copy_to_user(to, from, n);
 #else
 	return _copy_to_user(to, from, n);
lib/usercopy.c +1 -3
index b00a3a957de6..e2f0bf104a59 100644
@@ -12,15 +12,13 @@
 
 /* out-of-line parts */
 
-#if !defined(INLINE_COPY_FROM_USER)
+#if !defined(INLINE_COPY_USER)
 unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
 {
 	return _inline_copy_from_user(to, from, n);
 }
 EXPORT_SYMBOL(_copy_from_user);
-#endif
 
-#if !defined(INLINE_COPY_TO_USER)
 unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
 {
 	return _inline_copy_to_user(to, from, n);
rust/helpers/uaccess.c +1 -3
index aff22f16ab38..6e59cc9c665c 100644
@@ -14,15 +14,13 @@ rust_helper_copy_to_user(void __user *to, const void *from, unsigned long n)
 	return copy_to_user(to, from, n);
 }
 
-#ifdef INLINE_COPY_FROM_USER
+#ifdef INLINE_COPY_USER
 __rust_helper
 unsigned long rust_helper__copy_from_user(void *to, const void __user *from, unsigned long n)
 {
 	return _inline_copy_from_user(to, from, n);
 }
-#endif
 
-#ifdef INLINE_COPY_TO_USER
 __rust_helper
 unsigned long rust_helper__copy_to_user(void __user *to, const void *from, unsigned long n)
 {
Signature · 1 line
2.51.0
author
#4 lore raw

PATCH v2 3/3 uaccess: minimize INLINE_COPY_USER-related ifdefery

In reply to Yury Norov

Now that we've got the same config selecting inline vs outline copy_to_user() and copy_from_user(), we can simplify the corresponding logic in the uaccess.h.

Fixes 1f9a8286bc0c ("uaccess: always export _copy_[from|to]_user with CONFIG_RUST")
---
 include/linux/uaccess.h | 21 +++++++--------------
 1 file changed, 7 insertions(+), 14 deletions(-)
include/linux/uaccess.h +7 -14
index 6100f1046546..e0c3d6e29301 100644
@@ -190,10 +190,6 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)
 	memset(to + (n - res), 0, res);
 	return res;
 }
-#ifndef INLINE_COPY_USER
-extern __must_check unsigned long
-_copy_from_user(void *, const void __user *, unsigned long);
-#endif
 
 static inline __must_check unsigned long
 _inline_copy_to_user(void __user *to, const void *from, unsigned long n)
@@ -207,7 +203,13 @@ _inline_copy_to_user(void __user *to, const void *from, unsigned long n)
 	}
 	return n;
 }
-#ifndef INLINE_COPY_USER
+#ifdef INLINE_COPY_USER
+# define _copy_to_user _inline_copy_to_user
+# define _copy_from_user _inline_copy_from_user
+#else
+extern __must_check unsigned long
+_copy_from_user(void *, const void __user *, unsigned long);
+
 extern __must_check unsigned long
 _copy_to_user(void __user *, const void *, unsigned long);
 #endif
@@ -217,11 +219,7 @@ copy_from_user(void *to, const void __user *from, unsigned long n)
 {
 	if (!check_copy_size(to, n, false))
 		return n;
-#ifdef INLINE_COPY_USER
-	return _inline_copy_from_user(to, from, n);
-#else
 	return _copy_from_user(to, from, n);
-#endif
 }
 
 static __always_inline unsigned long __must_check
@@ -229,12 +227,7 @@ copy_to_user(void __user *to, const void *from, unsigned long n)
 {
 	if (!check_copy_size(from, n, true))
 		return n;
-
-#ifdef INLINE_COPY_USER
-	return _inline_copy_to_user(to, from, n);
-#else
 	return _copy_to_user(to, from, n);
-#endif
 }
 
 #ifndef copy_mc_to_kernel
Signature · 1 line
2.51.0
maintainer
#5 lore raw
In reply to Yury Norov
Quoted 5 lines · On Fri, 24 Apr 2026 22:08:54 -0400 Yury Norov <[email protected]> wrote:

The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.

Quoted 7 lines

Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.

The first patch of the series fixes rust/uaccess coppy_to_user() wrapper guarded with INLINE_COPY_FROM_USER. The 2nd patch switches codebase to the unified INLINE_COPY_USER. And the last patch cleans up ifdefery in the include/linux/uaccess.h

Seems sensible, thanks. I'll queue these up for test-n-review.

Arnd was involved in some of the Fixes: commits and might be interested in these changes, so Cc:.

#6 lore raw

Le 27/04/2026 à 17:58, Andrew Morton a écrit :

Quoted 15 lines · On Fri, 24 Apr 2026 22:08:54 -0400 Yury Norov <[email protected]> wrote:
Quoted 6 lines

The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.

Quoted 7 lines

Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.

The first patch of the series fixes rust/uaccess coppy_to_user() wrapper guarded with INLINE_COPY_FROM_USER. The 2nd patch switches codebase to the unified INLINE_COPY_USER. And the last patch cleans up ifdefery in the include/linux/uaccess.h

Quoted 5 lines

Seems sensible, thanks. I'll queue these up for test-n-review.

Arnd was involved in some of the Fixes: commits and might be interested in these changes, so Cc:.

I have an alternative as patch 2 in this series : https://lore.kernel.org/all/[email protected]/

Christophe