uaccess: unify inline vs outline copy_{from,to}_user() selection
Subsystems (19)
v2 cover uaccess: unify inline vs outline copy_{from,to}_user() selection
The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.
Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.
The first patch of the series fixes rust/uaccess coppy_to_user() wrapper guarded with INLINE_COPY_FROM_USER. The 2nd patch switches codebase to the unified INLINE_COPY_USER. And the last patch cleans up ifdefery in the include/linux/uaccess.h
---
v1: https://lore.kernel.org/all/[email protected]/
v2:
- reword cover letter (Andrew);
- add v2#1 to fix rust/uaccess explicitly (Alice);
Yury Norov (3):
rust: uaccess: use INLINE_COPY_TO_USER to guard copy_to_user()
uaccess: unify inline vs outline copy_{from,to}_user() selection
uaccess: minimize INLINE_COPY_USER-related ifdefery
arch/arc/include/asm/uaccess.h | 3 +-- arch/arm/include/asm/uaccess.h | 3 +-- arch/arm64/include/asm/uaccess.h | 3 +-- arch/hexagon/include/asm/uaccess.h | 3 +-- arch/loongarch/include/asm/uaccess.h | 3 +-- arch/m68k/include/asm/uaccess.h | 3 +-- arch/microblaze/include/asm/uaccess.h | 3 +-- arch/mips/include/asm/uaccess.h | 3 +-- arch/nios2/include/asm/uaccess.h | 3 +-- arch/openrisc/include/asm/uaccess.h | 3 +-- arch/parisc/include/asm/uaccess.h | 3 +-- arch/s390/include/asm/uaccess.h | 3 +-- arch/sh/include/asm/uaccess.h | 3 +-- arch/sparc/include/asm/uaccess_32.h | 3 +-- arch/sparc/include/asm/uaccess_64.h | 3 +-- arch/um/include/asm/uaccess.h | 3 +-- arch/xtensa/include/asm/uaccess.h | 3 +-- include/asm-generic/uaccess.h | 3 +-- include/linux/uaccess.h | 25 +++++++++---------------- lib/usercopy.c | 4 +--- rust/helpers/uaccess.c | 2 +- 21 files changed, 29 insertions(+), 56 deletions(-)
Signature · 1 line
2.51.0
PATCH v2 1/3 rust: uaccess: use INLINE_COPY_TO_USER to guard copy_to_user()
The copy_to_user() rust helper is only needed when the main kernel inlines the function. It is controlled by INLINE_COPY_TO_USER, but the rust helper is protected with INLINE_COPY_FROM_USER.
Fix that.
--- rust/helpers/uaccess.c | 2 ++ 1 file changed, 2 insertions(+)
rust/helpers/uaccess.c +2 -0
@@ -20,7 +20,9 @@ unsigned long rust_helper__copy_from_user(void *to, const void __user *from, uns {return _inline_copy_from_user (to, from, n); } +#endif +#ifdef INLINE_COPY_TO_USER __rust_helper unsigned long rust_helper__copy_to_user (void __user *to,const void *from,unsigned long n) {
Signature · 1 line
2.51.0
Re: [PATCH v2 1/3] rust: uaccess: use INLINE_COPY_TO_USER to guard copy_to_user()\
Quoted 10 lines · Yury Norov
The
copy_to_user()rust helper is only needed when the main kernel inlines the function. It is controlled byINLINE_COPY_TO_USER, but the rust helper is protected withINLINE_COPY_FROM_USER.Fix that.
Reported-by Christophe Leroy (CS GROUP) <[email protected]>Fixes d99dc586ca7c7 ("uaccess: decouple INLINE_COPY_FROM_USER and CONFIG_RUST")Signed-off-by Yury Norov <[email protected]>
PATCH v2 2/3 uaccess: unify inline vs outline copy_{from,to}_user() selection
In reply to Yury NorovThe kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.
Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.
Switch the codebase to the single unified INLINE_COPY_USER control.
--- arch/arc/include/asm/uaccess.h | 3 +-- arch/arm/include/asm/uaccess.h | 3 +-- arch/arm64/include/asm/uaccess.h | 3 +-- arch/hexagon/include/asm/uaccess.h | 3 +-- arch/loongarch/include/asm/uaccess.h | 3 +-- arch/m68k/include/asm/uaccess.h | 3 +-- arch/microblaze/include/asm/uaccess.h | 3 +-- arch/mips/include/asm/uaccess.h | 3 +-- arch/nios2/include/asm/uaccess.h | 3 +-- arch/openrisc/include/asm/uaccess.h | 3 +-- arch/parisc/include/asm/uaccess.h | 3 +-- arch/s390/include/asm/uaccess.h | 3 +-- arch/sh/include/asm/uaccess.h | 3 +-- arch/sparc/include/asm/uaccess_32.h | 3 +-- arch/sparc/include/asm/uaccess_64.h | 3 +-- arch/um/include/asm/uaccess.h | 3 +-- arch/xtensa/include/asm/uaccess.h | 3 +-- include/asm-generic/uaccess.h | 3 +-- include/linux/uaccess.h | 12 ++++++------ lib/usercopy.c | 4 +--- rust/helpers/uaccess.c | 4 +--- 21 files changed, 26 insertions(+), 48 deletions(-)
arch/arc/include/asm/uaccess.h +1 -2
@@ -628,8 +628,7 @@ static inline unsigned long __clear_user(void __user *to, unsigned long n)return res; } -#define INLINE_COPY_TO_USER -#define INLINE_COPY_FROM_USER +#define INLINE_COPY_USER #define __clear_user __clear_user
arch/arm/include/asm/uaccess.h +1 -2
@@ -616,8 +616,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n) }#define __clear_user (addr, n) (memset((void __force *)addr, 0, n), 0)#endif -#define INLINE_COPY_TO_USER -#define INLINE_COPY_FROM_USER +#define INLINE_COPY_USER static inline unsigned long __must_check clear_user (void __user *to,unsigned long n) {
arch/arm64/include/asm/uaccess.h +1 -2
@@ -456,8 +456,7 @@ do { \unsafe_copy_loop (__ucu_dst, __ucu_src, __ucu_len, u8, label); \ } while (0) -#define INLINE_COPY_TO_USER -#define INLINE_COPY_FROM_USER +#define INLINE_COPY_USER extern unsigned long __must_check __arch_clear_user (void __user *to,unsigned long n);static inline unsigned long __must_check __clear_user (void __user *to,unsigned long n)
arch/hexagon/include/asm/uaccess.h +1 -2
@@ -26,8 +26,7 @@ unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);unsigned long raw_copy_to_user (void __user *to,const void *from,unsigned long n); -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER __kernel_size_t __clear_user_hexagon (void __user *dest,unsigned long count);#define __clear_user (a, s) __clear_user_hexagon((a), (s))
arch/loongarch/include/asm/uaccess.h +1 -2
@@ -292,8 +292,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)return __copy_user ((__force void *)to, from, n); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER /* * __clear_user: - Zero a block of memory in user space, with less checking.
arch/m68k/include/asm/uaccess.h +1 -2
@@ -377,8 +377,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)return __constant_copy_to_user (to, from, n);return __generic_copy_to_user (to, from, n); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER #define __get_kernel_nofault (dst, src, type, err_label) \ do { \
arch/microblaze/include/asm/uaccess.h +1 -2
@@ -250,8 +250,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n) {return __copy_tofrom_user (to, (__force const void __user *)from, n); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER /* * Copy a null terminated string from userspace.
arch/mips/include/asm/uaccess.h +1 -2
@@ -433,8 +433,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)return __cu_len_r; } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER extern __kernel_size_t __bzero (void __user *addr,__kernel_size_t size);
arch/nios2/include/asm/uaccess.h +1 -2
@@ -57,8 +57,7 @@ extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);extern unsigned long raw_copy_to_user (void __user *to,const void *from,unsigned long n); -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER extern long strncpy_from_user (char *__to,const char __user *__from,long __len);
arch/openrisc/include/asm/uaccess.h +1 -2
@@ -218,8 +218,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long size) {return __copy_tofrom_user ((__force void *)to, from, size); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER extern unsigned long __clear_user (void __user *addr,unsigned long size);
arch/parisc/include/asm/uaccess.h +1 -2
@@ -197,7 +197,6 @@ unsigned long __must_check raw_copy_to_user(void __user *dst, const void *src, unsigned long len);unsigned long __must_check raw_copy_from_user (void *dst,const void __user *src,unsigned long len); -#define INLINE_COPY_TO_USER -#define INLINE_COPY_FROM_USER +#define INLINE_COPY_USER #endif /* __PARISC_UACCESS_H */
arch/s390/include/asm/uaccess.h +1 -2
@@ -30,8 +30,7 @@ void debug_user_asce(int exit);#define uaccess_kmsan_or_inline __always_inline#endif -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER static uaccess_kmsan_or_inline __must_check unsigned long raw_copy_from_user (void *to,const void __user *from,unsigned long size)
arch/sh/include/asm/uaccess.h +1 -2
@@ -95,8 +95,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n) {return __copy_user ((__force void *)to, from, n); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER /* * Clear the area and return remaining number of bytes
arch/sparc/include/asm/uaccess_32.h +1 -2
@@ -190,8 +190,7 @@ static inline unsigned long raw_copy_from_user(void *to, const void __user *fromreturn __copy_user ((__force void __user *) to, from, n); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER static inline unsigned long __clear_user (void __user *addr,unsigned long size) {
arch/sparc/include/asm/uaccess_64.h +1 -2
@@ -231,8 +231,7 @@ unsigned long __must_check raw_copy_from_user(void *to,unsigned long __must_check raw_copy_to_user (void __user *to,const void *from,unsigned long size); -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER unsigned long __must_check raw_copy_in_user(void __user *to, const void __user *from,
arch/um/include/asm/uaccess.h +1 -2
@@ -27,8 +27,7 @@ static inline int __access_ok(const void __user *ptr, unsigned long size);#define __access_ok __access_ok#define __clear_user __clear_user -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER #include <asm-generic/uaccess.h>
arch/xtensa/include/asm/uaccess.h +1 -2
@@ -237,8 +237,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)prefetch (from);return __xtensa_copy_user ((__force void *)to, from, n); } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER /* * We need to return the number of bytes not cleared. Our memset()
include/asm-generic/uaccess.h +1 -2
@@ -91,8 +91,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)memcpy ((void __force *)to, from, n); return 0; } -#define INLINE_COPY_FROM_USER -#define INLINE_COPY_TO_USER +#define INLINE_COPY_USER #endif /* CONFIG_UACCESS_MEMCPY */ /*
include/linux/uaccess.h +6 -6
@@ -84,7 +84,7 @@* the 6 functions (copy_{to,from}_user(), __copy_{to,from}_user_inatomic()) * that are used instead. Out of those, __... ones are inlined. Plain * copy_{to,from}_user() might or might not be inlined. If you want them -* inlined, have asm/uaccess.h define INLINE_COPY_{TO,FROM}_USER. +* inlined, have asm/uaccess.h define INLINE_COPY_USER. * * NOTE: only copy_from_user() zero-pads the destination in case of short copy. * Neither __copy_from_user() nor __copy_from_user_inatomic() zero anything
@@ -157,7 +157,7 @@ __copy_to_user(void __user *to, const void *from, unsigned long n) }/* -* Architectures that #define INLINE_COPY_TO_USER use this function +* Architectures that #define INLINE_COPY_USER use this function * directly in the normal copy_to/from_user(), the other ones go * through an extern _copy_to/from_user(), which expands the same code * here.
@@ -190,7 +190,7 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)memset (to + (n - res),0 , res);return res; } -#ifndef INLINE_COPY_FROM_USER +#ifndef INLINE_COPY_USER extern __must_check unsigned long _copy_from_user (void *,const void __user *,unsigned long );#endif
@@ -207,7 +207,7 @@ _inline_copy_to_user(void __user *to, const void *from, unsigned long n) }return n; } -#ifndef INLINE_COPY_TO_USER +#ifndef INLINE_COPY_USER extern __must_check unsigned long _copy_to_user (void __user *,const void *,unsigned long );#endif
@@ -217,7 +217,7 @@ copy_from_user(void *to, const void __user *from, unsigned long n) {if (!check_copy_size (to, n, false))return n; -#ifdef INLINE_COPY_FROM_USER +#ifdef INLINE_COPY_USER return _inline_copy_from_user (to, from, n);#else return _copy_from_user (to, from, n);
@@ -230,7 +230,7 @@ copy_to_user(void __user *to, const void *from, unsigned long n)if (!check_copy_size (from, n, true))return n; -#ifdef INLINE_COPY_TO_USER +#ifdef INLINE_COPY_USER return _inline_copy_to_user (to, from, n);#else return _copy_to_user (to, from, n);
lib/usercopy.c +1 -3
@@ -12,15 +12,13 @@/* out-of-line parts */ -#if !defined(INLINE_COPY_FROM_USER ) +#if !defined(INLINE_COPY_USER )unsigned long _copy_from_user (void *to,const void __user *from,unsigned long n) {return _inline_copy_from_user (to, from, n); }EXPORT_SYMBOL (_copy_from_user); -#endif -#if !defined(INLINE_COPY_TO_USER )unsigned long _copy_to_user (void __user *to,const void *from,unsigned long n) {return _inline_copy_to_user (to, from, n);
rust/helpers/uaccess.c +1 -3
@@ -14,15 +14,13 @@ rust_helper_copy_to_user(void __user *to, const void *from, unsigned long n)return copy_to_user (to, from, n); } -#ifdef INLINE_COPY_FROM_USER +#ifdef INLINE_COPY_USER __rust_helper unsigned long rust_helper__copy_from_user (void *to,const void __user *from,unsigned long n) {return _inline_copy_from_user (to, from, n); } -#endif -#ifdef INLINE_COPY_TO_USER __rust_helper unsigned long rust_helper__copy_to_user (void __user *to,const void *from,unsigned long n) {
Signature · 1 line
2.51.0
PATCH v2 3/3 uaccess: minimize INLINE_COPY_USER-related ifdefery
In reply to Yury NorovNow that we've got the same config selecting inline vs outline copy_to_user() and copy_from_user(), we can simplify the corresponding logic in the uaccess.h.
--- include/linux/uaccess.h | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-)
include/linux/uaccess.h +7 -14
@@ -190,10 +190,6 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)memset (to + (n - res),0 , res);return res; } -#ifndef INLINE_COPY_USER -extern __must_check unsigned long -_copy_from_user (void *,const void __user *,unsigned long ); -#endif static inline __must_check unsigned long _inline_copy_to_user (void __user *to,const void *from,unsigned long n)
@@ -207,7 +203,13 @@ _inline_copy_to_user(void __user *to, const void *from, unsigned long n) }return n; } -#ifndef INLINE_COPY_USER +#ifdef INLINE_COPY_USER +# define _copy_to_user _inline_copy_to_user +# define _copy_from_user _inline_copy_from_user +#else +extern __must_check unsigned long +_copy_from_user (void *,const void __user *,unsigned long ); +extern __must_check unsigned long _copy_to_user (void __user *,const void *,unsigned long );#endif
@@ -217,11 +219,7 @@ copy_from_user(void *to, const void __user *from, unsigned long n) {if (!check_copy_size (to, n, false))return n; -#ifdef INLINE_COPY_USER -return _inline_copy_from_user (to, from, n); -#else return _copy_from_user (to, from, n); -#endif } static __always_inline unsigned long __must_check
@@ -229,12 +227,7 @@ copy_to_user(void __user *to, const void *from, unsigned long n) {if (!check_copy_size (from, n, true))return n; - -#ifdef INLINE_COPY_USER -return _inline_copy_to_user (to, from, n); -#else return _copy_to_user (to, from, n); -#endif }#ifndef copy_mc_to_kernel
Signature · 1 line
2.51.0
Andrew Morton
maintainerQuoted 5 lines · On Fri, 24 Apr 2026 22:08:54 -0400 Yury Norov <[email protected]> wrote:
The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual
INLINE_COPY_FROM_USERandINLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.
Quoted 7 lines
Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.
The first patch of the series fixes
rust/uaccesscoppy_to_user()wrapper guarded withINLINE_COPY_FROM_USER. The 2nd patch switches codebase to the unifiedINLINE_COPY_USER. And the last patch cleans up ifdefery in theinclude/linux/uaccess.h
Seems sensible, thanks. I'll queue these up for test-n-review.
Arnd was involved in some of the Fixes: commits and might be interested in these changes, so Cc:.
Le 27/04/2026 à 17:58, Andrew Morton a écrit :
Quoted 15 lines · On Fri, 24 Apr 2026 22:08:54 -0400 Yury Norov <[email protected]> wrote:
Quoted 6 lines
The kernel allows arches to select between inline and outline implementations of the copy_{from,to}_user() by defining individual
INLINE_COPY_FROM_USERandINLINE_COPY_TO_USER, correspondingly. However, all arches enable or disable them always together.Quoted 7 lines
Without the real use-case for one helper being inlined while the other outlined, having independent controls is excessive and error prone.
The first patch of the series fixes
rust/uaccesscoppy_to_user()wrapper guarded withINLINE_COPY_FROM_USER. The 2nd patch switches codebase to the unifiedINLINE_COPY_USER. And the last patch cleans up ifdefery in theinclude/linux/uaccess.h
Quoted 5 lines
Seems sensible, thanks. I'll queue these up for test-n-review.
Arnd was involved in some of the Fixes: commits and might be interested in these changes, so Cc:.
I have an alternative as patch 2 in this series : https://lore.kernel.org/all/[email protected]/
Christophe