Skip to contentSkip to conversation

Enable strict percpu address space checks

PATCH Applied Uros Bizjak 5 participants 14 messages Dec 8, 2024 – Dec 13, 2024

Subsystems (9)

v3 of 4 Applied 6/6 Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Enable strict percpu address space checks

Enable strict percpu address space checks via x86 named address space qualifiers. Percpu variables are declared in __seg_gs/__seg_fs named AS and kept named AS qualified until they are dereferenced via percpu accessor. This approach enables various compiler checks for cross-namespace variable assignments.

Please note that current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword. The proposed patch by Dan Carpenter to implement __typeof_unqual__() handling in sparse is located at:

https://lore.kernel.org/lkml/[email protected]/

v2: - Add comment to remove test for __CHECKER__ once sparse learns
about __typeof_unqual__.
- Add Acked-by: tags.
v3: - Rename __per_cpu_qual to __percpu_qual.
- Add more Acked-by: tags.

Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Nadav Amit <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Dan Carpenter <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Peter Zijlstra <[email protected]>

Uros Bizjak (6):
x86/kgdb: Use IS_ERR_PCPU() macro
compiler.h: Introduce TYPEOF_UNQUAL() macro
percpu: Use TYPEOF_UNQUAL() in variable declarations
percpu: Use TYPEOF_UNQUAL() in *_cpu_ptr() accessors
percpu: Repurpose __percpu tag as a named address space qualifier percpu/x86: Enable strict percpu checks via named AS qualifiers

 arch/x86/include/asm/percpu.h  | 38 ++++++++++++++++++++++++---------
 arch/x86/kernel/kgdb.c         |  2 +-
 fs/bcachefs/util.h             |  2 +-
 include/asm-generic/percpu.h   | 39 ++++++++++++++++++++++------------
 include/linux/compiler.h       | 13 ++++++++++++
 include/linux/compiler_types.h |  2 +-
 include/linux/part_stat.h      |  2 +-
 include/linux/percpu-defs.h    |  6 +++---
 include/net/snmp.h             |  5 ++---
 init/Kconfig                   |  3 +++
 kernel/locking/percpu-rwsem.c  |  2 +-
 net/mpls/internal.h            |  4 ++--
 12 files changed, 82 insertions(+), 36 deletions(-)
Signature · 1 line
2.42.0
#1 lore raw

PATCH v3 1/6 x86/kgdb: Use IS_ERR_PCPU() macro

Use IS_ERR_PCPU() when checking the error pointer in the percpu address space. This macro adds intermediate cast to unsigned long when switching named address spaces.

The patch will avoid future build errors due to pointer address space mismatch with enabled strict percpu address space checks.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/x86/kernel/kgdb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/kgdb.c +1 -1
index 9c9faa1634fb..102641fd2172 100644
@@ -655,7 +655,7 @@ void kgdb_arch_late(void)
 		if (breakinfo[i].pev)
 			continue;
 		breakinfo[i].pev = register_wide_hw_breakpoint(&attr, NULL, NULL);
-		if (IS_ERR((void * __force)breakinfo[i].pev)) {
+		if (IS_ERR_PCPU(breakinfo[i].pev)) {
 			printk(KERN_ERR "kgdb: Could not allocate hw"
 			       "breakpoints\nDisabling the kernel debugger\n");
 			breakinfo[i].pev = NULL;
Signature · 1 line
2.42.0
#2 lore raw

PATCH v3 2/6 compiler.h: Introduce TYPEOF_UNQUAL() macro

In reply to Uros Bizjak

Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof operator when available, to return unqualified type of the expression.

Current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword.

Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Peter Zijlstra <[email protected]
---
 include/linux/compiler.h | 13 +++++++++++++
 init/Kconfig             |  3 +++
 2 files changed, 16 insertions(+)
include/linux/compiler.h +13 -0
index 469a64dd6495..ec0429d7a153 100644
@@ -321,6 +321,19 @@ static inline void *offset_to_ptr(const int *off)
  */
 #define prevent_tail_call_optimization()	mb()
 
+/*
+ * Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof
+ * operator when available, to return unqualified type of the exp.
+ *
+ * XXX: Remove test for __CHECKER__ once
+ * sparse learns about __typeof_unqual__.
+ */
+#if defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) && !defined(__CHECKER__)
+# define TYPEOF_UNQUAL(exp) __typeof_unqual__(exp)
+#else
+# define TYPEOF_UNQUAL(exp) __typeof__(exp)
+#endif
+
 #include <asm/rwonce.h>
 
 #endif /* __LINUX_COMPILER_H */
init/Kconfig +3 -0
index a20e6efd3f0f..c1f9eb3d5f2e 100644
@@ -894,6 +894,9 @@ config ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
 config CC_HAS_INT128
 	def_bool !$(cc-option,$(m64-flag) -D__SIZEOF_INT128__=0) && 64BIT
 
+config CC_HAS_TYPEOF_UNQUAL
+	def_bool $(success,echo 'int foo (int a) { __typeof_unqual__(a) b = a; return b; }' | $(CC) -x c - -S -o /dev/null)
+
 config CC_IMPLICIT_FALLTHROUGH
 	string
 	default "-Wimplicit-fallthrough=5" if CC_IS_GCC && $(cc-option,-Wimplicit-fallthrough=5)
Signature · 1 line
2.42.0
maintainer
#3 lore raw

Re: [PATCH v3 2/6] compiler.h: Introduce TYPEOF_UNQUAL() macro

Quoted 6 lines · Uros Bizjak

Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof operator when available, to return unqualified type of the expression.

Current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword.

Ooooh, new toys.

I suppose __unqual_scalar_typeof() wants to be using this when available?

#4 lore raw

Re: [PATCH v3 2/6] compiler.h: Introduce TYPEOF_UNQUAL() macro

Quoted 5 lines · Peter Zijlstra
Quoted 3 lines · Uros Bizjak

Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof operator when available, to return unqualified type of the expression.

Quoted 8 lines
Quoted 3 lines

Current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword.

Ooooh, new toys.

I suppose __unqual_scalar_typeof() wants to be using this when available?

Yes, the attached patch compiles and boots OK.

Uros.

include/linux/compiler_types.h +7 -2
index 5d6544545658..87a9ce3ebd13 100644
@@ -486,15 +486,19 @@ struct ftrace_likely_data {
  * __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
  *			       non-scalar types unchanged.
  */
+
+#if defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) && !defined(__CHECKER__)
+# define __unqual_scalar_typeof(x) __typeof_unqual__(x)
+#else
 /*
  * Prefer C11 _Generic for better compile-times and simpler code. Note: 'char'
  * is not type-compatible with 'signed char', and we define a separate case.
  */
-#define __scalar_type_to_expr_cases(type)				\
+ #define __scalar_type_to_expr_cases(type)				\
 		unsigned type:	(unsigned type)0,			\
 		signed type:	(signed type)0
 
-#define __unqual_scalar_typeof(x) typeof(				\
+ #define __unqual_scalar_typeof(x) typeof(				\
 		_Generic((x),						\
 			 char:	(char)0,				\
 			 __scalar_type_to_expr_cases(char),		\
@@ -503,6 +507,7 @@ struct ftrace_likely_data {
 			 __scalar_type_to_expr_cases(long),		\
 			 __scalar_type_to_expr_cases(long long),	\
 			 default: (x)))
+#endif
 
 /* Is this type a native word size -- useful for atomic operations */
 #define __native_word(t) \
#5 lore raw

Re: [PATCH v3 2/6] compiler.h: Introduce TYPEOF_UNQUAL() macro

In reply to Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
Quoted 3 lines · Uros Bizjak

Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof operator when available, to return unqualified type of the expression.

Quoted 8 lines
Quoted 3 lines

Current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword.

Ooooh, new toys.

I suppose __unqual_scalar_typeof() wants to be using this when available?

Not only that, the new toy enables clang to check kernel's address spaces in a generic way using address_space attribute.

Please find attached a follow-up patch that enables __percpu checks for all targets, supported by clang. Clang is a little bit pickier than gcc about named address space declarations (it warns for use of duplicated address space attribute), so the patch in addition to the obvious

+# define __percpu_qual __attribute__((address_space(3)))

also fixes a couple of macros that could result in a duplicated address space attribute.

The patch, applied as a follow-up to the series, survives allyesconfig compilation with clang-19 and produces a bootable kernel. The patch was tested only for x86_64 target, for other targets a couple of trivial fixes would be necessary (a cast or a substitution of typeof() with TYPEOF_UNQUAL()).

AFAICS, the same approach using clang's address_space attribute can be implemented to also check other address spaces: __user, __iommu and __rcu.

Uros.

include/asm-generic/percpu.h +6 -1
index 02aeca21479a..4109d828a564 100644
@@ -16,7 +16,12 @@
  * space qualifier).
  */
 #ifndef __percpu_qual
-# define __percpu_qual
+# if __has_attribute(address_space) && \
+     defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) && !defined(__CHECKER__)
+#  define __percpu_qual		__attribute__((address_space(3)))
+# else
+#  define __percpu_qual
+# endif
 #endif
 
 #ifdef CONFIG_SMP
include/linux/device.h +3 -3
index 667cb6db9019..1d6a55d5250a 100644
@@ -431,9 +431,9 @@ static inline int __devm_add_action_or_reset(struct device *dev, void (*action)(
  * RETURNS:
  * Pointer to allocated memory on success, NULL on failure.
  */
-#define devm_alloc_percpu(dev, type)      \
-	((typeof(type) __percpu *)__devm_alloc_percpu((dev), sizeof(type), \
-						      __alignof__(type)))
+#define devm_alloc_percpu(dev, type)				    \
+	((TYPEOF_UNQUAL(type) __percpu *)__devm_alloc_percpu((dev), \
+				sizeof(type), __alignof__(type)))
 
 void __percpu *__devm_alloc_percpu(struct device *dev, size_t size,
 				   size_t align);
include/linux/percpu.h +3 -3
index 52b5ea663b9f..c3bf040aba66 100644
@@ -148,13 +148,13 @@ extern void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved
 	alloc_hooks(pcpu_alloc_noprof(_size, _align, true, GFP_KERNEL))
 
 #define alloc_percpu_gfp(type, gfp)					\
-	(typeof(type) __percpu *)__alloc_percpu_gfp(sizeof(type),	\
+	(TYPEOF_UNQUAL(type) __percpu *)__alloc_percpu_gfp(sizeof(type), \
 						__alignof__(type), gfp)
 #define alloc_percpu(type)						\
-	(typeof(type) __percpu *)__alloc_percpu(sizeof(type),		\
+	(TYPEOF_UNQUAL(type) __percpu *)__alloc_percpu(sizeof(type),	\
 						__alignof__(type))
 #define alloc_percpu_noprof(type)					\
-	((typeof(type) __percpu *)pcpu_alloc_noprof(sizeof(type),	\
+	((TYPEOF_UNQUAL(type) __percpu *)pcpu_alloc_noprof(sizeof(type), \
 					__alignof__(type), false, GFP_KERNEL))
 
 extern void free_percpu(void __percpu *__pdata);
#6 lore raw

PATCH v3 3/6 percpu: Use TYPEOF_UNQUAL() in variable declarations

In reply to Uros Bizjak

Use TYPEOF_UNQUAL() to declare variables as a corresponding type without named address space qualifier to avoid "‘__seg_gs’ specified for auto variable ‘var’" errors.

Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Kent Overstreet <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc Jakub Kicinski <[email protected]>
Cc Paolo Abeni <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Waiman Long <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Brian Gerst <[email protected]>
---
 arch/x86/include/asm/percpu.h | 10 +++++-----
 fs/bcachefs/util.h            |  2 +-
 include/asm-generic/percpu.h  | 26 +++++++++++++-------------
 include/linux/part_stat.h     |  2 +-
 include/linux/percpu-defs.h   |  4 ++--
 include/net/snmp.h            |  5 ++---
 kernel/locking/percpu-rwsem.c |  2 +-
 net/mpls/internal.h           |  4 ++--
 8 files changed, 27 insertions(+), 28 deletions(-)
arch/x86/include/asm/percpu.h +5 -5
index e525cd85f999..666e4137b09f 100644
@@ -180,7 +180,7 @@ do {									\
 	__pcpu_type_##size pto_val__ = __pcpu_cast_##size(_val);	\
 									\
 	if (0) {		                                        \
-		typeof(_var) pto_tmp__;					\
+		TYPEOF_UNQUAL(_var) pto_tmp__;				\
 		pto_tmp__ = (_val);					\
 		(void)pto_tmp__;					\
 	}								\
@@ -219,7 +219,7 @@ do {									\
 	__pcpu_type_##size pto_val__ = __pcpu_cast_##size(_val);	\
 									\
 	if (0) {		                                        \
-		typeof(_var) pto_tmp__;					\
+		TYPEOF_UNQUAL(_var) pto_tmp__;				\
 		pto_tmp__ = (_val);					\
 		(void)pto_tmp__;					\
 	}								\
@@ -240,7 +240,7 @@ do {									\
 			 (val) == (typeof(val))-1)) ? (int)(val) : 0;	\
 									\
 	if (0) {							\
-		typeof(var) pao_tmp__;					\
+		TYPEOF_UNQUAL(var) pao_tmp__;				\
 		pao_tmp__ = (val);					\
 		(void)pao_tmp__;					\
 	}								\
@@ -273,7 +273,7 @@ do {									\
  */
 #define raw_percpu_xchg_op(_var, _nval)					\
 ({									\
-	typeof(_var) pxo_old__ = raw_cpu_read(_var);			\
+	TYPEOF_UNQUAL(_var) pxo_old__ = raw_cpu_read(_var);		\
 									\
 	raw_cpu_write(_var, _nval);					\
 									\
@@ -287,7 +287,7 @@ do {									\
  */
 #define this_percpu_xchg_op(_var, _nval)				\
 ({									\
-	typeof(_var) pxo_old__ = this_cpu_read(_var);			\
+	TYPEOF_UNQUAL(_var) pxo_old__ = this_cpu_read(_var);		\
 									\
 	do { } while (!this_cpu_try_cmpxchg(_var, &pxo_old__, _nval));	\
 									\
fs/bcachefs/util.h +1 -1
index fb02c1c36004..415a5803b8f4 100644
@@ -586,7 +586,7 @@ do {									\
 
 #define per_cpu_sum(_p)							\
 ({									\
-	typeof(*_p) _ret = 0;						\
+	TYPEOF_UNQUAL(*_p) _ret = 0;					\
 									\
 	int cpu;							\
 	for_each_possible_cpu(cpu)					\
include/asm-generic/percpu.h +13 -13
index 94cbd50cc870..50597b975a49 100644
@@ -74,7 +74,7 @@ do {									\
 
 #define raw_cpu_generic_add_return(pcp, val)				\
 ({									\
-	typeof(pcp) *__p = raw_cpu_ptr(&(pcp));				\
+	TYPEOF_UNQUAL(pcp) *__p = raw_cpu_ptr(&(pcp));			\
 									\
 	*__p += val;							\
 	*__p;								\
@@ -82,8 +82,8 @@ do {									\
 
 #define raw_cpu_generic_xchg(pcp, nval)					\
 ({									\
-	typeof(pcp) *__p = raw_cpu_ptr(&(pcp));				\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) *__p = raw_cpu_ptr(&(pcp));			\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	__ret = *__p;							\
 	*__p = nval;							\
 	__ret;								\
@@ -91,7 +91,7 @@ do {									\
 
 #define __cpu_fallback_try_cmpxchg(pcp, ovalp, nval, _cmpxchg)		\
 ({									\
-	typeof(pcp) __val, __old = *(ovalp);				\
+	TYPEOF_UNQUAL(pcp) __val, __old = *(ovalp);			\
 	__val = _cmpxchg(pcp, __old, nval);				\
 	if (__val != __old)						\
 		*(ovalp) = __val;					\
@@ -100,8 +100,8 @@ do {									\
 
 #define raw_cpu_generic_try_cmpxchg(pcp, ovalp, nval)			\
 ({									\
-	typeof(pcp) *__p = raw_cpu_ptr(&(pcp));				\
-	typeof(pcp) __val = *__p, ___old = *(ovalp);			\
+	TYPEOF_UNQUAL(pcp) *__p = raw_cpu_ptr(&(pcp));			\
+	TYPEOF_UNQUAL(pcp) __val = *__p, ___old = *(ovalp);		\
 	bool __ret;							\
 	if (__val == ___old) {						\
 		*__p = nval;						\
@@ -115,14 +115,14 @@ do {									\
 
 #define raw_cpu_generic_cmpxchg(pcp, oval, nval)			\
 ({									\
-	typeof(pcp) __old = (oval);					\
+	TYPEOF_UNQUAL(pcp) __old = (oval);				\
 	raw_cpu_generic_try_cmpxchg(pcp, &__old, nval);			\
 	__old;								\
 })
 
 #define __this_cpu_generic_read_nopreempt(pcp)				\
 ({									\
-	typeof(pcp) ___ret;						\
+	TYPEOF_UNQUAL(pcp) ___ret;					\
 	preempt_disable_notrace();					\
 	___ret = READ_ONCE(*raw_cpu_ptr(&(pcp)));			\
 	preempt_enable_notrace();					\
@@ -131,7 +131,7 @@ do {									\
 
 #define __this_cpu_generic_read_noirq(pcp)				\
 ({									\
-	typeof(pcp) ___ret;						\
+	TYPEOF_UNQUAL(pcp) ___ret;					\
 	unsigned long ___flags;						\
 	raw_local_irq_save(___flags);					\
 	___ret = raw_cpu_generic_read(pcp);				\
@@ -141,7 +141,7 @@ do {									\
 
 #define this_cpu_generic_read(pcp)					\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	if (__native_word(pcp))						\
 		__ret = __this_cpu_generic_read_nopreempt(pcp);		\
 	else								\
@@ -160,7 +160,7 @@ do {									\
 
 #define this_cpu_generic_add_return(pcp, val)				\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	unsigned long __flags;						\
 	raw_local_irq_save(__flags);					\
 	__ret = raw_cpu_generic_add_return(pcp, val);			\
@@ -170,7 +170,7 @@ do {									\
 
 #define this_cpu_generic_xchg(pcp, nval)				\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	unsigned long __flags;						\
 	raw_local_irq_save(__flags);					\
 	__ret = raw_cpu_generic_xchg(pcp, nval);			\
@@ -190,7 +190,7 @@ do {									\
 
 #define this_cpu_generic_cmpxchg(pcp, oval, nval)			\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	unsigned long __flags;						\
 	raw_local_irq_save(__flags);					\
 	__ret = raw_cpu_generic_cmpxchg(pcp, oval, nval);		\
include/linux/part_stat.h +1 -1
index ac8c44dd8237..c5e9cac0575e 100644
@@ -33,7 +33,7 @@ struct disk_stats {
 
 #define part_stat_read(part, field)					\
 ({									\
-	typeof((part)->bd_stats->field) res = 0;			\
+	TYPEOF_UNQUAL((part)->bd_stats->field) res = 0;			\
 	unsigned int _cpu;						\
 	for_each_possible_cpu(_cpu)					\
 		res += per_cpu_ptr((part)->bd_stats, _cpu)->field; \
include/linux/percpu-defs.h +2 -2
index 35842d1e3879..266297b21a5d 100644
@@ -320,7 +320,7 @@ static __always_inline void __this_cpu_preempt_check(const char *op) { }
 
 #define __pcpu_size_call_return(stem, variable)				\
 ({									\
-	typeof(variable) pscr_ret__;					\
+	TYPEOF_UNQUAL(variable) pscr_ret__;				\
 	__verify_pcpu_ptr(&(variable));					\
 	switch(sizeof(variable)) {					\
 	case 1: pscr_ret__ = stem##1(variable); break;			\
@@ -335,7 +335,7 @@ static __always_inline void __this_cpu_preempt_check(const char *op) { }
 
 #define __pcpu_size_call_return2(stem, variable, ...)			\
 ({									\
-	typeof(variable) pscr2_ret__;					\
+	TYPEOF_UNQUAL(variable) pscr2_ret__;				\
 	__verify_pcpu_ptr(&(variable));					\
 	switch(sizeof(variable)) {					\
 	case 1: pscr2_ret__ = stem##1(variable, __VA_ARGS__); break;	\
include/net/snmp.h +2 -3
index 468a67836e2f..4cb4326dfebe 100644
@@ -159,7 +159,7 @@ struct linux_tls_mib {
 
 #define __SNMP_ADD_STATS64(mib, field, addend) 				\
 	do {								\
-		__typeof__(*mib) *ptr = raw_cpu_ptr(mib);		\
+		TYPEOF_UNQUAL(*mib) *ptr = raw_cpu_ptr(mib);		\
 		u64_stats_update_begin(&ptr->syncp);			\
 		ptr->mibs[field] += addend;				\
 		u64_stats_update_end(&ptr->syncp);			\
@@ -176,8 +176,7 @@ struct linux_tls_mib {
 #define SNMP_INC_STATS64(mib, field) SNMP_ADD_STATS64(mib, field, 1)
 #define __SNMP_UPD_PO_STATS64(mib, basefield, addend)			\
 	do {								\
-		__typeof__(*mib) *ptr;				\
-		ptr = raw_cpu_ptr((mib));				\
+		TYPEOF_UNQUAL(*mib) *ptr = raw_cpu_ptr(mib);		\
 		u64_stats_update_begin(&ptr->syncp);			\
 		ptr->mibs[basefield##PKTS]++;				\
 		ptr->mibs[basefield##OCTETS] += addend;			\
kernel/locking/percpu-rwsem.c +1 -1
index 6083883c4fe0..d6964fc29f51 100644
@@ -184,7 +184,7 @@ EXPORT_SYMBOL_GPL(__percpu_down_read);
 
 #define per_cpu_sum(var)						\
 ({									\
-	typeof(var) __sum = 0;						\
+	TYPEOF_UNQUAL(var) __sum = 0;					\
 	int cpu;							\
 	compiletime_assert_atomic_type(__sum);				\
 	for_each_possible_cpu(cpu)					\
net/mpls/internal.h +2 -2
index b9f492ddf93b..83c629529b57 100644
@@ -33,7 +33,7 @@ struct mpls_dev {
 
 #define MPLS_INC_STATS_LEN(mdev, len, pkts_field, bytes_field)		\
 	do {								\
-		__typeof__(*(mdev)->stats) *ptr =			\
+		TYPEOF_UNQUAL(*(mdev)->stats) *ptr =			\
 			raw_cpu_ptr((mdev)->stats);			\
 		local_bh_disable();					\
 		u64_stats_update_begin(&ptr->syncp);			\
@@ -45,7 +45,7 @@ struct mpls_dev {
 
 #define MPLS_INC_STATS(mdev, field)					\
 	do {								\
-		__typeof__(*(mdev)->stats) *ptr =			\
+		TYPEOF_UNQUAL(*(mdev)->stats) *ptr =			\
 			raw_cpu_ptr((mdev)->stats);			\
 		local_bh_disable();					\
 		u64_stats_update_begin(&ptr->syncp);			\
Signature · 1 line
2.42.0
#7 lore raw

PATCH v3 4/6 percpu: Use TYPEOF_UNQUAL() in *_cpu_ptr() accessors

In reply to Uros Bizjak

Use TYPEOF_UNQUAL() macro to declare the return type of *_cpu_ptr() accessors in the generic named address space to avoid access to data from pointer to non-enclosed address space type of errors.

Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/x86/include/asm/percpu.h | 8 ++++++--
 include/linux/percpu-defs.h   | 2 +-
 2 files changed, 7 insertions(+), 3 deletions(-)
arch/x86/include/asm/percpu.h +6 -2
index 666e4137b09f..27f668660abe 100644
@@ -73,10 +73,14 @@
 	unsigned long tcp_ptr__ = raw_cpu_read_long(this_cpu_off);	\
 									\
 	tcp_ptr__ += (__force unsigned long)(_ptr);			\
-	(typeof(*(_ptr)) __kernel __force *)tcp_ptr__;			\
+	(TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)tcp_ptr__;		\
 })
 #else
-#define arch_raw_cpu_ptr(_ptr) ({ BUILD_BUG(); (typeof(_ptr))0; })
+#define arch_raw_cpu_ptr(_ptr)						\
+({									\
+	BUILD_BUG();							\
+	(TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)0;			\
+})
 #endif
 
 #define PER_CPU_VAR(var)	%__percpu_seg:(var)__percpu_rel
include/linux/percpu-defs.h +1 -1
index 266297b21a5d..2921ea97d242 100644
@@ -223,7 +223,7 @@ do {									\
 #define PERCPU_PTR(__p)							\
 ({									\
 	unsigned long __pcpu_ptr = (__force unsigned long)(__p);	\
-	(typeof(*(__p)) __force __kernel *)(__pcpu_ptr);		\
+	(TYPEOF_UNQUAL(*(__p)) __force __kernel *)(__pcpu_ptr);		\
 })
 
 #ifdef CONFIG_SMP
Signature · 1 line
2.42.0
#8 lore raw

PATCH v3 5/6 percpu: Repurpose __percpu tag as a named address space qualifier

In reply to Uros Bizjak

The patch introduces __percpu_qual define and repurposes __percpu tag as a named address space qualifier using the new define.

Arches can now conditionally define __percpu_qual as their named address space qualifier for percpu variables.

Cc Arnd Bergmann <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Brian Gerst <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
v3: - Rename __per_cpu_qual to __percpu_qual.

---
 include/asm-generic/percpu.h   | 13 +++++++++++++
 include/linux/compiler_types.h |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)
include/asm-generic/percpu.h +13 -0
index 50597b975a49..02aeca21479a 100644
@@ -6,6 +6,19 @@
 #include <linux/threads.h>
 #include <linux/percpu-defs.h>
 
+/*
+ * __percpu_qual is the qualifier for the percpu named address space.
+ *
+ * Most arches use generic named address space for percpu variables but
+ * some arches define percpu variables in different named address space
+ * (on the x86 arch, percpu variable may be declared as being relative
+ * to the %fs or %gs segments using __seg_fs or __seg_gs named address
+ * space qualifier).
+ */
+#ifndef __percpu_qual
+# define __percpu_qual
+#endif
+
 #ifdef CONFIG_SMP
 
 /*
include/linux/compiler_types.h +1 -1
index 981cc3d7e3aa..5d6544545658 100644
@@ -57,7 +57,7 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 #  define __user	BTF_TYPE_TAG(user)
 # endif
 # define __iomem
-# define __percpu	BTF_TYPE_TAG(percpu)
+# define __percpu	__percpu_qual BTF_TYPE_TAG(percpu)
 # define __rcu		BTF_TYPE_TAG(rcu)
 
 # define __chk_user_ptr(x)	(void)0
Signature · 1 line
2.42.0
#9 lore raw

PATCH v3 6/6 percpu/x86: Enable strict percpu checks via named AS qualifiers

In reply to Uros Bizjak

This patch declares percpu variables in __seg_gs/__seg_fs named AS and keeps them named AS qualified until they are dereferenced with percpu accessor. This approach enables various compiler check for cross-namespace variable assignments.

Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
v2: - Add comment to remove test for __CHECKER__ once sparse learns
about __typeof_unqual__.

---
 arch/x86/include/asm/percpu.h | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
arch/x86/include/asm/percpu.h +17 -3
index 27f668660abe..1ef08289e667 100644
@@ -95,9 +95,23 @@
 
 #endif /* CONFIG_SMP */
 
-#define __my_cpu_type(var)	typeof(var) __percpu_seg_override
-#define __my_cpu_ptr(ptr)	(__my_cpu_type(*(ptr))*)(__force uintptr_t)(ptr)
-#define __my_cpu_var(var)	(*__my_cpu_ptr(&(var)))
+/*
+ * XXX: Remove test for __CHECKER__ once
+ * sparse learns about __typeof_unqual__.
+ */
+#if defined(CONFIG_USE_X86_SEG_SUPPORT) && \
+    defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) && !defined(__CHECKER__)
+# define __my_cpu_type(var)	typeof(var)
+# define __my_cpu_ptr(ptr)	(ptr)
+# define __my_cpu_var(var)	(var)
+
+# define __percpu_qual		__percpu_seg_override
+#else
+# define __my_cpu_type(var)	typeof(var) __percpu_seg_override
+# define __my_cpu_ptr(ptr)	(__my_cpu_type(*(ptr))*)(__force uintptr_t)(ptr)
+# define __my_cpu_var(var)	(*__my_cpu_ptr(&(var)))
+#endif
+
 #define __percpu_arg(x)		__percpu_prefix "%" #x
 #define __force_percpu_arg(x)	__force_percpu_prefix "%" #x
 
Signature · 1 line
2.42.0
maintainer Acked-by on the v3 series
#10 lore raw
In reply to Uros Bizjak

Hi Uros,

Quoted 41 lines · Uros Bizjak

Enable strict percpu address space checks via x86 named address space qualifiers. Percpu variables are declared in __seg_gs/__seg_fs named AS and kept named AS qualified until they are dereferenced via percpu accessor. This approach enables various compiler checks for cross-namespace variable assignments.

Please note that current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword. The proposed patch by Dan Carpenter to implement __typeof_unqual__() handling in sparse is located at:

https://lore.kernel.org/lkml/[email protected]/

v2: - Add comment to remove test for __CHECKER__ once sparse learns
about __typeof_unqual__.
- Add Acked-by: tags.
v3: - Rename __per_cpu_qual to __percpu_qual.
- Add more Acked-by: tags.

Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Nadav Amit <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Dan Carpenter <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Peter Zijlstra <[email protected]>

Uros Bizjak (6):
x86/kgdb: Use IS_ERR_PCPU() macro
compiler.h: Introduce TYPEOF_UNQUAL() macro
percpu: Use TYPEOF_UNQUAL() in variable declarations
percpu: Use TYPEOF_UNQUAL() in *_cpu_ptr() accessors
percpu: Repurpose __percpu tag as a named address space qualifier percpu/x86: Enable strict percpu checks via named AS qualifiers

Quoted 17 lines
 arch/x86/include/asm/percpu.h  | 38 ++++++++++++++++++++++++---------
 arch/x86/kernel/kgdb.c         |  2 +-
 fs/bcachefs/util.h             |  2 +-
 include/asm-generic/percpu.h   | 39 ++++++++++++++++++++++------------
 include/linux/compiler.h       | 13 ++++++++++++
 include/linux/compiler_types.h |  2 +-
 include/linux/part_stat.h      |  2 +-
 include/linux/percpu-defs.h    |  6 +++---
 include/net/snmp.h             |  5 ++---
 init/Kconfig                   |  3 +++
 kernel/locking/percpu-rwsem.c  |  2 +-
 net/mpls/internal.h            |  4 ++--
 12 files changed, 82 insertions(+), 36 deletions(-)
Signature · 1 line
2.42.0

This all looks good to me.

Thanks,
Dennis

maintainer
#11 lore raw
In reply to Uros Bizjak
Quoted 11 lines · On Sun, 8 Dec 2024 21:45:15 +0100 Uros Bizjak <[email protected]> wrote:

Enable strict percpu address space checks via x86 named address space qualifiers. Percpu variables are declared in __seg_gs/__seg_fs named AS and kept named AS qualified until they are dereferenced via percpu accessor. This approach enables various compiler checks for cross-namespace variable assignments.

Please note that current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword. The proposed patch by Dan Carpenter to implement __typeof_unqual__() handling in sparse is located at:

google("what the hell is typeof_unequal") failed me.

I think it would be nice to include within the changelog (and code comments!) an explanation-for-others of what this thing is and why anyone would want to use it. Rather than assuming that all kernel developers are typeof() experts!

#12 lore raw
Quoted 9 lines · Andrew Morton
Quoted 6 lines · On Sun, 8 Dec 2024 21:45:15 +0100 Uros Bizjak <[email protected]> wrote:

Enable strict percpu address space checks via x86 named address space qualifiers. Percpu variables are declared in __seg_gs/__seg_fs named AS and kept named AS qualified until they are dereferenced via percpu accessor. This approach enables various compiler checks for cross-namespace variable assignments.

Quoted 7 lines
Quoted 5 lines

Please note that current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword. The proposed patch by Dan Carpenter to implement __typeof_unqual__() handling in sparse is located at:

google("what the hell is typeof_unequal") failed me.

It is not "typeof_unequal", but "typeof_unqual", as in "unqualified".

Apparently, google does not like expletives, googling for "What is typeof_unqual?" returns some very informative hits, e.g.:

https://en.cppreference.com/w/c/keyword/typeof_unqual
https://learn.microsoft.com/en-us/cpp/c-language/typeof-unqual-c?view=msvc-170
https://gcc.gnu.org/onlinedocs/gcc/Typeof.html
https://dev.to/pauljlucas/typeof-in-c23-55p2

Quoted 4 lines

I think it would be nice to include within the changelog (and code comments!) an explanation-for-others of what this thing is and why anyone would want to use it. Rather than assuming that all kernel developers are typeof() experts!

The comment above definition of TYPEOF_UNQUAL in [PATCH 2/6] summarises the above as:

+ * Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof
+ * operator when available, to return unqualified type of the exp.

which is basically what the standard says in its reference document.

Thanks,
Uros.

#13 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton
Quoted 6 lines · On Sun, 8 Dec 2024 21:45:15 +0100 Uros Bizjak <[email protected]> wrote:

Enable strict percpu address space checks via x86 named address space qualifiers. Percpu variables are declared in __seg_gs/__seg_fs named AS and kept named AS qualified until they are dereferenced via percpu accessor. This approach enables various compiler checks for cross-namespace variable assignments.

Quoted 7 lines
Quoted 5 lines

Please note that current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword. The proposed patch by Dan Carpenter to implement __typeof_unqual__() handling in sparse is located at:

google("what the hell is typeof_unequal") failed me.

I'm glad I'm not the only person who read that as "equal" instead of "qualified".

regards,
dan carpenter