Skip to contentSkip to conversation

Enable strict percpu address space checks

PATCH Superseded Uros Bizjak 3 participants 10 messages Dec 5, 2024 – Dec 6, 2024

Subsystems (8)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover Enable strict percpu address space checks

This patchset enables strict percpu address space checks via x86 named address space qualifiers. Percpu variables are declared in __seg_gs/__seg_fs named AS and kept named AS qualified until they are dereferenced via percpu accessor. This approach enables various compiler checks for cross-namespace variable assignments.

Please note that current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword.

v2: - Add comment to remove test for __CHECKER__ once sparse learns
about __typeof_unqual__.
- Add Acked-by: tags.

Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Nadav Amit <[email protected]>
Cc Brian Gerst <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Peter Zijlstra <[email protected]>

Uros Bizjak (6):
x86/kgdb: Use IS_ERR_PCPU() macro
compiler.h: Introduce TYPEOF_UNQUAL() macro
percpu: Use TYPEOF_UNQUAL() in variable declarations
percpu: Use TYPEOF_UNQUAL() in *_cpu_ptr() accessors
percpu: Repurpose __percpu tag as a named address space qualifier percpu/x86: Enable strict percpu checks via named AS qualifiers

 arch/x86/include/asm/percpu.h  | 38 ++++++++++++++++++++++---------
 arch/x86/kernel/kgdb.c         |  2 +-
 fs/bcachefs/util.h             |  2 +-
 include/asm-generic/percpu.h   | 41 +++++++++++++++++++++++-----------
 include/linux/compiler.h       | 13 +++++++++++
 include/linux/compiler_types.h |  2 +-
 include/linux/part_stat.h      |  2 +-
 include/linux/percpu-defs.h    |  6 ++---
 include/net/snmp.h             |  5 ++---
 init/Kconfig                   |  3 +++
 kernel/locking/percpu-rwsem.c  |  2 +-
 net/mpls/internal.h            |  4 ++--
 12 files changed, 84 insertions(+), 36 deletions(-)
Signature · 1 line
2.42.0
#1 lore raw

PATCH v2 1/6 x86/kgdb: Use IS_ERR_PCPU() macro

Use IS_ERR_PCPU() when checking the error pointer in the percpu address space. This macro adds intermediate cast to unsigned long when switching named address spaces.

The patch will avoid future build errors due to pointer address space mismatch with enabled strict percpu address space checks.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/x86/kernel/kgdb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/kgdb.c +1 -1
index 9c9faa1634fb..102641fd2172 100644
@@ -655,7 +655,7 @@ void kgdb_arch_late(void)
 		if (breakinfo[i].pev)
 			continue;
 		breakinfo[i].pev = register_wide_hw_breakpoint(&attr, NULL, NULL);
-		if (IS_ERR((void * __force)breakinfo[i].pev)) {
+		if (IS_ERR_PCPU(breakinfo[i].pev)) {
 			printk(KERN_ERR "kgdb: Could not allocate hw"
 			       "breakpoints\nDisabling the kernel debugger\n");
 			breakinfo[i].pev = NULL;
Signature · 1 line
2.42.0
#2 lore raw

PATCH v2 2/6 compiler.h: Introduce TYPEOF_UNQUAL() macro

In reply to Uros Bizjak

Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof operator when available, to return unqualified type of the expression.

Current version of sparse doesn't know anything about __typeof_unqual__() operator. Avoid the usage of __typeof_unqual__() when sparse checking is active to prevent sparse errors with unknowing keyword.

Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Peter Zijlstra <[email protected]
---
 include/linux/compiler.h | 13 +++++++++++++
 init/Kconfig             |  3 +++
 2 files changed, 16 insertions(+)
include/linux/compiler.h +13 -0
index 469a64dd6495..ec0429d7a153 100644
@@ -321,6 +321,19 @@ static inline void *offset_to_ptr(const int *off)
  */
 #define prevent_tail_call_optimization()	mb()
 
+/*
+ * Define TYPEOF_UNQUAL() to use __typeof_unqual__() as typeof
+ * operator when available, to return unqualified type of the exp.
+ *
+ * XXX: Remove test for __CHECKER__ once
+ * sparse learns about __typeof_unqual__.
+ */
+#if defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) && !defined(__CHECKER__)
+# define TYPEOF_UNQUAL(exp) __typeof_unqual__(exp)
+#else
+# define TYPEOF_UNQUAL(exp) __typeof__(exp)
+#endif
+
 #include <asm/rwonce.h>
 
 #endif /* __LINUX_COMPILER_H */
init/Kconfig +3 -0
index a20e6efd3f0f..c1f9eb3d5f2e 100644
@@ -894,6 +894,9 @@ config ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
 config CC_HAS_INT128
 	def_bool !$(cc-option,$(m64-flag) -D__SIZEOF_INT128__=0) && 64BIT
 
+config CC_HAS_TYPEOF_UNQUAL
+	def_bool $(success,echo 'int foo (int a) { __typeof_unqual__(a) b = a; return b; }' | $(CC) -x c - -S -o /dev/null)
+
 config CC_IMPLICIT_FALLTHROUGH
 	string
 	default "-Wimplicit-fallthrough=5" if CC_IS_GCC && $(cc-option,-Wimplicit-fallthrough=5)
Signature · 1 line
2.42.0
#3 lore raw

PATCH v2 3/6 percpu: Use TYPEOF_UNQUAL() in variable declarations

In reply to Uros Bizjak

Use TYPEOF_UNQUAL() to declare variables as a corresponding type without named address space qualifier to avoid "‘__seg_gs’ specified for auto variable ‘var’" errors.

Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Kent Overstreet <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc Jakub Kicinski <[email protected]>
Cc Paolo Abeni <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Waiman Long <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Brian Gerst <[email protected]>
---
 arch/x86/include/asm/percpu.h | 10 +++++-----
 fs/bcachefs/util.h            |  2 +-
 include/asm-generic/percpu.h  | 26 +++++++++++++-------------
 include/linux/part_stat.h     |  2 +-
 include/linux/percpu-defs.h   |  4 ++--
 include/net/snmp.h            |  5 ++---
 kernel/locking/percpu-rwsem.c |  2 +-
 net/mpls/internal.h           |  4 ++--
 8 files changed, 27 insertions(+), 28 deletions(-)
arch/x86/include/asm/percpu.h +5 -5
index e525cd85f999..666e4137b09f 100644
@@ -180,7 +180,7 @@ do {									\
 	__pcpu_type_##size pto_val__ = __pcpu_cast_##size(_val);	\
 									\
 	if (0) {		                                        \
-		typeof(_var) pto_tmp__;					\
+		TYPEOF_UNQUAL(_var) pto_tmp__;				\
 		pto_tmp__ = (_val);					\
 		(void)pto_tmp__;					\
 	}								\
@@ -219,7 +219,7 @@ do {									\
 	__pcpu_type_##size pto_val__ = __pcpu_cast_##size(_val);	\
 									\
 	if (0) {		                                        \
-		typeof(_var) pto_tmp__;					\
+		TYPEOF_UNQUAL(_var) pto_tmp__;				\
 		pto_tmp__ = (_val);					\
 		(void)pto_tmp__;					\
 	}								\
@@ -240,7 +240,7 @@ do {									\
 			 (val) == (typeof(val))-1)) ? (int)(val) : 0;	\
 									\
 	if (0) {							\
-		typeof(var) pao_tmp__;					\
+		TYPEOF_UNQUAL(var) pao_tmp__;				\
 		pao_tmp__ = (val);					\
 		(void)pao_tmp__;					\
 	}								\
@@ -273,7 +273,7 @@ do {									\
  */
 #define raw_percpu_xchg_op(_var, _nval)					\
 ({									\
-	typeof(_var) pxo_old__ = raw_cpu_read(_var);			\
+	TYPEOF_UNQUAL(_var) pxo_old__ = raw_cpu_read(_var);		\
 									\
 	raw_cpu_write(_var, _nval);					\
 									\
@@ -287,7 +287,7 @@ do {									\
  */
 #define this_percpu_xchg_op(_var, _nval)				\
 ({									\
-	typeof(_var) pxo_old__ = this_cpu_read(_var);			\
+	TYPEOF_UNQUAL(_var) pxo_old__ = this_cpu_read(_var);		\
 									\
 	do { } while (!this_cpu_try_cmpxchg(_var, &pxo_old__, _nval));	\
 									\
fs/bcachefs/util.h +1 -1
index fb02c1c36004..415a5803b8f4 100644
@@ -586,7 +586,7 @@ do {									\
 
 #define per_cpu_sum(_p)							\
 ({									\
-	typeof(*_p) _ret = 0;						\
+	TYPEOF_UNQUAL(*_p) _ret = 0;					\
 									\
 	int cpu;							\
 	for_each_possible_cpu(cpu)					\
include/asm-generic/percpu.h +13 -13
index 94cbd50cc870..50597b975a49 100644
@@ -74,7 +74,7 @@ do {									\
 
 #define raw_cpu_generic_add_return(pcp, val)				\
 ({									\
-	typeof(pcp) *__p = raw_cpu_ptr(&(pcp));				\
+	TYPEOF_UNQUAL(pcp) *__p = raw_cpu_ptr(&(pcp));			\
 									\
 	*__p += val;							\
 	*__p;								\
@@ -82,8 +82,8 @@ do {									\
 
 #define raw_cpu_generic_xchg(pcp, nval)					\
 ({									\
-	typeof(pcp) *__p = raw_cpu_ptr(&(pcp));				\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) *__p = raw_cpu_ptr(&(pcp));			\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	__ret = *__p;							\
 	*__p = nval;							\
 	__ret;								\
@@ -91,7 +91,7 @@ do {									\
 
 #define __cpu_fallback_try_cmpxchg(pcp, ovalp, nval, _cmpxchg)		\
 ({									\
-	typeof(pcp) __val, __old = *(ovalp);				\
+	TYPEOF_UNQUAL(pcp) __val, __old = *(ovalp);			\
 	__val = _cmpxchg(pcp, __old, nval);				\
 	if (__val != __old)						\
 		*(ovalp) = __val;					\
@@ -100,8 +100,8 @@ do {									\
 
 #define raw_cpu_generic_try_cmpxchg(pcp, ovalp, nval)			\
 ({									\
-	typeof(pcp) *__p = raw_cpu_ptr(&(pcp));				\
-	typeof(pcp) __val = *__p, ___old = *(ovalp);			\
+	TYPEOF_UNQUAL(pcp) *__p = raw_cpu_ptr(&(pcp));			\
+	TYPEOF_UNQUAL(pcp) __val = *__p, ___old = *(ovalp);		\
 	bool __ret;							\
 	if (__val == ___old) {						\
 		*__p = nval;						\
@@ -115,14 +115,14 @@ do {									\
 
 #define raw_cpu_generic_cmpxchg(pcp, oval, nval)			\
 ({									\
-	typeof(pcp) __old = (oval);					\
+	TYPEOF_UNQUAL(pcp) __old = (oval);				\
 	raw_cpu_generic_try_cmpxchg(pcp, &__old, nval);			\
 	__old;								\
 })
 
 #define __this_cpu_generic_read_nopreempt(pcp)				\
 ({									\
-	typeof(pcp) ___ret;						\
+	TYPEOF_UNQUAL(pcp) ___ret;					\
 	preempt_disable_notrace();					\
 	___ret = READ_ONCE(*raw_cpu_ptr(&(pcp)));			\
 	preempt_enable_notrace();					\
@@ -131,7 +131,7 @@ do {									\
 
 #define __this_cpu_generic_read_noirq(pcp)				\
 ({									\
-	typeof(pcp) ___ret;						\
+	TYPEOF_UNQUAL(pcp) ___ret;					\
 	unsigned long ___flags;						\
 	raw_local_irq_save(___flags);					\
 	___ret = raw_cpu_generic_read(pcp);				\
@@ -141,7 +141,7 @@ do {									\
 
 #define this_cpu_generic_read(pcp)					\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	if (__native_word(pcp))						\
 		__ret = __this_cpu_generic_read_nopreempt(pcp);		\
 	else								\
@@ -160,7 +160,7 @@ do {									\
 
 #define this_cpu_generic_add_return(pcp, val)				\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	unsigned long __flags;						\
 	raw_local_irq_save(__flags);					\
 	__ret = raw_cpu_generic_add_return(pcp, val);			\
@@ -170,7 +170,7 @@ do {									\
 
 #define this_cpu_generic_xchg(pcp, nval)				\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	unsigned long __flags;						\
 	raw_local_irq_save(__flags);					\
 	__ret = raw_cpu_generic_xchg(pcp, nval);			\
@@ -190,7 +190,7 @@ do {									\
 
 #define this_cpu_generic_cmpxchg(pcp, oval, nval)			\
 ({									\
-	typeof(pcp) __ret;						\
+	TYPEOF_UNQUAL(pcp) __ret;					\
 	unsigned long __flags;						\
 	raw_local_irq_save(__flags);					\
 	__ret = raw_cpu_generic_cmpxchg(pcp, oval, nval);		\
include/linux/part_stat.h +1 -1
index ac8c44dd8237..c5e9cac0575e 100644
@@ -33,7 +33,7 @@ struct disk_stats {
 
 #define part_stat_read(part, field)					\
 ({									\
-	typeof((part)->bd_stats->field) res = 0;			\
+	TYPEOF_UNQUAL((part)->bd_stats->field) res = 0;			\
 	unsigned int _cpu;						\
 	for_each_possible_cpu(_cpu)					\
 		res += per_cpu_ptr((part)->bd_stats, _cpu)->field; \
include/linux/percpu-defs.h +2 -2
index 35842d1e3879..266297b21a5d 100644
@@ -320,7 +320,7 @@ static __always_inline void __this_cpu_preempt_check(const char *op) { }
 
 #define __pcpu_size_call_return(stem, variable)				\
 ({									\
-	typeof(variable) pscr_ret__;					\
+	TYPEOF_UNQUAL(variable) pscr_ret__;				\
 	__verify_pcpu_ptr(&(variable));					\
 	switch(sizeof(variable)) {					\
 	case 1: pscr_ret__ = stem##1(variable); break;			\
@@ -335,7 +335,7 @@ static __always_inline void __this_cpu_preempt_check(const char *op) { }
 
 #define __pcpu_size_call_return2(stem, variable, ...)			\
 ({									\
-	typeof(variable) pscr2_ret__;					\
+	TYPEOF_UNQUAL(variable) pscr2_ret__;				\
 	__verify_pcpu_ptr(&(variable));					\
 	switch(sizeof(variable)) {					\
 	case 1: pscr2_ret__ = stem##1(variable, __VA_ARGS__); break;	\
include/net/snmp.h +2 -3
index 468a67836e2f..4cb4326dfebe 100644
@@ -159,7 +159,7 @@ struct linux_tls_mib {
 
 #define __SNMP_ADD_STATS64(mib, field, addend) 				\
 	do {								\
-		__typeof__(*mib) *ptr = raw_cpu_ptr(mib);		\
+		TYPEOF_UNQUAL(*mib) *ptr = raw_cpu_ptr(mib);		\
 		u64_stats_update_begin(&ptr->syncp);			\
 		ptr->mibs[field] += addend;				\
 		u64_stats_update_end(&ptr->syncp);			\
@@ -176,8 +176,7 @@ struct linux_tls_mib {
 #define SNMP_INC_STATS64(mib, field) SNMP_ADD_STATS64(mib, field, 1)
 #define __SNMP_UPD_PO_STATS64(mib, basefield, addend)			\
 	do {								\
-		__typeof__(*mib) *ptr;				\
-		ptr = raw_cpu_ptr((mib));				\
+		TYPEOF_UNQUAL(*mib) *ptr = raw_cpu_ptr(mib);		\
 		u64_stats_update_begin(&ptr->syncp);			\
 		ptr->mibs[basefield##PKTS]++;				\
 		ptr->mibs[basefield##OCTETS] += addend;			\
kernel/locking/percpu-rwsem.c +1 -1
index 6083883c4fe0..d6964fc29f51 100644
@@ -184,7 +184,7 @@ EXPORT_SYMBOL_GPL(__percpu_down_read);
 
 #define per_cpu_sum(var)						\
 ({									\
-	typeof(var) __sum = 0;						\
+	TYPEOF_UNQUAL(var) __sum = 0;					\
 	int cpu;							\
 	compiletime_assert_atomic_type(__sum);				\
 	for_each_possible_cpu(cpu)					\
net/mpls/internal.h +2 -2
index b9f492ddf93b..83c629529b57 100644
@@ -33,7 +33,7 @@ struct mpls_dev {
 
 #define MPLS_INC_STATS_LEN(mdev, len, pkts_field, bytes_field)		\
 	do {								\
-		__typeof__(*(mdev)->stats) *ptr =			\
+		TYPEOF_UNQUAL(*(mdev)->stats) *ptr =			\
 			raw_cpu_ptr((mdev)->stats);			\
 		local_bh_disable();					\
 		u64_stats_update_begin(&ptr->syncp);			\
@@ -45,7 +45,7 @@ struct mpls_dev {
 
 #define MPLS_INC_STATS(mdev, field)					\
 	do {								\
-		__typeof__(*(mdev)->stats) *ptr =			\
+		TYPEOF_UNQUAL(*(mdev)->stats) *ptr =			\
 			raw_cpu_ptr((mdev)->stats);			\
 		local_bh_disable();					\
 		u64_stats_update_begin(&ptr->syncp);			\
Signature · 1 line
2.42.0
#4 lore raw

PATCH v2 4/6 percpu: Use TYPEOF_UNQUAL() in *_cpu_ptr() accessors

In reply to Uros Bizjak

Use TYPEOF_UNQUAL() macro to declare the return type of *_cpu_ptr() accessors in the generic named address space to avoid access to data from pointer to non-enclosed address space type of errors.

Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/x86/include/asm/percpu.h | 8 ++++++--
 include/linux/percpu-defs.h   | 2 +-
 2 files changed, 7 insertions(+), 3 deletions(-)
arch/x86/include/asm/percpu.h +6 -2
index 666e4137b09f..27f668660abe 100644
@@ -73,10 +73,14 @@
 	unsigned long tcp_ptr__ = raw_cpu_read_long(this_cpu_off);	\
 									\
 	tcp_ptr__ += (__force unsigned long)(_ptr);			\
-	(typeof(*(_ptr)) __kernel __force *)tcp_ptr__;			\
+	(TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)tcp_ptr__;		\
 })
 #else
-#define arch_raw_cpu_ptr(_ptr) ({ BUILD_BUG(); (typeof(_ptr))0; })
+#define arch_raw_cpu_ptr(_ptr)						\
+({									\
+	BUILD_BUG();							\
+	(TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)0;			\
+})
 #endif
 
 #define PER_CPU_VAR(var)	%__percpu_seg:(var)__percpu_rel
include/linux/percpu-defs.h +1 -1
index 266297b21a5d..2921ea97d242 100644
@@ -223,7 +223,7 @@ do {									\
 #define PERCPU_PTR(__p)							\
 ({									\
 	unsigned long __pcpu_ptr = (__force unsigned long)(__p);	\
-	(typeof(*(__p)) __force __kernel *)(__pcpu_ptr);		\
+	(TYPEOF_UNQUAL(*(__p)) __force __kernel *)(__pcpu_ptr);		\
 })
 
 #ifdef CONFIG_SMP
Signature · 1 line
2.42.0
#5 lore raw

PATCH v2 5/6 percpu: Repurpose __percpu tag as a named address space qualifier

In reply to Uros Bizjak

The patch introduces per_cpu_qual define and repurposes __percpu tag as a named address space qualifier using the new define.

Arches can now conditionally define __per_cpu_qual as their named address space qualifier for percpu variables.

Cc Arnd Bergmann <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Brian Gerst <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 include/asm-generic/percpu.h   | 15 +++++++++++++++
 include/linux/compiler_types.h |  2 +-
 2 files changed, 16 insertions(+), 1 deletion(-)
include/asm-generic/percpu.h +15 -0
index 50597b975a49..3b93b168faa1 100644
@@ -6,6 +6,21 @@
 #include <linux/threads.h>
 #include <linux/percpu-defs.h>
 
+/*
+ * per_cpu_qual is the qualifier for the percpu named address space.
+ *
+ * Most arches use generic named address space for percpu variables but
+ * some arches define percpu variables in different named address space
+ * (on the x86 arch, percpu variable may be declared as being relative
+ * to the %fs or %gs segments using __seg_fs or __seg_gs named address
+ * space qualifier).
+ */
+#ifdef __per_cpu_qual
+# define per_cpu_qual __per_cpu_qual
+#else
+# define per_cpu_qual
+#endif
+
 #ifdef CONFIG_SMP
 
 /*
include/linux/compiler_types.h +1 -1
index 981cc3d7e3aa..877fe0c43c5d 100644
@@ -57,7 +57,7 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 #  define __user	BTF_TYPE_TAG(user)
 # endif
 # define __iomem
-# define __percpu	BTF_TYPE_TAG(percpu)
+# define __percpu	per_cpu_qual BTF_TYPE_TAG(percpu)
 # define __rcu		BTF_TYPE_TAG(rcu)
 
 # define __chk_user_ptr(x)	(void)0
Signature · 1 line
2.42.0
maintainer
#6 lore raw

Re: [PATCH v2 5/6] percpu: Repurpose __percpu tag as a named address space qualifier

Hi Uros,

Quoted 32 lines · Uros Bizjak

The patch introduces per_cpu_qual define and repurposes __percpu tag as a named address space qualifier using the new define.

Arches can now conditionally define __per_cpu_qual as their named address space qualifier for percpu variables.

Cc Arnd Bergmann <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Brian Gerst <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 include/asm-generic/percpu.h   | 15 +++++++++++++++
 include/linux/compiler_types.h |  2 +-
 2 files changed, 16 insertions(+), 1 deletion(-)
include/asm-generic/percpu.h +0 -0
index 50597b975a49..3b93b168faa1 100644
@@ -6,6 +6,21 @@
 #include <linux/threads.h>
 #include <linux/percpu-defs.h>
 

include/asm-generic/percpu.h line 18

Quoted 10 lines
include/asm-generic/percpu.h +10 -0
+/*
+ * per_cpu_qual is the qualifier for the percpu named address space.
+ *
+ * Most arches use generic named address space for percpu variables but
+ * some arches define percpu variables in different named address space
+ * (on the x86 arch, percpu variable may be declared as being relative
+ * to the %fs or %gs segments using __seg_fs or __seg_gs named address
+ * space qualifier).
+ */
+#ifdef __per_cpu_qual

I read through the series and I think my only nit would be here. Can we name this __percpu_qual? My thoughts are that it keeps it consistent with the old address space identifier and largely most of the core percpu stuff is defined with percpu as the naming scheme.

Quoted 24 lines
include/asm-generic/percpu.h +5 -0
+# define per_cpu_qual __per_cpu_qual
+#else
+# define per_cpu_qual
+#endif
+
 #ifdef CONFIG_SMP
 
 /*
include/linux/compiler_types.h +1 -1
index 981cc3d7e3aa..877fe0c43c5d 100644
@@ -57,7 +57,7 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 #  define __user	BTF_TYPE_TAG(user)
 # endif
 # define __iomem
-# define __percpu	BTF_TYPE_TAG(percpu)
+# define __percpu	per_cpu_qual BTF_TYPE_TAG(percpu)
 # define __rcu		BTF_TYPE_TAG(rcu)
 
 # define __chk_user_ptr(x)	(void)0
Signature · 1 line
2.42.0

Thanks,
Dennis

#7 lore raw

Re: [PATCH v2 5/6] percpu: Repurpose __percpu tag as a named address space qualifier

Quoted 4 lines · Dennis Zhou

I read through the series and I think my only nit would be here. Can we name this __percpu_qual? My thoughts are that it keeps it consistent with the old address space identifier and largely most of the core percpu stuff is defined with percpu as the naming scheme.

Yes lets keep it consistent.

#8 lore raw

Re: [PATCH v2 5/6] percpu: Repurpose __percpu tag as a named address space qualifier

In reply to Dennis Zhou
Quoted 36 lines · Dennis Zhou

Hi Uros,

Quoted 32 lines · Uros Bizjak

The patch introduces per_cpu_qual define and repurposes __percpu tag as a named address space qualifier using the new define.

Arches can now conditionally define __per_cpu_qual as their named address space qualifier for percpu variables.

Cc Arnd Bergmann <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Brian Gerst <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 include/asm-generic/percpu.h   | 15 +++++++++++++++
 include/linux/compiler_types.h |  2 +-
 2 files changed, 16 insertions(+), 1 deletion(-)
include/asm-generic/percpu.h +0 -0
index 50597b975a49..3b93b168faa1 100644
@@ -6,6 +6,21 @@
 #include <linux/threads.h>
 #include <linux/percpu-defs.h>
 
Quoted 15 lines

include/asm-generic/percpu.h line 18

Quoted 10 lines
include/asm-generic/percpu.h +10 -0
+/*
+ * per_cpu_qual is the qualifier for the percpu named address space.
+ *
+ * Most arches use generic named address space for percpu variables but
+ * some arches define percpu variables in different named address space
+ * (on the x86 arch, percpu variable may be declared as being relative
+ * to the %fs or %gs segments using __seg_fs or __seg_gs named address
+ * space qualifier).
+ */
+#ifdef __per_cpu_qual

I read through the series and I think my only nit would be here. Can we name this __percpu_qual? My thoughts are that it keeps it consistent with the old address space identifier and largely most of the core percpu stuff is defined with percpu as the naming scheme.

I based the approach on the definition of per_cpu_offset() a few lines bellow in include/asm-generic/percpu.h:

--q--
#ifndef __per_cpu_offset
extern unsigned long __per_cpu_offset[NR_CPUS];

#define per_cpu_offset(x) (__per_cpu_offset[x])
#endif
--/q--

Sure, we can call this __percpu_qual. So, the definition in arch/x86/include/asm/percpu.h would read as:

# define __percpu_qual      __percpu_seg_override
Quoted 4 lines

include/asm-generic/percpu.h line 22

Quoted 4 lines
include/asm-generic/percpu.h +4 -0
+# define per_cpu_qual __per_cpu_qual
+#else
+# define per_cpu_qual
+#endif

The above part could be recoded as:

#ifndef __percpu_qual

# define __percpu_qual
#endif

while the line below would become:

# define __percpu    __percpu_qual BTF_TYPE_TAG(percpu)
Quoted 16 lines
Quoted 16 lines
include/asm-generic/percpu.h +1 -0
+
 #ifdef CONFIG_SMP
 
 /*
include/linux/compiler_types.h +1 -1
index 981cc3d7e3aa..877fe0c43c5d 100644
@@ -57,7 +57,7 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 #  define __user     BTF_TYPE_TAG(user)
 # endif
 # define __iomem
-# define __percpu    BTF_TYPE_TAG(percpu)
+# define __percpu    per_cpu_qual BTF_TYPE_TAG(percpu)
 # define __rcu               BTF_TYPE_TAG(rcu)
 
Quoted 1 line

include/linux/compiler_types.h static inline void __chk_io_ptr(const volatile void __iomem *ptr) { } · line 63

Quoted 1 line
include/linux/compiler_types.h +0 -0
 # define __chk_user_ptr(x)   (void)0

Let me test these changes a bit, I'll send a v3 in a couple of days.

Thanks,
Uros.

#9 lore raw

PATCH v2 6/6 percpu/x86: Enable strict percpu checks via named AS qualifiers

In reply to Uros Bizjak

This patch declares percpu variables in __seg_gs/__seg_fs named AS and keeps them named AS qualified until they are dereferenced with percpu accessor. This approach enables various compiler check for cross-namespace variable assignments.

Cc Dennis Zhou <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
v2: - Add comment to remove test for __CHECKER__ once sparse learns
about __typeof_unqual__.

---
 arch/x86/include/asm/percpu.h | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
arch/x86/include/asm/percpu.h +17 -3
index 27f668660abe..6be1eafa76ec 100644
@@ -95,9 +95,23 @@
 
 #endif /* CONFIG_SMP */
 
-#define __my_cpu_type(var)	typeof(var) __percpu_seg_override
-#define __my_cpu_ptr(ptr)	(__my_cpu_type(*(ptr))*)(__force uintptr_t)(ptr)
-#define __my_cpu_var(var)	(*__my_cpu_ptr(&(var)))
+/*
+ * XXX: Remove test for __CHECKER__ once
+ * sparse learns about __typeof_unqual__.
+ */
+#if defined(CONFIG_USE_X86_SEG_SUPPORT) && \
+    defined(CONFIG_CC_HAS_TYPEOF_UNQUAL) && !defined(__CHECKER__)
+# define __my_cpu_type(var)	typeof(var)
+# define __my_cpu_ptr(ptr)	(ptr)
+# define __my_cpu_var(var)	(var)
+
+# define __per_cpu_qual		__percpu_seg_override
+#else
+# define __my_cpu_type(var)	typeof(var) __percpu_seg_override
+# define __my_cpu_ptr(ptr)	(__my_cpu_type(*(ptr))*)(__force uintptr_t)(ptr)
+# define __my_cpu_var(var)	(*__my_cpu_ptr(&(var)))
+#endif
+
 #define __percpu_arg(x)		__percpu_prefix "%" #x
 #define __force_percpu_arg(x)	__force_percpu_prefix "%" #x
 
Signature · 1 line
2.42.0