rcu/x86: Use per-cpu rcu preempt count
Subsystems (6)
Lai Jiangshan
reviewercover rcu/x86: Use per-cpu rcu preempt count
From: Lai Jiangshan <[email protected]>
X86 can access percpu data in a single instruction.
Use per-cpu rcu preempt count and make it able to be inlined.
patch 1-7: prepare
patch 8-10: implement PCPU_RCU_PREEMPT_COUNT
Lai Jiangshan (10):
lib: Use rcu_preempt_depth() to replace current->rcu_read_lock_nesting
rcu: Move rcu_preempt_depth_set() to rcupdate.h
rcu: Reorder tree_exp.h after tree_plugin.h
rcu: Add macros set_rcu_preempt_special() and
clear_rcu_preempt_special()
rcu: Make rcu_read_unlock_special() global
rcu: Rename marco __LINUX_RCU_H to __KERNEL_RCU_H sched/core: Add rcu_preempt_switch()
rcu: Implement PCPU_RCU_PREEMPT_COUNT framework
x86/rcu: Add rcu_preempt_count
x86/rcu: Add THUNK rcu_read_unlock_special_thunk
arch/x86/Kconfig | 1 + arch/x86/entry/thunk_64.S | 5 ++ arch/x86/include/asm/current.h | 3 + arch/x86/include/asm/rcu_preempt.h | 109 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/common.c | 7 +- include/linux/rcupdate.h | 36 ++++++++++ kernel/rcu/Kconfig | 8 +++ kernel/rcu/rcu.h | 15 +++- kernel/rcu/tree.c | 2 +- kernel/rcu/tree_exp.h | 2 +- kernel/rcu/tree_plugin.h | 41 +++++++---- kernel/sched/core.c | 2 + lib/locking-selftest.c | 6 +- 13 files changed, 211 insertions(+), 26 deletions(-) create mode 100644 arch/x86/include/asm/rcu_preempt.h
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 1/10 lib: Use rcu_preempt_depth() to replace current->rcu_read_lock_nesting
From: Lai Jiangshan <[email protected]>
Use the general wrapper rcu_preempt_depth() instead of the open code.
Prepare for enabling per-cpu rcu_preempt_count, in which case current->rcu_read_lock_nesting might not be synced with the real depth.
--- lib/locking-selftest.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
lib/locking-selftest.c +3 -3
@@ -1434,7 +1434,7 @@ static void dotest(void (*testcase_fn)(void), int expected, int lockclass_mask)#ifdef CONFIG_SMP int saved_mgd_count = current->migration_disabled;#endif -int saved_rcu_count = current->rcu_read_lock_nesting; +int saved_rcu_count =rcu_preempt_depth ();#endif WARN_ON (irqs_disabled ());
@@ -1476,9 +1476,9 @@ static void dotest(void (*testcase_fn)(void), int expected, int lockclass_mask)migrate_enable ();#endif -while (current->rcu_read_lock_nesting > saved_rcu_count) +while (rcu_preempt_depth () > saved_rcu_count)rcu_read_unlock (); -WARN_ON_ONCE (current->rcu_read_lock_nesting < saved_rcu_count); +WARN_ON_ONCE (rcu_preempt_depth () < saved_rcu_count);#endif #ifdef CONFIG_TRACE_IRQFLAGS
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 2/10 rcu: Move rcu_preempt_depth_set() to rcupdate.h
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Prepare for arch-specific-defined rcu_preempt_depth_set().
No functionality change intended, but it has to be defined as a macro as rcupdate.h is a very low level header included from areas that don't even know about the task struct "current".
--- include/linux/rcupdate.h | 1 + kernel/rcu/tree_plugin.h | 5 ----- 2 files changed, 1 insertion(+), 5 deletions(-)
include/linux/rcupdate.h +1 -0
@@ -78,6 +78,7 @@ void __rcu_read_unlock(void);* types of kernel builds, the rcu_read_lock() nesting depth is unknowable. */ #define rcu_preempt_depth () READ_ONCE(current->rcu_read_lock_nesting) +#define rcu_preempt_depth_set (val) WRITE_ONCE(current->rcu_read_lock_nesting, (val))#else /* #ifdef CONFIG_PREEMPT_RCU */
kernel/rcu/tree_plugin.h +0 -5
@@ -387,11 +387,6 @@ static int rcu_preempt_read_exit(void)return ret; } -static void rcu_preempt_depth_set (int val) -{ -WRITE_ONCE (current->rcu_read_lock_nesting, val); -} -/* * Preemptible RCU implementation for rcu_read_lock(). * Just increment ->rcu_read_lock_nesting, shared state will be updated
Signature · 1 line
2.19.1.6.gb485710b
Re: [PATCH 02/10] rcu: Move rcu_preempt_depth_set() to rcupdate.h
Quoted 11 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
Prepare for arch-specific-defined
rcu_preempt_depth_set().No functionality change intended, but it has to be defined as a macro as
rcupdate.his a very low level header included from areas that don't even know about the task struct "current".
Sorry I did not follow changelog. If some rcupdate.h includers do not know about task_struct, how does adding a macro that uses current help?
Thanks,
- Joel
Quoted 41 lines
Cc "Paul E. McKenney" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Lai Jiangshan <[email protected]>---
include/linux/rcupdate.h| 1 +kernel/rcu/tree_plugin.h| 5 -----2 files changed, 1 insertion(+), 5 deletions(-)include/linux/rcupdate.h +1
-0@@ -78,6 +78,7 @@ void __rcu_read_unlock(void);* types of kernel builds, the rcu_read_lock() nesting depth is unknowable. */ #define rcu_preempt_depth () READ_ONCE(current->rcu_read_lock_nesting) +#define rcu_preempt_depth_set (val) WRITE_ONCE(current->rcu_read_lock_nesting, (val))#else /* #ifdef CONFIG_PREEMPT_RCU */ kernel/rcu/tree_plugin.h +0
-6@@ -387,11 +387,6 @@ static int rcu_preempt_read_exit(void)return ret; } -static void rcu_preempt_depth_set (int val) -{ -WRITE_ONCE (current->rcu_read_lock_nesting, val); -} -/* * Preemptible RCU implementation for rcu_read_lock(). * Just increment ->rcu_read_lock_nesting, shared state will be updated -- 2.19.1.6.gb485710b
Lai Jiangshan
reviewerRe: [PATCH 02/10] rcu: Move rcu_preempt_depth_set() to rcupdate.h
Quoted 7 lines · Joel Fernandes
Quoted 7 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
Quoted 9 lines
Quoted 5 lines
Prepare for arch-specific-defined
rcu_preempt_depth_set().No functionality change intended, but it has to be defined as a macro as
rcupdate.his a very low level header included from areas that don't even know about the task struct "current".Sorry I did not follow changelog. If some
rcupdate.hincluders do not know abouttask_struct, how does adding a macro that uses current help?
Hello
This is how macro works and it expands blindly based on tokens on the usage-sites.
And rcu_preempt_depth() & rcu_preempt_depth_set() are not universally used wrappers, the user can simply also include linux/sched.h to make they work.
Thanks
Lai
Re: [PATCH 02/10] rcu: Move rcu_preempt_depth_set() to rcupdate.h
Quoted 22 lines · Lai Jiangshan
Quoted 7 lines · Joel Fernandes
Quoted 7 lines
Quoted 4 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
Quoted 9 lines
Quoted 5 lines
Prepare for arch-specific-defined
rcu_preempt_depth_set().No functionality change intended, but it has to be defined as a macro as
rcupdate.his a very low level header included from areas that don't even know about the task struct "current".Sorry I did not follow changelog. If some
rcupdate.hincluders do not know abouttask_struct, how does adding a macro that uses current help?
Quoted 8 lines
Hello
This is how macro works and it expands blindly based on tokens on the usage-sites.
And
rcu_preempt_depth()&rcu_preempt_depth_set()are not universally used wrappers, the user can simply also includelinux/sched.hto make they work.
Oh I see, so by hiding it in a macro, the code remains unexpanded. That makes sense..
Thanks.
Quoted 3 lines
Thanks
Lai
Re: [PATCH 02/10] rcu: Move rcu_preempt_depth_set() to rcupdate.h
In reply to Lai Jiangshan2024年4月1日 00:16,Lai Jiangshan <[email protected]> wrote:
Quoted 15 lines · Joel Fernandes
Quoted 13 lines
Quoted 12 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
Prepare for arch-specific-defined
rcu_preempt_depth_set().No functionality change intended, but it has to be defined as a macro as
rcupdate.his a very low level header included from areas that don't even know about the task struct "current".
Quoted 8 lines
Quoted 3 lines
Sorry I did not follow changelog. If some
rcupdate.hincluders do not know abouttask_struct, how does adding a macro that uses current help?Hello
This is how macro works and it expands blindly based on tokens on the usage-sites.
But ‘current’ still needs to be expanded at last, it seems to me that it only affects the including order of the header files?
Or what am I missing?
Quoted 8 lines
And
rcu_preempt_depth()&rcu_preempt_depth_set()are not universally used wrappers, the user can simply also includelinux/sched.hto make they work.Thanks
Lai
Re: [PATCH 02/10] rcu: Move rcu_preempt_depth_set() to rcupdate.h
2024年4月1日 19:40,Alan Huang <[email protected]> wrote:
Quoted 23 lines
2024年4月1日 00:16,Lai Jiangshan <[email protected]> wrote:
Quoted 21 lines · Joel Fernandes
Quoted 7 lines
Quoted 7 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
Quoted 9 lines
Quoted 5 lines
Prepare for arch-specific-defined
rcu_preempt_depth_set().No functionality change intended, but it has to be defined as a macro as
rcupdate.his a very low level header included from areas that don't even know about the task struct "current".Sorry I did not follow changelog. If some
rcupdate.hincluders do not know abouttask_struct, how does adding a macro that uses current help?Hello
Quoted 7 lines
Quoted 2 lines
This is how macro works and it expands blindly based on tokens on the usage-sites.
But ‘current’ still needs to be expanded at last, it seems to me that it only affects the including order of the header files?
Or what am I missing?
Get the missing part: if the user don’t need to use rcu_preempt_depth() and rcu_preempt_depth_set() but other parts of rcupdate.h, then the two has to be defined as a macro to avoid including linux/sched.h.
Sorry for the bother.
Quoted 8 lines
Quoted 8 lines
And
rcu_preempt_depth()&rcu_preempt_depth_set()are not universally used wrappers, the user can simply also includelinux/sched.hto make they work.Thanks
Lai
Lai Jiangshan
reviewerPATCH 3/10 rcu: Reorder tree_exp.h after tree_plugin.h
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Enable tree_exp.h using some rcu_preempt macros introduced in the next patch. The new macros touch core rcu_preempt fields and are better to be implemented in tree_plugin.h.
--- kernel/rcu/tree.c | 2 +- kernel/rcu/tree_plugin.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +1 -1
@@ -5245,6 +5245,6 @@ void __init rcu_init(void) }#include "tree_stall.h" -#include "tree_exp.h" #include "tree_nocb.h" #include "tree_plugin.h" +#include "tree_exp.h"
kernel/rcu/tree_plugin.h +1 -0
@@ -103,6 +103,7 @@ static void __init rcu_bootup_announce_oddness(void)#ifdef CONFIG_PREEMPT_RCU static void rcu_report_exp_rnp (struct rcu_node *rnp,bool wake); +static bool sync_rcu_exp_done (struct rcu_node *rnp);static void rcu_read_unlock_special (struct task_struct *t);/*
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 4/10 rcu: Add macros set_rcu_preempt_special() and clear_rcu_preempt_special()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Add wrappers accessing to t->rcu_read_unlock_special.b so that the wrappers can be adapted for synching the value with the later-introduced per-cpu rcu_preempt_count.
--- kernel/rcu/tree_exp.h | 2 +- kernel/rcu/tree_plugin.h | 20 +++++++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-)
kernel/rcu/tree_exp.h +1 -1
@@ -725,7 +725,7 @@ static void rcu_exp_handler(void *unused)raw_spin_lock_irqsave_rcu_node (rnp, flags);if (rnp->expmask & rdp->grpmask) {WRITE_ONCE (rdp->cpu_no_qs.b.exp, true); - t->rcu_read_unlock_special.b.exp_hint = true; +set_rcu_preempt_special (exp_hint); }raw_spin_unlock_irqrestore_rcu_node (rnp, flags);return ;
kernel/rcu/tree_plugin.h +13 -7
@@ -106,6 +106,14 @@ static void rcu_report_exp_rnp(struct rcu_node *rnp, bool wake);static bool sync_rcu_exp_done (struct rcu_node *rnp);static void rcu_read_unlock_special (struct task_struct *t); +#define set_rcu_preempt_special (reason) do { \ + WRITE_ONCE(current->rcu_read_unlock_special.b.reason, true); \ + } while (0) + +#define clear_rcu_preempt_special (reason) do { \ + WRITE_ONCE(current->rcu_read_unlock_special.b.reason, false); \ + } while (0) +/* * Tell them what RCU they are running. */
@@ -293,7 +301,7 @@ static void rcu_qs(void) TPS("cpuqs"));__this_cpu_write (rcu_data.cpu_no_qs.b.norm, false);barrier ();/* Coordinate with rcu_flavor_sched_clock_irq(). */ -WRITE_ONCE (current->rcu_read_unlock_special.b.need_qs, false); +clear_rcu_preempt_special (need_qs); } }
@@ -325,7 +333,7 @@ void rcu_note_context_switch(bool preempt)/* Possibly blocking in an RCU read-side critical section. */ rnp = rdp->mynode;raw_spin_lock_rcu_node (rnp); - t->rcu_read_unlock_special.b.blocked = true; +set_rcu_preempt_special (blocked); t->rcu_blocked_node = rnp;/*
@@ -399,7 +407,7 @@ void __rcu_read_lock(void)if (IS_ENABLED (CONFIG_PROVE_LOCKING ))WARN_ON_ONCE (rcu_preempt_depth () >RCU_NEST_PMAX );if (IS_ENABLED (CONFIG_RCU_STRICT_GRACE_PERIOD ) && rcu_state.gp_kthread) -WRITE_ONCE (current->rcu_read_unlock_special.b.need_qs, true); +set_rcu_preempt_special (need_qs);barrier ();/* critical section after entry code. */ }EXPORT_SYMBOL_GPL (__rcu_read_lock);
@@ -738,7 +746,7 @@ static void rcu_flavor_sched_clock_irq(int user) __this_cpu_read(rcu_data.cpu_no_qs.b.norm) && !t->rcu_read_unlock_special.b.need_qs && time_after(jiffies, rcu_state.gp_start + HZ)) - t->rcu_read_unlock_special.b.need_qs = true; +set_rcu_preempt_special (need_qs); }/*
@@ -751,12 +759,10 @@ static void rcu_flavor_sched_clock_irq(int user)*/ void exit_rcu (void ) { -struct task_struct *t = current; -if (unlikely (!list_empty (¤t->rcu_node_entry))) {rcu_preempt_depth_set (1 );barrier (); -WRITE_ONCE (t->rcu_read_unlock_special.b.blocked, true); +set_rcu_preempt_special (blocked); }else if (unlikely (rcu_preempt_depth ())) {rcu_preempt_depth_set (1 ); }else {
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 5/10 rcu: Make rcu_read_unlock_special() global
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Make it global so that it can be used in the future inlined rcu_read_unlock().
Make it exported so that the inlined rcu_read_unlock() can be used in modules.
Make it taking a void-argument so that the caller can reduce instructions obtaining the "current" task value and it can be used in THUNK later.
--- include/linux/rcupdate.h | 2 ++ kernel/rcu/tree_plugin.h | 7 ++++--- 2 files changed, 6 insertions(+), 3 deletions(-)
include/linux/rcupdate.h +2 -0
@@ -68,6 +68,8 @@ static inline bool same_state_synchronize_rcu(unsigned long oldstate1, unsigned#ifdef CONFIG_PREEMPT_RCU +void rcu_read_unlock_special (void ); +void __rcu_read_lock (void );void __rcu_read_unlock (void );
kernel/rcu/tree_plugin.h +4 -3
@@ -104,7 +104,6 @@ static void __init rcu_bootup_announce_oddness(void)static void rcu_report_exp_rnp (struct rcu_node *rnp,bool wake);static bool sync_rcu_exp_done (struct rcu_node *rnp); -static void rcu_read_unlock_special (struct task_struct *t);#define set_rcu_preempt_special (reason) do { \ WRITE_ONCE(current->rcu_read_unlock_special.b.reason, true); \
@@ -427,7 +426,7 @@ void __rcu_read_unlock(void)if (rcu_preempt_read_exit () ==0 ) {barrier ();// critical-section exit before .s check. if (unlikely (READ_ONCE (t->rcu_read_unlock_special.s))) -rcu_read_unlock_special (t); +rcu_read_unlock_special (); }if (IS_ENABLED (CONFIG_PROVE_LOCKING )) {int rrln =rcu_preempt_depth ();
@@ -627,8 +626,9 @@ static void rcu_preempt_deferred_qs_handler(struct irq_work *iwp)* notify RCU core processing or task having blocked during the RCU * read-side critical section. */ -static void rcu_read_unlock_special (struct task_struct *t) +void rcu_read_unlock_special (void ) { +struct task_struct *t = current;unsigned long flags;bool irqs_were_disabled; bool preempt_bh_were_disabled =
@@ -684,6 +684,7 @@ static void rcu_read_unlock_special(struct task_struct *t) }rcu_preempt_deferred_qs_irqrestore (t, flags); } +EXPORT_SYMBOL_GPL (rcu_read_unlock_special);/* * Check that the list of blocked tasks for the newly completed grace
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 6/10 rcu: Rename marco __LINUX_RCU_H to __KERNEL_RCU_H
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
to reflect the path of kernel/rcu/rcu.h.
--- kernel/rcu/rcu.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/rcu/rcu.h +3 -3
@@ -7,8 +7,8 @@* Author: Paul E. McKenney <[email protected]> */ -#ifndef __LINUX_RCU_H -#define __LINUX_RCU_H +#ifndef __KERNEL_RCU_H +#define __KERNEL_RCU_H#include <linux/slab.h> #include <trace/events/rcu.h>
@@ -668,4 +668,4 @@ int rcu_stall_notifier_call_chain(unsigned long val, void *v);static inline int rcu_stall_notifier_call_chain (unsigned long val,void *v) {return NOTIFY_DONE ; }#endif // #else // #if defined(CONFIG_RCU_STALL_COMMON) && defined(CONFIG_RCU_CPU_STALL_NOTIFIER) -#endif /* __LINUX_RCU_H */ +#endif /* __KERNEL_RCU_H */
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 7/10 sched/core: Add rcu_preempt_switch()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
When the per-cpu rcu_preempt_count is used, it has to be switched too on context-switching. And the instructions to switch the per-cpu rcu_preempt_count are meager, so it is inlined to avoid the overhead in the scheduler.
--- kernel/rcu/rcu.h | 5 +++++ kernel/sched/core.c | 2 ++ 2 files changed, 7 insertions(+)
kernel/rcu/rcu.h +5 -0
@@ -668,4 +668,9 @@ int rcu_stall_notifier_call_chain(unsigned long val, void *v);static inline int rcu_stall_notifier_call_chain (unsigned long val,void *v) {return NOTIFY_DONE ; }#endif // #else // #if defined(CONFIG_RCU_STALL_COMMON) && defined(CONFIG_RCU_CPU_STALL_NOTIFIER) +static inline void +rcu_preempt_switch (struct task_struct *prev,struct task_struct *next) +{ +} +#endif /* __KERNEL_RCU_H */
kernel/sched/core.c +2 -0
@@ -95,6 +95,7 @@#include "../workqueue_internal.h" #include "../../io_uring/io-wq.h" #include "../smpboot.h" +#include "../rcu/rcu.h" EXPORT_TRACEPOINT_SYMBOL_GPL (ipi_send_cpu);EXPORT_TRACEPOINT_SYMBOL_GPL (ipi_send_cpumask);
@@ -6727,6 +6728,7 @@ static void __sched notrace __schedule(unsigned int sched_mode)*/ ++*switch_count; +rcu_preempt_switch (prev, next);migrate_disable_switch (rq, prev);psi_sched_switch (prev, next, !task_on_rq_queued (prev));
Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerPATCH 8/10 rcu: Implement PCPU_RCU_PREEMPT_COUNT framework
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
When the arch code provides HAVE_PCPU_RCU_PREEMPT_COUNT and the corresponding functions, rcu_preempt core uses the functions to implement rcu_read_[un]lock, rcu_preempt_depth(), special bits, switching and so on.
--- include/linux/rcupdate.h | 33 +++++++++++++++++++++++++++++++++ kernel/rcu/Kconfig | 8 ++++++++ kernel/rcu/rcu.h | 4 ++++ kernel/rcu/tree_plugin.h | 8 ++++++++ 4 files changed, 53 insertions(+)
include/linux/rcupdate.h +33 -0
@@ -70,6 +70,8 @@ static inline bool same_state_synchronize_rcu(unsigned long oldstate1, unsignedvoid rcu_read_unlock_special (void ); +#ifndef CONFIG_PCPU_RCU_PREEMPT_COUNT +void __rcu_read_lock (void );void __rcu_read_unlock (void );
@@ -81,6 +83,37 @@ void __rcu_read_unlock(void);*/ #define rcu_preempt_depth () READ_ONCE(current->rcu_read_lock_nesting)#define rcu_preempt_depth_set (val) WRITE_ONCE(current->rcu_read_lock_nesting, (val)) +#define pcpu_rcu_preempt_special_set () do { } while (0) +#define pcpu_rcu_preempt_special_clear () do { } while (0) + +#else /* #ifndef CONFIG_PCPU_RCU_PREEMPT_COUNT */ + +#include <asm/rcu_preempt.h> + +static __always_inline void __rcu_read_lock(void) +{ +pcpu_rcu_preempt_count_add (1 ); +barrier (); +} + +static __always_inline void __rcu_read_unlock(void) +{ +barrier (); +if (unlikely (pcpu_rcu_preempt_count_dec_and_test ())) +pcpu_rcu_read_unlock_special (); +} + +static inline int rcu_preempt_depth (void ) +{ +return pcpu_rcu_preempt_count (); +} + +static inline void rcu_preempt_depth_set (int val) +{ +pcpu_rcu_preempt_count_set (val); +} + +#endif /* #else #ifndef CONFIG_PCPU_RCU_PREEMPT_COUNT */ #else /* #ifdef CONFIG_PREEMPT_RCU */
kernel/rcu/Kconfig +8 -0
@@ -345,4 +345,12 @@ config RCU_DOUBLE_CHECK_CB_TIME Say Y here if you need tighter callback-limit enforcement. Say N here if you are unsure. +config HAVE_PCPU_RCU_PREEMPT_COUNT + bool + +config PCPU_RCU_PREEMPT_COUNT + def_bool y + depends on PREEMPT_RCU && HAVE_PCPU_RCU_PREEMPT_COUNT + depends on !PROVE_LOCKING && !RCU_STRICT_GRACE_PERIOD + endmenu # "RCU Subsystem"
kernel/rcu/rcu.h +4 -0
@@ -671,6 +671,10 @@ static inline int rcu_stall_notifier_call_chain(unsigned long val, void *v) { restatic inline void rcu_preempt_switch (struct task_struct *prev,struct task_struct *next) { +#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT + prev->rcu_read_lock_nesting =rcu_preempt_depth (); +pcpu_rcu_preempt_switch (next->rcu_read_lock_nesting, next->rcu_read_unlock_special.s); +#endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT }#endif /* __KERNEL_RCU_H */
kernel/rcu/tree_plugin.h +8 -0
@@ -107,10 +107,13 @@ static bool sync_rcu_exp_done(struct rcu_node *rnp);#define set_rcu_preempt_special (reason) do { \ WRITE_ONCE(current->rcu_read_unlock_special.b.reason, true); \ + pcpu_rcu_preempt_special_set(); \ } while (0)#define clear_rcu_preempt_special (reason) do { \ WRITE_ONCE(current->rcu_read_unlock_special.b.reason, false); \ + if (!current->rcu_read_unlock_special.s) \ + pcpu_rcu_preempt_special_clear(); \ } while (0)/*
@@ -379,6 +382,8 @@ static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp)return READ_ONCE (rnp->gp_tasks) !=NULL ; } +#ifndef CONFIG_PCPU_RCU_PREEMPT_COUNT +/* limit value for ->rcu_read_lock_nesting. */ #define RCU_NEST_PMAX (INT_MAX / 2)
@@ -436,6 +441,8 @@ void __rcu_read_unlock(void) }EXPORT_SYMBOL_GPL (__rcu_read_unlock); +#endif /* #ifndef CONFIG_PCPU_RCU_PREEMPT_COUNT */ +/* * Advance a ->blkd_tasks-list pointer to the next entry, instead * returning NULL if at the end of the list.
@@ -489,6 +496,7 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct *t, unsigned long flags)return ; } t->rcu_read_unlock_special.s =0 ; +pcpu_rcu_preempt_special_clear ();if (special.b.need_qs) {if (IS_ENABLED (CONFIG_RCU_STRICT_GRACE_PERIOD )) { rdp->cpu_no_qs.b.norm = false;
Signature · 1 line
2.19.1.6.gb485710b
Frederic Weisbecker
maintainerRe: [PATCH 08/10] rcu: Implement PCPU_RCU_PREEMPT_COUNT framework
Quoted 6 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
When the arch code provides
HAVE_PCPU_RCU_PREEMPT_COUNTand the corresponding functions,rcu_preemptcore uses the functions to implement rcu_read_[un]lock,rcu_preempt_depth(), special bits, switching and so on.
The changelogs don't explain the reason for all of that. I'm guessing from the cover letter that the purpose is to save some instructions on calls to rcu_read_unlock() due to inline calls and per CPU optimized instructions but it's hard to guess from the actual individual patches, which are meants to be the actual git records.
Thanks.
Lai Jiangshan
reviewerPATCH 9/10 x86/rcu: Add rcu_preempt_count
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Implement PCPU_RCU_PREEMPT_COUNT for x86.
Mainly copied from asm/preempt.h
Make rcu_read_[un]lock() inlined for rcu-preempt. Make rcu_read_lock() only one instruction.
Make rcu_read_unlock() only two instructions in the fast path.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/current.h | 3 + arch/x86/include/asm/rcu_preempt.h | 107 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/common.c | 7 +- 4 files changed, 115 insertions(+), 3 deletions(-) create mode 100644 arch/x86/include/asm/rcu_preempt.h
arch/x86/Kconfig +1 -0
@@ -257,6 +257,7 @@ config X86 select HAVE_OBJTOOL if X86_64 select HAVE_OPTPROBES select HAVE_PAGE_SIZE_4KB + select HAVE_PCPU_RCU_PREEMPT_COUNT select HAVE_PCSPKR_PLATFORM select HAVE_PERF_EVENTS select HAVE_PERF_EVENTS_NMI
arch/x86/include/asm/current.h +3 -0
@@ -24,6 +24,9 @@ struct pcpu_hot {unsigned long top_of_stack;void *hardirq_stack_ptr;u16 softirq_pending; +#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT +int rcu_preempt_count; +#endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT #ifdef CONFIG_X86_64 bool hardirq_stack_inuse;#else
arch/x86/include/asm/rcu_preempt.h new file +107 -0
107 lines not on this page · raw message
arch/x86/kernel/cpu/common.c +4 -3
@@ -1992,9 +1992,10 @@ static __init int setup_clearcpuid(char *arg)__setup ("clearcpuid=" , setup_clearcpuid); DEFINE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot) = { - .current_task = &init_task, - .preempt_count =INIT_PREEMPT_COUNT , - .top_of_stack =TOP_OF_INIT_STACK , + .current_task = &init_task, + .preempt_count =INIT_PREEMPT_COUNT , + .top_of_stack =TOP_OF_INIT_STACK , + .rcu_preempt_count =RCU_PREEMPT_INIT , };EXPORT_PER_CPU_SYMBOL (pcpu_hot);EXPORT_PER_CPU_SYMBOL (const_pcpu_hot);
Signature · 1 line
2.19.1.6.gb485710b
kernel test robot
build botRe: [PATCH 09/10] x86/rcu: Add rcu_preempt_count
kernel test robot noticed the following build errors:
Hi Lai,
kernel test robot noticed the following build errors:
[auto build test ERROR on paulmck-rcu/dev]
[also build test ERROR on tip/locking/core tip/sched/core tip/x86/asm tip/master linus/master v6.9-rc1 next-20240328] [cannot apply to tip/x86/core tip/auto-latest]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Lai-Jiangshan/lib-Use-rcu_preempt_depth-to-replace-current-rcu_read_lock_nesting/20240328-155513 base: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev patch link: https://lore.kernel.org/r/20240328075318.83039-10-jiangshanlai%40gmail.com patch subject: [PATCH 09/10] x86/rcu: Add rcu_preempt_count config: x86_64-randconfig-161-20240328 (https://download.01.org/0day-ci/archive/20240329/[email protected]/config) compiler: gcc-10 (Ubuntu 10.5.0-1ubuntu1) 10.5.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240329/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All error/warnings (new ones prefixed by >>):
>> arch/x86/kernel/cpu/common.c:1998:3: error: 'struct pcpu_hot' has no member named 'rcu_preempt_count'; did you mean 'preempt_count'?
1998 | .rcu_preempt_count = RCU_PREEMPT_INIT,
| ^~~~~~~~~~~~~~~~~
| preempt_count
>> arch/x86/kernel/cpu/common.c:1998:23: error: 'RCU_PREEMPT_INIT' undeclared here (not in a function); did you mean 'RCUREF_INIT'?
1998 | .rcu_preempt_count = RCU_PREEMPT_INIT,
| ^~~~~~~~~~~~~~~~
| RCUREF_INIT
>> arch/x86/kernel/cpu/common.c:1998:23: warning: excess elements in struct initializer arch/x86/kernel/cpu/common.c:1998:23: note: (near initialization for 'pcpu_hot')
vim +1998 arch/x86/kernel/cpu/common.c
1993
1994 DEFINE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot) = {
1995 .current_task = &init_task,
1996 .preempt_count = INIT_PREEMPT_COUNT,
1997 .top_of_stack = TOP_OF_INIT_STACK,
> 1998 .rcu_preempt_count = RCU_PREEMPT_INIT,
1999 };
2000 EXPORT_PER_CPU_SYMBOL(pcpu_hot);
2001 EXPORT_PER_CPU_SYMBOL(const_pcpu_hot);
2002
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 09/10] x86/rcu: Add rcu_preempt_count
In reply to Lai Jiangshankernel test robot noticed the following build errors:
Hi Lai,
kernel test robot noticed the following build errors:
[auto build test ERROR on paulmck-rcu/dev]
[also build test ERROR on tip/locking/core tip/sched/core tip/x86/asm tip/master linus/master v6.9-rc1 next-20240328] [cannot apply to tip/x86/core tip/auto-latest]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Lai-Jiangshan/lib-Use-rcu_preempt_depth-to-replace-current-rcu_read_lock_nesting/20240328-155513 base: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev patch link: https://lore.kernel.org/r/20240328075318.83039-10-jiangshanlai%40gmail.com patch subject: [PATCH 09/10] x86/rcu: Add rcu_preempt_count config: i386-randconfig-141-20240328 (https://download.01.org/0day-ci/archive/20240329/[email protected]/config) compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240329/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> arch/x86/kernel/cpu/common.c:1998:23: error: use of undeclared identifier 'RCU_PREEMPT_INIT'
1998 | .rcu_preempt_count = RCU_PREEMPT_INIT,
| ^
1 error generated.
Kconfig warnings: (for reference only)
WARNING: unmet direct dependencies detected for DRM_I915_DEBUG_GEM
Depends on [n]: HAS_IOMEM [=y] && DRM_I915 [=y] && EXPERT [=y] && DRM_I915_WERROR [=n] Selected by [y]:
- DRM_I915_DEBUG [=y] && HAS_IOMEM [=y] && DRM_I915 [=y] && EXPERT [=y] && !COMPILE_TEST [=n]
vim +/RCU_PREEMPT_INIT +1998 arch/x86/kernel/cpu/common.c
1993
1994 DEFINE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot) = {
1995 .current_task = &init_task,
1996 .preempt_count = INIT_PREEMPT_COUNT,
1997 .top_of_stack = TOP_OF_INIT_STACK,
> 1998 .rcu_preempt_count = RCU_PREEMPT_INIT,
1999 };
2000 EXPORT_PER_CPU_SYMBOL(pcpu_hot);
2001 EXPORT_PER_CPU_SYMBOL(const_pcpu_hot);
2002
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Frederic Weisbecker
maintainerRe: [PATCH 09/10] x86/rcu: Add rcu_preempt_count
In reply to Lai JiangshanQuoted 54 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
Implement
PCPU_RCU_PREEMPT_COUNTfor x86.
Mainly copied fromasm/preempt.hMake rcu_read_[un]lock() inlined for rcu-preempt. Make
rcu_read_lock()only one instruction.
Makercu_read_unlock()only two instructions in the fast path.Cc "Paul E. McKenney" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/Kconfig | 1 + arch/x86/include/asm/current.h | 3 + arch/x86/include/asm/rcu_preempt.h | 107 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/common.c | 7 +- 4 files changed, 115 insertions(+), 3 deletions(-) create mode 100644 arch/x86/include/asm/rcu_preempt.harch/x86/Kconfig +1
-0@@ -257,6 +257,7 @@ config X86 select HAVE_OBJTOOL if X86_64 select HAVE_OPTPROBES select HAVE_PAGE_SIZE_4KB + select HAVE_PCPU_RCU_PREEMPT_COUNT select HAVE_PCSPKR_PLATFORM select HAVE_PERF_EVENTS select HAVE_PERF_EVENTS_NMIarch/x86/include/asm/current.h +3
-0@@ -24,6 +24,9 @@ struct pcpu_hot {unsigned long top_of_stack;void *hardirq_stack_ptr;u16 softirq_pending; +#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT +int rcu_preempt_count; +#endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT #ifdef CONFIG_X86_64 bool hardirq_stack_inuse;#else arch/x86/include/asm/rcu_preempt.h new file +1
-0@@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: GPL-2.0 */
arch/x86/include/asm/rcu_preempt.h line 11
Quoted 10 lines
arch/x86/include/asm/rcu_preempt.h new file +10
-0+#ifndef __ASM_RCU_PREEMPT_H +#define __ASM_RCU_PREEMPT_H + +#include <asm/rmwcc.h> +#include <asm/percpu.h> +#include <asm/current.h> + +#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT + +/* We use the MSB mostly because its available */
I think you can safely remove the "We " from all the comments :-)
arch/x86/include/asm/rcu_preempt.h line 12
Quoted 1 line
arch/x86/include/asm/rcu_preempt.h new file +1
-0+#define RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED 0x80000000
How about RCU_PREEMPT_UNLOCK_FASTPATH ?
arch/x86/include/asm/rcu_preempt.h line 19
Quoted 7 lines
arch/x86/include/asm/rcu_preempt.h new file +7
-0+ +/* +* We use the RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED bit as an inverted +* current->rcu_read_unlock_special.s such that a decrement hitting 0 +* means we can and should call rcu_read_unlock_special(). +*/ +#define RCU_PREEMPT_INIT (0 + RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED)
Or simply:
#define RCU_PREEMPT_INIT RCU_PREEMPT_UNLOCK_FASTPATH
Or you can even remove RCU_PREEMPT_INIT and use RCU_PREEMPT_UNLOCK_FASTPATH directly.
Quoted 4 lines
arch/x86/include/asm/rcu_preempt.h new file +4
-0+/* +* Because we keep RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED set when we do +* _not_ need to handle unlock-special for a fast-path decrement. +*/
arch/x86/include/asm/rcu_preempt.h line 103
Quoted 10 lines
arch/x86/include/asm/rcu_preempt.h new file +10
-0+static __always_inline bool pcpu_rcu_preempt_count_dec_and_test(void) +{ +return GEN_UNARY_RMWcc ("decl" ,__my_cpu_var (pcpu_hot.rcu_preempt_count), e, + __percpu_arg([var])); +} + +#define pcpu_rcu_read_unlock_special () \ +do { \ + rcu_read_unlock_special(); \ +} while (0)
Why not just call that directly?
Quoted 9 lines
arch/x86/include/asm/rcu_preempt.h new file +4
-0+ +#endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT + +#endif /* __ASM_RCU_PREEMPT_H */ arch/x86/kernel/cpu/common.c +0
-0
arch/x86/kernel/cpu/common.c static __init int setup_clearcpuid(char *arg) · line 1998
Quoted 10 lines
arch/x86/kernel/cpu/common.c +4
-3@@ -1992,9 +1992,10 @@ static __init int setup_clearcpuid(char *arg)__setup ("clearcpuid=" , setup_clearcpuid); DEFINE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot) = { - .current_task = &init_task, - .preempt_count =INIT_PREEMPT_COUNT , - .top_of_stack =TOP_OF_INIT_STACK , + .current_task = &init_task, + .preempt_count =INIT_PREEMPT_COUNT , + .top_of_stack =TOP_OF_INIT_STACK , + .rcu_preempt_count =RCU_PREEMPT_INIT ,
#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT ?
Thanks.
Quoted 6 lines
arch/x86/kernel/cpu/common.c +0
-0};EXPORT_PER_CPU_SYMBOL (pcpu_hot);EXPORT_PER_CPU_SYMBOL (const_pcpu_hot);Signature · 1 line
2.19.1.6.gb485710b
Lai Jiangshan
reviewerRe: [PATCH 09/10] x86/rcu: Add rcu_preempt_count
Hello, Frederic
Thanks for reviewing.
Quoted 4 lines · Frederic Weisbecker
Review
Quoted 2 lines
(quoted diff) +2
-0+ +/* We use the MSB mostly because its available */I think you can safely remove the "We " from all the comments :-)
The file is mainly copied from arch/x86/include/asm/preempt.h. I will rephrase sentences in later iterations.
Quoted 4 lines
Quoted 2 lines
+#define
RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED0x80000000How about
RCU_PREEMPT_UNLOCK_FASTPATH?
I'm not good at naming. But the MSB really means exactly the opposite of current->rcu_read_unlock_special and I think "UNLOCK_SPECIAL_INVERTED" fits the meaning.
Quoted 11 lines
arch/x86/include/asm/rcu_preempt.h line 19
Quoted 8 lines
arch/x86/include/asm/rcu_preempt.h new file +7
-0+ +/* +* We use the RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED bit as an inverted +* current->rcu_read_unlock_special.s such that a decrement hitting 0 +* means we can and should call rcu_read_unlock_special(). +*/ +#define RCU_PREEMPT_INIT (0 + RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED)Or simply:
Quoted 3 lines
#define
RCU_PREEMPT_INITRCU_PREEMPT_UNLOCK_FASTPATHOr you can even remove
RCU_PREEMPT_INITand useRCU_PREEMPT_UNLOCK_FASTPATHdirectly.
"0" means the initial rcu_preempt_count is 0 for the initial task.
Quoted 22 lines
Quoted 9 lines
arch/x86/include/asm/rcu_preempt.h new file +4
-0+ +#endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT + +#endif /* __ASM_RCU_PREEMPT_H */ arch/x86/kernel/cpu/common.c +0
-0arch/x86/kernel/cpu/common.c static __init int setup_clearcpuid(char *arg) · line 1998
Quoted 10 lines
arch/x86/kernel/cpu/common.c +4
-3@@ -1992,9 +1992,10 @@ static __init int setup_clearcpuid(char *arg)__setup ("clearcpuid=" , setup_clearcpuid); DEFINE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot) = { - .current_task = &init_task, - .preempt_count =INIT_PREEMPT_COUNT , - .top_of_stack =TOP_OF_INIT_STACK , + .current_task = &init_task, + .preempt_count =INIT_PREEMPT_COUNT , + .top_of_stack =TOP_OF_INIT_STACK , + .rcu_preempt_count =RCU_PREEMPT_INIT ,#ifdef
CONFIG_PCPU_RCU_PREEMPT_COUNT?
Quoted 1 line
Thanks.
Fixed in V2:
https://lore.kernel.org/lkml/[email protected]/
Thanks
Lai
Quoted 8 lines
Quoted 8 lines
arch/x86/kernel/cpu/common.c +0
-1};EXPORT_PER_CPU_SYMBOL (pcpu_hot);EXPORT_PER_CPU_SYMBOL (const_pcpu_hot); --2.19.1.6.gb485710b
Frederic Weisbecker
maintainerRe: [PATCH 09/10] x86/rcu: Add rcu_preempt_count
Quoted 15 lines · Lai Jiangshan
Hello, Frederic
Thanks for reviewing.
Quoted 4 lines · Frederic Weisbecker
Review
Quoted 2 lines
(quoted diff) +2
-0+ +/* We use the MSB mostly because its available */I think you can safely remove the "We " from all the comments :-)
The file is mainly copied from
arch/x86/include/asm/preempt.h. I will rephrase sentences in later iterations.Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
+#define
RCU_PREEMPT_UNLOCK_SPECIAL_INVERTED0x80000000How about
RCU_PREEMPT_UNLOCK_FASTPATH?I'm not good at naming. But the MSB really means exactly the opposite of
current->rcu_read_unlock_specialand I think "UNLOCK_SPECIAL_INVERTED" fits the meaning.
Right but I tend to think a constant should tell what something is, not what something is not.
FWIW, p->rcu_read_unlock_special could even be renamed to p->rcu_read_unlock_slowpath
Thanks.
Lai Jiangshan
reviewerPATCH 10/10 x86/rcu: Add THUNK rcu_read_unlock_special_thunk
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Add rcu_read_unlock_special_thunk(), so that the inlined rcu_read_unlock() doesn't need any code to save the caller-saved registers.
Make rcu_read_unlock() only two instructions in the slow path at the caller site.
--- arch/x86/entry/thunk_64.S | 5 +++++ arch/x86/include/asm/rcu_preempt.h | 4 +++- 2 files changed, 8 insertions(+), 1 deletion(-)
arch/x86/entry/thunk_64.S +5 -0
@@ -13,3 +13,8 @@ THUNK preempt_schedule_thunk, preempt_schedule THUNK preempt_schedule_notrace_thunk, preempt_schedule_notrace EXPORT_SYMBOL(preempt_schedule_thunk) EXPORT_SYMBOL(preempt_schedule_notrace_thunk) + +#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT +THUNK rcu_read_unlock_special_thunk, rcu_read_unlock_special +EXPORT_SYMBOL_GPL(rcu_read_unlock_special_thunk) +#endif /* #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT */
arch/x86/include/asm/rcu_preempt.h +3 -1
@@ -97,9 +97,11 @@ static __always_inline bool pcpu_rcu_preempt_count_dec_and_test(void) __percpu_arg([var])); } +extern asmlinkage void rcu_read_unlock_special_thunk(void); +#define pcpu_rcu_read_unlock_special () \ do { \ - rcu_read_unlock_special(); \ + asm volatile ("call rcu_read_unlock_special_thunk" : ASM_CALL_CONSTRAINT);\ } while (0)#endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT
Signature · 1 line
2.19.1.6.gb485710b
kernel test robot
build botRe: [PATCH 10/10] x86/rcu: Add THUNK rcu_read_unlock_special_thunk
kernel test robot noticed the following build errors:
Hi Lai,
kernel test robot noticed the following build errors:
[auto build test ERROR on paulmck-rcu/dev]
[also build test ERROR on tip/locking/core tip/sched/core tip/x86/asm tip/master linus/master v6.9-rc1 next-20240328] [cannot apply to tip/x86/core tip/auto-latest]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Lai-Jiangshan/lib-Use-rcu_preempt_depth-to-replace-current-rcu_read_lock_nesting/20240328-155513 base: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev patch link: https://lore.kernel.org/r/20240328075318.83039-11-jiangshanlai%40gmail.com patch subject: [PATCH 10/10] x86/rcu: Add THUNK rcu_read_unlock_special_thunk config: i386-alldefconfig (https://download.01.org/0day-ci/archive/20240329/[email protected]/config) compiler: gcc-12 (Ubuntu 12.3.0-9ubuntu2) 12.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240329/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
ld: init/main.o: in function `rest_init':
>> main.c:(.ref.text+0x96): undefined reference to `rcu_read_unlock_special_thunk' >> ld: main.c:(.ref.text+0x9d): undefined reference to `rcu_read_unlock_special_thunk' ld: arch/x86/events/intel/ds.o: in function `intel_pmu_drain_bts_buffer': >> ds.c:(.text+0x2cd9): undefined reference to `rcu_read_unlock_special_thunk' ld: arch/x86/kernel/nmi.o: in function `nmi_handle': >> nmi.c:(.text+0x2e9): undefined reference to `rcu_read_unlock_special_thunk' ld: arch/x86/kernel/alternative.o: in function `__text_poke': >> alternative.c:(.text+0x531): undefined reference to `rcu_read_unlock_special_thunk'
ld: arch/x86/mm/init.o:init.c:(.init.text+0xf1): more undefined references to `rcu_read_unlock_special_thunk' follow
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki