rcu: Assign higher prio to RCU threads if rcutorture is built-in
Subsystems (2)
PATCH v2 1/3 rcu: Assign higher prio to RCU threads if rcutorture is built-in
From: "Joel Fernandes (Google)" <[email protected]>
rcutorture boost tests fail even with CONFIG_RCU_BOOST set because rcutorture's threads are equal priority to the default RCU kthreads (RT class with priority of 1).
This patch checks if RCU torture is built into the kernel and if so, assigns a higher priority to the RCU threads. With this the rcutorture boost tests pass.
--- kernel/rcu/tree.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +5 -1
@@ -3884,12 +3884,16 @@ static int __init rcu_spawn_gp_kthread(void)struct task_struct *t;/* Force priority into range. */ -if (IS_ENABLED (CONFIG_RCU_BOOST ) && kthread_prio <1 ) +if (IS_ENABLED (CONFIG_RCU_BOOST ) && kthread_prio <2 + &&IS_BUILTIN (CONFIG_RCU_TORTURE_TEST )) + kthread_prio =2 ; +else if (IS_ENABLED (CONFIG_RCU_BOOST ) && kthread_prio <1 ) kthread_prio =1 ;else if (kthread_prio <0 ) kthread_prio =0 ;else if (kthread_prio >99 ) kthread_prio =99 ; +if (kthread_prio != kthread_prio_in)pr_alert ("rcu_spawn_gp_kthread(): Limited prio to %d from %d\n" , kthread_prio, kthread_prio_in);
Signature · 1 line
2.18.0.rc1.244.gcf134e6275-goog
PATCH v2 2/3 rcutorture: Add support to detect if boost kthread prio is too low
From: "Joel Fernandes (Google)" <[email protected]>
When rcutorture is built as a built-in, we can detect that and raise the priority of RCU's kthreads enough that rcutorture's boost threads will not be able to starve them. We do so in earlier patches.
However, if rcutorture is built as a module, we wouldn't have raised the RCU thread's priorities and we don't want to raise them for non-rcutorture-builtin kernels since we don't want to risk folks who may already be depending on the old thread priorities. For this reason, we can't test for boost if rcutorture is built as a module. This patch detects this situation and prints an message from the rcutorture module if such a scenario is detected so that the user knows what to do inorder to remedy it (that is manually pass rcutree.kthread_prio of a higher value so that boosting can be tested).
--- kernel/rcu/rcu.h | 2 ++ kernel/rcu/rcutorture.c | 32 ++++++++++++++++++++++++++++---- kernel/rcu/tree.c | 7 +++++++ 3 files changed, 37 insertions(+), 4 deletions(-)
kernel/rcu/rcu.h +2 -0
@@ -547,6 +547,7 @@ static inline void rcu_force_quiescent_state(void) { }static inline void rcu_bh_force_quiescent_state (void ) { }static inline void rcu_sched_force_quiescent_state (void ) { }static inline void show_rcu_gp_kthreads (void ) { } +static inline int rcu_get_gp_kthreads_prio (void ) {return 0 ; }#else /* #ifdef CONFIG_TINY_RCU */ unsigned long rcu_get_gp_seq (void );unsigned long rcu_bh_get_gp_seq (void );
@@ -555,6 +556,7 @@ unsigned long rcu_exp_batches_completed(void);unsigned long rcu_exp_batches_completed_sched (void );unsigned long srcu_batches_completed (struct srcu_struct *sp);void show_rcu_gp_kthreads (void ); +int rcu_get_gp_kthreads_prio (void );void rcu_force_quiescent_state (void );void rcu_bh_force_quiescent_state (void );void rcu_sched_force_quiescent_state (void );
kernel/rcu/rcutorture.c +28 -4
@@ -1786,6 +1786,32 @@ static void rcu_torture_barrier_cleanup(void) } } +static bool rcu_torture_can_boost (void ) +{ +static int boost_warn_once; +int prio; + +if (!(test_boost ==1 && cur_ops->can_boost) && test_boost !=2 ) +return false; + + prio =rcu_get_gp_kthreads_prio (); +if (!prio) +return false; + +if (prio <2 ) { +if (boost_warn_once ==1 ) +return false; + +pr_alert ("%s: WARN: RCU kthread priority too low to test boosting. " +"Skipping RCU boost test. Try passing rcutree.kthread_prio > 1 " +"on the kernel command line.\n" ,KBUILD_MODNAME ); + boost_warn_once =1 ; +return false; + } + +return true; +} +static enum cpuhp_state rcutor_hp; static void
@@ -1830,8 +1856,7 @@ rcu_torture_cleanup(void)torture_stop_kthread (rcu_torture_fqs, fqs_task);for (i =0 ; i < ncbflooders; i++)torture_stop_kthread (rcu_torture_cbflood, cbflood_task[i]); -if ((test_boost ==1 && cur_ops->can_boost) || - test_boost ==2 ) +if (rcu_torture_can_boost ())cpuhp_remove_state (rcutor_hp);/*
@@ -2055,8 +2080,7 @@ rcu_torture_init(void) test_boost_interval =1 ;if (test_boost_duration <2 ) test_boost_duration =2 ; -if ((test_boost ==1 && cur_ops->can_boost) || - test_boost ==2 ) { +if (rcu_torture_can_boost ()) { boost_starttime = jiffies + test_boost_interval *HZ ;
kernel/rcu/tree.c +7 -0
@@ -183,6 +183,13 @@ module_param(gp_init_delay, int, 0444);static int gp_cleanup_delay;module_param (gp_cleanup_delay, int,0444 ); +/* Retreive RCU kthreads priority for rcutorture */ +int rcu_get_gp_kthreads_prio (void ) +{ +return kthread_prio; +} +EXPORT_SYMBOL_GPL (rcu_get_gp_kthreads_prio); +/* * Number of grace periods between delays, normalized by the duration of * the delay. The longer the delay, the more the grace periods between
Signature · 1 line
2.18.0.rc1.244.gcf134e6275-goog
PATCH v2 3/3 rcutorture: Fix rcu_barrier successes counter
In reply to Joel FernandesFrom: "Joel Fernandes (Google)" <[email protected]>
rcutorture currently increments both successes and error for barrier test incase of errors. It should only increment the error counter incase of errors so make it do so.
Test: Introduced rcu_barrier errors by returning from the barrier callback without incrementing the callback counter.
--- kernel/rcu/rcutorture.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/rcu/rcutorture.c +3 -2
@@ -169,7 +169,7 @@ static long n_rcu_torture_boost_failure;static long n_rcu_torture_boosts;static atomic_long_t n_rcu_torture_timers;static long n_barrier_attempts; -static long n_barrier_successes; +static long n_barrier_successes;/* did rcu_barrier test succeed? */ static atomic_long_t n_cbfloods;static struct list_head rcu_torture_removed;
@@ -1722,8 +1722,9 @@ static int rcu_torture_barrier(void *arg)atomic_read (&barrier_cbs_invoked), n_barrier_cbs);WARN_ON_ONCE (1 ); + }else { + n_barrier_successes++; } - n_barrier_successes++;schedule_timeout_interruptible (HZ /10 ); }while (!torture_must_stop ());torture_kthread_stopping ("rcu_torture_barrier" );
Signature · 1 line
2.18.0.rc1.244.gcf134e6275-goog
Quoted 11 lines · Joel Fernandes
From: "Joel Fernandes (Google)" <[email protected]>
rcutorture boost tests fail even with
CONFIG_RCU_BOOSTset because rcutorture's threads are equal priority to the default RCU kthreads (RT class with priority of 1).This patch checks if RCU torture is built into the kernel and if so, assigns a higher priority to the RCU threads. With this the rcutorture boost tests pass.
Signed-off-by Joel Fernandes (Google) <[email protected]>
I queued all three for testing and further review. I reworked the commit logs a bit and also pulled the error string onto one line to make it easier on people grepping for the message. Please let me know if I messed anything up.
Thanx, Paul
Quoted 29 lines
--- kernel/rcu/tree.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/rcu/tree.c +5
-1@@ -3884,12 +3884,16 @@ static int __init rcu_spawn_gp_kthread(void)struct task_struct *t;/* Force priority into range. */ -if (IS_ENABLED (CONFIG_RCU_BOOST ) && kthread_prio <1 ) +if (IS_ENABLED (CONFIG_RCU_BOOST ) && kthread_prio <2 + &&IS_BUILTIN (CONFIG_RCU_TORTURE_TEST )) + kthread_prio =2 ; +else if (IS_ENABLED (CONFIG_RCU_BOOST ) && kthread_prio <1 ) kthread_prio =1 ;else if (kthread_prio <0 ) kthread_prio =0 ;else if (kthread_prio >99 ) kthread_prio =99 ; +if (kthread_prio != kthread_prio_in)pr_alert ("rcu_spawn_gp_kthread(): Limited prio to %d from %d\n" , kthread_prio, kthread_prio_in);Signature · 1 line
2.18.0.rc1.244.gcf134e6275-goog
Quoted 11 lines · Paul E. McKenney
Quoted 10 lines · Joel Fernandes
From: "Joel Fernandes (Google)" <[email protected]>
rcutorture boost tests fail even with
CONFIG_RCU_BOOSTset because rcutorture's threads are equal priority to the default RCU kthreads (RT class with priority of 1).This patch checks if RCU torture is built into the kernel and if so, assigns a higher priority to the RCU threads. With this the rcutorture boost tests pass.
Quoted 6 lines
Quoted 1 line
Signed-off-by Joel Fernandes (Google) <[email protected]>I queued all three for testing and further review. I reworked the commit logs a bit and also pulled the error string onto one line to make it easier on people grepping for the message. Please let me know if I messed anything up.
I double checked it and it looks good to me, thanks Paul!
Regards,
- Joel