Skip to contentSkip to conversation

handle kthread_prio kernel cmdline parameter consistently

PATCH In review Alison Chaiken 2 participants 6 messages Jan 12, 2022 – Jan 13, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover handle kthread_prio kernel cmdline parameter consistently

When CONFIG_PREEMPT_RT=y, RCU_SOFTIRQ work is moved to dedicated rcuc per-core threads. The rcutree.kthread_prio kernel cmdline parameter controls the priority of those threads.

A provided kthread_prio kernel cmdline parameter also elevates the priority of rcuog threads, but only if they are launched from rcu_spawn_gp_kthread(), not if they are launched from rcu_spawn_one_nocb_kthread(). Fix this inconsistency.

When CONFIG_RCU_NOCB_CPU=y, an rcu_nocbs command-line parameter offloads the work of rcuc on designated cores to new rcuop threads that are not pinned to the cores whose expired grace-period timer callbacks they run. While rcuop threads have the same function as rcuc threads, their priority is not controlled by the kthread_prio parameter. Add this feature and update the documentation accordingly.

Alison Chaiken (4):
RCU: move kthread_prio bounds-check to a separate function
RCU: make priority of grace-period thread consistent
RCU: elevate priority of offloaded callback threads
RCU: update documentation regarding kthread_prio cmdline parameter

 .../admin-guide/kernel-parameters.txt         |  2 +
 kernel/rcu/tree.c                             | 42 +++++++++++--------
 kernel/rcu/tree_nocb.h                        |  7 ++++
 3 files changed, 34 insertions(+), 17 deletions(-)
Signature · 1 line
2.32.0
#1 lore raw

PATCH 1/4 RCU: move kthread_prio bounds-check to a separate function

Move the bounds-check of the kthread_prio cmdline parameter to a new function in order to faciliate a different callsite.

---
 kernel/rcu/tree.c | 39 +++++++++++++++++++++++----------------
 1 file changed, 23 insertions(+), 16 deletions(-)
kernel/rcu/tree.c +23 -16
index 9a04550cc54b..15cabb288a1c 100644
@@ -4404,26 +4404,10 @@ static int rcu_pm_notify(struct notifier_block *self,
 static int __init rcu_spawn_gp_kthread(void)
 {
 	unsigned long flags;
-	int kthread_prio_in = kthread_prio;
 	struct rcu_node *rnp;
 	struct sched_param sp;
 	struct task_struct *t;
 
-	/* Force priority into range. */
-	if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 2
-	    && IS_BUILTIN(CONFIG_RCU_TORTURE_TEST))
-		kthread_prio = 2;
-	else if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 1)
-		kthread_prio = 1;
-	else if (kthread_prio < 0)
-		kthread_prio = 0;
-	else if (kthread_prio > 99)
-		kthread_prio = 99;
-
-	if (kthread_prio != kthread_prio_in)
-		pr_alert("rcu_spawn_gp_kthread(): Limited prio to %d from %d\n",
-			 kthread_prio, kthread_prio_in);
-
 	rcu_scheduler_fully_active = 1;
 	t = kthread_create(rcu_gp_kthread, NULL, "%s", rcu_state.name);
 	if (WARN_ONCE(IS_ERR(t), "%s: Could not start grace-period kthread, OOM is now expected behavior\n", __func__))
@@ -4548,6 +4532,28 @@ static void __init rcu_init_one(void)
 	}
 }
 
+/*
+ * Force priority from the kernel command-line into range.
+ */
+static void __init sanitize_kthread_prio(void)
+{
+	int kthread_prio_in = kthread_prio;
+
+	if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 2
+	    && IS_BUILTIN(CONFIG_RCU_TORTURE_TEST))
+		kthread_prio = 2;
+	else if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 1)
+		kthread_prio = 1;
+	else if (kthread_prio < 0)
+		kthread_prio = 0;
+	else if (kthread_prio > 99)
+		kthread_prio = 99;
+
+	if (kthread_prio != kthread_prio_in)
+		pr_alert("%s: Limited prio to %d from %d\n",
+			 __func__, kthread_prio, kthread_prio_in);
+}
+
 /*
  * Compute the rcu_node tree geometry from kernel parameters.  This cannot
  * replace the definitions in tree.h because those are needed to size
@@ -4708,6 +4714,7 @@ void __init rcu_init(void)
 
 	kfree_rcu_batch_init();
 	rcu_bootup_announce();
+	sanitize_kthread_prio();
 	rcu_init_geometry();
 	rcu_init_one();
 	if (dump_tree)
Signature · 1 line
2.32.0
maintainer
#2 lore raw
In reply to Alison Chaiken
Quoted 16 lines · Alison Chaiken

When CONFIG_PREEMPT_RT=y, RCU_SOFTIRQ work is moved to dedicated rcuc per-core threads. The rcutree.kthread_prio kernel cmdline parameter controls the priority of those threads.

A provided kthread_prio kernel cmdline parameter also elevates the priority of rcuog threads, but only if they are launched from rcu_spawn_gp_kthread(), not if they are launched from rcu_spawn_one_nocb_kthread(). Fix this inconsistency.

When CONFIG_RCU_NOCB_CPU=y, an rcu_nocbs command-line parameter offloads the work of rcuc on designated cores to new rcuop threads that are not pinned to the cores whose expired grace-period timer callbacks they run. While rcuop threads have the same function as rcuc threads, their priority is not controlled by the kthread_prio parameter. Add this feature and update the documentation accordingly.

Quoted 5 lines

Alison Chaiken (4):
RCU: move kthread_prio bounds-check to a separate function
RCU: make priority of grace-period thread consistent
RCU: elevate priority of offloaded callback threads
RCU: update documentation regarding kthread_prio cmdline parameter

I have queued these for review and testing, thank you!

2/4 appears to have been produced against a pre-merge-window mainline, so I had to adjust it a bit to queue it on -rcu. Please check to make sure that I did not mess anything up.

Thanx, Paul

#3 lore raw

PATCH 2/4 RCU: make priority of grace-period thread consistent

In reply to Alison Chaiken

The priority of RCU grace period threads is set to kthread_prio when they are launched from rcu_spawn_gp_kthread(). The same is not true of rcu_spawn_one_nocb_kthread(). Accordingly, add priority elevation to rcu_spawn_one_nocb_kthread().

---
 kernel/rcu/tree_nocb.h | 4 ++++
 1 file changed, 4 insertions(+)
kernel/rcu/tree_nocb.h +4 -0
index 8fdf44f8523f..08b5c0042979 100644
@@ -1183,7 +1183,9 @@ static void rcu_spawn_one_nocb_kthread(int cpu)
 	struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
 	struct rcu_data *rdp_gp;
 	struct task_struct *t;
+	struct sched_param sp;
 
+	sp.sched_priority = kthread_prio;
 	/*
 	 * If this isn't a no-CBs CPU or if it already has an rcuo kthread,
 	 * then nothing to do.
@@ -1199,6 +1201,8 @@ static void rcu_spawn_one_nocb_kthread(int cpu)
 		if (WARN_ONCE(IS_ERR(t), "%s: Could not start rcuo GP kthread, OOM is now expected behavior\n", __func__))
 			return;
 		WRITE_ONCE(rdp_gp->nocb_gp_kthread, t);
+		if (kthread_prio)
+			sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
 	}
 
 	/* Spawn the kthread for this CPU. */
Signature · 1 line
2.32.0
#4 lore raw

PATCH 3/4 RCU: elevate priority of offloaded callback threads

In reply to Alison Chaiken

When CONFIG_PREEMPT_RT=y, the rcutree.kthread_prio command-line parameter signals initialization code to boost the priority of rcuc callbacks to the designated value. With the additional CONFIG_RCU_NOCB_CPU=y configuration and an additional rcu_nocbs command-line parameter, the callbacks on the listed cores are offloaded to new rcuop threads that are not pinned to the cores whose post-grace-period work is performed. While the rcuop threads perform the same function as the rcuc threads they offload, the kthread_prio parameter only boosts the priority of the rcuc threads. Fix this inconsistency by elevating rcuop threads to the same priority as rcuc ones.

---
 kernel/rcu/tree.c      | 2 +-
 kernel/rcu/tree_nocb.h | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +1 -1
index 15cabb288a1c..ba9eaa9a72b0 100644
@@ -153,7 +153,7 @@ static void sync_sched_exp_online_cleanup(int cpu);
 static void check_cb_ovld_locked(struct rcu_data *rdp, struct rcu_node *rnp);
 static bool rcu_rdp_is_offloaded(struct rcu_data *rdp);
 
-/* rcuc/rcub kthread realtime priority */
+/* rcuc/rcub/rcuop kthread realtime priority */
 static int kthread_prio = IS_ENABLED(CONFIG_RCU_BOOST) ? 1 : 0;
 module_param(kthread_prio, int, 0444);
 
kernel/rcu/tree_nocb.h +3 -0
index 08b5c0042979..3343e42685dc 100644
@@ -1210,6 +1210,9 @@ static void rcu_spawn_one_nocb_kthread(int cpu)
 			"rcuo%c/%d", rcu_state.abbr, cpu);
 	if (WARN_ONCE(IS_ERR(t), "%s: Could not start rcuo CB kthread, OOM is now expected behavior\n", __func__))
 		return;
+
+	if (kthread_prio)
+		sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
 	WRITE_ONCE(rdp->nocb_cb_kthread, t);
 	WRITE_ONCE(rdp->nocb_gp_kthread, rdp_gp->nocb_gp_kthread);
 }
Signature · 1 line
2.32.0
#5 lore raw

PATCH 4/4 RCU: update documentation regarding kthread_prio cmdline parameter

In reply to Alison Chaiken

Inform readers that the priority of RCU no-callback threads will also be boosted.

---
 Documentation/admin-guide/kernel-parameters.txt | 2 ++
 1 file changed, 2 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +2 -0
index 1396fd2d9031..b844978e1bdf 100644
@@ -4439,6 +4439,8 @@
 			(the least-favored priority).  Otherwise, when
 			RCU_BOOST is not set, valid values are 0-99 and
 			the default is zero (non-realtime operation).
+			When RCU_NOCB_CPU is set, also adjust the
+			priority of NOCB callback kthreads.
 
 	rcutree.rcu_nocb_gp_stride= [KNL]
 			Set the number of NOCB callback kthreads in
Signature · 1 line
2.32.0