Skip to contentSkip to conversation

sched/core: introduce sched_core_idle_cpu()

PATCH Superseded Cruz Zhao 3 participants 3 messages Jun 24, 2023 – Jun 27, 2023

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Cruz Zhao

author
#0 lore raw

PATCH v2 1/1 sched/core: introduce sched_core_idle_cpu()

As core scheduling introduced, a new state of idle is defined as force idle, running idle task but nr_running greater than zero.

If a cpu is in force idle state, idle_cpu() will return zero. This result makes sense in certain scenarios, e.g., load balance. but this will cause error in other scenarios, e.g., tick_irq_exit() will not change ts->idle_active back to 1 because idle_cpu() returns 0 when force idle.

To solve this problem, we introduce sched_core_idle_cpu(), which returns 1 when force idle. We audit all users of idle_cpu(), and change idle_cpu() into sched_core_idle_cpu() in the following functions:
- showacpu()
- rcu_is_rcuc_kthread_starving()
- tick_irq_exit()

v1-->v2: replace idle_cpu() with sched_core_idle_cpu() in function showacpu() and rcu_is_rcuc_kthread_starving()

---
 drivers/tty/sysrq.c     |  2 +-
 include/linux/sched.h   |  2 ++
 kernel/rcu/tree_stall.h |  2 +-
 kernel/sched/core.c     | 13 +++++++++++++
 kernel/softirq.c        |  2 +-
 5 files changed, 18 insertions(+), 3 deletions(-)
drivers/tty/sysrq.c +1 -1
index b6e70c5cfa17..8a6586800385 100644
@@ -232,7 +232,7 @@ static void showacpu(void *dummy)
 	unsigned long flags;
 
 	/* Idle CPUs have no interesting backtrace. */
-	if (idle_cpu(smp_processor_id())) {
+	if (sched_core_idle_cpu(smp_processor_id())) {
 		pr_info("CPU%d: backtrace skipped as idling\n", smp_processor_id());
 		return;
 	}
include/linux/sched.h +2 -0
index b09a83bfad8b..73e61c0f10a7 100644
@@ -2430,9 +2430,11 @@ extern void sched_core_free(struct task_struct *tsk);
 extern void sched_core_fork(struct task_struct *p);
 extern int sched_core_share_pid(unsigned int cmd, pid_t pid, enum pid_type type,
 				unsigned long uaddr);
+extern int sched_core_idle_cpu(int cpu);
 #else
 static inline void sched_core_free(struct task_struct *tsk) { }
 static inline void sched_core_fork(struct task_struct *p) { }
+static inline int sched_core_idle_cpu(int cpu) { return idle_cpu(cpu); }
 #endif
 
 extern void sched_set_stop_task(int cpu, struct task_struct *stop);
kernel/rcu/tree_stall.h +1 -1
index b10b8349bb2a..6169faf30ecd 100644
@@ -418,7 +418,7 @@ static bool rcu_is_rcuc_kthread_starving(struct rcu_data *rdp, unsigned long *jp
 		return false;
 
 	cpu = task_cpu(rcuc);
-	if (cpu_is_offline(cpu) || idle_cpu(cpu))
+	if (cpu_is_offline(cpu) || sched_core_idle_cpu(cpu))
 		return false;
 
 	j = jiffies - READ_ONCE(rdp->rcuc_activity);
kernel/sched/core.c +13 -0
index 71c1a0f232b4..c80088956987 100644
@@ -7286,6 +7286,19 @@ struct task_struct *idle_task(int cpu)
 	return cpu_rq(cpu)->idle;
 }
 
+#ifdef CONFIG_SCHED_CORE
+int sched_core_idle_cpu(int cpu)
+{
+	struct rq *rq = cpu_rq(cpu);
+
+	if (sched_core_enabled(rq) && rq->curr == rq->idle)
+		return 1;
+
+	return idle_cpu(cpu);
+}
+
+#endif
+
 #ifdef CONFIG_SMP
 /*
  * This function computes an effective utilization for the given CPU, to be
kernel/softirq.c +1 -1
index c8a6913c067d..98b98991ce45 100644
@@ -630,7 +630,7 @@ static inline void tick_irq_exit(void)
 	int cpu = smp_processor_id();
 
 	/* Make sure that timer wheel updates are propagated */
-	if ((idle_cpu(cpu) && !need_resched()) || tick_nohz_full_cpu(cpu)) {
+	if ((sched_core_idle_cpu(cpu) && !need_resched()) || tick_nohz_full_cpu(cpu)) {
 		if (!in_hardirq())
 			tick_nohz_irq_exit();
 	}
Signature · 1 line
2.27.0
#1 lore raw
Quoted 34 lines · Cruz Zhao

As core scheduling introduced, a new state of idle is defined as force idle, running idle task but nr_running greater than zero.

If a cpu is in force idle state, idle_cpu() will return zero. This result makes sense in certain scenarios, e.g., load balance. but this will cause error in other scenarios, e.g., tick_irq_exit() will not change ts->idle_active back to 1 because idle_cpu() returns 0 when force idle.

To solve this problem, we introduce sched_core_idle_cpu(), which returns 1 when force idle. We audit all users of idle_cpu(), and change idle_cpu() into sched_core_idle_cpu() in the following functions:
- showacpu()
- rcu_is_rcuc_kthread_starving()
- tick_irq_exit()

v1-->v2: replace idle_cpu() with sched_core_idle_cpu() in function showacpu() and rcu_is_rcuc_kthread_starving()

---
 drivers/tty/sysrq.c     |  2 +-
 include/linux/sched.h   |  2 ++
 kernel/rcu/tree_stall.h |  2 +-
 kernel/sched/core.c     | 13 +++++++++++++
 kernel/softirq.c        |  2 +-
 5 files changed, 18 insertions(+), 3 deletions(-)
drivers/tty/sysrq.c +0 -0
index b6e70c5cfa17..8a6586800385 100644

drivers/tty/sysrq.c static void showacpu(void *dummy) · line 236

Quoted 6 lines
drivers/tty/sysrq.c +1 -1
@@ -232,7 +232,7 @@ static void showacpu(void *dummy)
 	unsigned long flags;
 
 	/* Idle CPUs have no interesting backtrace. */
-	if (idle_cpu(smp_processor_id())) {
+	if (sched_core_idle_cpu(smp_processor_id())) {
 		pr_info("CPU%d: backtrace skipped as idling\n", smp_processor_id());

Actually perhaps an idle injection's backtrace is worth dumping. I guess it might accidentally produce lockups and it's worth knowing the source then.

Though I don't have a strong opinion on that...

kernel/rcu/tree_stall.h static bool rcu_is_rcuc_kthread_starving(struct rcu_data *rdp, unsigned long *jp · line 421

Quoted 12 lines
drivers/tty/sysrq.c +0 -0
 		return;
 	}
kernel/rcu/tree_stall.h +1 -1
index b10b8349bb2a..6169faf30ecd 100644
@@ -418,7 +418,7 @@ static bool rcu_is_rcuc_kthread_starving(struct rcu_data *rdp, unsigned long *jp
 		return false;
 
 	cpu = task_cpu(rcuc);
-	if (cpu_is_offline(cpu) || idle_cpu(cpu))
+	if (cpu_is_offline(cpu) || sched_core_idle_cpu(cpu))

An idle injection could possibly starve the RCU boost kthread, and then it's worth knowing about it. I would suggest keeping idle_cpu() here.

kernel/softirq.c static inline void tick_irq_exit(void) · line 633

Quoted 13 lines
kernel/rcu/tree_stall.h +0 -0
 		return false;
 
 	j = jiffies - READ_ONCE(rdp->rcuc_activity);
kernel/softirq.c +1 -1
index c8a6913c067d..98b98991ce45 100644
@@ -630,7 +630,7 @@ static inline void tick_irq_exit(void)
 	int cpu = smp_processor_id();
 
 	/* Make sure that timer wheel updates are propagated */
-	if ((idle_cpu(cpu) && !need_resched()) || tick_nohz_full_cpu(cpu)) {
+	if ((sched_core_idle_cpu(cpu) && !need_resched()) ||  tick_nohz_full_cpu(cpu)) {

That one looks good.

Thanks!

#2 lore raw
Quoted 32 lines · Frederic Weisbecker
Quoted 12 lines
  drivers/tty/sysrq.c     |  2 +-
  include/linux/sched.h   |  2 ++
  kernel/rcu/tree_stall.h |  2 +-
  kernel/sched/core.c     | 13 +++++++++++++
  kernel/softirq.c        |  2 +-
  5 files changed, 18 insertions(+), 3 deletions(-)
drivers/tty/sysrq.c +0 -0
index b6e70c5cfa17..8a6586800385 100644

drivers/tty/sysrq.c static void showacpu(void *dummy) · line 236

Quoted 6 lines
drivers/tty/sysrq.c +1 -1
@@ -232,7 +232,7 @@ static void showacpu(void *dummy)
  	unsigned long flags;
 
  	/* Idle CPUs have no interesting backtrace. */
-	if (idle_cpu(smp_processor_id())) {
+	if (sched_core_idle_cpu(smp_processor_id())) {
  		pr_info("CPU%d: backtrace skipped as idling\n", smp_processor_id());

Actually perhaps an idle injection's backtrace is worth dumping. I guess it might accidentally produce lockups and it's worth knowing the source then.

Though I don't have a strong opinion on that...

Quoted 9 lines
drivers/tty/sysrq.c +0 -0
  		return;
  	}
kernel/rcu/tree_stall.h +0 -0
index b10b8349bb2a..6169faf30ecd 100644
@@ -418,7 +418,7 @@ static bool rcu_is_rcuc_kthread_starving(struct rcu_data *rdp, unsigned long *jp
  		return false;
 
Quoted 6 lines

kernel/rcu/tree_stall.h static bool rcu_is_rcuc_kthread_starving(struct rcu_data *rdp, unsigned long *jp · line 421

Quoted 3 lines
kernel/rcu/tree_stall.h +1 -1
  	cpu = task_cpu(rcuc);
-	if (cpu_is_offline(cpu) || idle_cpu(cpu))
+	if (cpu_is_offline(cpu) || sched_core_idle_cpu(cpu))

An idle injection could possibly starve the RCU boost kthread, and then it's worth knowing about it. I would suggest keeping idle_cpu() here.

Actually I think it should just be idle_cpu() for rcu_is_rcuc_kthread_starving() and showacpu() because "force idling" is different from "idling".

Force idling happens because there is something incompatible on the sibling runqueue in the core. That just makes the 2 runqueues on the core appear to be a single runqueue. The concept of "force idling" is more closer to "preemption of tasks on a single runqueue".

Considering that, I would vote for only converting the tick user. If force idling happens for too long, it'd be good to know that as Frederic also mentioned.

thanks,

- Joel