Skip to contentSkip to conversation

sched_ext: Fix finite-slice ticks on nohz_full

PATCH In review Andrea Righi 2 participants 4 messages Jul 8

Subsystems (3)
lore
Share & tools
lore

Andrea Righi

author reviewer
#0 lore raw

PATCH v3 1/1 sched_ext: Fix finite-slice ticks on nohz_full

The scheduler tick dependency for EXT tasks can become stale across context switches on nohz_full CPUs. An incoming finite-slice task may incorrectly keep the tick disabled because the scheduler still observes the outgoing task. The same can happen after an idle interval when the previous EXT task also had a finite slice. Likewise, when the last finite-slice EXT task is dequeued, its outgoing state can incorrectly keep the tick enabled after the CPU goes idle.

Fix this by directly asserting the scheduler tick dependency whenever a finite-slice EXT task is selected and by ignoring the outgoing EXT slice state when no EXT tasks are accounted on the runqueue.

Moreover, provide a kselftest that exercises both infinite-to-finite and finite-to-finite transitions across an idle interval and verifies that scheduler ticks are delivered correctly. The test is skipped when an allowed nohz_full CPU and a separate housekeeping CPU are not available.

Test case (using virtme-ng):

  $ vng -vb --config tools/testing/selftests/sched_ext/config \
        --configitem CONFIG_NO_HZ_FULL=y
  $ vng --append "nohz_full=all" -- \
        tools/testing/selftests/sched_ext/runner -t nohz_tick

Before:

  $ vng --append "nohz_full=all" -- tools/testing/selftests/sched_ext/runner -t nohz_tick
  ===== START =====
  TEST: nohz_tick
  DESCRIPTION: Verify finite EXT slices restart the NOHZ_FULL tick
  OUTPUT:
  ERR: nohz_tick.c:270
  Finite-slice worker received only 1 scheduler ticks
  not ok 1 nohz_tick #
  =====  END  =====

=============================

RESULTS:

PASSED: 0
SKIPPED: 0
FAILED: 1

Failed tests:
- nohz_tick

After:

  $ vng --append "nohz_full=all" -- tools/testing/selftests/sched_ext/runner -t nohz_tick
  ===== START =====
  TEST: nohz_tick
  DESCRIPTION: Verify finite EXT slices restart the NOHZ_FULL tick
  OUTPUT:
  CPU 1 received 6 finite-slice ticks
  ok 1 nohz_tick #
  =====  END  =====

=============================

RESULTS:

PASSED: 1
SKIPPED: 0
FAILED: 0

Changes in v3:
- Split the infinite/finite-slice paths so that the slice type is tested only once (Tejun)
- Adjust/rephrase code comments
- Link to v2: https://lore.kernel.org/all/[email protected]/

Changes in v2:
- Reassert the tick dependency for every selected finite-slice EXT task, fixing the finite-to-idle-to-finite case and extend the selftest to cover it (Tejun, sashiko AI)
- Ignore stale outgoing EXT slice state when no EXT tasks are accounted on the runqueue (sashiko AI).
- Clarify that the infinite-slice bypass scenario is unreachable because bypass mode always assigns finite slices (sashiko AI).
- Kill test workers automatically if their parent exits unexpectedly in the selfetst (sashiko AI)
- Link to v1: https://lore.kernel.org/all/[email protected]/

Andrea Righi (2):
sched_ext: Enable tick for finite slices on nohz_full selftests/sched_ext: Verify nohz_full tick behavior

 kernel/sched/ext/ext.c                            |  47 ++-
 tools/testing/selftests/sched_ext/Makefile        |   1 +
 tools/testing/selftests/sched_ext/nohz_tick.bpf.c |  65 ++++
 tools/testing/selftests/sched_ext/nohz_tick.c     | 347 ++++++++++++++++++++++
 4 files changed, 448 insertions(+), 12 deletions(-)
 create mode 100644 tools/testing/selftests/sched_ext/nohz_tick.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/nohz_tick.c
author reviewer
#1 lore raw

PATCH 1/2 sched_ext: Enable tick for finite slices on nohz_full

set_next_task_scx() updates the tick dependency before __schedule() updates rq->curr. When switching from a non-EXT task, such as idle, to an EXT task with a finite slice, sched_update_tick_dependency() checks the outgoing task and can allow the tick to remain stopped.

The dependency can also be lost without a slice-type transition. After a finite-slice task leaves the CPU idle, the enqueue path can clear the dependency against the idle rq->curr. SCX_RQ_CAN_STOP_TICK still records a finite slice, so another finite task skips the transition block and can run without the ticks needed to expire its slice.

The reverse mismatch can also happen when the last finite-slice EXT task is dequeued: sub_nr_running() updates the dependency before rq->curr changes, so the outgoing task state can keep the dependency set after the CPU goes idle.

Fix this by unconditionally enabling the scheduler tick whenever a finite-slice EXT task is selected on a nohz_full CPU. Moreover, when the last runnable EXT task leaves, ignore the outgoing EXT slice state so the generic scheduler can correctly re-evaluate and clear the tick dependency.

Fixes 22a920209ab6 ("sched_ext: Implement tickless support")
---
 kernel/sched/ext/ext.c | 47 +++++++++++++++++++++++++++++++-----------
 1 file changed, 35 insertions(+), 12 deletions(-)
kernel/sched/ext/ext.c +35 -12
index e75e2fd5ab7e3..3fe06d05ffdd5 100644
@@ -2978,24 +2978,38 @@ static void set_next_task_scx(struct rq *rq, struct task_struct *p, bool first)
 
 	/*
 	 * @p is getting newly scheduled or got kicked after someone updated its
-	 * slice. Refresh whether tick can be stopped. See scx_can_stop_tick().
+	 * slice. Update SCX_RQ_CAN_STOP_TICK to reflect whether the tick can be
+	 * stopped. See scx_can_stop_tick().
+	 *
+	 * Moreover, refresh the load_avgs just when transitioning in and out of
+	 * nohz. In the future, we might want to add a mechanism to update
+	 * load_avgs periodically on tick-stopped CPUs.
 	 */
-	if ((p->scx.slice == SCX_SLICE_INF) !=
-	    (bool)(rq->scx.flags & SCX_RQ_CAN_STOP_TICK)) {
-		if (p->scx.slice == SCX_SLICE_INF)
+	if (p->scx.slice == SCX_SLICE_INF) {
+		if (!(rq->scx.flags & SCX_RQ_CAN_STOP_TICK)) {
+			/*
+			 * Bypass mode always assigns finite slices, so @p
+			 * can't have an infinite slice while bypassing.
+			 * Therefore, sched_update_tick_dependency() can safely
+			 * evaluate the outgoing task.
+			 */
 			rq->scx.flags |= SCX_RQ_CAN_STOP_TICK;
-		else
-			rq->scx.flags &= ~SCX_RQ_CAN_STOP_TICK;
+			sched_update_tick_dependency(rq);
 
-		sched_update_tick_dependency(rq);
+			update_other_load_avgs(rq);
+		}
+	} else {
+		if (rq->scx.flags & SCX_RQ_CAN_STOP_TICK) {
+			rq->scx.flags &= ~SCX_RQ_CAN_STOP_TICK;
+			update_other_load_avgs(rq);
+		}
 
 		/*
-		 * For now, let's refresh the load_avgs just when transitioning
-		 * in and out of nohz. In the future, we might want to add a
-		 * mechanism which calls the following periodically on
-		 * tick-stopped CPUs.
+		 * @rq still references the outgoing scheduling context. A finite
+		 * slice is sufficient by itself to require the tick.
 		 */
-		update_other_load_avgs(rq);
+		if (tick_nohz_full_cpu(cpu_of(rq)))
+			tick_nohz_dep_set_cpu(cpu_of(rq), TICK_DEP_BIT_SCHED);
 	}
 }
 
@@ -4321,6 +4335,15 @@ bool scx_can_stop_tick(struct rq *rq)
 	if (p->sched_class != &ext_sched_class)
 		return true;
 
+	/*
+	 * @rq->curr may still reference an outgoing EXT task after it has been
+	 * dequeued. If no EXT tasks are accounted on @rq, ignore its stale
+	 * slice state. If another task is dispatched from a DSQ,
+	 * set_next_task_scx() will update the dependency for the incoming task.
+	 */
+	if (!rq->scx.nr_running)
+		return true;
+
 	if (scx_bypassing(sch, cpu_of(rq)))
 		return false;
 
Signature · 1 line
2.55.0
author reviewer
#2 lore raw

PATCH 2/2 selftests/sched_ext: Verify nohz_full tick behavior

In reply to Andrea Righi

Finite-slice EXT tasks need the periodic scheduler tick to expire their slices even when nohz_full is enabled.

Add a regression test that selects a nohz_full CPU and exercises both infinite-to-finite and finite-to-finite slice transitions across an idle interval. For each finite task, verify that its ops.tick() callback is invoked.

Skip the test when an allowed nohz_full CPU and a separate housekeeping CPU are not available.

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/nohz_tick.bpf.c       |  65 ++++
 tools/testing/selftests/sched_ext/nohz_tick.c | 347 ++++++++++++++++++
 3 files changed, 413 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/nohz_tick.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/nohz_tick.c
tools/testing/selftests/sched_ext/Makefile +1 -0
index 5d2dffca0e918..3cfe90e0f34fa 100644
@@ -176,6 +176,7 @@ auto-test-targets :=			\
 	maybe_null			\
 	minimal				\
 	non_scx_kfunc_deny		\
+	nohz_tick			\
 	numa				\
 	allowed_cpus			\
 	peek_dsq			\
tools/testing/selftests/sched_ext/nohz_tick.bpf.c new file +65 -0

65 lines not on this page · raw message

tools/testing/selftests/sched_ext/nohz_tick.c new file +347 -0

347 lines not on this page · raw message

Signature · 1 line
2.55.0