Add a deadline server for sched_ext tasks

From: Joel Fernandes <[email protected]>

Updating "ppos" on error conditions does not make much sense. The pattern is to return the error code directly without modifying the position, or modify the position on success and return the number of bytes written.

Since on success, the return value of apply is 0, there is no point in modifying ppos either. Fix it by removing all this and just returning error code or number of bytes written on success.

Acked-by Tejun Heo <[email protected]>

Reviewed-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

From: Joel Fernandes <[email protected]>

Currently the DL server interface for applying parameters checks CFS-internals to identify if the server is active. This is error-prone and makes it difficult when adding new servers in the future.

Fix it, by using dl_server_active() which is also used by the DL server code to determine if the DL server was started.

Acked-by Tejun Heo <[email protected]>

Reviewed-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/debug.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

kernel/sched/debug.c +2 -0

index 6cf9be6eea49a..e71f6618c1a6a 100644
@@ -354,6 +354,8 @@ static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu
 		return err;
 
 	scoped_guard (rq_lock_irqsave, rq) {
+		bool is_active;
+
 		runtime  = rq->fair_server.dl_runtime;
 		period = rq->fair_server.dl_period;
 

Hi!

Quoted 33 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

Currently the DL server interface for applying parameters checks CFS-internals to identify if the server is active. This is error-prone and makes it difficult when adding new servers in the future.

Fix it, by using dl_server_active() which is also used by the DL server code to determine if the DL server was started.

Acked-by Tejun Heo <[email protected]>

Reviewed-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/debug.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

kernel/sched/debug.c +2 -0

index 6cf9be6eea49a..e71f6618c1a6a 100644
@@ -354,6 +354,8 @@ static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu
 		return err;
 
 	scoped_guard (rq_lock_irqsave, rq) {
+		bool is_active;
+
 		runtime  = rq->fair_server.dl_runtime;
 		period = rq->fair_server.dl_period;
 

@@ -376,8 +378,11 @@ static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu
 			return  -EINVAL;
 		}
 

kernel/sched/debug.c static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu · line 385

Quoted 7 lines

kernel/sched/debug.c +5 -2

-		update_rq_clock(rq);
-		dl_server_stop(&rq->fair_server);
+		is_active = dl_server_active(&rq->fair_server);
+		if (is_active) {
+			update_rq_clock(rq);
+			dl_server_stop(&rq->fair_server);
+		}

Won't this reintroduce what bb4700adc3abe ("sched/deadline: Always stop dl-server before changing parameters") fixed?

From: Joel Fernandes <[email protected]>

Hotplugged CPUs coming online do an enqueue but are not a part of any root domain containing cpu_active() CPUs. So in this case, don't mess with accounting and we can retry later. Without this patch, we see crashes with sched_ext selftest's hotplug test due to divide by zero.

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/deadline.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

kernel/sched/deadline.c +0 -0

index 4aefb34a1d38b..f2f5b1aea8e2b 100644

Hi!

Quoted 20 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

Hotplugged CPUs coming online do an enqueue but are not a part of any root domain containing cpu_active() CPUs. So in this case, don't mess with accounting and we can retry later. Without this patch, we see crashes with sched_ext selftest's hotplug test due to divide by zero.

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/deadline.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

kernel/sched/deadline.c +0 -0

index 4aefb34a1d38b..f2f5b1aea8e2b 100644
@@ -1665,7 +1665,12 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	cpus = dl_bw_cpus(cpu);
 	cap = dl_bw_capacity(cpu);
 

kernel/sched/deadline.c int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio · line 1671

Quoted 5 lines

kernel/sched/deadline.c +4 -1

-	if (__dl_overflow(dl_b, cap, old_bw, new_bw))
+	/*
+	 * Hotplugged CPUs coming online do an enqueue but are not a part of any
+	 * root domain containing cpu_active() CPUs. So in this case, don't mess
+	 * with accounting and we can retry later.

Later when? It seems a little vague. :)

Quoted 23 lines · Juri Lelli

Hi!

Quoted 20 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

Hotplugged CPUs coming online do an enqueue but are not a part of any root domain containing cpu_active() CPUs. So in this case, don't mess with accounting and we can retry later. Without this patch, we see crashes with sched_ext selftest's hotplug test due to divide by zero.

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/deadline.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

kernel/sched/deadline.c +0 -0

index 4aefb34a1d38b..f2f5b1aea8e2b 100644
@@ -1665,7 +1665,12 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	cpus = dl_bw_cpus(cpu);
 	cap = dl_bw_capacity(cpu);
 

Quoted 7 lines

kernel/sched/deadline.c int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio · line 1671

Quoted 5 lines

kernel/sched/deadline.c +4 -1

-	if (__dl_overflow(dl_b, cap, old_bw, new_bw))
+	/*
+	 * Hotplugged CPUs coming online do an enqueue but are not a part of any
+	 * root domain containing cpu_active() CPUs. So in this case, don't mess
+	 * with accounting and we can retry later.

Later when? It seems a little vague. :)

Yeah, this comment is actually incorrect, we're not "retrying later" anymore (we used to do that in a previous version), now the params are applied via:

ext.c:handle_hotplug() -> dl_server_on() -> dl_server_apply_params()

Or via scx_enable() when an scx scheduler is loaded. So, I'm wondering if this condition is still needed. Will do some tests.

Quoted 38 lines · Andrea Righi

Quoted 23 lines · Juri Lelli

Hi!

Quoted 20 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

Hotplugged CPUs coming online do an enqueue but are not a part of any root domain containing cpu_active() CPUs. So in this case, don't mess with accounting and we can retry later. Without this patch, we see crashes with sched_ext selftest's hotplug test due to divide by zero.

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/deadline.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

kernel/sched/deadline.c +0 -0

index 4aefb34a1d38b..f2f5b1aea8e2b 100644
@@ -1665,7 +1665,12 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	cpus = dl_bw_cpus(cpu);
 	cap = dl_bw_capacity(cpu);
 

Quoted 7 lines

kernel/sched/deadline.c int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio · line 1671

Quoted 5 lines

kernel/sched/deadline.c +4 -1

-	if (__dl_overflow(dl_b, cap, old_bw, new_bw))
+	/*
+	 * Hotplugged CPUs coming online do an enqueue but are not a part of any
+	 * root domain containing cpu_active() CPUs. So in this case, don't mess
+	 * with accounting and we can retry later.

Later when? It seems a little vague. :)

Yeah, this comment is actually incorrect, we're not "retrying later" anymore (we used to do that in a previous version), now the params are applied via:

ext.c:handle_hotplug() -> dl_server_on() -> dl_server_apply_params()

Quoted 2 lines

Or via scx_enable() when an scx scheduler is loaded. So, I'm wondering if this condition is still needed. Will do some tests.

Looks like I can't reproduce the error with the hotplug kselftest anymore (and it was happening pretty quickly).

Then I guess we can drop this patch or maybe add a WARN_ON_ONCE(!cpus) just to safe?

From: Joel Fernandes <[email protected]>

Since we are adding more servers, make dl_server_update_idle_time() accept a server argument than a specific server.

Reviewed-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

From: Joel Fernandes <[email protected]>

Since we are adding more servers, make dl_server_update_idle_time() accept a server argument than a specific server.

Reviewed-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

From: Joel Fernandes <[email protected]>

sched_ext currently suffers starvation due to RT. The same workload when converted to EXT can get zero runtime if RT is 100% running, causing EXT processes to stall. Fix it by adding a DL server for EXT.

A kselftest is also provided later to verify:

./runner -t rt_stall
===== START =====
TEST: rt_stall
DESCRIPTION: Verify that RT tasks cannot stall SCHED_EXT tasks OUTPUT:
TAP version 13
1..1
ok 1 PASS: CFS task got more than 4.00% of runtime

[ arighi: drop ->balance() now that pick_task() has an rf argument ]

Cc Luigi De Matteis <[email protected]>

Co-developed-by Andrea Righi <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Quoted 17 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

sched_ext currently suffers starvation due to RT. The same workload when converted to EXT can get zero runtime if RT is 100% running, causing EXT processes to stall. Fix it by adding a DL server for EXT.

A kselftest is also provided later to verify:

./runner -t rt_stall
===== START =====
TEST: rt_stall
DESCRIPTION: Verify that RT tasks cannot stall SCHED_EXT tasks OUTPUT:
TAP version 13
1..1
ok 1 PASS: CFS task got more than 4.00% of runtime

Quoted 6 lines

[ arighi: drop ->balance() now that pick_task() has an rf argument ]

Cc Luigi De Matteis <[email protected]>

Co-developed-by Andrea Righi <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

I don't see anything preventing this to come after patch 11 so that all sched_ext changes are at the end. Am I correct? That'd make applying the patches easier. All the debug and deadline changes can be applied to sched/core and I can pull that and apply sched_ext changes on top.

kernel/sched/ext.c BTF_ID_FLAGS(func, scx_bpf_now) · line 6910

Quoted 5 lines · Andrea Righi

kernel/sched/ext.c +5 -0

+static struct task_struct *
+ext_server_pick_task(struct sched_dl_entity *dl_se, struct rq_flags *rf)
+{
+	return pick_task_scx(dl_se->rq, rf);
+}

I wonder whether we should tell pick_task_scx() to suppress the rq_modified_above() test in this case as a fair or RT task being enqueued has no reason to restart the picking process. While it will behave fine on retry, it's probably useful to be explicit here.

Quoted 11 lines · Tejun Heo

kernel/sched/ext.c BTF_ID_FLAGS(func, scx_bpf_now) · line 6910

Quoted 5 lines · Andrea Righi

kernel/sched/ext.c +5 -0

+static struct task_struct *
+ext_server_pick_task(struct sched_dl_entity *dl_se, struct rq_flags *rf)
+{
+	return pick_task_scx(dl_se->rq, rf);
+}

I wonder whether we should tell pick_task_scx() to suppress the rq_modified_above() test in this case as a fair or RT task being enqueued has no reason to restart the picking process. While it will behave fine on retry, it's probably useful to be explicit here.

Yeah, that's a valid point. Maybe we can add a new flag to rq->scx.flags? Something like SCX_RQ_DL_SERVER_PICK?

Quoted 13 lines · Andrea Righi

Quoted 11 lines · Tejun Heo

kernel/sched/ext.c BTF_ID_FLAGS(func, scx_bpf_now) · line 6910

Quoted 5 lines · Andrea Righi

kernel/sched/ext.c +5 -0

+static struct task_struct *
+ext_server_pick_task(struct sched_dl_entity *dl_se, struct rq_flags *rf)
+{
+	return pick_task_scx(dl_se->rq, rf);
+}

I wonder whether we should tell pick_task_scx() to suppress the rq_modified_above() test in this case as a fair or RT task being enqueued has no reason to restart the picking process. While it will behave fine on retry, it's probably useful to be explicit here.

Quoted 2 lines

Yeah, that's a valid point. Maybe we can add a new flag to rq->scx.flags? Something like SCX_RQ_DL_SERVER_PICK?

We can factor out the internals of pick_task_scx() into a separate function and add a flag there?

From: Joel Fernandes <[email protected]>

sched_ext currently suffers starvation due to RT. The same workload when converted to EXT can get zero runtime if RT is 100% running, causing EXT processes to stall. Fix it by adding a DL server for EXT.

A kselftest is also provided later to verify:

./runner -t rt_stall
===== START =====
TEST: rt_stall
DESCRIPTION: Verify that RT tasks cannot stall SCHED_EXT tasks OUTPUT:
TAP version 13
1..1
ok 1 PASS: CFS task got more than 4.00% of runtime

[ arighi: drop ->balance() now that pick_task() has an rf argument ]

Cc Luigi De Matteis <[email protected]>

Co-developed-by Andrea Righi <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/core.c     |  3 +++
 kernel/sched/deadline.c |  2 +-
 kernel/sched/ext.c      | 51 +++++++++++++++++++++++++++++++++++++++--
 kernel/sched/sched.h    |  2 ++
 4 files changed, 55 insertions(+), 3 deletions(-)

kernel/sched/core.c +3 -0

index 096e8d03d85e7..31a9c9381c63f 100644
@@ -8679,6 +8679,9 @@ void __init sched_init(void)
 		hrtick_rq_init(rq);
 		atomic_set(&rq->nr_iowait, 0);
 		fair_server_init(rq);
+#ifdef CONFIG_SCHED_CLASS_EXT
+		ext_server_init(rq);
+#endif
 
 #ifdef CONFIG_SCHED_CORE
 		rq->core = rq;

kernel/sched/deadline.c +0 -0

index 0680e0186577a..3c1fd2190949e 100644

kernel/sched/deadline.c +1 -1

 	 * workload because it is running fair work.
 	 */
-	if (dl_se == &rq->fair_server)
+	if (dl_se->dl_server)
 		return;
 
 #ifdef CONFIG_RT_GROUP_SCHED

Hi!

Quoted 49 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

sched_ext currently suffers starvation due to RT. The same workload when converted to EXT can get zero runtime if RT is 100% running, causing EXT processes to stall. Fix it by adding a DL server for EXT.

A kselftest is also provided later to verify:

./runner -t rt_stall
===== START =====
TEST: rt_stall
DESCRIPTION: Verify that RT tasks cannot stall SCHED_EXT tasks OUTPUT:
TAP version 13
1..1
ok 1 PASS: CFS task got more than 4.00% of runtime

[ arighi: drop ->balance() now that pick_task() has an rf argument ]

Cc Luigi De Matteis <[email protected]>

Co-developed-by Andrea Righi <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

---
 kernel/sched/core.c     |  3 +++
 kernel/sched/deadline.c |  2 +-
 kernel/sched/ext.c      | 51 +++++++++++++++++++++++++++++++++++++++--
 kernel/sched/sched.h    |  2 ++
 4 files changed, 55 insertions(+), 3 deletions(-)

kernel/sched/core.c +3 -0

index 096e8d03d85e7..31a9c9381c63f 100644
@@ -8679,6 +8679,9 @@ void __init sched_init(void)
 		hrtick_rq_init(rq);
 		atomic_set(&rq->nr_iowait, 0);
 		fair_server_init(rq);
+#ifdef CONFIG_SCHED_CLASS_EXT
+		ext_server_init(rq);
+#endif
 
 #ifdef CONFIG_SCHED_CORE
 		rq->core = rq;

kernel/sched/deadline.c +0 -0

index 0680e0186577a..3c1fd2190949e 100644

kernel/sched/deadline.c static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64 · line 1504

Quoted 1 line

kernel/sched/deadline.c +0 -0

@@ -1504,7 +1504,7 @@ static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64
 	 * The fair server (sole dl_server) does not account for real-time

Fair server is not alone anymore. :))

Please update the comment as well.

Quoted 7 lines

kernel/sched/deadline.c +1 -1

 	 * workload because it is running fair work.
 	 */
-	if (dl_se == &rq->fair_server)
+	if (dl_se->dl_server)
 		return;
 
 #ifdef CONFIG_RT_GROUP_SCHED

...

Quoted 3 lines

kernel/sched/deadline.c +0 -0

@@ -1487,6 +1499,11 @@ static bool dequeue_task_scx(struct rq *rq, struct task_struct *p, int deq_flags
 	sub_nr_running(rq, 1);
 

kernel/sched/deadline.c static bool dequeue_task_scx(struct rq *rq, struct task_struct *p, int deq_flags · line 1506

Quoted 6 lines

kernel/sched/deadline.c +5 -0

 	dispatch_dequeue(rq, p);
+
+	/* Stop the server if this was the last task */
+	if (rq->scx.nr_running == 0)
+		dl_server_stop(&rq->ext_server);
+

Do we want to use the delayed stop behavior for scx-server as we have for fair-server? Wonder if it's a matter of removing this explicit stop and wait for a full period to elapse as we do for fair. It should reduce timer reprogramming overhead for scx as well.

Hi Juri,

Quoted 3 lines · Juri Lelli

Hi!

On 17/10/25 11:25, Andrea Righi wrote:

Quoted 14 lines

kernel/sched/ext.c static bool dequeue_task_scx(struct rq *rq, struct task_struct *p, int deq_flags · line 1506

Quoted 9 lines

kernel/sched/ext.c +5 -0

@@ -1487,6 +1499,11 @@ static bool dequeue_task_scx(struct rq *rq, struct task_struct *p, int deq_flags
 	sub_nr_running(rq, 1);
 
 	dispatch_dequeue(rq, p);
+
+	/* Stop the server if this was the last task */
+	if (rq->scx.nr_running == 0)
+		dl_server_stop(&rq->ext_server);
+

Do we want to use the delayed stop behavior for scx-server as we have for fair-server? Wonder if it's a matter of removing this explicit stop and wait for a full period to elapse as we do for fair. It should reduce timer reprogramming overhead for scx as well.

So, IIUC we could just remove this explicit dl_server_stop() and the server would naturally stop at the end of its current deadline period, if there are still no runnable tasks, right?

In that case it's worth a try.

From: Joel Fernandes <[email protected]>

When a sched_ext server is loaded, tasks in CFS are converted to run in sched_ext class. Add support to modify the ext server parameters similar to how the fair server parameters are modified.

Re-use common code between ext and fair servers as needed.

[ arighi: Use dl_se->dl_server to determine if dl_se is a DL server, as
suggested by PeterZ. ]

Co-developed-by Andrea Righi <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

---

kernel/sched/debug.c +4 -4

@@ -373,25 +375,25 @@ static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu
 		}
 
 		if (runtime > period ||
-		    period > fair_server_period_max ||
-		    period < fair_server_period_min) {
+		    period > dl_server_period_max ||
+		    period < dl_server_period_min) {
 			return  -EINVAL;
 		}
 
-		is_active = dl_server_active(&rq->fair_server);
+		is_active = dl_server_active(dl_se);
 		if (is_active) {
 			update_rq_clock(rq);
-			dl_server_stop(&rq->fair_server);
+			dl_server_stop(dl_se);

Hi!

Quoted 15 lines · Andrea Righi

From: Joel Fernandes <[email protected]>

When a sched_ext server is loaded, tasks in CFS are converted to run in sched_ext class. Add support to modify the ext server parameters similar to how the fair server parameters are modified.

Re-use common code between ext and fair servers as needed.

[ arighi: Use dl_se->dl_server to determine if dl_se is a DL server, as
suggested by PeterZ. ]

Co-developed-by Andrea Righi <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

---

...

Quoted 22 lines

kernel/sched/debug.c +5 -5

@@ -373,25 +375,25 @@ static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu
 		}
 
 		if (runtime > period ||
-		    period > fair_server_period_max ||
-		    period < fair_server_period_min) {
+		    period > dl_server_period_max ||
+		    period < dl_server_period_min) {
 			return  -EINVAL;
 		}
 
-		is_active = dl_server_active(&rq->fair_server);
+		is_active = dl_server_active(dl_se);
 		if (is_active) {
 			update_rq_clock(rq);
-			dl_server_stop(&rq->fair_server);
+			dl_server_stop(dl_se);
 		}
 
-		retval = dl_server_apply_params(&rq->fair_server, runtime, period, 0);
+		retval = dl_server_apply_params(dl_se, runtime, period, 0);
 

kernel/sched/debug.c static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu · line 393

Quoted 5 lines

kernel/sched/debug.c +2 -2

 		if (!runtime)
-			printk_deferred("Fair server disabled in CPU %d, system may crash due to starvation.\n",
-					cpu_of(rq));
+			printk_deferred("%s server disabled on CPU %d, system may crash due to starvation.\n",
+					server == &rq->fair_server ? "Fair" : "Ext", cpu_of(rq));

Guess this might get convoluted if are ever going to add an additional dl-server, but I fail to see that happening atm (to service what?).

Reviewed-by Juri Lelli <[email protected]>

Thanks,
Juri

During switching from sched_ext to FAIR tasks and vice-versa, we need support for removing the bandwidth contribution of either DL server. Add support for the same.

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

---
 kernel/sched/deadline.c | 31 +++++++++++++++++++++++++++++++
 kernel/sched/sched.h    |  1 +
 2 files changed, 32 insertions(+)

kernel/sched/deadline.c +20 -0

index 3c1fd2190949e..d585be4014427 100644
@@ -1684,6 +1684,12 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 		dl_rq_change_utilization(rq, dl_se, new_bw);
 	}
 
+	/* Clear these so that the dl_server is reinitialized */
+	if (new_bw == 0) {
+		dl_se->dl_defer = 0;
+		dl_se->dl_server = 0;
+	}
+
 	dl_se->dl_runtime = runtime;
 	dl_se->dl_deadline = period;
 	dl_se->dl_period = period;

@@ -1697,6 +1703,31 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio
 	return retval;
 }
 
+/**
+ * dl_server_remove_params - Remove bandwidth reservation for a DL server
+ * @dl_se: The DL server entity to remove bandwidth for
+ *
+ * This function removes the bandwidth reservation for a DL server entity,
+ * cleaning up all bandwidth accounting and server state.
+ *
+ * Returns: 0 on success, negative error code on failure
+ */
+int dl_server_remove_params(struct sched_dl_entity *dl_se)
+{
+	if (!dl_se->dl_server)
+		return 0; /* Already disabled */
+

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

---

 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

tools/testing/selftests/sched_ext/Makefile +1 -0

index 5fe45f9c5f8fd..c9255d1499b6e 100644
@@ -183,6 +183,7 @@ auto-test-targets :=                        \
        select_cpu_dispatch_bad_dsq     \
        select_cpu_dispatch_dbl_dsp     \
        select_cpu_vtime                \
+       rt_stall                        \
        test_example                    \
 
 testcase-targets := $(addsuffix .o,$(addprefix $(SCXOBJ_DIR)/,$(auto-test-targets)))

tools/testing/selftests/sched_ext/rt_stall.bpf.c new file +23 -0

new file mode 100644
index 0000000000000..80086779dd1eb
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A scheduler that verified if RT tasks can stall SCHED_EXT tasks.
+ *
+ * Copyright (c) 2025 NVIDIA Corporation.
+ */
+
+#include <scx/common.bpf.h>
+
+char _license[] SEC("license") = "GPL";
+
+UEI_DEFINE(uei);
+
+void BPF_STRUCT_OPS(rt_stall_exit, struct scx_exit_info *ei)
+{
+       UEI_RECORD(uei, ei);
+}
+
+SEC(".struct_ops.link")
+struct sched_ext_ops rt_stall_ops = {
+       .exit                   = (void *)rt_stall_exit,
+       .name                   = "rt_stall",
+};

tools/testing/selftests/sched_ext/rt_stall.c new file +103 -0

new file mode 100644
index 0000000000000..e9a0def9ee323
@@ -0,0 +1,214 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 NVIDIA Corporation.
+ */
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sched.h>
+#include <sys/prctl.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <linux/sched.h>
+#include <signal.h>
+#include <bpf/bpf.h>
+#include <scx/common.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#include "rt_stall.bpf.skel.h"
+#include "scx_test.h"
+#include "../kselftest.h"
+
+#define CORE_ID                0       /* CPU to pin tasks to */
+#define RUN_TIME        5      /* How long to run the test in seconds */
+
+/* Simple busy-wait function for test tasks */
+static void process_func(void)
+{
+       while (1) {
+               /* Busy wait */
+               for (volatile unsigned long i = 0; i < 10000000UL; i++)
+                       ;
+       }
+}
+
+/* Set CPU affinity to a specific core */
+static void set_affinity(int cpu)
+{
+       cpu_set_t mask;
+
+       CPU_ZERO(&mask);
+       CPU_SET(cpu, &mask);
+       if (sched_setaffinity(0, sizeof(mask), &mask) != 0) {
+               perror("sched_setaffinity");
+               exit(EXIT_FAILURE);
+       }
+}
+
+/* Set task scheduling policy and priority */
+static void set_sched(int policy, int priority)
+{
+       struct sched_param param;
+
+       param.sched_priority = priority;
+       if (sched_setscheduler(0, policy, &param) != 0) {
+               perror("sched_setscheduler");
+               exit(EXIT_FAILURE);
+       }
+}
+
+/* Get process runtime from /proc/<pid>/stat */
+static float get_process_runtime(int pid)
+{
+       char path[256];
+       FILE *file;
+       long utime, stime;
+       int fields;
+
+       snprintf(path, sizeof(path), "/proc/%d/stat", pid);
+       file = fopen(path, "r");
+       if (file == NULL) {
+               perror("Failed to open stat file");
+               return -1;
+       }
+
+       /* Skip the first 13 fields and read the 14th and 15th */
+       fields = fscanf(file,
+                       "%*d %*s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %lu %lu",
+                       &utime, &stime);
+       fclose(file);
+
+       if (fields != 2) {
+               fprintf(stderr, "Failed to read stat file\n");
+               return -1;
+       }
+
+       /* Calculate the total time spent in the process */
+       long total_time = utime + stime;
+       long ticks_per_second = sysconf(_SC_CLK_TCK);
+       float runtime_seconds = total_time * 1.0 / ticks_per_second;
+
+       return runtime_seconds;
+}
+
+static enum scx_test_status setup(void **ctx)
+{
+       struct rt_stall *skel;
+
+       skel = rt_stall__open();
+       SCX_FAIL_IF(!skel, "Failed to open");
+       SCX_ENUM_INIT(skel);
+       SCX_FAIL_IF(rt_stall__load(skel), "Failed to load skel");

tools/testing/selftests/sched_ext/rt_stall.c new file +35 -0

+
+       ksft_print_header();
+       ksft_set_plan(1);
+
+       /* Create and set up a EXT task */
+       cfs_pid = fork();
+       if (cfs_pid == 0) {
+               set_affinity(CORE_ID);
+               process_func();
+               exit(0);
+       } else if (cfs_pid < 0) {
+               perror("fork for EXT task");
+               ksft_exit_fail();
+       }
+
+       /* Create an RT task */
+       rt_pid = fork();
+       if (rt_pid == 0) {
+               set_affinity(CORE_ID);
+               set_sched(SCHED_FIFO, 50);
+               process_func();
+               exit(0);
+       } else if (rt_pid < 0) {
+               perror("fork for RT task");
+               ksft_exit_fail();
+       }
+
+       /* Let the processes run for the specified time */
+       sleep(RUN_TIME);
+
+       /* Get runtime for the EXT task */
+       cfs_runtime = get_process_runtime(cfs_pid);
+       if (cfs_runtime != -1)
+               ksft_print_msg("Runtime of EXT task (PID %d) is %f seconds\n",
+                              cfs_pid, cfs_runtime);

tools/testing/selftests/sched_ext/rt_stall.c new file +55 -1

+       /* Kill the processes */
+       kill(cfs_pid, SIGKILL);
+       kill(rt_pid, SIGKILL);
+       waitpid(cfs_pid, NULL, 0);
+       waitpid(rt_pid, NULL, 0);
+
+       /* Verify that the scx task got enough runtime */
+       actual_ratio = cfs_runtime / (cfs_runtime + rt_runtime);
+       ksft_print_msg("EXT task got %.2f%% of total runtime\n", actual_ratio * 100);
+
+       if (actual_ratio >= expected_min_ratio) {
+               ksft_test_result_pass("PASS: EXT task got more than %.2f%% of runtime\n",
+                                     expected_min_ratio * 100);
+               return true;
+       }
+       ksft_test_result_fail("FAIL: EXT task got less than %.2f%% of runtime\n",
+                             expected_min_ratio * 100);
+       return false;
+}
+
+static enum scx_test_status run(void *ctx)
+{
+       struct rt_stall *skel = ctx;
+       struct bpf_link *link;
+       bool res;
+
+       link = bpf_map__attach_struct_ops(skel->maps.rt_stall_ops);
+       SCX_FAIL_IF(!link, "Failed to attach scheduler");
+
+       res = sched_stress_test();
+
+       SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_NONE));
+       bpf_link__destroy(link);
+
+       if (!res)
+               ksft_exit_fail();
+
+       return SCX_TEST_PASS;
+}
+
+static void cleanup(void *ctx)
+{
+       struct rt_stall *skel = ctx;
+
+       rt_stall__destroy(skel);
+}
+
+struct scx_test rt_stall = {
+       .name = "rt_stall",
+       .description = "Verify that RT tasks cannot stall SCHED_EXT tasks",
+       .setup = setup,
+       .run = run,
+       .cleanup = cleanup,
+};
+REGISTER_SCX_TEST(&rt_stall)
--

2.51.0

Quoted 10 lines · Andrea Righi

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

---

Nits listed below, but otherwise:

Reviewed-by Emil Tsalapatis <[email protected]>

Code review aside, on my VM the test alternates between 4.81% and 5.20% for me so it's working as expected.

Quoted 158 lines

 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

tools/testing/selftests/sched_ext/Makefile +1 -0

index 5fe45f9c5f8fd..c9255d1499b6e 100644
@@ -183,6 +183,7 @@ auto-test-targets :=                        \
        select_cpu_dispatch_bad_dsq     \
        select_cpu_dispatch_dbl_dsp     \
        select_cpu_vtime                \
+       rt_stall                        \
        test_example                    \
 
 testcase-targets := $(addsuffix .o,$(addprefix $(SCXOBJ_DIR)/,$(auto-test-targets)))

tools/testing/selftests/sched_ext/rt_stall.bpf.c new file +23 -0

new file mode 100644
index 0000000000000..80086779dd1eb
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A scheduler that verified if RT tasks can stall SCHED_EXT tasks.
+ *
+ * Copyright (c) 2025 NVIDIA Corporation.
+ */
+
+#include <scx/common.bpf.h>
+
+char _license[] SEC("license") = "GPL";
+
+UEI_DEFINE(uei);
+
+void BPF_STRUCT_OPS(rt_stall_exit, struct scx_exit_info *ei)
+{
+       UEI_RECORD(uei, ei);
+}
+
+SEC(".struct_ops.link")
+struct sched_ext_ops rt_stall_ops = {
+       .exit                   = (void *)rt_stall_exit,
+       .name                   = "rt_stall",
+};

tools/testing/selftests/sched_ext/rt_stall.c new file +103 -0

new file mode 100644
index 0000000000000..e9a0def9ee323
@@ -0,0 +1,214 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 NVIDIA Corporation.
+ */
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sched.h>
+#include <sys/prctl.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <linux/sched.h>
+#include <signal.h>
+#include <bpf/bpf.h>
+#include <scx/common.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#include "rt_stall.bpf.skel.h"
+#include "scx_test.h"
+#include "../kselftest.h"
+
+#define CORE_ID                0       /* CPU to pin tasks to */
+#define RUN_TIME        5      /* How long to run the test in seconds */
+
+/* Simple busy-wait function for test tasks */
+static void process_func(void)
+{
+       while (1) {
+               /* Busy wait */
+               for (volatile unsigned long i = 0; i < 10000000UL; i++)
+                       ;
+       }
+}
+
+/* Set CPU affinity to a specific core */
+static void set_affinity(int cpu)
+{
+       cpu_set_t mask;
+
+       CPU_ZERO(&mask);
+       CPU_SET(cpu, &mask);
+       if (sched_setaffinity(0, sizeof(mask), &mask) != 0) {
+               perror("sched_setaffinity");
+               exit(EXIT_FAILURE);
+       }
+}
+
+/* Set task scheduling policy and priority */
+static void set_sched(int policy, int priority)
+{
+       struct sched_param param;
+
+       param.sched_priority = priority;
+       if (sched_setscheduler(0, policy, &param) != 0) {
+               perror("sched_setscheduler");
+               exit(EXIT_FAILURE);
+       }
+}
+
+/* Get process runtime from /proc/<pid>/stat */
+static float get_process_runtime(int pid)
+{
+       char path[256];
+       FILE *file;
+       long utime, stime;
+       int fields;
+
+       snprintf(path, sizeof(path), "/proc/%d/stat", pid);
+       file = fopen(path, "r");
+       if (file == NULL) {
+               perror("Failed to open stat file");
+               return -1;
+       }
+
+       /* Skip the first 13 fields and read the 14th and 15th */
+       fields = fscanf(file,
+                       "%*d %*s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %lu %lu",
+                       &utime, &stime);
+       fclose(file);
+
+       if (fields != 2) {
+               fprintf(stderr, "Failed to read stat file\n");
+               return -1;
+       }
+
+       /* Calculate the total time spent in the process */
+       long total_time = utime + stime;
+       long ticks_per_second = sysconf(_SC_CLK_TCK);
+       float runtime_seconds = total_time * 1.0 / ticks_per_second;
+
+       return runtime_seconds;
+}
+
+static enum scx_test_status setup(void **ctx)
+{
+       struct rt_stall *skel;
+
+       skel = rt_stall__open();
+       SCX_FAIL_IF(!skel, "Failed to open");
+       SCX_ENUM_INIT(skel);
+       SCX_FAIL_IF(rt_stall__load(skel), "Failed to load skel");

tools/testing/selftests/sched_ext/rt_stall.c line 113

Quoted 10 lines

tools/testing/selftests/sched_ext/rt_stall.c new file +10 -0

+
+       *ctx = skel;
+
+       return SCX_TEST_PASS;
+}
+
+static bool sched_stress_test(void)
+{
+       float cfs_runtime, rt_runtime, actual_ratio;
+       int cfs_pid, rt_pid;

I think it should be cfs_pid -> ext_pid, cfs_runtime -> ext_runtime

tools/testing/selftests/sched_ext/rt_stall.c line 114

Quoted 1 line

tools/testing/selftests/sched_ext/rt_stall.c new file +1 -0

+       float expected_min_ratio = 0.04; /* 4% */

Maybe add a comment that explains the 4% value? As in, we're expecting it to be around 5% so 0.04 accounts for values close enough but below < 5%.

Quoted 36 lines

tools/testing/selftests/sched_ext/rt_stall.c new file +35 -0

+
+       ksft_print_header();
+       ksft_set_plan(1);
+
+       /* Create and set up a EXT task */
+       cfs_pid = fork();
+       if (cfs_pid == 0) {
+               set_affinity(CORE_ID);
+               process_func();
+               exit(0);
+       } else if (cfs_pid < 0) {
+               perror("fork for EXT task");
+               ksft_exit_fail();
+       }
+
+       /* Create an RT task */
+       rt_pid = fork();
+       if (rt_pid == 0) {
+               set_affinity(CORE_ID);
+               set_sched(SCHED_FIFO, 50);
+               process_func();
+               exit(0);
+       } else if (rt_pid < 0) {
+               perror("fork for RT task");
+               ksft_exit_fail();
+       }
+
+       /* Let the processes run for the specified time */
+       sleep(RUN_TIME);
+
+       /* Get runtime for the EXT task */
+       cfs_runtime = get_process_runtime(cfs_pid);
+       if (cfs_runtime != -1)
+               ksft_print_msg("Runtime of EXT task (PID %d) is %f seconds\n",
+                              cfs_pid, cfs_runtime);

tools/testing/selftests/sched_ext/rt_stall.c line 159

Quoted 10 lines

tools/testing/selftests/sched_ext/rt_stall.c new file +10 -0

+       else
+               ksft_exit_fail_msg("Error getting runtime for EXT task (PID %d)\n", cfs_pid);
+
+       /* Get runtime for the RT task */
+       rt_runtime = get_process_runtime(rt_pid);
+       if (rt_runtime != -1)
+               ksft_print_msg("Runtime of RT task (PID %d) is %f seconds\n", rt_pid, rt_runtime);
+       else
+               ksft_exit_fail_msg("Error getting runtime for RT task (PID %d)\n", rt_pid);
+

Minor, but why not

if (rt_runtime == -1)
ksft_exit_fail_msg("Error getting runtime for RT task (PID
%d)\n", rt_pid);
ksft_print_msg("Runtime of RT task (PID %d) is %f seconds\n", rt_pid, rt_runtime);

since ksft_exit_fail_msg never returns?

Quoted 52 lines

tools/testing/selftests/sched_ext/rt_stall.c new file +51 -0

+       /* Kill the processes */
+       kill(cfs_pid, SIGKILL);
+       kill(rt_pid, SIGKILL);
+       waitpid(cfs_pid, NULL, 0);
+       waitpid(rt_pid, NULL, 0);
+
+       /* Verify that the scx task got enough runtime */
+       actual_ratio = cfs_runtime / (cfs_runtime + rt_runtime);
+       ksft_print_msg("EXT task got %.2f%% of total runtime\n", actual_ratio * 100);
+
+       if (actual_ratio >= expected_min_ratio) {
+               ksft_test_result_pass("PASS: EXT task got more than %.2f%% of runtime\n",
+                                     expected_min_ratio * 100);
+               return true;
+       }
+       ksft_test_result_fail("FAIL: EXT task got less than %.2f%% of runtime\n",
+                             expected_min_ratio * 100);
+       return false;
+}
+
+static enum scx_test_status run(void *ctx)
+{
+       struct rt_stall *skel = ctx;
+       struct bpf_link *link;
+       bool res;
+
+       link = bpf_map__attach_struct_ops(skel->maps.rt_stall_ops);
+       SCX_FAIL_IF(!link, "Failed to attach scheduler");
+
+       res = sched_stress_test();
+
+       SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_NONE));
+       bpf_link__destroy(link);
+
+       if (!res)
+               ksft_exit_fail();
+
+       return SCX_TEST_PASS;
+}
+
+static void cleanup(void *ctx)
+{
+       struct rt_stall *skel = ctx;
+
+       rt_stall__destroy(skel);
+}
+
+struct scx_test rt_stall = {
+       .name = "rt_stall",
+       .description = "Verify that RT tasks cannot stall SCHED_EXT tasks",
+       .setup = setup,

Quoted 8 lines

tools/testing/selftests/sched_ext/rt_stall.c new file +4 -1

+       .run = run,
+       .cleanup = cleanup,
+};
+REGISTER_SCX_TEST(&rt_stall)
--

2.51.0

Hi Emil,

Quoted 6 lines

tools/testing/selftests/sched_ext/rt_stall.c line 113

Quoted 4 lines

tools/testing/selftests/sched_ext/rt_stall.c new file +4 -0

+static bool sched_stress_test(void)
+{
+       float cfs_runtime, rt_runtime, actual_ratio;
+       int cfs_pid, rt_pid;

I think it should be cfs_pid -> ext_pid, cfs_runtime -> ext_runtime

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

Quoted 8 lines · Andrea Righi

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Quoted 7 lines

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

Does this pass consistently for you?
For a loop of 1000 runs I'm getting total runtime numbers for the EXT task of:

   0.000 -    0.261 |  (7)
   0.261 -    0.522 | ###### (86)
   0.522 -    4.437 |  (0)
   4.437 -    4.698 |  (1)
   4.698 -    4.959 | ################### (257)
   4.959 -    5.220 | ################################################## (649)

I'll try to see what's going wrong here...

Hi Christian,

Quoted 28 lines · Christian Loehle

Quoted 8 lines · Andrea Righi

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Quoted 7 lines

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

Does this pass consistently for you?
For a loop of 1000 runs I'm getting total runtime numbers for the EXT task of:

   0.000 -    0.261 |  (7)
   0.261 -    0.522 | ###### (86)
   0.522 -    4.437 |  (0)
   4.437 -    4.698 |  (1)
   4.698 -    4.959 | ################### (257)
   4.959 -    5.220 | ################################################## (649)

Quoted 1 line

I'll try to see what's going wrong here...

Is that 1000 runs of total_bw? Yeah, the small ones don't look right at

all, unless they're caused by some errors in the measurement (or something wrong in the test itself). Still better than without the dl_server, but it'd be nice to understand what's going on. :)

I'll try to reproduce that on my side as well.

Thanks,
-Andrea

Hi Christian,

Quoted 28 lines · Christian Loehle

Quoted 8 lines · Andrea Righi

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Quoted 7 lines

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

Does this pass consistently for you?
For a loop of 1000 runs I'm getting total runtime numbers for the EXT task of:

   0.000 -    0.261 |  (7)
   0.261 -    0.522 | ###### (86)
   0.522 -    4.437 |  (0)
   4.437 -    4.698 |  (1)
   4.698 -    4.959 | ################### (257)
   4.959 -    5.220 | ################################################## (649)

Quoted 1 line

I'll try to see what's going wrong here...

Is that 1000 runs of total_bw? Yeah, the small ones don't look right at all, unless they're caused by some errors in the measurement (or something wrong in the test itself). Still better than without the dl_server, but it'd be nice to understand what's going on. :)

I'll try to reproduce that on my side as well.

Quoted 38 lines · Andrea Righi

Hi Christian,

Quoted 21 lines · Christian Loehle

Quoted 8 lines · Andrea Righi

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Quoted 7 lines

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

Does this pass consistently for you?
For a loop of 1000 runs I'm getting total runtime numbers for the EXT task of:

Quoted 8 lines

   0.000 -    0.261 |  (7)
   0.261 -    0.522 | ###### (86)
   0.522 -    4.437 |  (0)
   4.437 -    4.698 |  (1)
   4.698 -    4.959 | ################### (257)
   4.959 -    5.220 | ################################################## (649)

I'll try to see what's going wrong here...

Is that 1000 runs of total_bw? Yeah, the small ones don't look right at all, unless they're caused by some errors in the measurement (or something wrong in the test itself). Still better than without the dl_server, but it'd be nice to understand what's going on. :)

Quoted 2 lines

I'll try to reproduce that on my side as well.

Yes it's pretty much
for i in $(seq 0 999); do ./runner -t rt_stall ; sleep 10; done

I also tried to increase the runtime of the test, but results look the same so I assume the DL server isn't running in the fail cases.

Quoted 42 lines · Christian Loehle

Quoted 40 lines · Andrea Righi

Hi Christian,

Quoted 21 lines · Christian Loehle

Quoted 8 lines · Andrea Righi

Add a selftest to validate the correct behavior of the deadline server for the ext_sched_class.

[ Joel: Replaced occurences of CFS in the test with EXT. ]

Co-developed-by Joel Fernandes <[email protected]>

Signed-off-by Joel Fernandes <[email protected]>

Signed-off-by Andrea Righi <[email protected]>

Quoted 7 lines

---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/rt_stall.bpf.c        |  23 ++
 tools/testing/selftests/sched_ext/rt_stall.c  | 214 ++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/rt_stall.c

Does this pass consistently for you?
For a loop of 1000 runs I'm getting total runtime numbers for the EXT task of:

Quoted 8 lines

   0.000 -    0.261 |  (7)
   0.261 -    0.522 | ###### (86)
   0.522 -    4.437 |  (0)
   4.437 -    4.698 |  (1)
   4.698 -    4.959 | ################### (257)
   4.959 -    5.220 | ################################################## (649)

I'll try to see what's going wrong here...

Is that 1000 runs of total_bw? Yeah, the small ones don't look right at all, unless they're caused by some errors in the measurement (or something wrong in the test itself). Still better than without the dl_server, but it'd be nice to understand what's going on. :)

I'll try to reproduce that on my side as well.

Quoted 6 lines

Yes it's pretty much
for i in $(seq 0 999); do ./runner -t rt_stall ; sleep 10; done

I also tried to increase the runtime of the test, but results look the same so I assume the DL server isn't running in the fail cases.

FWIW the below fixes the issue and also explains why runtime of the test was irrelevant.

I wonder if we should let the test do FAIR->EXT->FAIR->EXT or something like that, the change would be minimal and coverage improved significantly IMO.

-----8<-----

kernel/sched/ext.c +2 -0

index c5f3c39972b6..ed48c681c4c2 100644
@@ -2568,6 +2568,8 @@ static void dl_server_on(struct rq *rq, bool switch_all)
 
        err = dl_server_init_params(&rq->ext_server);
        WARN_ON_ONCE(err);
+       if (rq->scx.nr_running)
+               dl_server_start(&rq->ext_server);
 
        rq_unlock_irqrestore(rq, &rf);
 }