Skip to contentSkip to conversation

sched/rt: check if curr can be pushed/pulled somewhere else in advance

PATCH Superseded Wanpeng Li 4 participants 19 messages Oct 27, 2014 – Oct 31, 2014 Scheduler

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Wanpeng Li

author
#0 lore raw

PATCH 1/6 sched/rt: check if curr can be pushed/pulled somewhere else in advance

This patch checks if current can be pushed/pulled somewhere else in advance to make logic clear.

- If current can't be migrated, useless to reschedule, let's hope task can move out.
- If task is migratable, so let's not schedule it and see if it can be pushed or pulled somewhere else.

---
 kernel/sched/rt.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
kernel/sched/rt.c +10 -4
index 87ea5bf..ca1b7c7 100644
@@ -1351,16 +1351,22 @@ out:
 
 static void check_preempt_equal_prio(struct rq *rq, struct task_struct *p)
 {
-	if (rq->curr->nr_cpus_allowed == 1)
+	/*
+	 * Current can't be migrated, useless to reschedule,
+	 * let's hope p can move out.
+	 */
+	if (rq->curr->nr_cpus_allowed == 1 ||
+		!cpupri_find(&rq->rd->cpupri, rq->curr, NULL))
 		return;
 
+	/*
+	 * p is migratable, so let's not schedule it and
+	 * see if it is pushed or pulled somewhere else.
+	 */
 	if (p->nr_cpus_allowed != 1
 	    && cpupri_find(&rq->rd->cpupri, p, NULL))
 		return;
 
-	if (!cpupri_find(&rq->rd->cpupri, rq->curr, NULL))
-		return;
-
 	/*
 	 * There appears to be other cpus that can accept
 	 * current and none to run 'p', so lets reschedule
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH 2/6 sched/dl: fix yield task artificial overrun

The yield semantic of deadline class is to reduce remaining runtime to zero, and then update_curr_dl() will stop it. However, comsumed bandwidth is reduced from the budget of yield task again even if it has already been set to zero which leads to artificial overrun. This patch fix it by reduce remaining runtime to zero if there is still remaining runtime after comsumed bandwidth is accumulated.

---
 kernel/sched/deadline.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
kernel/sched/deadline.c +7 -4
index abfaf3d..d4ffc1e 100644
@@ -568,7 +568,12 @@ static
 int dl_runtime_exceeded(struct rq *rq, struct sched_dl_entity *dl_se)
 {
 	int dmiss = dl_time_before(dl_se->deadline, rq_clock(rq));
-	int rorun = dl_se->runtime <= 0;
+	int rorun;
+
+	if (dl_se->dl_yielded && dl_se->runtime > 0)
+		dl_se->runtime = 0;
+
+	rorun = dl_se->runtime <= 0;
 
 	if (!rorun && !dmiss)
 		return 0;
@@ -897,10 +902,8 @@ static void yield_task_dl(struct rq *rq)
 	 * it and the bandwidth timer will wake it up and will give it
 	 * new scheduling parameters (thanks to dl_yielded=1).
 	 */
-	if (p->dl.runtime > 0) {
+	if (p->dl.runtime > 0)
 		rq->curr->dl.dl_yielded = 1;
-		p->dl.runtime = 0;
-	}
 	update_curr_dl(rq);
 }
 
Signature · 1 line
1.9.1
#2 lore raw

Re: [PATCH 2/6] sched/dl: fix yield task artificial overrun

Hi,

Quoted 7 lines · Wanpeng Li

The yield semantic of deadline class is to reduce remaining runtime to zero, and then update_curr_dl() will stop it. However, comsumed bandwidth is reduced from the budget of yield task again even if it has already been set to zero which leads to artificial overrun. This patch fix it by reduce remaining runtime to zero if there is still remaining runtime after comsumed bandwidth is accumulated.

Oh, right. But, how about what below instead (with a proper comment and changelog)?

Thanks,

- Juri

Quoted 1 line

From 108ecdff52b154ea2c79d4aac552ddf1ead871c7 Mon Sep 17 00:00:00 2001

From: Juri Lelli <[email protected]>
Date: Wed, 29 Oct 2014 16:09:06 +0000
Subject: [PATCH] sched/deadline: fix artificial overrun introduced by
yield_task_dl

---
 kernel/sched/deadline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/deadline.c +1 -1
index 2e31a30..db6ad38 100644
@@ -633,7 +633,7 @@ static void update_curr_dl(struct rq *rq)
 
 	sched_rt_avg_update(rq, delta_exec);
 
-	dl_se->runtime -= delta_exec;
+	dl_se->runtime -= dl_se->dl_yielded ? 0 : delta_exec;
 	if (dl_runtime_exceeded(rq, dl_se)) {
 		__dequeue_task_dl(rq, curr, 0);
 		if (likely(start_dl_timer(dl_se, curr->dl.dl_boosted)))
Signature · 1 line
2.1.2
Quoted 36 lines
---
 kernel/sched/deadline.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
kernel/sched/deadline.c +7 -4
index abfaf3d..d4ffc1e 100644
@@ -568,7 +568,12 @@ static
 int dl_runtime_exceeded(struct rq *rq, struct sched_dl_entity *dl_se)
 {
 	int dmiss = dl_time_before(dl_se->deadline, rq_clock(rq));
-	int rorun = dl_se->runtime <= 0;
+	int rorun;
+
+	if (dl_se->dl_yielded && dl_se->runtime > 0)
+		dl_se->runtime = 0;
+
+	rorun = dl_se->runtime <= 0;
 
 	if (!rorun && !dmiss)
 		return 0;
@@ -897,10 +902,8 @@ static void yield_task_dl(struct rq *rq)
 	 * it and the bandwidth timer will wake it up and will give it
 	 * new scheduling parameters (thanks to dl_yielded=1).
 	 */
-	if (p->dl.runtime > 0) {
+	if (p->dl.runtime > 0)
 		rq->curr->dl.dl_yielded = 1;
-		p->dl.runtime = 0;
-	}
 	update_curr_dl(rq);
 }
 
#3 lore raw

Re: [PATCH 2/6] sched/dl: fix yield task artificial overrun

Hi Juri,
2014/10/30 0:33, Juri Lelli:

Quoted 33 lines

Hi,

Quoted 7 lines · Wanpeng Li

The yield semantic of deadline class is to reduce remaining runtime to zero, and then update_curr_dl() will stop it. However, comsumed bandwidth is reduced from the budget of yield task again even if it has already been set to zero which leads to artificial overrun. This patch fix it by reduce remaining runtime to zero if there is still remaining runtime after comsumed bandwidth is accumulated.

Oh, right. But, how about what below instead (with a proper comment and changelog)?

Thanks,

- Juri

From 108ecdff52b154ea2c79d4aac552ddf1ead871c7 Mon Sep 17 00:00:00 2001
From: Juri Lelli <[email protected]>
Date: Wed, 29 Oct 2014 16:09:06 +0000
Subject: [PATCH] sched/deadline: fix artificial overrun introduced by
yield_task_dl

---
  kernel/sched/deadline.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/deadline.c +0 -0
index 2e31a30..db6ad38 100644

kernel/sched/deadline.c static void update_curr_dl(struct rq *rq) · line 636

Quoted 5 lines
kernel/sched/deadline.c +1 -1
@@ -633,7 +633,7 @@ static void update_curr_dl(struct rq *rq)
 
  	sched_rt_avg_update(rq, delta_exec);
 
-	dl_se->runtime -= delta_exec;
+	dl_se->runtime -= dl_se->dl_yielded ? 0 : delta_exec;

+ if (dl_se->dl_yielded && dl_se->runtime > 0)
+ dl_se->runtime = 0;

Maybe this can be moved to update_curr_dl().

I think the consumed bandwidth still should be reduced from remaining runtime even if yield, then the remaining runtime will be reset to 0 if there is still remaining runtime as what my patch do. What's your option?

Regards,
Wanpeng Li

Quoted 3 lines
kernel/sched/deadline.c +0 -0
  	if (dl_runtime_exceeded(rq, dl_se)) {
  		__dequeue_task_dl(rq, curr, 0);
  		if (likely(start_dl_timer(dl_se, curr->dl.dl_boosted)))
#4 lore raw

Re: [PATCH 2/6] sched/dl: fix yield task artificial overrun

Hi,

Quoted 42 lines · Wanpeng Li

Hi Juri,
2014/10/30 0:33, Juri Lelli:

Quoted 38 lines

Hi,

Quoted 7 lines · Wanpeng Li

The yield semantic of deadline class is to reduce remaining runtime to zero, and then update_curr_dl() will stop it. However, comsumed bandwidth is reduced from the budget of yield task again even if it has already been set to zero which leads to artificial overrun. This patch fix it by reduce remaining runtime to zero if there is still remaining runtime after comsumed bandwidth is accumulated.

Oh, right. But, how about what below instead (with a proper comment and changelog)?

Thanks,

- Juri

From 108ecdff52b154ea2c79d4aac552ddf1ead871c7 Mon Sep 17 00:00:00 2001
From: Juri Lelli <[email protected]>
Date: Wed, 29 Oct 2014 16:09:06 +0000
Subject: [PATCH] sched/deadline: fix artificial overrun introduced by
yield_task_dl

---
  kernel/sched/deadline.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/deadline.c +1 -1
index 2e31a30..db6ad38 100644
@@ -633,7 +633,7 @@ static void update_curr_dl(struct rq *rq)
 
  	sched_rt_avg_update(rq, delta_exec);
 
-	dl_se->runtime -= delta_exec;
+	dl_se->runtime -= dl_se->dl_yielded ? 0 : delta_exec;

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (dl_se->dl_yielded && dl_se->runtime > 0)
+		dl_se->runtime = 0;

We already did this in yield_task_dl(). In update_curr_dl() we now just have to make sure we don't steal some more runtime from the task that yielded (but we do all other accounting).

Thanks,

- Juri

Quoted 14 lines

Maybe this can be moved to update_curr_dl().

I think the consumed bandwidth still should be reduced from remaining runtime even if yield, then the remaining runtime will be reset to 0 if there is still remaining runtime as what my patch do. What's your option?

Regards,
Wanpeng Li

Quoted 3 lines
kernel/sched/deadline.c +0 -0
  	if (dl_runtime_exceeded(rq, dl_se)) {
  		__dequeue_task_dl(rq, curr, 0);
  		if (likely(start_dl_timer(dl_se, curr->dl.dl_boosted)))
author
#5 lore raw

PATCH 3/6 sched/dl: add deadline rq status print

In reply to Wanpeng Li

This patch add deadline rq status print.

---
 kernel/sched/deadline.c | 9 +++++++++
 kernel/sched/debug.c    | 7 +++++++
 kernel/sched/sched.h    | 1 +
 3 files changed, 17 insertions(+)
kernel/sched/deadline.c +9 -0
index d4ffc1e..aa7c27f 100644
@@ -1682,3 +1682,12 @@ const struct sched_class dl_sched_class = {
 	.switched_from		= switched_from_dl,
 	.switched_to		= switched_to_dl,
 };
+
+#ifdef CONFIG_SCHED_DEBUG
+extern void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq);
+
+void print_dl_stats(struct seq_file *m, int cpu)
+{
+	print_dl_rq(m, cpu, &cpu_rq(cpu)->dl);
+}
+#endif /* CONFIG_SCHED_DEBUG */
kernel/sched/debug.c +7 -0
index ce33780..eeb6046 100644
@@ -261,6 +261,12 @@ void print_rt_rq(struct seq_file *m, int cpu, struct rt_rq *rt_rq)
 #undef P
 }
 
+void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
+{
+	SEQ_printf(m, "\ndl_rq[%d]:\n", cpu);
+	SEQ_printf(m, "  .%-30s: %ld\n", "dl_nr_running", dl_rq->dl_nr_running);
+}
+
 extern __read_mostly int sched_clock_running;
 
 static void print_cpu(struct seq_file *m, int cpu)
@@ -329,6 +335,7 @@ do {									\
 	spin_lock_irqsave(&sched_debug_lock, flags);
 	print_cfs_stats(m, cpu);
 	print_rt_stats(m, cpu);
+	print_dl_stats(m, cpu);
 
 	print_rq(m, rq, cpu);
 	spin_unlock_irqrestore(&sched_debug_lock, flags);
kernel/sched/sched.h +1 -0
index 6130251..2c2ef3c 100644
@@ -1502,6 +1502,7 @@ extern struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq);
 extern struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq);
 extern void print_cfs_stats(struct seq_file *m, int cpu);
 extern void print_rt_stats(struct seq_file *m, int cpu);
+extern void print_dl_stats(struct seq_file *m, int cpu);
 
 extern void init_cfs_rq(struct cfs_rq *cfs_rq);
 extern void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq);
Signature · 1 line
1.9.1
#6 lore raw

Re: [PATCH 3/6] sched/dl: add deadline rq status print

Hi,

Quoted 2 lines · Wanpeng Li

This patch add deadline rq status print.

Don't know if useful, but we could add this, just to be consistent.

Thanks,

- Juri

Quoted 62 lines
---
 kernel/sched/deadline.c | 9 +++++++++
 kernel/sched/debug.c    | 7 +++++++
 kernel/sched/sched.h    | 1 +
 3 files changed, 17 insertions(+)
kernel/sched/deadline.c +9 -0
index d4ffc1e..aa7c27f 100644
@@ -1682,3 +1682,12 @@ const struct sched_class dl_sched_class = {
 	.switched_from		= switched_from_dl,
 	.switched_to		= switched_to_dl,
 };
+
+#ifdef CONFIG_SCHED_DEBUG
+extern void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq);
+
+void print_dl_stats(struct seq_file *m, int cpu)
+{
+	print_dl_rq(m, cpu, &cpu_rq(cpu)->dl);
+}
+#endif /* CONFIG_SCHED_DEBUG */
kernel/sched/debug.c +7 -0
index ce33780..eeb6046 100644
@@ -261,6 +261,12 @@ void print_rt_rq(struct seq_file *m, int cpu, struct rt_rq *rt_rq)
 #undef P
 }
 
+void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
+{
+	SEQ_printf(m, "\ndl_rq[%d]:\n", cpu);
+	SEQ_printf(m, "  .%-30s: %ld\n", "dl_nr_running", dl_rq->dl_nr_running);
+}
+
 extern __read_mostly int sched_clock_running;
 
 static void print_cpu(struct seq_file *m, int cpu)
@@ -329,6 +335,7 @@ do {									\
 	spin_lock_irqsave(&sched_debug_lock, flags);
 	print_cfs_stats(m, cpu);
 	print_rt_stats(m, cpu);
+	print_dl_stats(m, cpu);
 
 	print_rq(m, rq, cpu);
 	spin_unlock_irqrestore(&sched_debug_lock, flags);
kernel/sched/sched.h +1 -0
index 6130251..2c2ef3c 100644
@@ -1502,6 +1502,7 @@ extern struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq);
 extern struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq);
 extern void print_cfs_stats(struct seq_file *m, int cpu);
 extern void print_rt_stats(struct seq_file *m, int cpu);
+extern void print_dl_stats(struct seq_file *m, int cpu);
 
 extern void init_cfs_rq(struct cfs_rq *cfs_rq);
 extern void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq);
#7 lore raw

Re: [PATCH 3/6] sched/dl: add deadline rq status print

Hi Juri,
2014/10/30 0:52, Juri Lelli:

Quoted 6 lines

Hi,

Quoted 2 lines · Wanpeng Li

This patch add deadline rq status print.

Don't know if useful, but we could add this, just to be consistent.

Yes, to keep it consistent. Maybe more things can be print if they are necessary in the future.

Regards,
Wanpeng Li

Quoted 72 lines

Thanks,

- Juri

Quoted 55 lines
---
  kernel/sched/deadline.c | 9 +++++++++
  kernel/sched/debug.c    | 7 +++++++
  kernel/sched/sched.h    | 1 +
  3 files changed, 17 insertions(+)
kernel/sched/deadline.c +9 -0
index d4ffc1e..aa7c27f 100644
@@ -1682,3 +1682,12 @@ const struct sched_class dl_sched_class = {
  	.switched_from		= switched_from_dl,
  	.switched_to		= switched_to_dl,
  };
+
+#ifdef CONFIG_SCHED_DEBUG
+extern void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq);
+
+void print_dl_stats(struct seq_file *m, int cpu)
+{
+	print_dl_rq(m, cpu, &cpu_rq(cpu)->dl);
+}
+#endif /* CONFIG_SCHED_DEBUG */
kernel/sched/debug.c +7 -0
index ce33780..eeb6046 100644
@@ -261,6 +261,12 @@ void print_rt_rq(struct seq_file *m, int cpu, struct rt_rq *rt_rq)
  #undef P
  }
 
+void print_dl_rq(struct seq_file *m, int cpu, struct dl_rq *dl_rq)
+{
+	SEQ_printf(m, "\ndl_rq[%d]:\n", cpu);
+	SEQ_printf(m, "  .%-30s: %ld\n", "dl_nr_running", dl_rq->dl_nr_running);
+}
+
  extern __read_mostly int sched_clock_running;
 
  static void print_cpu(struct seq_file *m, int cpu)
@@ -329,6 +335,7 @@ do {									\
  	spin_lock_irqsave(&sched_debug_lock, flags);
  	print_cfs_stats(m, cpu);
  	print_rt_stats(m, cpu);
+	print_dl_stats(m, cpu);
 
  	print_rq(m, rq, cpu);
  	spin_unlock_irqrestore(&sched_debug_lock, flags);
kernel/sched/sched.h +0 -0
index 6130251..2c2ef3c 100644

kernel/sched/sched.h extern struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq); · line 1508

Quoted 7 lines
kernel/sched/sched.h +1 -0
@@ -1502,6 +1502,7 @@ extern struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq);
  extern struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq);
  extern void print_cfs_stats(struct seq_file *m, int cpu);
  extern void print_rt_stats(struct seq_file *m, int cpu);
+extern void print_dl_stats(struct seq_file *m, int cpu);
 
  extern void init_cfs_rq(struct cfs_rq *cfs_rq);
  extern void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#8 lore raw

PATCH 4/6 sched/dl: push task away if the deadline is equal to curr during wakeup

In reply to Wanpeng Li

This patch pushes task away if the dealine of the task is equal to current during wake up. The same behavior as rt class.

---
 kernel/sched/deadline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/deadline.c +1 -1
index aa7c27f..97141e2 100644
@@ -1492,7 +1492,7 @@ static void task_woken_dl(struct rq *rq, struct task_struct *p)
 	    p->nr_cpus_allowed > 1 &&
 	    dl_task(rq->curr) &&
 	    (rq->curr->nr_cpus_allowed < 2 ||
-	     dl_entity_preempt(&rq->curr->dl, &p->dl))) {
+	     !dl_entity_preempt(&p->dl, &rq->curr->dl))) {
 		push_dl_tasks(rq);
 	}
 }
Signature · 1 line
1.9.1
#9 lore raw

Re: [PATCH 4/6] sched/dl: push task away if the deadline is equal to curr during wakeup

Hi,

Quoted 3 lines · Wanpeng Li

This patch pushes task away if the dealine of the task is equal to current during wake up. The same behavior as rt class.

Right.

Thanks,

- Juri

Quoted 19 lines
---
 kernel/sched/deadline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/deadline.c +1 -1
index aa7c27f..97141e2 100644
@@ -1492,7 +1492,7 @@ static void task_woken_dl(struct rq *rq, struct task_struct *p)
 	    p->nr_cpus_allowed > 1 &&
 	    dl_task(rq->curr) &&
 	    (rq->curr->nr_cpus_allowed < 2 ||
-	     dl_entity_preempt(&rq->curr->dl, &p->dl))) {
+	     !dl_entity_preempt(&p->dl, &rq->curr->dl))) {
 		push_dl_tasks(rq);
 	}
 }
author
#10 lore raw

PATCH 5/6 sched/dl: reschedule if successfully pull earlier deadline task

In reply to Wanpeng Li

Reschedule if successfully pull earlier deadline task.

---
 kernel/sched/deadline.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +2 -2
index 97141e2..21de865 100644
@@ -1583,8 +1583,8 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
+			resched_curr(rq);
 #endif
 }
 
Signature · 1 line
1.9.1
#11 lore raw

Re: [PATCH 5/6] sched/dl: reschedule if successfully pull earlier deadline task

Hi Juri, could you review 5/6 and 6/6? 5/6 is the same behavior as rt class. ;-) 2014/10/27 9:41, Wanpeng Li:

Quoted 22 lines

Reschedule if successfully pull earlier deadline task.

---
  kernel/sched/deadline.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +2 -2
index 97141e2..21de865 100644
@@ -1583,8 +1583,8 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
  	 * this is the right place to try to pull some other one
  	 * from an overloaded cpu, if any.
  	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
+			resched_curr(rq);
  #endif
  }
 
#12 lore raw

Re: [PATCH 5/6] sched/dl: reschedule if successfully pull earlier deadline task

In reply to Wanpeng Li

Hi,

Quoted 12 lines · Wanpeng Li

Reschedule if successfully pull earlier deadline task.

---
 kernel/sched/deadline.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +0 -0
index 97141e2..21de865 100644

kernel/sched/deadline.c static void switched_from_dl(struct rq *rq, struct task_struct *p) · line 1587

Quoted 7 lines
kernel/sched/deadline.c +2 -2
@@ -1583,8 +1583,8 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
+			resched_curr(rq);

Right, we have to reschedule. But, how about we fully align to -rt?

Peter, is the task_on_rq_queued() check for cases in which p is not enqueued but its class changes after a setscheduler/rt_mutex_setprio?

Thanks,

- Juri

Quoted 1 line

From 404749782e493cb6019193f347163b640ebd552d Mon Sep 17 00:00:00 2001

From: Juri Lelli <[email protected]>
Date: Thu, 30 Oct 2014 10:15:40 +0000
Subject: [PATCH] sched/deadline: reschedule from switched_from_dl() after a
successful pull

In switched_from_dl() we have to issue a resched if we successfully pulled some task from other cpus. This patch also aligns the behaviour with -rt.

---
 kernel/sched/deadline.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +5 -2
index db6ad38..e8943fc 100644
@@ -1623,8 +1623,11 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!task_on_rq_queued(p) || rq->dl.dl_nr_running)
+		return;
+
+	if (pull_dl_task(rq))
+		resched_curr(rq);
 #endif
 }
 
Signature · 1 line
2.1.2
author
#13 lore raw

Re: [PATCH 5/6] sched/dl: reschedule if successfully pull earlier deadline task

Hi Juri,

Quoted 33 lines · Juri Lelli

Hi,

Quoted 12 lines · Wanpeng Li

Reschedule if successfully pull earlier deadline task.

---
 kernel/sched/deadline.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +0 -0
index 97141e2..21de865 100644

kernel/sched/deadline.c static void switched_from_dl(struct rq *rq, struct task_struct *p) · line 1587

Quoted 7 lines
kernel/sched/deadline.c +2 -2
@@ -1583,8 +1583,8 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
+			resched_curr(rq);

Right, we have to reschedule. But, how about we fully align to -rt?

Peter, is the task_on_rq_queued() check for cases in which p is not enqueued but its class changes after a setscheduler/rt_mutex_setprio?

Thanks,

- Juri

Quoted 1 line

From 404749782e493cb6019193f347163b640ebd552d Mon Sep 17 00:00:00 2001

Quoted 9 lines

From: Juri Lelli <[email protected]>
Date: Thu, 30 Oct 2014 10:15:40 +0000
Subject: [PATCH] sched/deadline: reschedule from switched_from_dl() after a successful pull

In switched_from_dl() we have to issue a resched if we successfully pulled some task from other cpus. This patch also aligns the behaviour with -rt.

Good idea.

Quoted 1 line

I will send out version two of this patchset with the update of 2/6 and 5/6 with *your Suggested-by*. Thanks for your great proposal. ;-)

Regards,
Wanpeng Li

Quoted 25 lines

---
kernel/sched/deadline.c | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +5 -2
index db6ad38..e8943fc 100644
@@ -1623,8 +1623,11 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!task_on_rq_queued(p) || rq->dl.dl_nr_running)
+		return;
+
+	if (pull_dl_task(rq))
+		resched_curr(rq);
 #endif
 }
 
Signature · 1 line
2.1.2
#14 lore raw

Re: [PATCH 5/6] sched/dl: reschedule if successfully pull earlier deadline task

Hi,

Quoted 45 lines · Wanpeng Li

Hi Juri,

Quoted 33 lines · Juri Lelli

Hi,

Quoted 12 lines · Wanpeng Li

Reschedule if successfully pull earlier deadline task.

---
 kernel/sched/deadline.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +0 -0
index 97141e2..21de865 100644

kernel/sched/deadline.c static void switched_from_dl(struct rq *rq, struct task_struct *p) · line 1587

Quoted 7 lines
kernel/sched/deadline.c +2 -2
@@ -1583,8 +1583,8 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
+			resched_curr(rq);

Right, we have to reschedule. But, how about we fully align to -rt?

Peter, is the task_on_rq_queued() check for cases in which p is not enqueued but its class changes after a setscheduler/rt_mutex_setprio?

Thanks,

- Juri

Quoted 1 line

From 404749782e493cb6019193f347163b640ebd552d Mon Sep 17 00:00:00 2001

Quoted 9 lines

From: Juri Lelli <[email protected]>
Date: Thu, 30 Oct 2014 10:15:40 +0000
Subject: [PATCH] sched/deadline: reschedule from switched_from_dl() after a successful pull

In switched_from_dl() we have to issue a resched if we successfully pulled some task from other cpus. This patch also aligns the behaviour with -rt.

Quoted 7 lines

Good idea.

Quoted 1 line

I will send out version two of this patchset with the update of 2/6 and 5/6 with *your Suggested-by*. Thanks for your great proposal. ;-)

Well, I've actually *wrote* (especially 2/6 is a different fix w.r.t. your version) and *tested* the updates . But, I'm not really getting anxious about signing patches, so fair enough.

Thanks,

- Juri

Quoted 29 lines

Regards,
Wanpeng Li

Quoted 25 lines

---
kernel/sched/deadline.c | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)
kernel/sched/deadline.c +5 -2
index db6ad38..e8943fc 100644
@@ -1623,8 +1623,11 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 * this is the right place to try to pull some other one
 	 * from an overloaded cpu, if any.
 	 */
-	if (!rq->dl.dl_nr_running)
-		pull_dl_task(rq);
+	if (!task_on_rq_queued(p) || rq->dl.dl_nr_running)
+		return;
+
+	if (pull_dl_task(rq))
+		resched_curr(rq);
 #endif
 }
 
Signature · 1 line
2.1.2
maintainer
#15 lore raw

Re: [PATCH 5/6] sched/dl: reschedule if successfully pull earlier deadline task

In reply to Juri Lelli
Quoted 2 lines · Juri Lelli

Peter, is the task_on_rq_queued() check for cases in which p is not enqueued but its class changes after a setscheduler/rt_mutex_setprio?

Yes, in particular if the task wasn't on the rq we won't change the rq by changing the class and therefore we don't need to attempt to pull.

---
commit da7a735e51f9622eb3e1672594d4a41da01d7e4f

Author: Peter Zijlstra <[email protected]>
Date:   Mon Jan 17 17:03:27 2011 +0100

sched: Fix switch_from_fair()

When a task is taken out of the fair class we must ensure the vruntime is properly normalized because when we put it back in it will assume to be normalized.

The case that goes wrong is when changing away from the fair class while sleeping. Sleeping tasks have non-normalized vruntime in order to make sleeper-fairness work. So treat the switch away from fair as a wakeup and preserve the relative vruntime.

Also update sysrq-n to call the ->switch_{to,from} methods.

LKML-Reference: <new-submission>

author
#16 lore raw

PATCH 6/6 sched/dl: don't check CONFIG_SMP in switched_from_dl

In reply to Wanpeng Li

There are both UP and SMP version of pull_dl_task(), so don't need to check CONFIG_SMP in switched_from_dl();

---
 kernel/sched/deadline.c | 2 --
 1 file changed, 2 deletions(-)
kernel/sched/deadline.c +0 -2
index 21de865..35d6dd5 100644
@@ -1577,7 +1577,6 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 
 	__dl_clear_params(p);
 
-#ifdef CONFIG_SMP
 	/*
 	 * Since this might be the only -deadline task on the rq,
 	 * this is the right place to try to pull some other one
@@ -1585,7 +1584,6 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 */
 	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
 			resched_curr(rq);
-#endif
 }
 
 /*
Signature · 1 line
1.9.1
#17 lore raw

Re: [PATCH 6/6] sched/dl: don't check CONFIG_SMP in switched_from_dl

Hi,

Quoted 22 lines · Wanpeng Li

There are both UP and SMP version of pull_dl_task(), so don't need to check CONFIG_SMP in switched_from_dl();

---
 kernel/sched/deadline.c | 2 --
 1 file changed, 2 deletions(-)
kernel/sched/deadline.c +0 -1
index 21de865..35d6dd5 100644
@@ -1577,7 +1577,6 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 
 	__dl_clear_params(p);
 
-#ifdef CONFIG_SMP
 	/*
 	 * Since this might be the only -deadline task on the rq,
 	 * this is the right place to try to pull some other one

kernel/sched/deadline.c static void switched_from_dl(struct rq *rq, struct task_struct *p) · line 1589

Quoted 7 lines
kernel/sched/deadline.c +0 -1
@@ -1585,7 +1584,6 @@ static void switched_from_dl(struct rq *rq, struct task_struct *p)
 	 */
 	if (!rq->dl.dl_nr_running && pull_dl_task(rq))
 			resched_curr(rq);
-#endif
 }
 
 /*

Yes, we can remove it.

Thanks,

- Juri

#18 lore raw
In reply to Wanpeng Li

Cc Kirill,
2014/10/27 9:41, Wanpeng Li:

Quoted 44 lines

This patch checks if current can be pushed/pulled somewhere else in advance to make logic clear.

- If current can't be migrated, useless to reschedule, let's hope task can move out.
- If task is migratable, so let's not schedule it and see if it can be pushed or pulled somewhere else.

---
  kernel/sched/rt.c | 14 ++++++++++----
  1 file changed, 10 insertions(+), 4 deletions(-)
kernel/sched/rt.c +10 -4
index 87ea5bf..ca1b7c7 100644
@@ -1351,16 +1351,22 @@ out:
 
  static void check_preempt_equal_prio(struct rq *rq, struct task_struct *p)
  {
-	if (rq->curr->nr_cpus_allowed == 1)
+	/*
+	 * Current can't be migrated, useless to reschedule,
+	 * let's hope p can move out.
+	 */
+	if (rq->curr->nr_cpus_allowed == 1 ||
+		!cpupri_find(&rq->rd->cpupri, rq->curr, NULL))
  		return;
 
+	/*
+	 * p is migratable, so let's not schedule it and
+	 * see if it is pushed or pulled somewhere else.
+	 */
  	if (p->nr_cpus_allowed != 1
  	    && cpupri_find(&rq->rd->cpupri, p, NULL))
  		return;
 
-	if (!cpupri_find(&rq->rd->cpupri, rq->curr, NULL))
-		return;
-
  	/*
  	 * There appears to be other cpus that can accept
  	 * current and none to run 'p', so lets reschedule