sched/deadline: support dl task migration during cpu hotplug

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

 kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +38 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
 	return hrtimer_active(&dl_se->dl_timer);
 }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
 /*
  * This is the bandwidth enforcement timer callback. If here, we know
  * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
 	update_rq_clock(rq);
 	dl_se->dl_throttled = 0;
 	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);
+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);
+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);
+
+		goto unlock;
+	}
+
 	if (task_on_rq_queued(p)) {
 		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
 		if (dl_task(rq->curr))

kernel/sched/deadline.c +0 -0

 	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
 	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
 			task, later_mask);

Quoted 7 lines

kernel/sched/deadline.c +1 -2

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 2 lines

kernel/sched/deadline.c +2 -0

+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Quoted 7 lines · Juri Lelli

Quoted 7 lines

kernel/sched/deadline.c +1 -2

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 2 lines

kernel/sched/deadline.c +2 -0

+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Ah, good point that, we must somehow find the right root domain to 'restore' the task to. Now I'm not entirely sure we still have this. Lemme ponder that.

Quoted 7 lines · Juri Lelli

Quoted 7 lines

kernel/sched/deadline.c +1 -2

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 2 lines

kernel/sched/deadline.c +2 -0

+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Ah, good point that, we must somehow find the right root domain to 'restore' the task to. Now I'm not entirely sure we still have this. Lemme ponder that.

Quoted 17 lines · Peter Zijlstra

Quoted 7 lines · Juri Lelli

Quoted 7 lines

kernel/sched/deadline.c +1 -2

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 2 lines

kernel/sched/deadline.c +2 -0

+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Quoted 3 lines

Ah, good point that, we must somehow find the right root domain to 'restore' the task to. Now I'm not entirely sure we still have this. Lemme ponder that.

Ah, we should be able to find this by looking at the cpuset cgroup information. The cpuset cgroup knows the available cpumask of this task, which we can translate to the correct root domain in two separate ways (either run up the cpuset cgroup hierarchy and find the highest domain with balancing enabled, or look at whatever the rq->rd is for any one of the allowed CPUs of the immediate cgroup this task belongs to).

Hi,

Quoted 94 lines · Wanpeng Li

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

 kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +38 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
 	return hrtimer_active(&dl_se->dl_timer);
 }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
 /*
  * This is the bandwidth enforcement timer callback. If here, we know
  * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
 	update_rq_clock(rq);
 	dl_se->dl_throttled = 0;
 	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);
+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);
+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);
+
+		goto unlock;
+	}
+
 	if (task_on_rq_queued(p)) {
 		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
 		if (dl_task(rq->curr))

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 8 lines

kernel/sched/deadline.c +3 -2

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Thanks,

- Juri

Quoted 4 lines

kernel/sched/deadline.c +0 -0

 	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
 	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
 			task, later_mask);

Hi,

Quoted 99 lines · Wanpeng Li

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

  kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +38 -2

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
  	return hrtimer_active(&dl_se->dl_timer);
  }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
  /*
   * This is the bandwidth enforcement timer callback. If here, we know
   * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
  	update_rq_clock(rq);
  	dl_se->dl_throttled = 0;
  	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);
+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);
+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);
+
+		goto unlock;
+	}
+
  	if (task_on_rq_queued(p)) {
  		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
  		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
  	 * We have to consider system topology and task affinity
  	 * first, then we can look for a suitable cpu.
  	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 3 lines

kernel/sched/deadline.c +3 -0

+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Thanks,

- Juri

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 4 lines

kernel/sched/deadline.c +0 -0

  	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
  	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
  			task, later_mask);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hi Juri,

Quoted 102 lines · Juri Lelli

Hi,

Quoted 99 lines · Wanpeng Li

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

  kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +38 -2

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
  	return hrtimer_active(&dl_se->dl_timer);
  }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
  /*
   * This is the bandwidth enforcement timer callback. If here, we know
   * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
  	update_rq_clock(rq);
  	dl_se->dl_throttled = 0;
  	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);
+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);
+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);
+
+		goto unlock;
+	}
+
  	if (task_on_rq_queued(p)) {
  		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
  		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
  	 * We have to consider system topology and task affinity
  	 * first, then we can look for a suitable cpu.
  	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 3 lines

kernel/sched/deadline.c +3 -0

+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Sorry, what's clustered scheduling?

Regards,
Wanpeng Li

Quoted 14 lines

Thanks,

- Juri

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 4 lines

kernel/sched/deadline.c +0 -0

  	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
  	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
  			task, later_mask);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hi,

Quoted 112 lines · Wanpeng Li

Hi Juri,

Quoted 102 lines · Juri Lelli

Hi,

Quoted 99 lines · Wanpeng Li

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

   kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
   1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +38 -2

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
   	return hrtimer_active(&dl_se->dl_timer);
   }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
   /*
    * This is the bandwidth enforcement timer callback. If here, we know
    * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
   	update_rq_clock(rq);
   	dl_se->dl_throttled = 0;
   	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);
+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);
+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);
+
+		goto unlock;
+	}
+
   	if (task_on_rq_queued(p)) {
   		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
   		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
   	 * We have to consider system topology and task affinity
   	 * first, then we can look for a suitable cpu.
   	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 3 lines

kernel/sched/deadline.c +3 -0

+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Quoted 2 lines

Sorry, what's clustered scheduling?

It's a scheduling configuration in which you restrict tasks to run in disjoint subsets of system CPUs. Translated to what we have, it's what you get when you create exclusive cpusets (each one gets a rd) and associate tasks to them.

My concern in what above is that you may end up breaking this setup if you don't consider the sd->span when one of the CPUs of your cpuset is off. But, Pang Xunlei patches may solve this, I still have to check :/.

Thanks,

- Juri

Quoted 17 lines

Regards,
Wanpeng Li

Quoted 13 lines

Thanks,

- Juri

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 4 lines

kernel/sched/deadline.c +0 -0

   	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
   	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
   			task, later_mask);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hi Juri,

Quoted 122 lines · Juri Lelli

Hi,

Quoted 112 lines · Wanpeng Li

Hi Juri,

Quoted 102 lines · Juri Lelli

Hi,

Quoted 99 lines · Wanpeng Li

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

   kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
   1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +38 -2

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
   	return hrtimer_active(&dl_se->dl_timer);
   }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
   /*
    * This is the bandwidth enforcement timer callback. If here, we know
    * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
   	update_rq_clock(rq);
   	dl_se->dl_throttled = 0;
   	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);
+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);
+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);
+
+		goto unlock;
+	}
+
   	if (task_on_rq_queued(p)) {
   		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
   		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
   	 * We have to consider system topology and task affinity
   	 * first, then we can look for a suitable cpu.
   	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 3 lines

kernel/sched/deadline.c +3 -0

+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Quoted 2 lines

Sorry, what's clustered scheduling?

It's a scheduling configuration in which you restrict tasks to run in disjoint subsets of system CPUs. Translated to what we have, it's what you get when you create exclusive cpusets (each one gets a rd) and associate tasks to them.

Quoted 4 lines

My concern in what above is that you may end up breaking this setup if you don't consider the sd->span when one of the CPUs of your cpuset is off. But, Pang Xunlei patches may solve this, I still have to check :/.

Thanks for your explanation. Could you point out which one of Pang's patchset solve this? ;-)

Regards,
Wanpeng Li

Quoted 22 lines

Thanks,

- Juri

Quoted 17 lines

Regards,
Wanpeng Li

Quoted 13 lines

Thanks,

- Juri

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 4 lines

kernel/sched/deadline.c +0 -0

   	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
   	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
   			task, later_mask);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoted 26 lines

Quoted 25 lines

Quoted 14 lines

Quoted 6 lines

Quoted 6 lines

kernel/sched/deadline.c +0 -2

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
   	 * We have to consider system topology and task affinity
   	 * first, then we can look for a suitable cpu.
   	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1228

Quoted 3 lines

kernel/sched/deadline.c +3 -0

+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);

So, here you consider the span only when the task_rq is online, but there might be others cpus still online belonging to the same rd->span. And you have to consider them when migrating. Actually, migration must still be restricted to the online cpus of task's original rd->span, or I fear you can break clustered scheduling.

Quoted 2 lines

Sorry, what's clustered scheduling?

It's a scheduling configuration in which you restrict tasks to run in disjoint subsets of system CPUs. Translated to what we have, it's what you get when you create exclusive cpusets (each one gets a rd) and associate tasks to them.

My concern in what above is that you may end up breaking this setup if you don't consider the sd->span when one of the CPUs of your cpuset is off. But, Pang Xunlei patches may solve this, I still have to check :/.

Quoted 3 lines

Thanks for your explanation. Could you point out which one of Pang's patchset solve this? ;-)

https://lkml.org/lkml/2014/11/17/443 may help with this, although I still have to properly look at it.

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

 kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +15 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
 	return hrtimer_active(&dl_se->dl_timer);
 }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
 /*
  * This is the bandwidth enforcement timer callback. If here, we know
  * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
 	update_rq_clock(rq);
 	dl_se->dl_throttled = 0;
 	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*

kernel/sched/deadline.c +7 -2

+
+		goto unlock;
+	}
+
 	if (task_on_rq_queued(p)) {
 		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
 		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);
 	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
 	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
 			task, later_mask);

В Ср, 12/11/2014 в 09:06 +0800, Wanpeng Li пишет:

Quoted 71 lines

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

 kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +19 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
 	return hrtimer_active(&dl_se->dl_timer);
 }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
 /*
  * This is the bandwidth enforcement timer callback. If here, we know
  * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
 	update_rq_clock(rq);
 	dl_se->dl_throttled = 0;
 	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,

kernel/sched/deadline.c again: · line 565

Quoted 6 lines

kernel/sched/deadline.c +6 -0

+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);

later_rq is not locked here, but you activate p on it and you do unlock below.

kernel/sched/deadline.c again: · line 570

Quoted 6 lines

kernel/sched/deadline.c +5 -0

+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);

		^^^^^

kernel/sched/deadline.c again: · line 574

Quoted 4 lines

kernel/sched/deadline.c +4 -0

+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);

		^^^^^^

Quoted 19 lines

kernel/sched/deadline.c +7 -2

+
+		goto unlock;
+	}
+
 	if (task_on_rq_queued(p)) {
 		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
 		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);
 	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
 	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
 			task, later_mask);

Hi Kirill,

Quoted 72 lines · Kirill Tkhai

В Ср, 12/11/2014 в 09:06 +0800, Wanpeng Li пишет:

Quoted 71 lines

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

 kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +19 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
 	return hrtimer_active(&dl_se->dl_timer);
 }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
 /*
  * This is the bandwidth enforcement timer callback. If here, we know
  * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
 	update_rq_clock(rq);
 	dl_se->dl_throttled = 0;
 	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*
+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,

Quoted 8 lines

kernel/sched/deadline.c again: · line 565

Quoted 6 lines

kernel/sched/deadline.c +6 -0

+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);

later_rq is not locked here, but you activate p on it and you do unlock below.

Great catch! How about add double_lock_balance(rq, later_rq); here?

Regards,
Wanpeng Li

Quoted 36 lines

kernel/sched/deadline.c again: · line 570

Quoted 6 lines

kernel/sched/deadline.c +5 -0

+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);

		^^^^^

kernel/sched/deadline.c again: · line 574

Quoted 4 lines

kernel/sched/deadline.c +4 -0

+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);

		^^^^^^

Quoted 19 lines

kernel/sched/deadline.c +7 -2

+
+		goto unlock;
+	}
+
 	if (task_on_rq_queued(p)) {
 		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
 		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
 	 * We have to consider system topology and task affinity
 	 * first, then we can look for a suitable cpu.
 	 */
-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);
 	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
 	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
 			task, later_mask);

В Чт, 13/11/2014 в 06:56 +0800, Wanpeng Li пишет:

Quoted 81 lines

Hi Kirill,

Quoted 79 lines · Kirill Tkhai

В Ср, 12/11/2014 в 09:06 +0800, Wanpeng Li пишет:

Quoted 67 lines

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

  kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +15 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
  	return hrtimer_active(&dl_se->dl_timer);
  }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
  /*
   * This is the bandwidth enforcement timer callback. If here, we know
   * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
  	update_rq_clock(rq);
  	dl_se->dl_throttled = 0;
  	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*

kernel/sched/deadline.c again: · line 565

Quoted 10 lines

kernel/sched/deadline.c +10 -0

+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);

later_rq is not locked here, but you activate p on it and you do unlock below.

Quoted 1 line

Great catch! How about add double_lock_balance(rq, later_rq); here?

This sounds good.

Quoted 28 lines

Regards,
Wanpeng Li

Quoted 24 lines

kernel/sched/deadline.c again: · line 570

Quoted 5 lines

kernel/sched/deadline.c +5 -0

+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);

		^^^^^

kernel/sched/deadline.c again: · line 574

Quoted 4 lines

kernel/sched/deadline.c +4 -0

+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);

		^^^^^^

Quoted 11 lines

kernel/sched/deadline.c +4 -0

+
+		goto unlock;
+	}
+
  	if (task_on_rq_queued(p)) {
  		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
  		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
  	 * We have to consider system topology and task affinity
  	 * first, then we can look for a suitable cpu.
  	 */

Quoted 8 lines

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 8 lines

kernel/sched/deadline.c +3 -2

-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);
  	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
  	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
  			task, later_mask);

Also, we should think about the following situation.

DL task is left on dead rq. In your scheme it will be moved by the timer. But what will be if somebody changes the class of the task (before timer)?

In this case the task still remains on dead rq.

We should handle this situation in some way.

Kirill

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hi Kirill,

Quoted 82 lines · Kirill Tkhai

В Чт, 13/11/2014 в 06:56 +0800, Wanpeng Li пишет:

Quoted 81 lines

Hi Kirill,

Quoted 79 lines · Kirill Tkhai

В Ср, 12/11/2014 в 09:06 +0800, Wanpeng Li пишет:

Quoted 67 lines

I observe that dl task can't be migrated to other cpus during cpu hotplug, in addition, task may/may not be running again if cpu is added back. The root cause which I found is that dl task will be throtted and removed from dl rq after comsuming all budget, which leads to stop task can't pick it up from dl rq and migrate to other cpus during hotplug.

The method to reproduce:
schedtool -E -t 50000:100000 -e ./test
Actually test is just a simple for loop. Then observe which cpu the test task is on.
echo 0 > /sys/devices/system/cpu/cpuN/online

This patch adds the dl task migration during cpu hotplug by finding a most suitable later deadline rq after dl timer fire if current rq is offline, if fail to find a suitable later deadline rq then fallback to any eligible online cpu in order that the deadline task will come back to us, and the push/pull mechanism should then move it around properly.

Signed-off-by Wanpeng Li <[email protected]>

---
v4 -> v5:
* remove raw_spin_unlock(&rq->lock)
* cleanup codes, spotted by Peterz
* cleanup patch description
v3 -> v4:
* use tsk_cpus_allowed wrapper
* fix compile error
v2 -> v3:
* don't get_task_struct
* if cannot preempt any rq, fallback to pick any online cpus
* use cpu_active_mask as original later_mask if cpu is offline
v1 -> v2:
* push the task to another cpu in dl_task_timer() if rq is offline.

  kernel/sched/deadline.c | 43 +++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 41 insertions(+), 2 deletions(-)

kernel/sched/deadline.c +15 -0

index f3d7776..7c31906 100644
@@ -487,6 +487,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se, bool boosted)
  	return hrtimer_active(&dl_se->dl_timer);
  }
 
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
  /*
   * This is the bandwidth enforcement timer callback. If here, we know
   * a task is not on its dl_rq, since the fact that the timer was running

@@ -538,6 +539,43 @@ again:
  	update_rq_clock(rq);
  	dl_se->dl_throttled = 0;
  	dl_se->dl_yielded = 0;
+
+	/*
+	 * So if we find that the rq the task was on is no longer
+	 * available, we need to select a new rq.
+	 */
+	if (unlikely(!rq->online)) {
+		struct rq *later_rq = NULL;
+
+		later_rq = find_lock_later_rq(p, rq);
+
+		if (!later_rq) {
+			int cpu;
+
+			/*

kernel/sched/deadline.c again: · line 565

Quoted 10 lines

kernel/sched/deadline.c +10 -0

+			 * If cannot preempt any rq, fallback to pick any
+			 * online cpu.
+			 */
+			cpu = cpumask_any_and(cpu_active_mask,
+					tsk_cpus_allowed(p));
+			if (cpu >= nr_cpu_ids) {
+				pr_warn("fail to find any online cpu and task will never come back\n");
+				goto unlock;
+			}
+			later_rq = cpu_rq(cpu);

later_rq is not locked here, but you activate p on it and you do unlock below.

Quoted 2 lines

Quoted 1 line

Great catch! How about add double_lock_balance(rq, later_rq); here?

This sounds good.

I will do this in next version. ;-)

Quoted 38 lines

Quoted 28 lines

Regards,
Wanpeng Li

Quoted 24 lines

kernel/sched/deadline.c again: · line 570

Quoted 5 lines

kernel/sched/deadline.c +5 -0

+		}
+
+		deactivate_task(rq, p, 0);
+		set_task_cpu(p, later_rq->cpu);
+		activate_task(later_rq, p, 0);

		^^^^^

kernel/sched/deadline.c again: · line 574

Quoted 4 lines

kernel/sched/deadline.c +4 -0

+
+		resched_curr(later_rq);
+
+		double_unlock_balance(rq, later_rq);

		^^^^^^

Quoted 11 lines

kernel/sched/deadline.c +4 -0

+
+		goto unlock;
+	}
+
  	if (task_on_rq_queued(p)) {
  		enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
  		if (dl_task(rq->curr))

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
  	 * We have to consider system topology and task affinity
  	 * first, then we can look for a suitable cpu.
  	 */

Quoted 8 lines

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 8 lines

kernel/sched/deadline.c +3 -2

-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);
  	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
  	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
  			task, later_mask);

Also, we should think about the following situation.

Quoted 2 lines

DL task is left on dead rq. In your scheme it will be moved by the timer. But what will be if somebody changes the class of the task (before timer)?

I think timer will be cancelled in switched_from_dl().

Regards,
Wanpeng Li

Quoted 12 lines

In this case the task still remains on dead rq.

We should handle this situation in some way.

Kirill

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoted 14 lines

Quoted 14 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

kernel/sched/deadline.c +0 -0

@@ -1185,8 +1223,9 @@ static int find_later_rq(struct task_struct *task)
  	 * We have to consider system topology and task affinity
  	 * first, then we can look for a suitable cpu.
  	 */

Quoted 8 lines

Quoted 8 lines

kernel/sched/deadline.c static int find_later_rq(struct task_struct *task) · line 1231

Quoted 8 lines

kernel/sched/deadline.c +3 -2

-	cpumask_copy(later_mask, task_rq(task)->rd->span);
-	cpumask_and(later_mask, later_mask, cpu_active_mask);
+	cpumask_copy(later_mask, cpu_active_mask);
+	if (likely(task_rq(task)->online))
+		cpumask_and(later_mask, later_mask, task_rq(task)->rd->span);
  	cpumask_and(later_mask, later_mask, &task->cpus_allowed);
  	best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
  			task, later_mask);

Also, we should think about the following situation.

Quoted 4 lines

Quoted 2 lines

DL task is left on dead rq. In your scheme it will be moved by the timer. But what will be if somebody changes the class of the task (before timer)?

I think timer will be cancelled in switched_from_dl().

Yeah, but nobody will move this task to alive rq.

Quoted 7 lines

Regards,
Wanpeng Li

Quoted 3 lines

In this case the task still remains on dead rq.

We should handle this situation in some way.

Quoted 10 lines

Quoted 9 lines

Kirill

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoted 6 lines

Quoted 4 lines

Also, we should think about the following situation.

DL task is left on dead rq. In your scheme it will be moved by the timer. But what will be if somebody changes the class of the task (before timer)?

I think timer will be cancelled in switched_from_dl().

Yeah, but nobody will move this task to alive rq.

Quoted 3 lines

Quoted 3 lines

In this case the task still remains on dead rq.

We should handle this situation in some way.

Quoted 10 lines

Quoted 9 lines

Kirill

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/