Skip to contentSkip to conversation

sched: move cfs task on a CPU with higher capacity

PATCH Hillf Danton 2 participants 6 messages Nov 4, 2014

lore
Share & tools
lore
#0 lore raw
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

+ /*
+ * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.

Why specify one task instead of not less than one?

if cfs.h_nr_running = 0 (which should not occurs at that point), we don't need to do more test to check if it's worth moving the task because there is no task to move.

I wonder if you can please shed light on the case that the dst_cpu is newly idle.

Thanks
Hillf

#1 lore raw
Quoted 4 lines · Hillf Danton
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

+ /*
+ * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Why specify one task instead of not less than one?

if cfs.h_nr_running = 0 (which should not occurs at that point), we don't need to do more test to check if it's worth moving the task because there is no task to move.

I wonder if you can please shed light on the case that the dst_cpu is newly idle.

The main problem if we do the test only for newly idle case, is that we are not sure to move the task because we must rely on the wakeup/sleep sequence of other tasks on an idle CPU in order to trig the migration (periodic background task as an example). So we might never move the task whereas idle CPUs are available

Regards,
Vincent

Quoted 4 lines

Thanks
Hillf

#2 lore raw
Quoted 12 lines · Hillf Danton
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

+ /*
+ * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.

Why specify one task instead of not less than one?

if cfs.h_nr_running = 0 (which should not occurs at that point), we don't need to do more test to check if it's worth moving the task because there is no task to move.

Quoted 9 lines
Quoted 2 lines

I wonder if you can please shed light on the case that the dst_cpu is newly idle.

The main problem if we do the test only for newly idle case, is that we are not sure to move the task because we must rely on the wakeup/sleep sequence of other tasks on an idle CPU in order to trig the migration (periodic background task as an example). So we might never move the task whereas idle CPUs are available

So no task is migrated in the newly idle case, if I understand the above correctly.

Hillf

#3 lore raw
Quoted 15 lines · Hillf Danton
Quoted 15 lines · Hillf Danton
Quoted 12 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

+ /*
+ * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.

Why specify one task instead of not less than one?

if cfs.h_nr_running = 0 (which should not occurs at that point), we don't need to do more test to check if it's worth moving the task because there is no task to move.

I wonder if you can please shed light on the case that the dst_cpu is newly idle.

Quoted 8 lines
Quoted 6 lines

The main problem if we do the test only for newly idle case, is that we are not sure to move the task because we must rely on the wakeup/sleep sequence of other tasks on an idle CPU in order to trig the migration (periodic background task as an example). So we might never move the task whereas idle CPUs are available

So no task is migrated in the newly idle case, if I understand the above correctly.

A task can be moved in both idle and newly idle. If we rely only on newly idle and we have only idle CPUs, we can never move task. In the same way, if we rely only on idle case and a CPU never stays idle long enough to trig the idle load balance, we will never move the task. I agree that for the latter, we might wonder if it's worth moving the task. This is your concern ?

Vincent

Quoted 8 lines

Hillf

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#4 lore raw
Quoted 12 lines
Quoted 11 lines
Quoted 9 lines
Quoted 2 lines

I wonder if you can please shed light on the case that the dst_cpu is newly idle.

The main problem if we do the test only for newly idle case, is that we are not sure to move the task because we must rely on the wakeup/sleep sequence of other tasks on an idle CPU in order to trig the migration (periodic background task as an example). So we might never move the task whereas idle CPUs are available

So no task is migrated in the newly idle case, if I understand the above correctly.

Quoted 7 lines

A task can be moved in both idle and newly idle. If we rely only on newly idle and we have only idle CPUs, we can never move task. In the same way, if we rely only on idle case and a CPU never stays idle long enough to trig the idle load balance, we will never move the task. I agree that for the latter, we might wonder if it's worth moving the task. This is your concern ?

I concern if the only-one cfs task is migrated to a newly-idle CPU in your code:
+ /*
+ * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.
+ * It's worth migrating the task if the src_cpu's capacity is reduced
+ * because of other sched_class or IRQs whereas capacity stays
+ * available on dst_cpu.
+ */
+ if ((env->idle != CPU_NOT_IDLE) &&
+ (env->src_rq->cfs.h_nr_running == 1)) {
+
due to the comment:
/*
* Increment the failure counter only on periodic balance.
* We do not want newidle balance, which can be very
* frequent, pollute the failure counter causing
* excessive cache_hot migrations and active balances. */
if (idle != CPU_NEWLY_IDLE)
sd->nr_balance_failed++;

Hillf

#5 lore raw
Quoted 31 lines · Hillf Danton
Quoted 12 lines
Quoted 11 lines
Quoted 9 lines
Quoted 2 lines

I wonder if you can please shed light on the case that the dst_cpu is newly idle.

The main problem if we do the test only for newly idle case, is that we are not sure to move the task because we must rely on the wakeup/sleep sequence of other tasks on an idle CPU in order to trig the migration (periodic background task as an example). So we might never move the task whereas idle CPUs are available

So no task is migrated in the newly idle case, if I understand the above correctly.

Quoted 7 lines

A task can be moved in both idle and newly idle. If we rely only on newly idle and we have only idle CPUs, we can never move task. In the same way, if we rely only on idle case and a CPU never stays idle long enough to trig the idle load balance, we will never move the task. I agree that for the latter, we might wonder if it's worth moving the task. This is your concern ?

I concern if the only-one cfs task is migrated to a newly-idle CPU in your code:

(quoted diff) +9 -0
+       /*
+        * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.
+        * It's worth migrating the task if the src_cpu's capacity is reduced
+        * because of other sched_class or IRQs whereas capacity stays
+        * available on dst_cpu.
+        */
+       if ((env->idle != CPU_NOT_IDLE) &&
+                       (env->src_rq->cfs.h_nr_running == 1)) {
+

due to the comment:

Quoted 9 lines

/*
* Increment the failure counter only on periodic balance.
* We do not want newidle balance, which can be very
* frequent, pollute the failure counter causing
* excessive cache_hot migrations and active balances. */
if (idle != CPU_NEWLY_IDLE)
sd->nr_balance_failed++;

I understand the code above as only idle CPU increases the nr_balance_failed and can generate an active load balance but newly idle CPUs don't increase it because it can occur quite often and would generate excessive active migration.
This patch will not pollute nr_balance_failed as it will clear it once the task has moved on another CPU with full capacity. If a newly idle load balance occurs before, the nr_balance_failed will also be cleared

Vincent

Quoted 2 lines

Hillf