SCHED_DEADLINE with CPU affinity
Am 13.01.2020 10:22 schrieb Juri Lelli:
Quoted 3 lines
Hi,
Sorry for the delay in repling (Xmas + catching-up w/ emails).
No worries
Quoted 14 lines
Quoted 7 lines
I fear I have not understood quite well yet why this "workaround" leads to (presumably) the same results as
set_affinitywould. From what I have read, I understand it as follows: Forsched_dead, admission control tries to guarantee that the requested policy can be executed. To do so, it analyzes the current workload situation, taking especially the number of cores into account.Quoted 6 lines
Now, with a pre-configured set, the kernel knows which tasks will run on which core, therefore it's able to judge wether a process can be deadline scheduled or not. But when using the default way, you could start your processes as
SCHED_OTHER, setSCHED_DEADLINEas policy and later many of them could suddenly callset_affinity, desiring to run on the same core, therefore provoking collisions.
Quoted 4 lines
But setting affinity would still have to pass admission control, and should fail in the case you are describing (IIUC).
https://elixir.bootlin.com/linux/latest/source/kernel/sched/core.c#L5433
Well, no, that's not what I meant.
I understand that the kernel currently rejects the combination of set_affinity and sched_setattr.
My question, basically is: Why does it work with exclusive cpu-sets?
As I wrote above, I assume that the difference is that the kernel knows which programs will run on which core beforehand and therefore can check the rules of admission control, whereas without exclusive cpu_sets it could happen any time that certain (other) deadline applications decide to switch cores manually, causing collisions with a deadline task already running on this core.
You originally wrote that this solution is "currently" required; that's why assume that in theory the admission control check could also be done dynamically when sched_setattr or set_affinity are called (after each other, without exclusive cpu sets).
Have I been clear enough now? Basically I want to know why cpusets+sched_deadline works whereas set_affinity+sched_deadline is rejected, although both seem to lead to the same result.
P.
Quoted 50 lines · On 14/01/20 10:44, [email protected] wrote:
Am 13.01.2020 10:22 schrieb Juri Lelli:
Quoted 3 lines
Hi,
Sorry for the delay in repling (Xmas + catching-up w/ emails).
No worries
Quoted 15 lines
Quoted 7 lines
I fear I have not understood quite well yet why this "workaround" leads to (presumably) the same results as
set_affinitywould. From what I have read, I understand it as follows: Forsched_dead, admission control tries to guarantee that the requested policy can be executed. To do so, it analyzes the current workload situation, taking especially the number of cores into account.Quoted 7 lines
Now, with a pre-configured set, the kernel knows which tasks will run on which core, therefore it's able to judge wether a process can be deadline scheduled or not. But when using the default way, you could start your processes as
SCHED_OTHER, setSCHED_DEADLINEas policy and later many of them could suddenly callset_affinity, desiring to run on
the same core, therefore provoking collisions.Quoted 4 lines
But setting affinity would still have to pass admission control, and should fail in the case you are describing (IIUC).
https://elixir.bootlin.com/linux/latest/source/kernel/sched/core.c#L5433
Well, no, that's not what I meant.
I understand that the kernel currently rejects the combination ofset_affinityandsched_setattr.
My question, basically is: Why does it work with exclusive cpu-sets?As I wrote above, I assume that the difference is that the kernel knows which
programs will run on which core beforehand and therefore can check the rules of admission control, whereas without exclusivecpu_setsit could happen
any time that certain (other) deadline applications decide to switch cores manually,
causing collisions with a deadline task already running on this core.You originally wrote that this solution is "currently" required; that's why assume that
in theory the admission control check could also be done dynamically whensched_setattrorset_affinityare called (after each other, without exclusive cpu sets).
Quoted 5 lines
Have I been clear enough now? Basically I want to know why cpusets+sched_deadline
works whereas set_affinity+sched_deadline is rejected, although both seem to lead
to the same result.
Oh, OK, I think I now got the question (sorry :-).
So, (exclusive) cpusets define "isolated" domains of CPUs among which DEADLINE tasks can freely migrate following the global EDF rule:
https://elixir.bootlin.com/linux/latest/source/Documentation/scheduler/sched-deadline.rst#L413
Relaxing this constraint and allowing users to define per-task affinities (via setaffinity) can lead to situations where affinity masks of different tasks overlap, and this creates problem for the admission control checks that are currently implemented.
Theory has been developed already that tackles the problem of overlapping affinities, but that is not yet implemented (it adds complexity that would have to be supported by strong usecases).
Best,
Juri