Skip to contentSkip to conversation

PlugSched-6.4 for 2.6.18-rc2

RFC Al Boldi 3 participants 15 messages Jul 24, 2006 – Jul 27, 2006

lore
Share & tools
lore

Al Boldi

author
#0 lore raw
Quoted 8 lines · Peter Williams

This version removes the hard/soft CPU rate caps from the SPA schedulers.

A patch for 2.6.18-rc2 is available at:

<http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.pat
ch?download>

Quoted 7 lines

Very Brief Documentation:

You can select a default scheduler at kernel build time. If you wish to boot with a scheduler other than the default it can be selected at boot time by adding:

cpusched=<scheduler>

Any reason dynsched couldn't be merged with plugsched?

Quoted 9 lines

to the boot command line where <scheduler> is one of: ingosched, ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs or zaphod. If you don't change the default when you build the kernel the default scheduler will be ingosched (which is the normal scheduler).

The scheduler in force on a running system can be determined by the contents of:

/proc/scheduler

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

Worth a try, and should be easy to implement.

Quoted 3 lines

Control parameters for the scheduler can be read/set via files in:

/sys/cpusched/<scheduler>/

Thanks for the most important out-of-tree patch that makes 2.6 reasonable.

--
Al

#1 lore raw
Quoted 10 lines · Al Boldi
Quoted 9 lines · Peter Williams

This version removes the hard/soft CPU rate caps from the SPA schedulers.

A patch for 2.6.18-rc2 is available at:

<http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.pat
ch?download>

Very Brief Documentation:

Quoted 7 lines
Quoted 5 lines

You can select a default scheduler at kernel build time. If you wish to boot with a scheduler other than the default it can be selected at boot time by adding:

cpusched=<scheduler>

Any reason dynsched couldn't be merged with plugsched?

None that I know of (but I'm not familiar with dynsched). Patches to add it to the mix would be accepted and once in I would try to keep it in step with kernel changes.

Quoted 6 lines
Quoted 6 lines

to the boot command line where <scheduler> is one of: ingosched, ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs or zaphod. If you don't change the default when you build the kernel the default scheduler will be ingosched (which is the normal scheduler).

Quoted 6 lines
Quoted 4 lines

The scheduler in force on a running system can be determined by the contents of:

/proc/scheduler

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

Quoted 8 lines

Worth a try, and should be easy to implement.

Quoted 3 lines

Control parameters for the scheduler can be read/set via files in:

/sys/cpusched/<scheduler>/

Thanks for the most important out-of-tree patch that makes 2.6 reasonable.

My pleasure,
Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
author
#2 lore raw
In reply to Al Boldi
Quoted 16 lines · Peter Williams
Quoted 15 lines · Al Boldi
Quoted 14 lines · Peter Williams

This version removes the hard/soft CPU rate caps from the SPA schedulers.

A patch for 2.6.18-rc2 is available at:

<http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.
pat ch?download>

Very Brief Documentation:

You can select a default scheduler at kernel build time. If you wish to boot with a scheduler other than the default it can be selected at boot time by adding:

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

cpusched=<scheduler>

Any reason dynsched couldn't be merged with plugsched?

None that I know of (but I'm not familiar with dynsched). Patches to add it to the mix would be accepted and once in I would try to keep it in step with kernel changes.

I thought dynsched patches against plugsched, what else is needed?

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

to the boot command line where <scheduler> is one of: ingosched, ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs or zaphod. If you don't change the default when you build the kernel the default scheduler will be ingosched (which is the normal scheduler).

The scheduler in force on a running system can be determined by the contents of:

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

/proc/scheduler

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

Thanks!

--
Al

#3 lore raw
Quoted 17 lines · Al Boldi
Quoted 16 lines · Peter Williams
Quoted 15 lines · Al Boldi
Quoted 14 lines · Peter Williams

This version removes the hard/soft CPU rate caps from the SPA schedulers.

A patch for 2.6.18-rc2 is available at:

<http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.
pat ch?download>

Very Brief Documentation:

You can select a default scheduler at kernel build time. If you wish to boot with a scheduler other than the default it can be selected at boot time by adding:

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line

cpusched=<scheduler>

Any reason dynsched couldn't be merged with plugsched?

None that I know of (but I'm not familiar with dynsched). Patches to add it to the mix would be accepted and once in I would try to keep it in step with kernel changes.

I thought dynsched patches against plugsched, what else is needed?

Hopefully, nothing but it may be necessary to modify the plugsched interface if dynsched can't be implemented against it "as is". E.g. both staircase and nicksched needed changes to what was required for ingosched and the SPA schedulers.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

to the boot command line where <scheduler> is one of: ingosched, ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs or zaphod. If you don't change the default when you build the kernel the default scheduler will be ingosched (which is the normal scheduler).

The scheduler in force on a running system can be determined by the contents of:

Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
Quoted 1 line

/proc/scheduler

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-). The schedulers would probably have to have a common idea of how the run queue works though and this would restrict the choice of schedulers.

I have no intentions (at the moment) of going down this path myself.

However, I am thinking about making it possible to switch between the various SPA schedulers on a running system. A extension to this could be to attempt automatic selection of which scheduler to use possibly based on which users are logged in.

Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
author
#4 lore raw
Quoted 50 lines · Peter Williams
Quoted 19 lines · Al Boldi
Quoted 18 lines · Peter Williams
Quoted 17 lines · Al Boldi
Quoted 15 lines · Peter Williams

This version removes the hard/soft CPU rate caps from the SPA schedulers.

A patch for 2.6.18-rc2 is available at:

<http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc
2. pat ch?download>

Very Brief Documentation:

You can select a default scheduler at kernel build time. If you wish to boot with a scheduler other than the default it can be selected at boot time by adding:

cpusched=<scheduler>

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Any reason dynsched couldn't be merged with plugsched?

None that I know of (but I'm not familiar with dynsched). Patches to add it to the mix would be accepted and once in I would try to keep it in step with kernel changes.

I thought dynsched patches against plugsched, what else is needed?

Hopefully, nothing but it may be necessary to modify the plugsched interface if dynsched can't be implemented against it "as is". E.g. both staircase and nicksched needed changes to what was required for ingosched and the SPA schedulers.

Quoted 17 lines
Quoted 16 lines
Quoted 6 lines
Quoted 6 lines

to the boot command line where <scheduler> is one of: ingosched, ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs or zaphod. If you don't change the default when you build the kernel the default scheduler will be ingosched (which is the normal scheduler).

Quoted 8 lines
Quoted 4 lines

The scheduler in force on a running system can be determined by the contents of:

/proc/scheduler

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

Quoted 8 lines
Quoted 1 line

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

What's wrong with multiple run-queues on UP?

Quoted 3 lines

The schedulers would probably have to
have a common idea of how the run queue works though and this would restrict the choice of schedulers.

Probably.

Quoted 6 lines

I have no intentions (at the moment) of going down this path myself.

However, I am thinking about making it possible to switch between the various SPA schedulers on a running system. A extension to this could be to attempt automatic selection of which scheduler to use possibly based on which users are logged in.

Thanks a lot!

--
Al

#5 lore raw
Quoted 10 lines · Al Boldi
Quoted 6 lines · Peter Williams

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

What's wrong with multiple run-queues on UP?

On an SMP system, you can have one CPU doing one class of scheduling (long timeslice for computational, for example), while another CPU is dedicated to doing RT scheduling, and so on. It's not clear to me that "different classes per CPU" makes any real sense on a UP....

author
#6 lore raw
Quoted 9 lines · [email protected] wrote:
Quoted 8 lines · Al Boldi
Quoted 6 lines · Peter Williams

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

Quoted 6 lines
Quoted 1 line

What's wrong with multiple run-queues on UP?

On an SMP system, you can have one CPU doing one class of scheduling (long timeslice for computational, for example), while another CPU is dedicated to doing RT scheduling, and so on. It's not clear to me that "different classes per CPU" makes any real sense on a UP....

Conceptually there should be no difference between UP and MP.

Think HyperThreading.

Thanks!

--
Al

#7 lore raw
Quoted 10 lines · Al Boldi
Quoted 4 lines · [email protected] wrote:

On an SMP system, you can have one CPU doing one class of scheduling (long timeslice for computational, for example), while another CPU is dedicated to doing RT scheduling, and so on. It's not clear to me that "different classes per CPU" makes any real sense on a UP....

Conceptually there should be no difference between UP and MP.

Think HyperThreading.

Which is why a UP kernel can schedule on both sides of an HT core.

Yeah, I got it now. ;)

An HT core still *has* "the other instruction stream" it can schedule differetly. You can't say "We'll schedule this one this way and that other one that way" when there *is* no "that other one".

(And if you look at the current code, you'll realize that HT is conceptually different from both UP *and* MP - go look at the places where the *current* scheduler is HT-aware, and how that was a big win over when it thought each HT was a fully capable MP......)

#8 lore raw
In reply to Al Boldi
Quoted 4 lines · Al Boldi
Quoted 3 lines · Peter Williams
Quoted 2 lines · Al Boldi
Quoted 1 line · Peter Williams

Al Boldi wrote:

[bits deleted]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

Quoted 8 lines
Quoted 6 lines

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

What's wrong with multiple run-queues on UP?

A really high likelihood of starvation of some tasks.

Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
author
#9 lore raw
Quoted 23 lines · Peter Williams
Quoted 4 lines · Al Boldi
Quoted 3 lines · Peter Williams
Quoted 2 lines · Al Boldi
Quoted 1 line · Peter Williams

Al Boldi wrote:

[bits deleted]

Quoted 15 lines
Quoted 14 lines
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

Quoted 3 lines
Quoted 1 line

What's wrong with multiple run-queues on UP?

A really high likelihood of starvation of some tasks.

Maybe you are thinking of running independent run-queues, in which case it would probably be unwise to run multiple RQs on a single CPU.

But I was more thinking of a run-queue of run-queues, with the masterRQ scheduling slaveRQs, each RQ possible running its own scheduling semantic.

Thanks!

--
Al

#10 lore raw
Quoted 19 lines · Al Boldi
Quoted 18 lines · Peter Williams
Quoted 4 lines · Al Boldi
Quoted 3 lines · Peter Williams
Quoted 2 lines · Al Boldi
Quoted 1 line · Peter Williams

Al Boldi wrote:

[bits deleted]

Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

Quoted 4 lines
Quoted 2 lines
Quoted 1 line

What's wrong with multiple run-queues on UP?

A really high likelihood of starvation of some tasks.

Maybe you are thinking of running independent run-queues, in which case it would probably be unwise to run multiple RQs on a single CPU.

No. I'm thinking about different schedulers on a single run queue. I don't think that it's a good idea.

Quoted 2 lines

But I was more thinking of a run-queue of run-queues, with the masterRQ scheduling slaveRQs, each RQ possible running its own scheduling semantic.

I think that you need to think a bit harder about the consequences of such a system. The word "chaos" springs to mind.

Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
author
#11 lore raw
Quoted 18 lines · Peter Williams
Quoted 17 lines · Al Boldi
Quoted 17 lines
Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

It may be really great, to allow schedulers perPid parent, thus allowing the stacking of different scheduler semantics. This could aid flexibility a lot.

I'm don't understand what you mean here. Could you elaborate?

Quoted 8 lines
Quoted 1 line

i.e: Boot the kernel with spa_no_frills, then start X with spa_ws.

It's probably not a good idea to have different schedulers managing the same resource. The way to do different scheduling per process is to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. (possibly extended) within each scheduler. On the other hand, on an SMP system, having a different scheduler on each run queue (or sub set of queues) might be interesting :-).

What's wrong with multiple run-queues on UP?

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

A really high likelihood of starvation of some tasks.

Maybe you are thinking of running independent run-queues, in which case it would probably be unwise to run multiple RQs on a single CPU.

No. I'm thinking about different schedulers on a single run queue. I don't think that it's a good idea.

Running different scheds on a single RQ at the same time on the same resource would be rather odd. That's why independent RQs are necessary even on SMP. OTOH, running independent RQs on UP doesn't make much sense, unless there is a way to relate them.

Quoted 6 lines
Quoted 3 lines

But I was more thinking of a run-queue of run-queues, with the masterRQ scheduling slaveRQs, each RQ possibly running its own scheduling semantic.

I think that you need to think a bit harder about the consequences of such a system. The word "chaos" springs to mind.

Are you sure?

MultiDimensional RunQueues spring to mind.

Thanks!

--
Al

#12 lore raw
Quoted 4 lines · Al Boldi

Running different scheds on a single RQ at the same time on the same resource would be rather odd. That's why independent RQs are necessary even on SMP. OTOH, running independent RQs on UP doesn't make much sense, unless there is a way to relate them.

Exactly..

(But now I'm confused why you said SMP and UP were conceptually the same a few notes back...)

author
#13 lore raw
Quoted 10 lines · [email protected] wrote:
Quoted 4 lines · Al Boldi

Running different scheds on a single RQ at the same time on the same resource would be rather odd. That's why independent RQs are necessary even on SMP. OTOH, running independent RQs on UP doesn't make much sense, unless there is a way to relate them.

Exactly..

(But now I'm confused why you said SMP and UP were conceptually the same a few notes back...)

The important part here is 'unless there is a way to relate them', at which point UP and MP should be conceptually the same, while possibly differing in the implementation details.

Thanks!

--
Al

#14 lore raw
Quoted 3 lines · Al Boldi

The important part here is 'unless there is a way to relate them', at which point UP and MP should be conceptually the same, while possibly differing in the implementation details.

Oh, OK. I think we're actually in agreement, just using different terms for the same thing - what you were considering multiple related queues, I'd consider one unified queue with subqueuing of some sort.. ;)