dynsched - different cpu schedulers per cpuset
Hej,
we're a student group witch is working on a research project concerning the ability to switch the cpu scheduler of the linux kernel at runtime.
We use Peter Williams Plugsched patch [1] to get an interface for the different scheduler implementations. Some month ago we started to modify the code to allow different scheduler running on each cpu on an SMP system. The cpu<->scheduler mapping is controlled via cpusets. Thus you can switch the scheduler for a cpuset containing multiple cpus and keep the rest untouched.
The project is hosted on Sourceforge [2] and the current patch applies against 2.6.18 patched with plugsched.
Threre are still lots of issues - especially the migration of tasks between cpus with different schedulers is quite buggy.
Switching the scheduler on non smp configured systems works fine (tested
x86)
Refer the project instruction site [3] for further information and usage / patch instructions.
We'd be quite happy, if someone could take a look at what we've done to gain some feedback/suggestions about the used techniques and implementation. Some changes are already queued.
The project documentation (description/benchmarks/usage/bugs) is in progress and will be completed in about 4 weeks.
Thanks a lot.
cheers,
Felix
[1] http://sourceforge.net/projects/cpuse/
[2] http://sourceforge.net/projects/dynsched/
[3] http://dynsched.sourceforge.net
Signature · 4 lines
Felix Obenhuber felixatobenhuber.de www.obenhuber.de/felix GPG: F696D489 Sat Nov 18 15:56:31 CET 2006
Quoted 3 lines · Felix
The cpu<->scheduler mapping is controlled via cpusets. Thus you can switch the scheduler for a cpuset containing multiple cpus and keep the rest untouched.
I don't have comments on the main focus of this work - schedulers are not my expertise.
I just noticed this lkml post because of my interest in cpusets.
You should take a look at the work of Paul Menage (added to the cc list), who is splitting the cpuset code into:
1) a generic "container" mechanism,
2) separate CPU and Memory "controllers", and
3) various other additional "controllers".
See Paul Menage's most recent patch proposal at:
http://lkml.org/lkml/2006/11/17/217
Subject: [PATCH 0/6] Multi-hierarchy Process Containers
Date: Fri, 17 Nov 2006 11:11:59 -0800
The container mechanism uses a virtual file system derived from the cpuset code to provide a file system style (hierarchical names and classic Unix style file and directory permissions) naming of a partitioning of the tasks on a system.
By partitioning here, I mean a division of the tasks into several subsets, aka partition elements, which are non-overlapping and covering.
That is, each task is in one and only one of the partition elements, these partitions elements are named by the directories in the container file system, and the regular files in the container file system provide per-element attributes.
Then kernel facilities that can be considered as providing attributes for and control of subsets of tasks is represented as a controller, and attached to such a container.
Your dynamic scheduler mechanisms appear (from what I can tell after a brief glance) to be a candidate for being such a controller.
The upshot of this is that, if your work should proceed and eventually be considered for inclusion in the kernel (I have --no-- idea if that would be a good idea, either for the purposes of your student group, or for the kernel itself) then it would likely (if Menage's work is accepted) need to be recast as a "controller" in Menage's terms, not as an extension to cpusets.
If Menage succeeds, that should not actually be that big of a change, either semantically, or in coding details.
Good luck.
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.925.600.0401Quoted 4 lines · pj
See Paul Menage's most recent patch proposal at:
http://lkml.org/lkml/2006/11/17/217
Subject: [PATCH 0/6] Multi-hierarchy Process Containers
Date: Fri, 17 Nov 2006 11:11:59 -0800
I'm behind the times. Paul Menage's most recent proposal is at:
http://lkml.org/lkml/2006/11/23/95 Subject: [PATCH 0/7] Generic Process Containers (+ ResGroups/BeanCounters) Date: Thu, 23 Nov 2006 04:08:48 -0800
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.925.600.0401Quoted 3 lines · Paul Jackson
Your dynamic scheduler mechanisms appear (from what I can tell after a brief glance) to be a candidate for being such a controller.
Possibly, if it was some kind of multi-level scheduler - i.e. a top-level scheduler picks which container to run, and then a configurable per-container scheduler picks a task from that container.
But (having glanced at the code even less than you) it sounded like it was intended to be a single level scheduler, configured on a per-cpu basis. In that case tying it to (exclusive) cpusets sounds like it might be more reasonable.
Paul.
Quoted 3 lines
Possibly, if it was some kind of multi-level scheduler - i.e. a top-level scheduler picks which container to run, and then a configurable per-container scheduler picks a task from that container.
What do you think about Plugsched? Peter Williams introduced a simple scheduler interface for scheduler testing purposes. It should be possible to integrate into your CPU Controller. The CPU Controller seems to be quite suited for that purpose, as far as I can appraise after reading some of your documentation.
Unfortunately, I didn't know about CKRM some months ago, when we decided to use cpusets. As I wrote, Dynsched is a student project at sunset and it's future is quite ambiguous. Maybe we'll find some time to take a look at.
Quoted 4 lines
But (having glanced at the code even less than you) it sounded like it was intended to be a single level scheduler, configured on a per-cpu basis. In that case tying it to (exclusive) cpusets sounds like it might be more reasonable.
You're right. We tie up the scheduler to cpus with some per_cpu functionality, and chose cpusets for management, cause we didn't want add another interface (e.g sysfs files for each cpu). Cpusets is quite adequate for our purpose, but there are lots of issues we're currently dealing with. Think about (not exclusive) subsets with different schedulers...
Felix