CKRM after a major overhaul
Quoted 10 lines · Chandra Seetharaman
Quoted 6 lines · Dave Hansen
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:
CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.
What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?
We think it is ready for -mm. But, want to go through a review cycle in lkml before i request Andrew for that.
IMHO, it would be a good idea to decouple the current implementation and reconnect them via an open mapper/wrapper to allow a more flexible/open approach to resource management, which may ease its transition into mainline, due to a step-by-step instead of an all-or-none approach.
Thanks!
--
Al
Quoted 6 lines · Al Boldi
Quoted 5 lines · Chandra Seetharaman
Quoted 4 lines · Dave Hansen
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:
CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?
We think it is ready for -mm. But, want to go through a review cycle in lkml before i request Andrew for that.
IMHO, it would be a good idea to decouple the current implementation and reconnect them via an open mapper/wrapper to allow a more flexible/open
I am not understanding your comment, can you please elaborate.
Quoted 2 lines
approach to resource management, which may ease its transition into mainline, due to a step-by-step instead of an all-or-none approach.
BTW, the design does allow step by step approach to resource management. You can add individual resource control one at a time, or even turn on only the resources you are interested in.
Quoted 12 lines
Thanks!
--
Al-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 4 lines
----------------------------------------------------------------------
Chandra Seetharaman | Be careful what you choose....
- [email protected] | .......you may get it.
----------------------------------------------------------------------Quoted 17 lines · Al Boldi
Quoted 10 lines · Chandra Seetharaman
Quoted 6 lines · Dave Hansen
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:
CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.
What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?
We think it is ready for -mm. But, want to go through a review cycle in lkml before i request Andrew for that.
IMHO, it would be a good idea to decouple the current implementation and reconnect them via an open mapper/wrapper to allow a more flexible/open approach to resource management, which may ease its transition into mainline, due to a step-by-step instead of an all-or-none approach.
Quoted 4 lines
Thanks!
--
Al
Hi Al,
I'm sorry, I don't understand what you're suggesting. Could you please
elaborate on how you think it should be decoupled?
Thanks,
-Matt Helsley
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Al Boldi
i.e: it should be possible to run the RCs w/o CKRM.
The current design pins the RCs on CKRM, when in fact this is not necessary. One way to decouple them, could be to pin them against pid, thus allowing an RC to leverage the pid hierarchy w/o the need for CKRM. And only when finer RM control is necessary, should CKRM come into play, by dynamically adjusting the RC to achieve the desired effect.
This model works well in universities, where you associate some resource when a student logs in, or a virtualised environment (like a UML or vserver), where you attach resource to the root process.
It doesn't work with web servers, database servers etc.,, where the main application will be forking tasks for different set of end users. In that case you have to group tasks that are not related to one another and attach resources to them.
Having a unified interface gives the system administrator ability to group the tasks as they see them in real life (a department or important transactions or just critical apps in a desktop).
It also has the added advantage that the resource controller writer do not have to spend their time in coming up with an interface for their controller. On the other hand if they do, the user finally ends up with multiple interface (/proc, sysfs, configfs, /dev etc.,) to do their resource management.
Quoted 6 lines
Thanks!
--
Al
Signature · 4 lines
----------------------------------------------------------------------
Chandra Seetharaman | Be careful what you choose....
- [email protected] | .......you may get it.
----------------------------------------------------------------------Quoted 13 lines · Chandra Seetharaman
Quoted 7 lines · Al Boldi
i.e: it should be possible to run the RCs w/o CKRM.
The current design pins the RCs on CKRM, when in fact this is not necessary. One way to decouple them, could be to pin them against pid, thus allowing an RC to leverage the pid hierarchy w/o the need for CKRM.
And only when finer RM control is necessary, should CKRM come into
play, by dynamically adjusting the RC to achieve the desired effect.This model works well in universities, where you associate some resource when a student logs in, or a virtualised environment (like a UML or vserver), where you attach resource to the root process.
Quoted 8 lines
It doesn't work with web servers, database servers etc.,, where the main application will be forking tasks for different set of end users. In that case you have to group tasks that are not related to one another and attach resources to them.
Having a unified interface gives the system administrator ability to group the tasks as they see them in real life (a department or important transactions or just critical apps in a desktop).
So, why drag this unified interface around when it is only needed in certain models. The underlying interface via pid comes for free and should be leveraged as such to yield a low overhead implementation. Then maybe, when a more complex model is involved should CKRM come into play.
Quoted 5 lines
It also has the added advantage that the resource controller writer do not have to spend their time in coming up with an interface for their controller. On the other hand if they do, the user finally ends up with multiple interface (/proc, sysfs, configfs, /dev etc.,) to do their resource management.
So, maybe what is needed is an abstract parent RC that implements this interface and lets the child RCs implement the specifics, and allows CKRM to connect to the parent RC to allow finer RM control when a specific model requires it.
Thanks!
--
Al
Quoted 19 lines · Al Boldi
Quoted 18 lines · Chandra Seetharaman
Quoted 7 lines · Al Boldi
i.e: it should be possible to run the RCs w/o CKRM.
The current design pins the RCs on CKRM, when in fact this is not necessary. One way to decouple them, could be to pin them against pid, thus allowing an RC to leverage the pid hierarchy w/o the need for CKRM.
And only when finer RM control is necessary, should CKRM come into
play, by dynamically adjusting the RC to achieve the desired effect.This model works well in universities, where you associate some resource when a student logs in, or a virtualised environment (like a UML or vserver), where you attach resource to the root process.
It doesn't work with web servers, database servers etc.,, where the main application will be forking tasks for different set of end users. In that case you have to group tasks that are not related to one another and attach resources to them.
Quoted 8 lines
Quoted 3 lines
Having a unified interface gives the system administrator ability to group the tasks as they see them in real life (a department or important transactions or just critical apps in a desktop).
So, why drag this unified interface around when it is only needed in certain models. The underlying interface via pid comes for free and should be leveraged as such to yield a low overhead implementation. Then maybe, when a more complex model is involved should CKRM come into play.
Assuming I'm not misinterpretting your brief description above:
The interface "via pid" does not come for free. You'd essentially
attach the shares structures to the task and implement inheritance and hierarchy of those shares during fork -- hardly lower overhead when you consider that in most cases the number of tasks is going to be much larger than the number of classes. Furthermore this would mean duplicating the loops in ckrm_alloc_class, ckrm_free_class, ckrm_register_controller, and ckrm_unregister_controller. I suspect the loops would be deeper, more complex, execute more frequently, and have a much wider performance impact when you consider that we'd be dealing with the task struct directly instead of a class. The class structure effectively factors most of the loops out of the fork() and exit() paths and into mkdir() rmdir() calls that create and remove classes. The remaining loops in fork() and exit() paths are proportional to the number of resource controllers -- currently limitedto 8 by CKRM_MAX_RES_CTLRS.
Classes also have an advantage when it comes to administrating resource
management -- they are created and destroyed by an administrator and hence are easier to control. In contrast, the resource management decisions associated purely with tasks would disappear with the task. In many cases a task would be too short-lived for an administrator to manually intervene even if swarms of these tasks are created. Having this orthogonal hierarchy gives us the opportunity to manage all of these situations via a common interface and factors out overhead from the per-task solution you seem to be advocating.
I'm willing to discuss your ideas without patches but I think patches (even if incomplete) would be clearer.
Quoted 10 lines
Quoted 5 lines
It also has the added advantage that the resource controller writer do not have to spend their time in coming up with an interface for their controller. On the other hand if they do, the user finally ends up with multiple interface (/proc, sysfs, configfs, /dev etc.,) to do their resource management.
So, maybe what is needed is an abstract parent RC that implements this interface and lets the child RCs implement the specifics, and allows CKRM to connect to the parent RC to allow finer RM control when a specific model requires it.
I'm not sure what advantage that would give compared to CKRM as it
stands now -- it sounds much more complex. Could you give an example of what kind of interfaces you're suggesting?
Cheers,
-Matt Helsley
Quoted 31 lines · Matt Helsley
Quoted 24 lines · Al Boldi
Quoted 14 lines · Chandra Seetharaman
Quoted 8 lines · Al Boldi
i.e: it should be possible to run the RCs w/o CKRM.
The current design pins the RCs on CKRM, when in fact this is not necessary. One way to decouple them, could be to pin them against pid, thus allowing an RC to leverage the pid hierarchy w/o the need for CKRM. And only when finer RM control is necessary, should CKRM come into play, by dynamically adjusting the RC to achieve the desired effect.
This model works well in universities, where you associate some resource when a student logs in, or a virtualised environment (like a UML or vserver), where you attach resource to the root process.
Quoted 8 lines
It doesn't work with web servers, database servers etc.,, where the main application will be forking tasks for different set of end users. In that case you have to group tasks that are not related to one another and attach resources to them.
Having a unified interface gives the system administrator ability to group the tasks as they see them in real life (a department or important transactions or just critical apps in a desktop).
Quoted 5 lines
So, why drag this unified interface around when it is only needed in certain models. The underlying interface via pid comes for free and should be leveraged as such to yield a low overhead implementation. Then maybe, when a more complex model is involved should CKRM come into play.
Quoted 5 lines
Assuming I'm not misinterpretting your brief description above:
The interface "via pid" does not come for free. You'd essentially
attach the shares structures to the task and implement inheritance and hierarchy of those shares during fork
No, attach the shares struct to the parent RC, and allow it to take advantage of the free pid hierarchy.
Quoted 2 lines
I'm willing to discuss your ideas without patches but I think patches (even if incomplete) would be clearer.
The discussion here is more about design rather than implementation.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
It also has the added advantage that the resource controller writer do not have to spend their time in coming up with an interface for their controller. On the other hand if they do, the user finally ends up with multiple interface (/proc, sysfs, configfs, /dev etc.,) to do their resource management.
Quoted 8 lines
Quoted 4 lines
So, maybe what is needed is an abstract parent RC that implements this interface and lets the child RCs implement the specifics, and allows CKRM to connect to the parent RC to allow finer RM control when a specific model requires it.
I'm not sure what advantage that would give compared to CKRM as it
stands now -- it sounds much more complex. Could you give an example of what kind of interfaces you're suggesting?
Nothing wrong w/ CKRM per se, other than its monolithic approach.
The suggestion here would be to modularize CKRM by removing dependencies, effectively splitting CKRM into 3 parts:
RM --- RC parent (no-op) / | \ RC child (ntask, cpu, mem, .....)
So it could be possible to:
1. Load the RC parent to provide for simple stats based on pid-hierarchy.
2. Load an RC child for rc-enforcement.
3. Load the RM for finer control across different tasks by way of an orthogonal hierarchy.
Although this may look more complex than the monolithic approach, it is in fact lots simpler, due to its "division of labor" approach.
Thanks!
--
Al
Quoted 22 lines · Al Boldi
Quoted 21 lines · Matt Helsley
Quoted 20 lines · Al Boldi
Quoted 19 lines · Chandra Seetharaman
Quoted 8 lines · Al Boldi
i.e: it should be possible to run the RCs w/o CKRM.
The current design pins the RCs on CKRM, when in fact this is not necessary. One way to decouple them, could be to pin them against pid, thus allowing an RC to leverage the pid hierarchy w/o the need for CKRM. And only when finer RM control is necessary, should CKRM come into play, by dynamically adjusting the RC to achieve the desired effect.
This model works well in universities, where you associate some resource when a student logs in, or a virtualised environment (like a UML or vserver), where you attach resource to the root process.
It doesn't work with web servers, database servers etc.,, where the main application will be forking tasks for different set of end users. In that case you have to group tasks that are not related to one another and attach resources to them.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Having a unified interface gives the system administrator ability to group the tasks as they see them in real life (a department or important transactions or just critical apps in a desktop).
So, why drag this unified interface around when it is only needed in certain models. The underlying interface via pid comes for free and
The "pid tree" approach will not allow ISPs to provide workload management capabilities inside a virual server also.
Quoted 1 line
Quoted 1 line
Quoted 1 line
should be leveraged as such to yield a low overhead implementation.
As you can see "pid based resource control" does not lead to a low overhead implementation.
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
Then maybe, when a more complex model is involved should CKRM come into play.
Assuming I'm not misinterpretting your brief description above:
The interface "via pid" does not come for free. You'd essentially
attach the shares structures to the task and implement inheritance and hierarchy of those shares during fork
Quoted 7 lines
No, attach the shares struct to the parent RC, and allow it to take advantage of the free pid hierarchy.
Quoted 2 lines
I'm willing to discuss your ideas without patches but I think patches (even if incomplete) would be clearer.
The discussion here is more about design rather than implementation.
"pid based RC" does sound as easily understandable design. But IMHO, we should consider how the implementation will be (in this context comparing it with CKRM). As Matt pointed it may not be any less complex.
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
It also has the added advantage that the resource controller writer do not have to spend their time in coming up with an interface for their controller. On the other hand if they do, the user finally ends up with multiple interface (/proc, sysfs, configfs, /dev etc.,) to do their resource management.
Quoted 8 lines
Quoted 4 lines
So, maybe what is needed is an abstract parent RC that implements this interface and lets the child RCs implement the specifics, and allows CKRM to connect to the parent RC to allow finer RM control when a specific model requires it.
I'm not sure what advantage that would give compared to CKRM as it
stands now -- it sounds much more complex. Could you give an example of what kind of interfaces you're suggesting?
Quoted 9 lines
Nothing wrong w/ CKRM per se, other than its monolithic approach.
The suggestion here would be to modularize CKRM by removing dependencies, effectively splitting CKRM into 3 parts:
RM --- RC parent (no-op) / | \ RC child (ntask, cpu, mem, .....)
If the "RC parent" is _only_ going to allow attaching resource shares with a "pid hierarchy", then how can an RM attach unrelated tasks to any resource share ?
CKRM brings in grouping of unrelated tasks, which IMO is not possible with the "pid tree" approach. On the other hand, CKRM takes care of different scenarios.
Quoted 14 lines
So it could be possible to:
1. Load the RC parent to provide for simple stats based on pid-hierarchy.
2. Load an RC child for rc-enforcement.
3. Load the RM for finer control across different tasks by way of an orthogonal hierarchy.Although this may look more complex than the monolithic approach, it is in fact lots simpler, due to its "division of labor" approach.
Thanks!
--
Al
Signature · 4 lines
----------------------------------------------------------------------
Chandra Seetharaman | Be careful what you choose....
- [email protected] | .......you may get it.
----------------------------------------------------------------------