RT scheduler config, suggestions and questions
Folks,
I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG:
sched_rt_period
sched_rt_ratio
I do not believe that is correct. I know that we do not want to expose scheduler knobs
in general but theses are not the heuristic kind of knobs. There is no way the scheduler can magically figure out what the correct setting should be here.
Also shouldn't those new RT features that recently went be configurable and _disabled_ by default ? For example "RT watchdog" and "RT throttling" actually seem very questionable. SCHED_FIFO is clearly defined as "
A SCHED_FIFO process runs until either it is blocked by an I/O request, it is preempted
by a higher priority process, or it calls sched_yield(2).
"
Both the watchdog and the throttling are clearly braking that rule. I think it's good to have those features but not enabled by default and certainly not with sysctls that disable them hidden under debugging. How about this:
- We introduce Kconfig options for them ?
- Expose all rt sysctls outside of #ifdef DEBUG
I can kook up some patches if that sounds ok.
----
btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread. Single setting per user is not enough. Unless a use has a single RT task.
Max
Quoted 9 lines · Max Krasnyanskiy
Folks,
I just realized that in latest Linus' tree following sysctls are under
SCHED_DEBUG:
sched_rt_period
sched_rt_ratioI do not believe that is correct. I know that we do not want to expose scheduler knobs in general but theses are not the heuristic kind of knobs. There is no way the scheduler can magically figure out what the correct setting should be here.
Yeah, since fixed.
Quoted 7 lines
Also shouldn't those new RT features that recently went be configurable and _disabled_ by default ? For example "RT watchdog" and "RT throttling" actually seem very questionable.
SCHED_FIFOis clearly defined as
"
ASCHED_FIFOprocess runs until either it is blocked by an I/O request, it is preempted by a higher priority process, or it callssched_yield(2).
"
The watchdog is disabled by default, the bandwidth is .95s every 1s, which is mainly a safe-guard against run-away real-time tasks. As long as real-time usage stays within those limits nothing happens. If you don't like it set sched_rt_runtime [*] to -1.
[*] provided in the interface changes posted a few days ago.
Quoted 5 lines
Both the watchdog and the throttling are clearly braking that rule. I think it's good to have those features but not enabled by default and certainly not with sysctls that disable them hidden under debugging.
How about this:
- We introduceKconfigoptions for them ?
I don't see why this would be needed.
Quoted 1 line
- Expose all rt sysctls outside of #ifdef DEBUG
Already did this somewhere along the line.
Quoted 2 lines
btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread.
It is.
Quoted 1 line
Single setting per user is not enough. Unless a use has a single RT task.
?
Quoted 8 lines · Peter Zijlstra
Quoted 2 lines
btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread.
It is.
Quoted 1 line
Single setting per user is not enough. Unless a use has a single RT task.
?
Ah, its starting to make sense, you want it configurable per thread, not process wide. Yes, I see how that is useful, just no idea how to expose that to user-space atm.
Quoted 4 lines · Peter Zijlstra
Quoted 2 lines · Peter Zijlstra
Quoted 2 lines
btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread.
Quoted 8 lines
Quoted 4 lines
It is.
Quoted 1 line
Single setting per user is not enough. Unless a use has a single RT task.
?
Ah, its starting to make sense, you want it configurable per thread, not process wide. Yes, I see how that is useful, just no idea how to expose that to user-space atm.
Yes. That's what I meant. I don't think overall per process setting is that useful. Per thread though would be useful.
How to expose that to the user-space ? The best option in my opinion is to extend
struct sched_param. That way both sched_setparam() and pthread_attr_setschedparam() can be used to set new attributes and it's backwards compatible. Something like:
struct sched_param {
...
unsigned int sched_rt_limit;
unsigned int sched_rt_...;
};Max
Quoted 21 lines · Max Krasnyanskiy
Quoted 4 lines · Peter Zijlstra
Quoted 2 lines · Peter Zijlstra
Quoted 2 lines
btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread.
Quoted 8 lines
Quoted 4 lines
It is.
Quoted 1 line
Single setting per user is not enough. Unless a use has a single RT task.
?
Ah, its starting to make sense, you want it configurable per thread, not process wide. Yes, I see how that is useful, just no idea how to expose that to user-space atm.
Yes. That's what I meant. I don't think overall per process setting is that useful. Per thread though would be useful.
How to expose that to the user-space ? The best option in my opinion is to extend struct
sched_param. That way bothsched_setparam()andpthread_attr_setschedparam()can be used to set new attributes and it's backwards compatible. Something like:
Quoted 5 lines
struct sched_param {
...
unsigned int sched_rt_limit;
unsigned int sched_rt_...;
};I'm somewhat afraid to touch sched_param, but it seems I need to change it anyway soonish, so I'd better figure out how to do that.
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Max Krasnyanskiy
Folks,
I just realized that in latest Linus' tree following sysctls are under
SCHED_DEBUG:
sched_rt_period
sched_rt_ratioI do not believe that is correct. I know that we do not want to expose scheduler knobs
in general but theses are not the heuristic kind of knobs. There is no way the scheduler can magically figure out what the correct setting should be here.Yeah, since fixed.
Super. I guess it's not in mainline yet. Or it wasn't yesterday.
Quoted 10 lines
Quoted 5 lines
Also shouldn't those new RT features that recently went be configurable and _disabled_ by default ? For example "RT watchdog" and "RT throttling" actually seem very questionable.
SCHED_FIFOis clearly defined as "
ASCHED_FIFOprocess runs until either it is blocked by an I/O request, it is preempted
by a higher priority process, or it callssched_yield(2).
"The watchdog is disabled by default, the bandwidth is .95s every 1s, which is mainly a safe-guard against run-away real-time tasks. As long as real-time usage stays within those limits nothing happens. If you don't like it set
sched_rt_runtime[*] to -1.
Oh. From looking at the code I assumed I need to set sched_rt_ratio to 65536. I guess you changed things around a bit. I'll look at the latest changes.
Quoted 5 lines
Quoted 3 lines
Both the watchdog and the throttling are clearly braking that rule. I think it's good to have those features but not enabled by default and certainly not with sysctls that disable them hidden under debugging. How about this:
- We introduceKconfigoptions for them ?I don't see why this would be needed.
Reduces code size and RT scheduler complexity for the cases when these features are not required.
For example my preference would for RT scheduler to be as lean as possible to avoid any kind of overhead and cache pollution. I know both watchdog and throttling end up being a single if() when disabled via sysctl, but it all adds up.
Quoted 4 lines
Quoted 1 line
- Expose all rt sysctls outside of #ifdef DEBUG
Already did this somewhere along the line.
Quoted 8 lines
Quoted 2 lines
btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread.
It is.
Quoted 1 line
Single setting per user is not enough. Unless a use has a single RT task.
?
Currently it's per process. Most RT apps will most likely have several RT threads.
Max