sched: rt-bandwidth fixes

It fixes an accounting bug where we would continue accumulating runtime even though the bandwidth control is disabled. This would lead to very long throttle periods once bandwidth control gets turned on again.

Signed-off-by Peter Zijlstra <[email protected]>

---
 kernel/sched_rt.c |   11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

Index: linux-2.6/kernel/sched_rt.c
===================================================================

kernel/sched_rt.c +0 -4

@@ -438,9 +438,6 @@ static int sched_rt_runtime_exceeded(str
 {
 	u64 runtime = sched_rt_runtime(rt_rq);
 
-	if (runtime == RUNTIME_INF)
-		return 0;
-
 	if (rt_rq->rt_throttled)
 		return rt_rq_throttled(rt_rq);
 

@@ -491,9 +488,11 @@ static void update_curr_rt(struct rq *rq
 		rt_rq = rt_rq_of_se(rt_se);
 
 		spin_lock(&rt_rq->rt_runtime_lock);
-		rt_rq->rt_time += delta_exec;

Quoted 26 lines · Peter Zijlstra

It fixes an accounting bug where we would continue accumulating runtime even though the bandwidth control is disabled. This would lead to very long throttle periods once bandwidth control gets turned on again.

Signed-off-by Peter Zijlstra <[email protected]>

---
 kernel/sched_rt.c |   11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

Index: linux-2.6/kernel/sched_rt.c
===================================================================

kernel/sched_rt.c +0 -3

@@ -438,9 +438,6 @@ static int sched_rt_runtime_exceeded(str
 {
 	u64 runtime = sched_rt_runtime(rt_rq);
 
-	if (runtime == RUNTIME_INF)
-		return 0;
-
 	if (rt_rq->rt_throttled)
 		return rt_rq_throttled(rt_rq);
 

@@ -491,9 +488,11 @@ static void update_curr_rt(struct rq *rq
 		rt_rq = rt_rq_of_se(rt_se);
 

kernel/sched_rt.c static void update_curr_rt(struct rq *rq · line 498

Quoted 12 lines

kernel/sched_rt.c +5 -3

 		spin_lock(&rt_rq->rt_runtime_lock);
-		rt_rq->rt_time += delta_exec;
-		if (sched_rt_runtime_exceeded(rt_rq))
-			resched_task(curr);
+		if (sched_rt_runtime(rt_rq) != RUNTIME_INF) {
+			rt_rq->rt_time += delta_exec;
+			if (sched_rt_runtime_exceeded(rt_rq))
+				resched_task(curr);
+		}
 		spin_unlock(&rt_rq->rt_runtime_lock);
 	}
 }

This will make 'disabled' case more expensive, will it not ? I mean now we'll have to run balance_runtime() even if throttling is disabled.

Do you guys mind if I make this stuff configurable ? ie Just like CONFIG_RT_GROUP_SCHED we could add CONFIG_RT_BANDWIDTH_THROTTLE.

Disable bandwidth control by default.

Signed-off-by Peter Zijlstra <[email protected]>

---
 kernel/sched.c |   17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

Index: linux-2.6/kernel/sched.c
===================================================================

The fixes look good to me, but this enabling of infinite RT task lockups is not an improvement.

The thing is, i got far more bugreports about locked up RT tasks where the lockup was unintentional, than real bugreports about anyone _intending_ for the whole box to come to a grinding halt because a high-prio RT tasks is monopolizing the CPU.

In fact there's only been this artificial test so far.

So could you please just increase the chunking to 10 seconds or so, from the current 1 second? Anyone locking up the system for more than 10 seconds via an RT task has to deal with many other issues already.

I.e. keep the system borderline debuggable (up to 10 seconds delays are _not_ nice so people will notice) - but it's still a marked improvement from completly locked up desktops.

And those who really need longer than 10 second periods can set it higher, or even (if they want to live dangerously or run POSIX conformance tests) make it infinite (set it to -1) - and will have to deal with other things like the softlockup watchdog as well.

Ok?

From fc21334298056c1e0d6428d3abe46b104188a05e Mon Sep 17 00:00:00 2001

Quoted 12 lines · Peter Zijlstra

Disable bandwidth control by default.

Signed-off-by Peter Zijlstra <[email protected]>

---
 kernel/sched.c |   17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

Index: linux-2.6/kernel/sched.c
===================================================================

kernel/sched.c static __read_mostly int scheduler_runni · line 829

Quoted 8 lines

kernel/sched.c +2 -2

@@ -824,9 +824,9 @@ static __read_mostly int scheduler_runni
 
 /*
  * part of the period that we allow rt tasks to run in us.
- * default: 0.95s
+ * default: inf
  */
-int sysctl_sched_rt_runtime = 950000;
+int sysctl_sched_rt_runtime = -1;

The fixes look good to me, but this enabling of infinite RT task lockups is not an improvement.

The thing is, i got far more bugreports about locked up RT tasks where the lockup was unintentional, than real bugreports about anyone _intending_ for the whole box to come to a grinding halt because a high-prio RT tasks is monopolizing the CPU.

In fact there's only been this artificial test so far.

So could you please just increase the chunking to 10 seconds or so, from the current 1 second? Anyone locking up the system for more than 10 seconds via an RT task has to deal with many other issues already.

I.e. keep the system borderline debuggable (up to 10 seconds delays are _not_ nice so people will notice) - but it's still a marked improvement from completly locked up desktops.

And those who really need longer than 10 second periods can set it higher, or even (if they want to live dangerously or run POSIX conformance tests) make it infinite (set it to -1) - and will have to deal with other things like the softlockup watchdog as well.

Ok?

[...] Let's retain our API specifications and backwards compatibilty by default. [...]

Quoted 1 line · Nick Piggin

[...] Let's retain our API specifications and backwards compatibilty by default. [...]

I agree with you that the 1 second default was a bit too tight - and we should definitely change that (and it's changed already).

So changing the "allow RT tasks up to 10 seconds uninterrupted CPU monopolization" is OK to me - it still keeps runaway CPU loops (which are in the vast majority) debuggable, while allowing common-sense RT task usage.

But changing that back to the other extreme: "allow lockups by default" is unreasonable IMO - especially in the face of rtlimit that allows unprivileged tasks to gain RT privileges.

As an experiment try running a 100% CPU using SCHED_FIFO:99 RT task. It does not result in a usable Linux system - it interacts with too many normal system activities. It is a very, very special mode of operation and anyone using Linux in such a way has to take precautions and has to tune things specially anyway. (has to turn off the softlockup watchdog, has to make sure IO requests do not time out artificially, etc.)

You wont even get normal keyboard or console behavior in most cases.

Furthermore, if by "API specifications" you mean POSIX - to get a conformant POSIX run one has to change a lot of things on a typical Linux system anyway. APIs and utilities have to be crippled to be "POSIX compliant".

In other words: we use common sense when thinking about specifications. The kernel's defaults are about being reasonable by default.

I have no _strong_ feelings about it, but i dont see the practical value in going beyond 10 seconds - as it turns a rather useful robustness feature off by default (and keeps it untested, etc.).

Quoted 2 lines · Nick Piggin

[...] Let's retain our API specifications and backwards compatibilty by default. [...]

I agree with you that the 1 second default was a bit too tight - and we should definitely change that (and it's changed already).

So changing the "allow RT tasks up to 10 seconds uninterrupted CPU monopolization" is OK to me - it still keeps runaway CPU loops (which are in the vast majority) debuggable, while allowing common-sense RT task usage.

But changing that back to the other extreme: "allow lockups by default" is unreasonable IMO - especially in the face of rtlimit that allows unprivileged tasks to gain RT privileges.

As an experiment try running a 100% CPU using SCHED_FIFO:99 RT task. It does not result in a usable Linux system - it interacts with too many normal system activities. It is a very, very special mode of operation and anyone using Linux in such a way has to take precautions and has to tune things specially anyway. (has to turn off the softlockup watchdog, has to make sure IO requests do not time out artificially, etc.) You wont even get normal keyboard or console behavior in most cases.

Furthermore, if by "API specifications" you mean POSIX - to get a conformant POSIX run one has to change a lot of things on a typical Linux system anyway. APIs and utilities have to be crippled to be "POSIX compliant".

In other words: we use common sense when thinking about specifications. The kernel's defaults are about being reasonable by default.

I have no _strong_ feelings about it, but i dont see the practical value in going beyond 10 seconds - as it turns a rather useful robustness feature off by default (and keeps it untested, etc.).

Quoted 6 lines · Ingo Molnar

Quoted 2 lines · Nick Piggin

[...] Let's retain our API specifications and backwards compatibilty by default. [...]

I agree with you that the 1 second default was a bit too tight - and we should definitely change that (and it's changed already).

I do not agree that it is too tight, it is just plain wrong.

Quoted 4 lines

So changing the "allow RT tasks up to 10 seconds uninterrupted CPU monopolization" is OK to me - it still keeps runaway CPU loops (which are in the vast majority) debuggable, while allowing common-sense RT task usage.

RT tasks have always been debuggable by using a simple watchdog thread. As I said before, someone who develops a non-trivial RT app without a watchdog thread or isolated CPU basically doesn't deserve the honour of us breaking our API to cater for their idiocity.

But even for those people, we now have the sysrq trigger too. And also we'll still have the rt throttle sysctl that can be changed at runtime.

There are so many options... "oh but maybe they didn't research the options either so let's break our APIs instead" is not common sense IMO.

Quoted 3 lines

But changing that back to the other extreme: "allow lockups by default" is unreasonable IMO - especially in the face of rtlimit that allows unprivileged tasks to gain RT privileges.

No, it's not "allow lockups by default". It is "follow the API and backwards compatibility by default".

If some distro has gone and given all users RTPRIO rlimit by default and allowed unprivileged users to lock up the system, it is not the problem of the upstream kernel. That distro can set the rt throttle default if it wants to. Or provide a watchdog thread for debugging RT tasks.

Quoted 7 lines

As an experiment try running a 100% CPU using SCHED_FIFO:99 RT task. It does not result in a usable Linux system - it interacts with too many normal system activities. It is a very, very special mode of operation and anyone using Linux in such a way has to take precautions and has to tune things specially anyway. (has to turn off the softlockup watchdog, has to make sure IO requests do not time out artificially, etc.) You wont even get normal keyboard or console behavior in most cases.

This is exactly what *real* RT app/system developers do. I'm not talking about an untuned desktop system!!

Quoted 4 lines

Furthermore, if by "API specifications" you mean POSIX - to get a conformant POSIX run one has to change a lot of things on a typical Linux system anyway. APIs and utilities have to be crippled to be "POSIX compliant".

By that argument we can break any userspace API for any reason.

Quoted 2 lines

In other words: we use common sense when thinking about specifications. The kernel's defaults are about being reasonable by default.

It's not common sense to change this. It would be perfectly valid to engineer a realtime process that uses a peak of say 90% of the CPU with a 10% margin for safety and other services. Now they only have 5%.

Or a realtime app could definitely use the CPU adaptively up to 100% but still unable to tolerate an unexpected preemption.

I don't know how you can change this so significantly and be so sure of yourself that you won't break anything (actually you already have one reported breakage in this thread).

Quoted 3 lines

I have no _strong_ feelings about it, but i dont see the practical value in going beyond 10 seconds - as it turns a rather useful robustness feature off by default (and keeps it untested, etc.).

I feel strongly about it.

The primary issue is that we have broken the API from both specification and previous implementation, the answer is yes. That *you* can't see any reason to use the API in that way kind of pales in comparison with all due respect. Especially as you already got a counter example of someone's app that broke.

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

and what do you mean by "trivial to solve in user-space"?

Quoted 6 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Disagree with what? That it's a problem to basically break the guarantee realtime SCHED_ policies have previously provided?

Quoted 1 line

and what do you mean by "trivial to solve in user-space"?

I mean that if some distro has turned on the RT scheduling ulimit by default and now finds themselves with a local DoS for unpriviliged users as a result, then either that distro should just make their init scripts set the throttle and break the API themselves, or they should start a watchdog at a higher priority than unprivileged user can set.

Quoted 10 lines · Nick Piggin

Quoted 6 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Disagree with what? That it's a problem to basically break the guarantee realtime SCHED_ policies have previously provided?

I think you are sticking to the rigid letter of some standard without seeing the bigger picture.

Firstly, please realize that to do a "successful" POSIX or other conformance run a default Linux distribution has to be tweaked and often crippled literally dozens and often hundreds of ways. In this case you also have to add one more entry to /etc/sysctl.conf, to allow RT tasks to monopolize CPU time. So you can still get the POSIX sticker if you want to - nothing changed about that.

Secondly, my big picture point is that our task is to make Linux more useful and more usable by default. You seem to be arguing that RT tasks should be allowed by default to monopolize all CPU time forever, and i disagree with that proposition.

But do _you_ actually use such runaway CPU-monopolizing RT tasks? Try it one day and you'll quickly meet various practical problems. Let a SCHED_FIFO:99 RT task run long enough and on all the main distributions you will get:

  BUG: soft lockup - CPU#1 stuck for 61s! [bash:3659]

But monopolizing any resource in a 100% way (which you are arguing for) is just not a generic Linux system and for years (seeing all the practical problems with it) we tried various methods to contain SCHED_FIFO tasks in the scheduler, none was really acceptable for mainline.

Peter's changes were clean and useful at last. There's lots of apps that use SCHED_FIFO for a short burst of activity, and 100% of the ones i know do not want to run for longer than 10 seconds.

Thirdly, your argument can only be consistent if you also argue for the softlockup watchdog to be disabled. Do you make that point?

Quoted 7 lines

Quoted 1 line

and what do you mean by "trivial to solve in user-space"?

I mean that if some distro has turned on the RT scheduling ulimit by default and now finds themselves with a local DoS for unpriviliged users as a result, then either that distro should just make their init scripts set the throttle and break the API themselves, or they should start a watchdog at a higher priority than unprivileged user can set.

... but that's by far not the only usecase. Very frequently i've seen bugreports from people with runaway RT tasks (which tasks were running as root) where that runaway behavior was completely unintended. Audio apps or other apps getting into a loop and locking up the system.

Worse than that, such bugs prevented the system from being debugged by plain users. A runaway RT task that monopolizes the CPU will lock it up completely, requiring a hard reset or a power cycle. That can lose data, etc. If we allow it to lock up the CPU for up to 10 seconds it will still be noticed if that is unintentional (the system is very slow), but the problem can be debugged.

By making RT tasks not lock up like that by default and allowing them to 'only' monopolize the CPU up to 10 seconds, we make the system more debuggable and more useful in general. It is a quite reasonable proposition that makes Linux useful in general, and you seem to be ignoring that practical angle altogether. It's not about allowing user-space rtprio-rlimit driven apps to not run away, it's about allowing _any_ RT task to be throttled by default if they run away.

On the other side of the equation, what exact application do you know that absolutely relies on being able to monopolize all CPU time in excess of 10 seconds? I havent heard much about that usecase. Why does that particular RT app do it, because that behavior sounds _very_ weird to me.

If it's some embedded system or other special-purpose app then it can tweak the sysctl no problem. (it will have to do it anyway, to turn off the softlockup watchdog)

If it's some general purpose Linux app, exactly which one is it? If it's an OSS app please give me an URL to its source code, we need to fix it urgently. Running for more than 10 seconds wastes power like mad and is generally a very un-nice thing to do.

All in one, since the 'buggy RT app runs into a loop and monopolizes the CPU' case is much more common, i do think that supporting that usecase is the better choice for a default.

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Quoted 5 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

Quoted 1 line

I disagree

Your arguments were along the line of:

* It probably doesn't break anything (except we had somebody report that it breaks their app)

* If it does break something then they must be doing something stupid (I refuted that because there are several legitimate ways to use rt scheduling that is broken by this)

* We have many other APIs and tools that don't conform to posix (why is that a reason to break this one?)

* We should break the API to cater for stupid users and distros who create local DoS and/or lock up their boxes (except this is trivial to solve by setting sysctls or having a watchdog or using sysrq)

Quoted 10 lines · Nick Piggin

Quoted 6 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Your arguments were along the line of:

Quoted 2 lines

* It probably doesn't break anything (except we had somebody report that it breaks their app)

I'm a real-time oldtimer. An application which hogs the CPU for 9.9 seconds with SCHED_FIFO priority is just broken. It's broken beyond all limits, whether POSIX allows to do that or Linux obeyed the request of the braindamaged application design.

Quoted 6 lines

* If it does break something then they must be doing something stupid (I refuted that because there are several legitimate ways to use rt scheduling that is broken by this)

* We have many other APIs and tools that don't conform to posix (why is that a reason to break this one?)

Simply because we use common sense instead of following every single POSIX brainfart by the letter.

Quoted 3 lines

* We should break the API to cater for stupid users and distros who create local DoS and/or lock up their boxes (except this is trivial to solve by setting sysctls or having a watchdog or using sysrq)

For the vast majority of users and RT developers a sane default of sanity measures is useful and sensible.

If someone wants to shoot himself in the foot then it's not an unreasonable request that he needs to disable the safety guards before pulling the trigger.

Oh with this much handwaving from you old timers I feel much better about it ;) I bet before the bug report and change to 10s, any application that hogged the CPU for more than 0.9 seconds was just broken too, right? But 10s is more than enough for everybody?

Quoted 2 lines

Simply because we use common sense instead of following every single POSIX brainfart by the letter.

How is that a brainfart? It is simple, relatively unambiguous, and not arbitrary. You really say the POSIX specified behaviour is "a brainfart", but adding an arbitrary 10s throttle "but the process might be preempted and lose the CPU to a lower priority task if it uses 10s of consecutive CPU time" would eliminate that brainfart? I have to laugh.

root is allowed to shoot themselves in the foot. root is the safeguard.

Quoted 6 lines

Oh with this much handwaving from you old timers I feel much better about it ;) I bet before the bug report and change to 10s, any application that hogged the CPU for more than 0.9 seconds was just broken too, right? But 10s is more than enough for everybody?

Actually, any real-time application which hogs the CPU at a high real-time priority for more than one second is probably doing something broken. The whole point of high real-time priorities is to do something really fast, get in and get out. Usually such routines are measured in milliseconds or microseconds.

Think about it *this* way --- what would you think of some device driver which hogged an interrupt for a full second, never mind 10 seconds. You'd say it was broken, right? Now consider that a high real-time priority thread might be running at a higher priority than interrupt handlers, and in fact could preempt interrupt handlers....

Quoted 8 lines

Quoted 2 lines

Simply because we use common sense instead of following every single POSIX brainfart by the letter.

How is that a brainfart? It is simple, relatively unambiguous, and not arbitrary. You really say the POSIX specified behaviour is "a brainfart", but adding an arbitrary 10s throttle "but the process might be preempted and lose the CPU to a lower priority task if it uses 10s of consecutive CPU time" would eliminate that brainfart? I have to laugh.

We've not followed POSIX before when it hasn't made sense. For example, "df" and "du" report its output in kilobytes, instead of 512 byte sectors, per POSIX's demands.

Quoted 1 line

root is allowed to shoot themselves in the foot. root is the safeguard.

We've done things before to make things harder for root; for example we've restricted what /dev/mem can do. And root can always lift the ulimit.

- Ted

Sorry, the world of embedded programming is sometime stranger than in theory. Normally it would not happen that a real-time process locks the CPU for more than 1 sec. But in some circumstances, especially FPGA initialisation and long term measurements it is possible that the real-time process locks the cpu for more than a, sometime for more than 10 sec. If the embedded program has designed it in that way, this behaviour is desired.

What coming at next? A device driver manager, which kills any driver which use to much CPU resource? Or throttle/kicks off the responsible driver if the hardware generates to many interrupts?

Quoted 11 lines · Thomas Gleixner

Quoted 10 lines · Nick Piggin

Quoted 6 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Your arguments were along the line of:

Quoted 7 lines

Quoted 2 lines

* It probably doesn't break anything (except we had somebody report that it breaks their app)

I'm a real-time oldtimer. An application which hogs the CPU for 9.9 seconds with SCHED_FIFO priority is just broken. It's broken beyond all limits, whether POSIX allows to do that or Linux obeyed the request of the braindamaged application design.

Oh with this much handwaving from you old timers I feel much better about it ;) I bet before the bug report and change to 10s, any application that hogged the CPU for more than 0.9 seconds was just broken too, right? But 10s is more than enough for everybody?

I may not be an old timer, but I can say the kernel is just broken if it deliberately deviates from standards to undocumented behaviour, and even more so if it changes from working to broken behaviour for reasons that can be worked around in userspace (eg. running a higher priority watchdog).

Quoted 7 lines

Quoted 6 lines

* If it does break something then they must be doing something stupid (I refuted that because there are several legitimate ways to use rt scheduling that is broken by this)

* We have many other APIs and tools that don't conform to posix (why is that a reason to break this one?)

Quoted 2 lines

Simply because we use common sense instead of following every single POSIX brainfart by the letter.

How is that a brainfart? It is simple, relatively unambiguous, and not arbitrary. You really say the POSIX specified behaviour is "a brainfart", but adding an arbitrary 10s throttle "but the process might be preempted and lose the CPU to a lower priority task if it uses 10s of consecutive CPU time" would eliminate that brainfart? I have to laugh.

Quoted 6 lines

Quoted 3 lines

* We should break the API to cater for stupid users and distros who create local DoS and/or lock up their boxes (except this is trivial to solve by setting sysctls or having a watchdog or using sysrq)

For the vast majority of users and RT developers a sane default of sanity measures is useful and sensible.

You seriously develop complex rt tasks without having at least a simple watchdog task?

Quoted 3 lines

If someone wants to shoot himself in the foot then it's not an unreasonable request that he needs to disable the safety guards before pulling the trigger.

root is allowed to shoot themselves in the foot. root is the safeguard.

Well, we might have a public opinion poll, whether a system is declared frozen after 1, 10 or 100 seconds. Even a one second unresponsivness shows up on the kernel bugzilla and you request that unlimited unresponsivness w/o a chance to debug it is the sane default.

Quoted 5 lines · Thomas Gleixner

Well, we might have a public opinion poll, whether a system is declared frozen after 1, 10 or 100 seconds. Even a one second unresponsivness shows up on the kernel bugzilla and you request that unlimited unresponsivness w/o a chance to debug it is the sane default.

That assumes single CPU. With multiple CPUs and not all hogged the system should be still responsive?

Quoted 7 lines · Andi Kleen

Quoted 5 lines · Thomas Gleixner

Well, we might have a public opinion poll, whether a system is declared frozen after 1, 10 or 100 seconds. Even a one second unresponsivness shows up on the kernel bugzilla and you request that unlimited unresponsivness w/o a chance to debug it is the sane default.

Quoted 2 lines

That assumes single CPU. With multiple CPUs and not all hogged the system should be still responsive?

Right.

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

While there's obviously CPU
time available on other CPUs, you cannot run 'top', the desktop will freeze, work flows of the system can be stuck, etc, etc..

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

The load balancer will not notice that a particular CPU is busy with real time tasks?

Quoted 7 lines · Andi Kleen

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

The load balancer will not notice that a particular CPU is busy with real time tasks?

Not currently, working on that though.

Quoted 7 lines · Andi Kleen

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

The load balancer will not notice that a particular CPU is busy with real time tasks?

Not currently, working on that though.

Quoted 6 lines · Peter Zijlstra

Quoted 5 lines · Andi Kleen

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

Quoted 4 lines

Quoted 2 lines

The load balancer will not notice that a particular CPU is busy with real time tasks?

Not currently, working on that though.

I wonder if it would make sense to break affinities in extreme case? With that even the workqueues would work again.

Then people can no longer assume stuff like queue_work_on() etc.. works. Users of such code might depend on it actually running on the specified cpu.

Quoted 3 lines

Then people can no longer assume stuff like queue_work_on() etc.. works. Users of such code might depend on it actually running on the specified cpu.

If they assume that they're already buggy because CPU hot unplug will break affinities.

Quoted 5 lines · Peter Zijlstra

Quoted 4 lines · Andi Kleen

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

Quoted 3 lines

Quoted 2 lines

The load balancer will not notice that a particular CPU is busy with real time tasks?

Not currently, working on that though.

I wonder if it would make sense to break affinities in extreme case? With that even the workqueues would work again.

Quoted 6 lines · Andi Kleen

Quoted 5 lines · Peter Zijlstra

Quoted 4 lines · Andi Kleen

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

Quoted 6 lines

Quoted 3 lines

Quoted 2 lines

The load balancer will not notice that a particular CPU is busy with real time tasks?

Not currently, working on that though.

I wonder if it would make sense to break affinities in extreme case? With that even the workqueues would work again.

Please lets not break affinity :).

Quoted 9 lines · Max Krasnyansky

Quoted 8 lines · Andi Kleen

Quoted 7 lines · Peter Zijlstra

Quoted 6 lines · Andi Kleen

Quoted 4 lines

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog.

The load balancer will not notice that a particular CPU is busy with real time tasks?

Quoted 6 lines

Quoted 4 lines

Quoted 1 line

Not currently, working on that though.

I wonder if it would make sense to break affinities in extreme case? With that even the workqueues would work again.

Please lets not break affinity :).

correct, breaking affinity is a rather stupid idea.

Quoted 8 lines · Nick Piggin

Quoted 7 lines · Andi Kleen

Quoted 5 lines · Thomas Gleixner

Well, we might have a public opinion poll, whether a system is declared frozen after 1, 10 or 100 seconds. Even a one second unresponsivness shows up on the kernel bugzilla and you request that unlimited unresponsivness w/o a chance to debug it is the sane default.

Quoted 4 lines

Quoted 2 lines

That assumes single CPU. With multiple CPUs and not all hogged the system should be still responsive?

Right.

Wrong.

Even if the system has multiple CPUs, and even if just a single CPU is fully utilized by an RT task, without the rt-limit the system will still lock up in practice due to various other factors: workqueues and tasks being 'stuck' on CPUs that host an RT hog. While there's obviously CPU time available on other CPUs, you cannot run 'top', the desktop will freeze, work flows of the system can be stuck, etc, etc..

With the rt limit in place, it's all pretty smooth and debuggable. Even with all CPUs hogged by SCHED_FIFO prio 99 the system is laggy but debuggable - the user can run 'top' and can resolve the situation.

Really, this reply of yours shows something startling: that despite this many mails you still have never actually tried to run the scenario you are complaining about: you have never tried to run a CPU hog high-prio RT task on a Linux system before, and you have never observed the effects it has on general system stability and debuggability.

This fundamental lack of experience weakens all your arguments and i dont even know why you are arguing about it. Do you perhaps have some customer application/workload you are worried about? If you have then please tell us about the exact specifics - this handwaving about compliance really makes little sense.

In other words: in our car the air-bag continues to be enabled by default, and if someone wants to use the car for stunts the air-bag can be disabled via that handy sysctl.

In any case i think i'm going to ignore this thread from now on, nothing new has been said really, just the general tone of discussion is deteriorating.

You are also very late with raising objections in any
case - the rt-limit feature has been posted 10 months ago and went upstream 8 months ago - two full kernel cycles have been completed with this change in place and a third one has almost been finished.

There is no customer issue and there is no handwaving about compliance;

Quoted 2 lines · Nick Piggin

There is no customer issue and there is no handwaving about compliance;

well, the reason i'm asking is that i cannot for anything in the world imagine you being so upset about _anything_ but something that involves benchmark runs ;-)

And what does SCHED_FIFO RT policy scheduling have to do with performance and benchmarks? Nothing usually in the real world, except for this little known fact: a common 'tuning' for TPC database benchmarks is to run all DB threads as SCHED_FIFO to squeeze the last 0.1% of performance out of the setup.

So - and i'm taking an educated guess here - is SCHED_FIFO+TPC performance perhaps one of the factors that played a role in you initiating this thread? If yes then it's obviously an incredibly broken use of SCHED_FIFO and we can add the sysctl tuning to the long list of dozens of other tunings that happen before a TPC run anyway.

Hm?

This fundamental lack of experience weakens all your arguments and i dont even know why you are arguing about it.

Quoted 20 lines · Nick Piggin

Quoted 18 lines · Thomas Gleixner

Quoted 6 lines · Nick Piggin

Quoted 5 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

Quoted 6 lines

Quoted 1 line

I disagree

Your arguments were along the line of:

* It probably doesn't break anything (except we had somebody report that it breaks their app)

I'm a real-time oldtimer. An application which hogs the CPU for 9.9 seconds with SCHED_FIFO priority is just broken. It's broken beyond all limits, whether POSIX allows to do that or Linux obeyed the request of the braindamaged application design.

Quoted 4 lines

Oh with this much handwaving from you old timers I feel much better about it ;) I bet before the bug report and change to 10s, any application that hogged the CPU for more than 0.9 seconds was just broken too, right? But 10s is more than enough for everybody?

Well, we might have a public opinion poll, whether a system is declared frozen after 1, 10 or 100 seconds.

Even a one second
unresponsivness shows up on the kernel bugzilla and you request that unlimited unresponsivness w/o a chance to debug it is the sane default.

An one second RT CPU hog is just a broken application, nothing else. Your precious customer use case is simply crap.

Real-time is about determinism and not about the allowance to fuck up a system at will. If a system failed to prevent the fuckup once then this is not at all a guarantee that it allows to do that forever.

Especially not in the Open Source space, where developers are still allowed to use their brain and apply common sense to prevent such a wreckage and abuse. Still, your not yet specified use case can continue to do stupid things forever with the simple tweak that it needs to declare itself broken by turning off the kernel sanity checks.

Quoted 5 lines

I may not be an old timer, but I can say the kernel is just broken if it deliberately deviates from standards to undocumented behaviour, and even more so if it changes from working to broken behaviour for reasons that can be worked around in userspace (eg. running a higher priority watchdog).

Right. I appreciate the nitpicking janitor of the most important POSIX feature:

"The unlimited right to monopolize the CPU for any given timeframe."

Get your brain together. Just because it worked before and POSIX allows it is not an argument at all that it is something useful. If you want to do this you still can do it by resetting the limit.

Your request to enforce that stupid and braindead behaviour on everyone is simply annyoing.

Quoted 7 lines

Quoted 7 lines

Quoted 6 lines

* If it does break something then they must be doing something stupid (I refuted that because there are several legitimate ways to use rt scheduling that is broken by this)

* We have many other APIs and tools that don't conform to posix (why is that a reason to break this one?)

Quoted 8 lines

Quoted 2 lines

Simply because we use common sense instead of following every single POSIX brainfart by the letter.

How is that a brainfart? It is simple, relatively unambiguous, and not arbitrary. You really say the POSIX specified behaviour is "a brainfart", but adding an arbitrary 10s throttle "but the process might be preempted and lose the CPU to a lower priority task if it uses 10s of consecutive CPU time" would eliminate that brainfart? I have to laugh.

No, I did not say that. All I said is that giving the normal and common sense capable user/developer the chance to debug a runaway task w/o rebooting the system via the power off button is a sensible and useful default.

Your request to default to a possibly unusable system serves some yet to be explained higher goal, which is definitely out of the scope of common sense.

You still did not explain why this behaviour is useful and your handwaving vs. some (probably closed source) customer application is not an argument at all.

Quoted 9 lines

Quoted 6 lines

Quoted 3 lines

* We should break the API to cater for stupid users and distros who create local DoS and/or lock up their boxes (except this is trivial to solve by setting sysctls or having a watchdog or using sysrq)

For the vast majority of users and RT developers a sane default of sanity measures is useful and sensible.

You seriously develop complex rt tasks without having at least a simple watchdog task?

Dude, don't tell me how to design and debug a real time system.

It's not about me, but about the general usability and debuggability of Linux even in extreme situations, e.g. an unvoluntary runaway task, which we see even from time to time in bug reports. Having a sensible default guard is helping in the common case and denying it is just a selfserving attitude to keep some braindamaged customer niche application alive. Linux and Open Source is not about the customer application, it is about having a sane and safe environment for 99% of the use cases. Your pretious CPU hog SCHED_FIFO application is an engineering brainfart which is really not relevant to any community decision of a sane and per default safe guarded OS.

Quoted 5 lines

Quoted 3 lines

If someone wants to shoot himself in the foot then it's not an unreasonable request that he needs to disable the safety guards before pulling the trigger.

root is allowed to shoot themselves in the foot. root is the safeguard.

Sure. You are allowed to shoot yourself in the foot as well. Does the gun manufacturer omit safety guards just because you are allowed to and just because the 1990 version of the gun did not have that safety guard ?

Again. Common sense is way more important than some green table specification and some esoteric customer application.

Quoted 8 lines · Nick Piggin

Quoted 5 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Your arguments were along the line of:

Quoted 2 lines

* It probably doesn't break anything (except we had somebody report that it breaks their app)

I'm a real-time oldtimer. An application which hogs the CPU for 9.9 seconds with SCHED_FIFO priority is just broken. It's broken beyond all limits, whether POSIX allows to do that or Linux obeyed the request of the braindamaged application design.

From my point of view, as an RT user, any kernel that supports SMP yet can't

Quoted 6 lines

* If it does break something then they must be doing something stupid (I refuted that because there are several legitimate ways to use rt scheduling that is broken by this)

* We have many other APIs and tools that don't conform to posix (why is that a reason to break this one?)

Simply because we use common sense instead of following every single POSIX brainfart by the letter.

Quoted 3 lines

* We should break the API to cater for stupid users and distros who create local DoS and/or lock up their boxes (except this is trivial to solve by setting sysctls or having a watchdog or using sysrq)

For the vast majority of users and RT developers a sane default of sanity measures is useful and sensible.

If someone wants to shoot himself in the foot then it's not an unreasonable request that he needs to disable the safety guards before pulling the trigger.

Quoted 10 lines · Thomas Gleixner

Quoted 8 lines · Nick Piggin

Quoted 5 lines · Ingo Molnar

Quoted 3 lines · Nick Piggin

So... no reply to this? I'm really wondering how it's OK to break documented standards and previous Linux behaviour by default for something that it is trivial to solve in userspace? [...]

I disagree

Your arguments were along the line of:

Quoted 8 lines

Quoted 2 lines

* It probably doesn't break anything (except we had somebody report that it breaks their app)

I'm a real-time oldtimer. An application which hogs the CPU for 9.9 seconds with SCHED_FIFO priority is just broken. It's broken beyond all limits, whether POSIX allows to do that or Linux obeyed the request of the braindamaged application design.

Well, I've been working on RT hardware (mostly) and software since 1977. With all due respect, thats crapola. I for one have this requirement and there is _no_ way around it in my world. In fact it's the kernel thats broke by stealing precious usecs from me.

What I would suggest is this.

1) Keep the default as the infinite for those that know what they are doing.

2) Change the sysctl scripts in the distros to set the default to a sane time that will protect the users.

An RT app that would break the 10s limit would probably be using busybox anyway, so the default for that would be what the kernel comes up with.

The default the 99% of users would have, is what the distro set it to for them.

This seems like a sane solution to satisfy both camps.

Quoted 12 lines · Peter Zijlstra

Disable bandwidth control by default.

Signed-off-by Peter Zijlstra <[email protected]>

---
 kernel/sched.c |   17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

Index: linux-2.6/kernel/sched.c
===================================================================

kernel/sched.c static __read_mostly int scheduler_runni · line 829

Quoted 8 lines

kernel/sched.c +2 -2

@@ -824,9 +824,9 @@ static __read_mostly int scheduler_runni
 
 /*
  * part of the period that we allow rt tasks to run in us.
- * default: 0.95s
+ * default: inf
  */
-int sysctl_sched_rt_runtime = 950000;
+int sysctl_sched_rt_runtime = -1;

The fixes look good to me, but this enabling of infinite RT task lockups is not an improvement.

The thing is, i got far more bugreports about locked up RT tasks where the lockup was unintentional, than real bugreports about anyone _intending_ for the whole box to come to a grinding halt because a high-prio RT tasks is monopolizing the CPU.

In fact there's only been this artificial test so far.

So could you please just increase the chunking to 10 seconds or so, from the current 1 second? Anyone locking up the system for more than 10 seconds via an RT task has to deal with many other issues already.

I.e. keep the system borderline debuggable (up to 10 seconds delays are _not_ nice so people will notice) - but it's still a marked improvement from completly locked up desktops.

And those who really need longer than 10 second periods can set it higher, or even (if they want to live dangerously or run POSIX conformance tests) make it infinite (set it to -1) - and will have to deal with other things like the softlockup watchdog as well.

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Quoted 5 lines · Steven Rostedt

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

yeah, agreed, this is a reasonable suggestion. Peter, do you agree?

Quoted 7 lines · Ingo Molnar

Quoted 5 lines · Steven Rostedt

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Quoted 1 line

yeah, agreed, this is a reasonable suggestion. Peter, do you agree?

Seems reasonable. But I still think it should be disabled by default (it might not get caught in testing for example).

Quoted 8 lines · Nick Piggin

Quoted 7 lines · Ingo Molnar

Quoted 5 lines · Steven Rostedt

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Quoted 4 lines

Quoted 1 line

yeah, agreed, this is a reasonable suggestion. Peter, do you agree?

Seems reasonable. But I still think it should be disabled by default (it might not get caught in testing for example).

Perhaps we should default it to 1sec, that way it would be hit more often, and educate the users of this now feature.

Quoted 9 lines · Steven Rostedt

Quoted 8 lines · Nick Piggin

Quoted 7 lines · Ingo Molnar

Quoted 5 lines · Steven Rostedt

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Quoted 7 lines

Quoted 4 lines

Quoted 1 line

yeah, agreed, this is a reasonable suggestion. Peter, do you agree?

Seems reasonable. But I still think it should be disabled by default (it might not get caught in testing for example).

Perhaps we should default it to 1sec, that way it would be hit more often, and educate the users of this now feature.

There only one sane default, as far as I can see.

Before anybody attacks me again because I haven't got my brain together or am an annoying standards nitpicker:

I'm very well aware of the consequences of unlimited hogging of the CPU. And I know exactly why people might want rt throttling. But just think for a minute the _negative_ consequences of changing the API and remember that is close to the #1 rule of Linux development to not break user API.

And put it this way: the sysctl is right there. Any distro that cares about this problem will probably find this thread as #1 hit and work out how to enable the sysctl and break the API if they are happy to do that. On the flip side, not every application development or deployment is even going to know about this, and it may not be trivial to catch in testing, so it could cause failures in the field.

I've always thought that the policy settings belong in the distro, and the kernel should never enforce a policy (by setting this as default, it is enforcing a policy, even though an RT user can change it).

Quoted 4 lines · Steven Rostedt

I've always thought that the policy settings belong in the distro, and the kernel should never enforce a policy (by setting this as default, it is enforcing a policy, even though an RT user can change it).

The kernel has always done a certain amount of "default policy".

What do you think things like "swappiness" etc are? Or things like oevrcommit settings? They're all policies, and there is always a default one. So in that sense the kernel always has - and fundamentally _must_ - set some kind of policy.

And the default policy should generally be the one that makes sense for most people. Quite frankly, if it's an issue where all normal distros would basically be expected to set a value, then that value should _be_ the default policy, and none of the normal distros should ever need to worry.

Whether this case is one such, I dunno. Quite frankly, I don't think it's even _nearly_ important enough to get this kind of noise.

Quoted 15 lines · Linus Torvalds

Quoted 4 lines · Steven Rostedt

I've always thought that the policy settings belong in the distro, and the kernel should never enforce a policy (by setting this as default, it is enforcing a policy, even though an RT user can change it).

The kernel has always done a certain amount of "default policy".

What do you think things like "swappiness" etc are? Or things like oevrcommit settings? They're all policies, and there is always a default one. So in that sense the kernel always has - and fundamentally _must_ - set some kind of policy.

Quoted 8 lines

And the default policy should generally be the one that makes sense for most people. Quite frankly, if it's an issue where all normal distros would basically be expected to set a value, then that value should _be_ the default policy, and none of the normal distros should ever need to worry.

Whether this case is one such, I dunno. Quite frankly, I don't think it's even _nearly_ important enough to get this kind of noise.

I guess the reason that this is getting so much noise over other default policies, is that this default policy is changing a well known definition:
The meaning of FIFO.

By making the default policy limit the time an RT task runs, we have, in essence, changed a user API. Applications that expect to be able to run uninterrupted by SCHED_OTHER tasks, will now break.

No one is arguing that this new feature is not useful. The argument is, should the kernel set the default policy of an old well known scheduling policy to something different than what is expected?

Distros set SE Linux on by default, should the kernel do that too?

-- Steve

Quoted 1 line · Steven Rostedt

I've always thought that the policy settings belong in the distro, and the kernel should never enforce a policy (by setting this as default, it is enforcing a policy, even though an RT user can change it).

The kernel has always done a certain amount of "default policy".

What do you think things like "swappiness" etc are? Or things like oevrcommit settings? They're all policies, and there is always a default one. So in that sense the kernel always has - and fundamentally _must_ - set some kind of policy.

And the default policy should generally be the one that makes sense for most people. Quite frankly, if it's an issue where all normal distros would basically be expected to set a value, then that value should _be_ the default policy, and none of the normal distros should ever need to worry.

Whether this case is one such, I dunno. Quite frankly, I don't think it's even _nearly_ important enough to get this kind of noise.

Linus

More and more are wanting and now finding the Linux kernel to be more RT capable. I seem to remember way back you saying it was one thing you didn't really care much about one way or the other. Thats OK. But, you _are_ the man.

"just to protect a few _supposedly_ bad programmers???"

And I'm not really interested. Quite frankly, I suspect the "we want to run something like pulseaudio with RT priorities" camp is the more common one, and in that context I understand limiting SCHED_FIFO sounds perfectly understandable.

Quoted 5 lines · Linus Torvalds

And I'm not really interested. Quite frankly, I suspect the "we want to run something like pulseaudio with RT priorities" camp is the more common one, and in that context I understand limiting SCHED_FIFO sounds perfectly understandable.

The fact that it actually limits a SCHED_FIFO task group, over a single task thread does bother me a little.

Quoted 7 lines · Steven Rostedt

Quoted 5 lines · Linus Torvalds

And I'm not really interested. Quite frankly, I suspect the "we want to run something like pulseaudio with RT priorities" camp is the more common one, and in that context I understand limiting SCHED_FIFO sounds perfectly understandable.

Quoted 2 lines

The fact that it actually limits a SCHED_FIFO task group, over a single task thread does bother me a little.

It bothers me some too. You have to patch/re-compile the kernel if you need to turn it off and don't have SCHED_DEBUG enabled (not free).

Quoted 3 lines · Mike Galbraith

It bothers me some too. You have to patch/re-compile the kernel if you need to turn it off and don't have SCHED_DEBUG enabled (not free).

/proc/sys/kernel/sched_rt_{runtime,period}_us don't require SCHED_DEBUG. If they are in any way non-functional on SCHED_DEBUG=n then that's a clear bug.

Quoted 4 lines

More and more are wanting and now finding the Linux kernel to be more RT capable. I seem to remember way back you saying it was one thing you didn't really care much about one way or the other. Thats OK. But, you _are_ the man.

The thing is, the reason I dislike RT is that so many people have so different understanding of what RT means.

Quite frankly, I think that the people who are complaining (like you) think that RT means "hard realtime". You think about literally specialized devices.

A lot of _other_ people think that RT means "good audio latency", where it really is a lot softer.

And neither camp seems to ever admit that they are just a small camp, and that the other camp exists or is even valid.

And I'm not really interested. Quite frankly, I suspect the "we want to run something like pulseaudio with RT priorities" camp is the more common one, and in that context I understand limiting SCHED_FIFO sounds perfectly understandable.

As to your

Quoted 1 line

"just to protect a few _supposedly_ bad programmers???"

quite frankly, most programmers aren't "supposedly bad". And if you think that the hard-RT "real man" programmers aren't bad, I really have nothing to say.

Linus

And I'm not really interested. Quite frankly, I suspect the "we want to run something like pulseaudio with RT priorities" camp is the more common one, and in that context I understand limiting SCHED_FIFO sounds perfectly understandable.

quite frankly, most programmers aren't "supposedly bad". And if you think that the hard-RT "real man" programmers aren't bad, I really have nothing to say.

Quoted 3 lines · Steven Rostedt

I've always thought that the policy settings belong in the distro, and the kernel should never enforce a policy (by setting this as default, it is enforcing a policy, even though an RT user can change it).

The kernel has always done a certain amount of "default policy".

What do you think things like "swappiness" etc are? Or things like oevrcommit settings? They're all policies, and there is always a default one. So in that sense the kernel always has - and fundamentally _must_ - set some kind of policy.

And the default policy should generally be the one that makes sense for most people. Quite frankly, if it's an issue where all normal distros would basically be expected to set a value, then that value should _be_ the default policy, and none of the normal distros should ever need to worry.

Whether this case is one such, I dunno. Quite frankly, I don't think it's even _nearly_ important enough to get this kind of noise.

Quoted 7 lines · Ingo Molnar

Quoted 5 lines · Steven Rostedt

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

yeah, agreed, this is a reasonable suggestion. Peter, do you agree?

Seems reasonable. But I still think it should be disabled by default (it might not get caught in testing for example).

Quoted 11 lines · Nick Piggin

Quoted 9 lines · Ingo Molnar

Quoted 6 lines · Steven Rostedt

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Quoted 4 lines

Quoted 1 line

yeah, agreed, this is a reasonable suggestion. Peter, do you agree?

Seems reasonable. But I still think it should be disabled by default (it might not get caught in testing for example).

I cannot believe you guys are still arguing about this and calling each other stupid/incompetent/braindead and such (not this particular email but all the stuff before) :)

Seems to me like leaving RT throttling disabled by default is a reasonable compromise. Several people suggested that and the advantage is that it does not change the definition of SCHED_FIFO/RR by default.

I personally do not care that much what the default is. If Fedora, for example, starts enabling it by default I'll still have to change it. So it's not much different from enabled by default in the kernel.

Max

My biggest concern about adding a limit to FIFO is that an RT developer would spend weeks trying to debug their system wondering why their planned CPU RT hog, is being preempted by a non-RT task.

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Peter, another question. Is this limit for a single RT task running, or all RT tasks. I'm assuming here that it is a single RT task. If you have 20 RT tasks all running, would this let non RT tasks in? In that case, this could be even a bigger issues.

Quoted 9 lines · Steven Rostedt

My biggest concern about adding a limit to FIFO is that an RT developer would spend weeks trying to debug their system wondering why their planned CPU RT hog, is being preempted by a non-RT task.

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Should be easy enough to do -

Quoted 4 lines

Peter, another question. Is this limit for a single RT task running, or all RT tasks. I'm assuming here that it is a single RT task. If you have 20 RT tasks all running, would this let non RT tasks in? In that case, this could be even a bigger issues.

No its not per task. Its per group (and trivially the !group case is one group).

Quoted 14 lines · Peter Zijlstra

Quoted 9 lines · Steven Rostedt

My biggest concern about adding a limit to FIFO is that an RT developer would spend weeks trying to debug their system wondering why their planned CPU RT hog, is being preempted by a non-RT task.

For this, if this time limit does kick in, we should at the very least print something out to let the user know this happened. After all, this is more of a safety net anyway, and if we are hitting the limit, the user should be notified. Perhaps even tell the user that if this behaviour is expected, to up the sysctl <var> by more.

Should be easy enough to do -

Quoted 7 lines

Quoted 4 lines

Peter, another question. Is this limit for a single RT task running, or all RT tasks. I'm assuming here that it is a single RT task. If you have 20 RT tasks all running, would this let non RT tasks in? In that case, this could be even a bigger issues.

No its not per task. Its per group (and trivially the !group case is one group).

Does this mean, if I have 100 RT tasks, that will together run for 10secs secs, they will only run for 9.5secs?

This looks like an even bigger issue. Now we don't have one RT FIFO CPU hog, we are now hitting 100 RT FIFO tasks that try to get a bunch done in 10 secs.