O13int for interactivity

Changes:

Reverted the child penalty to 95 as new changes help this from hurting

Changed the logic behind loss of interactive credits to those that burn off all their sleep_avg

Now all tasks get proportionately more sleep as their relative bonus drops off. This has the effect of detecting a change from a cpu burner to an interactive task more rapidly as in O10.

The _major_ change in this patch is that tasks on uninterruptible sleep do not earn any sleep avg during that sleep; it is not voluntary sleep so they should not get it. This has the effect of stopping cpu hogs from gaining dynamic priority during periods of heavy I/O. Very good for the jerks you may see in X or audio skips when you start a whole swag of disk intensive cpu hogs (eg make -j large number). I've simply dropped all their sleep_avg, but weighting it may be more appropriate. This has the side effect that pure disk tasks (eg cp) have relatively low priority which is why weighting may be better. We shall see.

Please test this one extensively. It should _not_ affect I/O throughput per se, but I'd like to see some of the I/O benchmarks on this. I do not want to have detrimental effects elsewhere.

patch-O12.3-O13int applies on top of 2.6.0-test2-mm4 that has been patched with O12.3int and is available on my site, and a full patch against 2.6.0-test2 called patch-test2-O13int is here:

http://kernel.kolivas.org/2.5

patch-O12.3-O13int:

kernel/sched.c +16 -14

@@ -78,7 +78,7 @@
 #define MAX_TIMESLICE		(200 * HZ / 1000)
 #define TIMESLICE_GRANULARITY	(HZ/40 ?: 1)
 #define ON_RUNQUEUE_WEIGHT	30
-#define CHILD_PENALTY		90
+#define CHILD_PENALTY		95
 #define PARENT_PENALTY		100
 #define EXIT_WEIGHT		3
 #define PRIO_BONUS_RATIO	25

@@ -365,6 +365,9 @@ static void recalc_task_prio(task_t *p, 
 	unsigned long long __sleep_time = now - p->timestamp;
 	unsigned long sleep_time;
 
+	if (!p->sleep_avg)
+		p->interactive_credit--;
+
 	if (__sleep_time > NS_MAX_SLEEP_AVG)
 		sleep_time = NS_MAX_SLEEP_AVG;
 	else

@@ -384,17 +387,19 @@ static void recalc_task_prio(task_t *p, 
 					JIFFIES_TO_NS(JUST_INTERACTIVE_SLEEP(p));
 		else {
 			/*
-			 * Tasks with interactive credits get boosted more
-			 * rapidly if their bonus has dropped off. Other
-			 * tasks are limited to one timeslice worth of
-			 * sleep avg.
+			 * The lower the sleep avg a task has the more
+			 * rapidly it will rise with sleep time. Tasks
+			 * without interactive_credit are limited to
+			 * one timeslice worth of sleep avg bonus.
 			 */
-			if (p->interactive_credit > 0)
-				sleep_time *= (MAX_BONUS + 1 -
+			sleep_time *= (MAX_BONUS + 1 -
 					(NS_TO_JIFFIES(p->sleep_avg) *
 					MAX_BONUS / MAX_SLEEP_AVG));
-			else if (sleep_time > JIFFIES_TO_NS(task_timeslice(p)))
-				sleep_time = JIFFIES_TO_NS(task_timeslice(p));
+
+			if (p->interactive_credit < 0 &&
+				sleep_time > JIFFIES_TO_NS(task_timeslice(p)))
+					sleep_time =
+						JIFFIES_TO_NS(task_timeslice(p));
 
 			/*
 			 * This code gives a bonus to interactive tasks.

@@ -435,20 +440,26 @@ static inline void activate_task(task_t 
 	recalc_task_prio(p, now);
 
 	/*
-	 * Tasks which were woken up by interrupts (ie. hw events)
-	 * are most likely of interactive nature. So we give them
-	 * the credit of extending their sleep time to the period
-	 * of time they spend on the runqueue, waiting for execution
-	 * on a CPU, first time around:
+	 * This checks to make sure it's not an uninterruptible task
+	 * that is now waking up.

kernel/sched.c +5 -0

+		/*
+		 * Tasks which were woken up by interrupts (ie. hw events)
+		 * are most likely of interactive nature. So we give them
+		 * the credit of extending their sleep time to the period
+		 * of time they spend on the runqueue, waiting for execution

kernel/sched.c +1 -0

+	}
 
 	p->timestamp = now;
 

@@ -572,8 +583,15 @@ repeat_lock_task:
 				task_rq_unlock(rq, &flags);
 				goto repeat_lock_task;

kernel/sched.c +1 -1

+			}
 			if (sync)
 				__activate_task(p, rq);
 			else {

@@ -1326,7 +1344,6 @@ void scheduler_tick(int user_ticks, int 
 		p->prio = effective_prio(p);
 		p->time_slice = task_timeslice(p);
 		p->first_time_slice = 0;
-		p->interactive_credit--;
 
 		if (!rq->expired_timestamp)
 			rq->expired_timestamp = jiffies;

kernel/sched.c static inline void activate_task(task_t · line 446

Quoted 1 line · Con Kolivas

kernel/sched.c +1 -0

+	if (!p->activated){

Con, I will probably be wrong, but in [1] you are testing "activated != 0" and [2] is setting "activated = -1", which _is_ != 0 and thus would enter the "if->else" branch and do "activated = 1" in [3].

Perhaps you meant to set "activated = 0" in [2]???

Note I've not read the rest of the scheduler code, so perhaps the "activated = 0" is in another place... just in case, I prefer asking.

Changes:

Reverted the child penalty to 95 as new changes help this from hurting

Changed the logic behind loss of interactive credits to those that burn off all their sleep_avg

Now all tasks get proportionately more sleep as their relative bonus drops off. This has the effect of detecting a change from a cpu burner to an interactive task more rapidly as in O10.

The _major_ change in this patch is that tasks on uninterruptible sleep do not earn any sleep avg during that sleep; it is not voluntary sleep so they should not get it. This has the effect of stopping cpu hogs from gaining dynamic priority during periods of heavy I/O. Very good for the jerks you may see in X or audio skips when you start a whole swag of disk intensive cpu hogs (eg make -j large number). I've simply dropped all their sleep_avg, but weighting it may be more appropriate. This has the side effect that pure disk tasks (eg cp) have relatively low priority which is why weighting may be better. We shall see.

IMHO, absolute cut off is a very bad idea (btdt, and it _sucked rocks_).

The last thing in the world you want to do is to remove differentiation between tasks... try to classify them and make them all the same within their class. For grins, take away all remaining differentiation, and run a hefty parallel make.

Quoted 6 lines · Mike Galbraith

IMHO, absolute cut off is a very bad idea (btdt, and it _sucked rocks_).

The last thing in the world you want to do is to remove differentiation between tasks... try to classify them and make them all the same within their class. For grins, take away all remaining differentiation, and run a hefty parallel make.

I didn't fully understand you but I get your drift. I have a better solution in the works anyway but I wanted this tested.

Changes:

Reverted the child penalty to 95 as new changes help this from hurting

Changed the logic behind loss of interactive credits to those that burn off all their sleep_avg

Now all tasks get proportionately more sleep as their relative bonus drops off. This has the effect of detecting a change from a cpu burner to an interactive task more rapidly as in O10.

The _major_ change in this patch is that tasks on uninterruptible sleep do not earn any sleep avg during that sleep; it is not voluntary sleep so they should not get it. This has the effect of stopping cpu hogs from gaining dynamic priority during periods of heavy I/O. Very good for the jerks you may see in X or audio skips when you start a whole swag of disk intensive cpu hogs (eg make -j large number). I've simply dropped all their sleep_avg, but weighting it may be more appropriate. This has the side effect that pure disk tasks (eg cp) have relatively low priority which is why weighting may
be better. We shall see.

Please test this one extensively. It should _not_ affect I/O throughput per se, but I'd like to see some of the I/O benchmarks on this. I do not want to have detrimental effects elsewhere.

Quoted 11 lines · Con Kolivas

Changes:

Reverted the child penalty to 95 as new changes help this from hurting

Changed the logic behind loss of interactive credits to those that burn off all their sleep_avg

Now all tasks get proportionately more sleep as their relative bonus drops off. This has the effect of detecting a change from a cpu burner to an interactive task more rapidly as in O10.

Quoted 9 lines

The _major_ change in this patch is that tasks on uninterruptible sleep do not earn any sleep avg during that sleep; it is not voluntary sleep so they should not get it. This has the effect of stopping cpu hogs from gaining dynamic priority during periods of heavy I/O. Very good for the jerks you may see in X or audio skips when you start a whole swag of disk intensive cpu hogs (eg make -j large number). I've simply dropped all their sleep_avg, but weighting it may be more appropriate. This has the side effect that pure disk tasks (eg cp) have relatively low priority which is why weighting may be better. We shall see.

I don't think this is a good idea. Uninterruptible does not mean its not a voluntary sleep. Its more to do with how a syscall is implemented. I don't think it should be treated any differently to any other type of sleep.

Any task which calls schedule in kernel context is sleeping volintarily
- if implicity due to having called a blocking syscall.

Quoted 3 lines

Please test this one extensively. It should _not_ affect I/O throughput per se, but I'd like to see some of the I/O benchmarks on this. I do not want to have detrimental effects elsewhere.

Well the reason it can affect IO thoughput is for example when there is an IO bound process and a CPU hog on the same processor: the longer the IO process has to wait (after being woken) before being run, the more chance the disk will fall idle for a longer period. And of course the CPU uncontended case is somewhat uninteresting when it comes to a CPU scheduler.

Quoted 36 lines · Nick Piggin

Quoted 11 lines · Con Kolivas

Changes:

Reverted the child penalty to 95 as new changes help this from hurting

Changed the logic behind loss of interactive credits to those that burn off all their sleep_avg

Now all tasks get proportionately more sleep as their relative bonus drops off. This has the effect of detecting a change from a cpu burner to an interactive task more rapidly as in O10.

Quoted 10 lines

The _major_ change in this patch is that tasks on uninterruptible sleep do not earn any sleep avg during that sleep; it is not voluntary sleep so they should not get it. This has the effect of stopping cpu hogs from gaining dynamic priority during periods of heavy I/O. Very good for the jerks you may see in X or audio skips when you start a whole swag of disk intensive cpu hogs (eg make -j large number). I've simply dropped all their sleep_avg, but weighting it may be more appropriate. This has the side effect that pure disk tasks (eg cp) have relatively low priority which is why weighting may be better. We shall see.

I don't think this is a good idea. Uninterruptible does not mean its not a voluntary sleep. Its more to do with how a syscall is implemented. I don't think it should be treated any differently to any other type of sleep.

Any task which calls schedule in kernel context is sleeping volintarily
- if implicity due to having called a blocking syscall.

Quoted 4 lines

Please test this one extensively. It should _not_ affect I/O throughput per se, but I'd like to see some of the I/O benchmarks on this. I do not want to have detrimental effects elsewhere.

Quoted 7 lines

Well the reason it can affect IO thoughput is for example when there is an IO bound process and a CPU hog on the same processor: the longer the IO process has to wait (after being woken) before being run, the more chance the disk will fall idle for a longer period. And of course the CPU uncontended case is somewhat uninteresting when it comes to a CPU scheduler.

I've already posted a better solution in O13.1

Quoted 1 line

I've already posted a better solution in O13.1

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Quoted 6 lines · Nick Piggin

Quoted 2 lines

I've already posted a better solution in O13.1

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Sure I'm open to any and all ideas. Cpu hogs occasionally do significant I/O. Up until that time they have been only losing sleep_avg as they have spent no time sleeping; and this is what gives them a lower dynamic priority. During uninterruptible sleep all of a sudden they are seen as sleeping even though they are cpu hogs waiting on I/O. Witness the old standard, a kernel compile. The very first time you launch a make -j something, the higher the something, the longer all the jobs wait on I/O, the better the dynamic priority they get, which they shouldn't.

No, this is not just a "fix the scheduler so you don't feel -j kernel compiles" as it happens with any cpu hog starving other tasks, and the longer the cpu hogs wait on I/O the worse it is. This change causes a _massive_ improvement for that test case which usually brings the machine to a standstill the size of which is dependent on the number of cpu hogs and the size of their I/O wait. I don't think the latest incarnation should be a problem. In my limited testing I've not found any difference in throughput but I don't have a major testbed at my disposal, nor time to use one if it was offered which is why I requested more testing.

Thoughts?

The idea I'm proposing, however poorly formed, is that if we allow some "excessive" oscillation early on in the life of a process, we may be able to more quickly get processes to NEAR its correct priority, OR get its CPU time over the course of three times being run for the underdamped case to be about the same as it would be if we knew in advance what the priority should be. But in the underdamped case, the priority would continue to oscillate up and down around the correct level, because we are intentionally overshooting the mark each time we adjust priority.

This may not be related, but something that pops into my mind is a numerical method called Newton's Method. It's a way to solve for roots of an equation, and it involved derivatives, and I don't quite remember how it works. But in any event, the results are less accurate than, say, bisection, but you get to the answer MUCH more quickly.

Here's a kooky idea...

I'm not sure about this detail, but I would guess that the int schedulers are trying to determine relatively stable priority values for processes. A process does not instantly come to its correct priority level, because it gets there based on accumulation of behavioral patterns.

Well, it occurs to me that we could benefit from situations where priority changes are underdamped. The results would sometimes be an oscillation in priority levels. In the short term, a given process may be given different amounts of CPU time when it is run, although in the long term, it should average out.

At the same time, certain tasks can only be judged correctly over the long term, like X, for example. Its long-term behavior is interactive, but now and then, it will become a CPU hog, and we want to LET it.

The idea I'm proposing, however poorly formed, is that if we allow some "excessive" oscillation early on in the life of a process, we may be able to more quickly get processes to NEAR its correct priority, OR get its CPU time over the course of three times being run for the underdamped case to be about the same as it would be if we knew in advance what the priority should be. But in the underdamped case, the priority would continue to oscillate up and down around the correct level, because we are intentionally overshooting the mark each time we adjust priority.

Here's a kooky idea...

I'm not sure about this detail, but I would guess that the int schedulers are trying to determine relatively stable priority values for processes. A process does not instantly come to its correct priority level, because it gets there based on accumulation of behavioral patterns.

Well, it occurs to me that we could benefit from situations where priority changes are underdamped. The results would sometimes be an oscillation in priority levels. In the short term, a given process may be given different amounts of CPU time when it is run, although in the long term, it should average out.

At the same time, certain tasks can only be judged correctly over the long term, like X, for example. Its long-term behavior is interactive, but now and then, it will become a CPU hog, and we want to LET it.

The idea I'm proposing, however poorly formed, is that if we allow some "excessive" oscillation early on in the life of a process, we may be able to more quickly get processes to NEAR its correct priority, OR get its CPU time over the course of three times being run for the underdamped case to be about the same as it would be if we knew in advance what the priority should be. But in the underdamped case, the priority would continue to oscillate up and down around the correct level, because we are intentionally overshooting the mark each time we adjust priority.

This may not be related, but something that pops into my mind is a numerical method called Newton's Method. It's a way to solve for roots of an equation, and it involved derivatives, and I don't quite remember how it works. But in any event, the results are less accurate than, say, bisection, but you get to the answer MUCH more quickly.

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Quoted 4 lines · Nick Piggin

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Footnote: I was thinking of using this to also _elevate_ the dynamic priority of tasks waking from interruptible sleep as well which may help throughput.

Quoted 8 lines · Con Kolivas

Quoted 4 lines · Nick Piggin

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Footnote: I was thinking of using this to also _elevate_ the dynamic

priority

Quoted 2 lines

of tasks waking from interruptible sleep as well which may help throughput.

Con, an uninterruptible sleep is one which is not be woken by a signal, an interruptible sleep is one which is. There is no other connotation. What happens when read/write syscalls are changed to be interruptible? I'm not saying this will happen... but come to think of it, NFS probably has interruptible read/write.

In short: make the same policy for an interruptible and an uninterruptible sleep.

Quoting Nick Piggin <[email protected]>:

Quoted 25 lines

Quoted 10 lines · Con Kolivas

Quoted 5 lines · Nick Piggin

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Footnote: I was thinking of using this to also _elevate_ the dynamic

priority

Quoted 3 lines

of tasks waking from interruptible sleep as well which may help throughput.

Con, an uninterruptible sleep is one which is not be woken by a signal, an interruptible sleep is one which is. There is no other connotation. What happens when read/write syscalls are changed to be interruptible? I'm not saying this will happen... but come to think of it, NFS probably has interruptible read/write.

Quoted 3 lines

In short: make the same policy for an interruptible and an uninterruptible sleep.

That's the policy that has always existed...

Interesting that I have only seen the desired effect and haven't noticed any side effect from this change so far. I'll keep experimenting as much as possible (as if I wasn't going to) and see what the testers find as well.

Quoted 20 lines · Con Kolivas

Quoting Nick Piggin <[email protected]>:

Quoted 17 lines

Quoted 10 lines · Con Kolivas

Quoted 5 lines · Nick Piggin

No, this still special-cases the uninterruptible sleep. Why is this needed? What is being worked around? There is probably a way to attack the cause of the problem.

Footnote: I was thinking of using this to also _elevate_ the dynamic

priority

Quoted 1 line

of tasks waking from interruptible sleep as well which may help

throughput.

Quoted 12 lines

Quoted 11 lines

Quoted 2 lines

Con, an uninterruptible sleep is one which is not be woken by a signal, an interruptible sleep is one which is. There is no other connotation. What happens when read/write syscalls are changed to be interruptible? I'm not saying this will happen... but come to think of it, NFS probably has interruptible read/write.

In short: make the same policy for an interruptible and an uninterruptible sleep.

Quoted 7 lines

That's the policy that has always existed...

Interesting that I have only seen the desired effect and haven't noticed any

side effect from this change so far. I'll keep experimenting as much as possible (as if I wasn't going to) and see what the testers find as well.

Oh, I'm not saying that your change is outright wrong, on the contrary I'd say you have a better feel for what is needed than I do, but if you are finding
that the uninterruptible sleep case needs some tweaking then the same tweak should be applied to all sleep cases. If there really is a difference, then its
just a fluke that the sleep paths in question use the type of sleep you are testing for, and nothing more profound than that.

Quoting Nick Piggin <[email protected]>:

Quoted 8 lines

Oh, I'm not saying that your change is outright wrong, on the contrary I'd say you have a better feel for what is needed than I do, but if you are finding that the uninterruptible sleep case needs some tweaking then the same tweak should be applied to all sleep cases. If there really is a difference, then its just a fluke that the sleep paths in question use the type of sleep you are testing for, and nothing more profound than that.

Ah I see. It was from my observations of the behaviour of tasks in D that found it was the period spent in D that was leading to unfairness. The same tweak can't be applied to the rest of the sleeps because that inactivates everything. So it is a fluke that the thing I'm trying to penalise is what tasks in uninterruptible sleep do, but it is by backward observation of D tasks, not random chance.

Quoted 1 line · Con Kolivas

Quoting Nick Piggin <[email protected]>:

Yes yes, but we come to the same conclusion no matter why you have decided to make the change ;) namely that you're only papering over a flaw in the scheduler!

What happens in the same sort of workload that is using interruptible sleeps?
Say the same make -j NFS mounted interrruptible (I think?).

I didn't really understand your answer a few emails ago... please just
reiterate: if the problem is that processes sleeping too long on IO get too high a priority, then give all processes the same boost after they have slept for half a second?

Also, why is this a problem exactly? Is there a difference between a process that would be a CPU hog but for its limited disk bandwidth, and a process that isn't a CPU hog? Disk IO aside, they are exactly the same thing to the CPU scheduler, aren't they?

_wants_ to be a CPU hog, but can't due to disk

Quoted 8 lines · Nick Piggin

Quoted 2 lines · Con Kolivas

Quoting Nick Piggin <[email protected]>:

Yes yes, but we come to the same conclusion no matter why you have decided to make the change ;) namely that you're only papering over a flaw in the scheduler!

This would take a redesign in the interactivity estimator. I worked on one for a while but decided it best to stick to one infrastructure and tune it as much as possible; especially in this stage of 2.6 blah blah...

Quoted 4 lines

What happens in the same sort of workload that is using interruptible sleeps?
Say the same make -j NFS mounted interrruptible (I think?).

Dunno. Can't say. I've only ever seen NFS D but I don't have enough test material...

Quoted 10 lines

I didn't really understand your answer a few emails ago... please just
reiterate: if the problem is that processes sleeping too long on IO get too high a priority, then give all processes the same boost after they have slept for half a second?

Also, why is this a problem exactly? Is there a difference between a process that would be a CPU hog but for its limited disk bandwidth, and a process that isn't a CPU hog? Disk IO aside, they are exactly the same thing to the CPU scheduler, aren't they?

Quoted 2 lines

_wants_ to be a CPU hog, but can't due to disk

You're on the right track; I'll try and explain differently.

A truly interactive task has periods of sleeping irrespective of disk activity. It is the time spent sleeping that the estimator uses to decide "this task is interactive, improve it's dynamic priority by 5". A true cpu hog (eg cc1) never sleeps intentionally and the estimator sees this as "I'm a hog; drop my priority by 5". Now if the cpu hog sleeps while waiting on disk i/o the estimator suddenly decides to elevate it's priority. If it gets to maximum boost and then stops doing I/O and goes back to being a hog it now starts starving other processes till it's dynamic priority drops enough again. As I said it's a design quirk (bug?) and _limiting_ how high the priority goes if the sleep is due to I/O would be ideal but I don't have a simple way to tell that apart from knowing that the sleep was UNINTERRUPTIBLE. This is not as bad as it sounds as for the most part it still is counted as sleep except that it can't ever get maximum priority boost to be a sustained starver.

What you are doing is restricting some range so it can adapt more quickly right? So you still have the problem in the cases where you are not restricting this range.

Quoted 4 lines · Nick Piggin

What you are doing is restricting some range so it can adapt more quickly right? So you still have the problem in the cases where you are not restricting this range.

Avoiding it becoming interactive in the first place is the answer. Anything more rapid and X dies dead as soon as you start moving a window for example, and new apps are seen as cpu hogs during startup and will take _forever_ to start under load. It's a tricky juggling act and I keep throwing more balls at it.

Quoted 5 lines · Con Kolivas

Quoted 3 lines · Nick Piggin

What you are doing is restricting some range so it can adapt more quickly right? So you still have the problem in the cases where you are not restricting this range.

Quoted 5 lines

Avoiding it becoming interactive in the first place is the answer. Anything more rapid and X dies dead as soon as you start moving a window for example, and new apps are seen as cpu hogs during startup and will take _forever_ to start under load. It's a tricky juggling act and I keep throwing more balls at it.

Well, what if you give less boost for sleeping?

Quoted 14 lines · Nick Piggin

Quoted 12 lines · Con Kolivas

Quoted 4 lines · Nick Piggin

What you are doing is restricting some range so it can adapt more quickly right? So you still have the problem in the cases where you are not restricting this range.

Avoiding it becoming interactive in the first place is the answer. Anything more rapid and X dies dead as soon as you start moving a window for example, and new apps are seen as cpu hogs during startup and will take _forever_ to start under load. It's a tricky juggling act and I keep throwing more balls at it.

Quoted 2 lines

Well, what if you give less boost for sleeping?

Then it takes longer to become interactive. Take 2.6.0-test2 vanilla - audio apps can take up to a minute to be seen as fully interactive; whether this is a problem for your hardware or not is another matter but clearly they are interactive using <1% cpu time on the whole.

Quoted 4 lines · Con Kolivas

Then it takes longer to become interactive. Take 2.6.0-test2 vanilla - audio apps can take up to a minute to be seen as fully interactive; whether this is a problem for your hardware or not is another matter but clearly they are interactive using <1% cpu time on the whole.

I think this is a big problem, a minute is much too long. I guess its taking this long to build up because X needs a great deal of inertia so that it can stay in a highly interactive state right?

If so then it seems the interactivity estimator does not have enough information to work properly for X. In which case maybe X needs to be reniced, or backboosted, or have _something_ done to help out.

Quoted 11 lines · Nick Piggin

Quoted 5 lines · Con Kolivas

Then it takes longer to become interactive. Take 2.6.0-test2 vanilla - audio apps can take up to a minute to be seen as fully interactive; whether this is a problem for your hardware or not is another matter but clearly they are interactive using <1% cpu time on the whole.

I think this is a big problem, a minute is much too long. I guess its taking this long to build up because X needs a great deal of inertia so that it can stay in a highly interactive state right?

Quoted 4 lines

If so then it seems the interactivity estimator does not have enough information to work properly for X. In which case maybe X needs to be reniced, or backboosted, or have _something_ done to help out.

Well we're in agreement there. That's what all this work I've done is about. You'll see I've not been just tweaking numbers.

I know you haven't been just tweaking numbers ;) But in the case of the patch that provides different behaviour depending on whether a sleep is interruptible or not really smelt of papering over symptoms. Now it might be that nothing better can be done without move invasive changes, but I just thought I'd voice my concerns.

Oh, and remember that your desktop load is devoid of make -j big compiles, so that is not a requisite for good interactivity.

Quoted 3 lines · Nick Piggin

What you are doing is restricting some range so it can adapt more quickly right? So you still have the problem in the cases where you are not restricting this range.

Avoiding it becoming interactive in the first place is the answer. Anything more rapid and X dies dead as soon as you start moving a window for example, and new apps are seen as cpu hogs during startup and will take _forever_ to start under load. It's a tricky juggling act and I keep throwing more balls at it.

generally that's a sign that the approach might not be the best one.

Lets face it: we're trying to estimate behavior here. Result: There ALWAYS will be mistakes in that estimator. The more complex the estimator the fewer such cases you will have, but the more mis-estimated such cases will be.
The only way to really deal with estimators is to *ALSO* make the price you pay on mis-estimation acceptable. For the scheduler that most likely means that you can't punish as hard as we do now, nor give bonuses as much as we do now.

If cc1 is doing a lot of waiting on IO, I fail to see how it should be called a CPU hog. OK I'll stop being difficult! I understand the problem is that its behaviour suddenly changes from IO bound to CPU hog, right? Then it seems like the scheduler's problem is that it doesn't adapt quickly enough to this change.

What you are doing is restricting some range so it can adapt more quickly right? So you still have the problem in the cases where you are not restricting this range.

For this, I reiterate my suggestion to intentionally over-shoot the mark. If you do it right, a process will run an inappropriate length of time only every other time slice until the oscillation dies down.

Quoted 3 lines

For this, I reiterate my suggestion to intentionally over-shoot the mark. If you do it right, a process will run an inappropriate length of time only every other time slice until the oscillation dies down.

Your thoughts are fine, and to some degree I do what you're proscribing, but I take into account the behaviour of real processes in the real world and their effect on scheduling fairness.

Quoted 6 lines · Con Kolivas

Quoted 3 lines

For this, I reiterate my suggestion to intentionally over-shoot the mark. If you do it right, a process will run an inappropriate length of time only every other time slice until the oscillation dies down.

Your thoughts are fine, and to some degree I do what you're proscribing,

but I

Quoted 1 line

take into account the behaviour of real processes in the real world and

their

Quoted 1 line

effect on scheduling fairness.

And I know you know a lot more about how real processes behave than I do. I'm not saying (or thinking) anything negative about you. I'm just trying to throw random thoughts into the mix just in case some small part of what I say is useful inspiration for someone else such as yourself.

It is probably the case that the idea I suggest is BS and makes no real difference or makes it worse anyhow. :)

Quoting Timothy Miller <[email protected]>:

Quoted 16 lines

Quoted 6 lines · Con Kolivas

Quoted 3 lines

For this, I reiterate my suggestion to intentionally over-shoot the mark. If you do it right, a process will run an inappropriate length of time only every other time slice until the oscillation dies down.

Your thoughts are fine, and to some degree I do what you're proscribing,

but I

Quoted 1 line

take into account the behaviour of real processes in the real world and

their

Quoted 5 lines

Quoted 1 line

effect on scheduling fairness.

And I know you know a lot more about how real processes behave than I do. I'm not saying (or thinking) anything negative about you. I'm just trying to throw random thoughts into the mix just in case some small part of what I say is useful inspiration for someone else such as yourself.

It is probably the case that the idea I suggest is BS and makes no real difference or makes it worse anyhow. :)

Nowhere do I recall saying your ideas were BS nor did I say you should stop throwing ideas at me. All thoughts are appreciated and considered. I'm pretty sure I said I do what you're suggesting anyway, bound by the limits of when those changes induce unfairness.

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Quoted 2 lines

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".
Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."
There is are "I have nothing to do right now, and I'm okay with that" sleeps, and there are "I have requested more work, and it should hurry up and get here" sleeps.

Quoted 3 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

Quoted 10 lines · Rob Landley

Quoted 2 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Quoted 1 line

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall. But in some cases I think there are times when you may not be able to use a non blocking call. And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Quoted 16 lines · Nick Piggin

Quoted 12 lines · Rob Landley

Quoted 2 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

Quoted 5 lines

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall. But in some cases I think there are times when you may not be able to use a non blocking call. And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Ah, but there is something there. Take the X and xmms's gl thread thingy I posted a while back. (X runs long enough to expire in the presence of a couple of low priority cpu hogs. gl thread, which is a mondo cpu hog, and normally runs and runs and runs at cpu hog priority, suddenly acquires extreme interactive priority, and X, which is normally sleepy suddenly becomes permanently runnable at cpu hog priority) The gl thread starts sleeping because X isn't getting enough cpu to be able to get it's work done and go to sleep. The gl thread isn't voluntarily sleeping, and X isn't voluntarily running. The behavior change is forced upon both.

Quoted 29 lines · Mike Galbraith

Quoted 18 lines · Nick Piggin

Quoted 8 lines · Rob Landley

Quoted 2 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Quoted 5 lines

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

Quoted 5 lines

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall. But in some cases I think there are times when you may not be able to use a non blocking call. And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Quoted 1 line

Ah, but there is something there. Take the X and xmms's gl thread thingy I posted a while back. (X runs long enough to expire in the presence of a couple of low priority cpu hogs. gl thread, which is a mondo cpu hog, and normally runs and runs and runs at cpu hog priority, suddenly acquires extreme interactive priority, and X, which is normally sleepy suddenly becomes permanently runnable at cpu hog priority) The gl thread starts sleeping because X isn't getting enough cpu to be able to get it's work done and go to sleep. The gl thread isn't voluntarily sleeping, and X isn't voluntarily running. The behavior change is forced upon both.

It does... It is I tell ya!

Look, the gl thread is probably _very_ explicitly asking to sleep. No I don't know how X works, but I have an idea that select is generally used as an event notification, right?

Now the gl thread is essentially saying "wait until X finishes the work I've given it, or I get some other event": ie. "put me to sleep until this fd becomes readable".

OK maybe your scenario is a big problem. Its not due to any imagined semantics in the way things are sleeping. Its due to the scheduler.

And no, X isn't intentionally sleeping. Its being preempted which is obviously not intentional.

Quoted 2 lines · Nick Piggin

And no, X isn't intentionally sleeping. Its being preempted which is obviously not intentional.

Right. Every time X wakes the gl thread, he'll lose the cpu. Once the gl thread passes X in priority, X is pretty much doomed. (hmm... sane [hard] backboost will probably prevent that)

Quoted 8 lines · Mike Galbraith

Quoted 2 lines · Nick Piggin

And no, X isn't intentionally sleeping. Its being preempted which is obviously not intentional.

Right. Every time X wakes the gl thread, he'll lose the cpu. Once the gl thread passes X in priority, X is pretty much doomed. (hmm... sane [hard] backboost will probably prevent that)

Isn't 2.4 doing exactly that for pipes and such?

Quoted 26 lines · Mike Galbraith

Quoted 13 lines · Nick Piggin

Quoted 11 lines · Rob Landley

Quoted 2 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Quoted 8 lines

Quoted 1 line

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall. But in some cases I think there are times when you may not be able to use a non blocking call. And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Quoted 2 lines

Ah, but there is something there. Take the X and xmms's gl thread thingy I posted a while back. (X runs long enough to expire in the presence of a couple of low priority cpu hogs. gl thread, which is a mondo cpu hog, and normally runs and runs and runs at cpu hog priority, suddenly acquires extreme interactive priority, and X, which is normally sleepy suddenly becomes permanently runnable at cpu hog priority) The gl thread starts sleeping because X isn't getting enough cpu to be able to get it's work done and go to sleep. The gl thread isn't voluntarily sleeping, and X isn't voluntarily running. The behavior change is forced upon both.

It does... It is I tell ya!

Look, the gl thread is probably _very_ explicitly asking to sleep. No I don't know how X works, but I have an idea that select is generally used as an event notification, right?

Now the gl thread is essentially saying "wait until X finishes the work I've given it, or I get some other event": ie. "put me to sleep until this fd becomes readable".

OK maybe your scenario is a big problem. Its not due to any imagined semantics in the way things are sleeping. Its due to the scheduler.

Quoted 2 lines · Nick Piggin

Mike Galbraith wrote:

Quoted 6 lines

Quoted 3 lines

Ah, but there is something there. Take the X and xmms's gl thread thingy I posted a while back. (X runs long enough to expire in the presence of a couple of low priority cpu hogs. gl thread, which is a mondo cpu hog, and normally runs and runs and runs at cpu hog priority, suddenly acquires extreme interactive priority, and X, which is normally sleepy suddenly becomes permanently runnable at cpu hog priority) The gl thread starts sleeping because X isn't getting enough cpu to be able to get it's work done and go to sleep. The gl thread isn't voluntarily sleeping, and X isn't voluntarily running. The behavior change is forced upon both.

Quoted 5 lines

It does... It is I tell ya!

Look, the gl thread is probably _very_ explicitly asking to sleep. No I don't know how X works, but I have an idea that select is generally used as an event notification, right?

Oh, sure, it blocks because it asks for it... but not because it _wants_ to :) It wants to create work for X fast enough to make a nice stutter free bit of eye-candy.

Quoted 4 lines

Now the gl thread is essentially saying "wait until X finishes the work I've given it, or I get some other event": ie. "put me to sleep until this fd becomes readable".

Yes. Voluntary or involuntary is just a matter of point of view.

Quoted 3 lines

OK maybe your scenario is a big problem. Its not due to any imagined semantics in the way things are sleeping. Its due to the scheduler.

It's due to the scheduler to a point... only in that it doesn't recognize the problem and correct it (that might be pretty hard to do). If my hardware were fast enough that X could get the work done in the allotted time, the problem wouldn't arise in the first place. I bet it's fairly hard to reproduce on a really fast box. It happens easily on my box because the combination of X and the gl thread need most of what my hardware has to offer.

I think backboost was very nice. I'd say Con could probably get a lot further if that was in but its not going to happen now.

Quoted 10 lines · Rob Landley

Quoted 2 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Quoted 4 lines

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall.

But in some cases
I think there are times when you may not be able to use a non blocking call.

And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Quoted 18 lines · Nick Piggin

Quoted 16 lines · Rob Landley

Quoted 3 lines · Nick Piggin

But by employing the kernel's services in the shape of a blocking syscall, all sleeps are intentional.

Wrong. Some sleeps indicate "I have run out of stuff to do right now, I'm going to wait for a timer or another process or something to wake me up with new work".

Some sleeps indicate "ideally this would run on an enormous ramdisk attached to gigabit ethernet, but hard drives and internet connections are just too slow so my true CPU-hogness is hidden by the fact I'm running on a PC instead of a mainframe."

Quoted 3 lines

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall.

So you can then block on poll instead, you mean?

Quoted 8 lines

But in some cases
I think there are times when you may not be able to use a non blocking call.

And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Audio playback, video playback, animated gifs in your web browser, and even first person shooters have built in rate limiting. (Okay, games can go to an insanely high framerate, but usually they achieve "good enough" and are happy with that unless you're doing a benchmark with them.)

There is a certain rate of work they do, and if they can manage that they stop working. On a system with twice as much CPU power and disks twice as fast, kmail shouldn't use significantly more CPU keeping up with my typing. These are "interactive" tasks.

Bug gzip, tar, gcc, and most cron jobs, are a different type of task. They have nobuilt-in rate limiting. On a system with twice as much CPU and disks twice as fast, they finish twice as quickly. They never voluntarily go idle until they exit; when they're idle it just means they hit a bottleneck. The system can never be "fast enough" that these quiesce themselves for a while because they've run out of work just now.

These are hogs, often both of CPU time and I/O bandwidth. Being blocked on I/O does not stop them from being hogs, it just means they're juggling their hoggishness.

An mpeg player has times when it's neither blocked on CPU or on I/O, it's waiting until it's time to display the next frame.

Now some of this could be viewed as a spooler problem, where there's a slow output device (the screen, the sound card, etc) and if you wanted to you could precompute stuff into a big memory wasting buffer and then instead of skipping because you're not getting scheduled fast enough you're skipping because your precomputed buffer got swapped to disk. But the difference here is that xmms or xine could do their output generation much faster than they are, if they wanted to. The output device could be sped up. Your animated gif can cycle too fast to see, you can fast-forward through your movie, you can play an mpeg so it sounds like chip and dale on helium... But they don't, they intentionally rate limit the output, and what they want in return is low latency when the rate limiting is up.

When you're rate limiting the output, you want to accurately control the rate. You don't want it to be too fast (timers are great at this), and you don't want it to be too slow (you get skips or miss frames).

That's what Con's detecting. It's a heuristic. But it's a good heuristic. A process that plays nice and yields the CPU regularly gets a priority boost. (That's always BEEN a heuristic.)

The current scheduler code has moved a bit beyond this, but this is the bit I was talking about when I disagreed with you earlier.

Quoted 4 lines

Quoted 2 lines

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall.

So you can then block on poll instead, you mean?

Well if thats what you intend, yes. Or set poll to be non-blocking.

Quoted 3 lines

These are hogs, often both of CPU time and I/O bandwidth. Being blocked on I/O does not stop them from being hogs, it just means they're juggling their hoggishness.

This is the CPU scheduler though. A program could be a disk/network hog and use a few % cpu. Its obviously not a cpu hog, and should get the cpu again soon after it is woken. Sooner than non running cpu hogs, anyway.

Quoted 6 lines

That's what Con's detecting. It's a heuristic. But it's a good heuristic. A process that plays nice and yields the CPU regularly gets a priority boost. (That's always BEEN a heuristic.)

The current scheduler code has moved a bit beyond this, but this is the bit I was talking about when I disagreed with you earlier.

Yeah, I know Con is trying to detect this. Its just that detecting it using TASK_INTERRUPTIBLE/TASK_UNINTERRUPTIBLE may not be the best way.

Suddenly your kernel compile on an NFS mount becomes interactive for example.

Then again, the way things are, Con might not have any other option.

Mostly I agree with what you've said above.

Quoted 3 lines · Nick Piggin

Quoted 3 lines

Quoted 3 lines

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall.

Quoted 4 lines

Quoted 2 lines

So you can then block on poll instead, you mean?

Well if thats what you intend, yes. Or set poll to be non-blocking.

So you're still blocking for an unknown amount of time waiting for your outstanding requests to get serviced, now you're just hiding it to intentionally give the scheduler less information to work with.

Quoted 10 lines

Quoted 5 lines

These are hogs, often both of CPU time and I/O bandwidth. Being blocked on I/O does not stop them from being hogs, it just means they're juggling their hoggishness.

This is the CPU scheduler though. A program could be a disk/network hog and use a few % cpu. Its obviously not a cpu hog, and should get the cpu again soon after it is woken. Sooner than non running cpu hogs, anyway.

A program that waits for a known amount of time (I.E. on a timer) cares about when it gets woken up. A program that blocks on an event that's going to take an unknown amount of time can't be too upset if its wakeup is after an unknown amount of time.

Beyond that there's blocking for input from the user (latency matters) and blocking for input from something else (latency doesn't matter), but we can't tell that directly and have to fake our way around it with heuristics.

Quoted 7 lines

Quoted 7 lines

That's what Con's detecting. It's a heuristic. But it's a good heuristic. A process that plays nice and yields the CPU regularly gets a priority boost. (That's always BEEN a heuristic.)

The current scheduler code has moved a bit beyond this, but this is the bit I was talking about when I disagreed with you earlier.

Quoted 4 lines

Yeah, I know Con is trying to detect this. Its just that detecting it using TASK_INTERRUPTIBLE/TASK_UNINTERRUPTIBLE may not be the best way.

Okay, if this isn't the "best way", then what is? You have yet to suggest an alternative, and this heuristic is obviously better than nothing.

Quoted 4 lines

Suddenly your kernel compile on an NFS mount becomes interactive for example.

Translation: Suppose the heuristics fail. If it can't fail, it's not a heuristic, is it? Failure of heuristics must be survivable. The kernel compile IS a rampant CPU hog, and if it's mis-identified as interactive for some reason it'll get demoted again after using up too many time slices. In the mean time, your PVR (think home-browed Tivo clone) skips recording your buffy rerun. This is something to be minimized, but the scheduler isn't psychic. If it happens to once out of every million hours of use, you're going to see more hard drive failures due and dying power supplies than problems caused by this. (This is not sufficient for running a nuclear power plant or automated factory, but those guys need hard realtime anyway, which this isn't pretending to be.)

Quoted 4 lines

Then again, the way things are, Con might not have any other option.

You're welcome to suggest a better alternative, but criticizing the current approach without suggesting any alternative at all may not be that helpful.

I have been hearing of people complaining the scheduler is worse than 2.4 so its not entirely obvious to me. But yeah lots of it is trial and error, so I'm not saying Con is wasting his time.

Quoted 3 lines · Nick Piggin

I have been hearing of people complaining the scheduler is worse than 2.4 so its not entirely obvious to me. But yeah lots of it is trial and error, so I'm not saying Con is wasting his time.

I've been watching Con and Ingo's efforts with the process scheduler and i haven't seen people complaining that the process scheduler is worse. They have complained that interactive processes seem to have more latency. Con has rightly questioned whether that might be because the process scheduler has less control over CPU time allocation than in 2.4. Remember that the process scheduler only manages the CPU time not spent in I/O and other overhead.

If there is something in BIO chewing cycles it will wreak havoc with latency no matter what you do about process scheduling. The work on BIO to improve bandwidth and reduce latency was Herculean but the growing performance gap between CPU and I/O is a formidable challenge.

Quoted 15 lines · jw schultz

Quoted 4 lines · Nick Piggin

I have been hearing of people complaining the scheduler is worse than 2.4 so its not entirely obvious to me. But yeah lots of it is trial and error, so I'm not saying Con is wasting his time.

I've been watching Con and Ingo's efforts with the process scheduler and i haven't seen people complaining that the process scheduler is worse. They have complained that interactive processes seem to have more latency. Con has rightly questioned whether that might be because the process scheduler has less control over CPU time allocation than in 2.4. Remember that the process scheduler only manages the CPU time not spent in I/O and other overhead.

Quoted 6 lines

If there is something in BIO chewing cycles it will wreak havoc with latency no matter what you do about process scheduling. The work on BIO to improve bandwidth and reduce latency was Herculean but the growing performance gap between CPU and I/O is a formidable challenge.

In thinking about this from the aspect of what I do here, this makes quite a bit of sense. In running 2.6.0-test3, with anticipatory scheduler, it appears the i/o intensive tasks are being pushed back in favor of interactivity, perhaps a bit too aggressively. An amanda estimate phase, which turns tar loose on the drives, had to be advanced to a -10 niceness for the whole tree of processes amanda spawns before it began to impact the setiathome use as shown by the nice display in gkrellm. Normally there is a period for maybe 20 minutes before the tape drive fires up where the machine is virtually unusable due to gzip hogging things, like the cpu, during which time seti could just as easily be swapped out. It remained at around 60%!

It did not hog/lag near as badly as usual, and the amanda run was over an hour longer than it would have been in 2.4.22-rc2.

It is my opinion that all this should have been at setiathomes expense, which is also rather cpu intensive, but it didn't seem to be without lots of forceing. This is what the original concept of niceness was all about. Or at least that was my impression. From what it feels like here, it seems the i/o stuff is whats being choked, and choked pretty badly when using the anticipatory scheduler.

I've read rumors that a boottime option can switch it to somethng else, so what do I do to switch it from the anticipatory scheduler to whatever the alternate is?, so that I can get a feel for the other methods and results.

Quoted 45 lines · Gene Heskett

Quoted 14 lines · jw schultz

Quoted 3 lines · Nick Piggin

I have been hearing of people complaining the scheduler is worse than 2.4 so its not entirely obvious to me. But yeah lots of it is trial and error, so I'm not saying Con is wasting his time.

I've been watching Con and Ingo's efforts with the process scheduler and i haven't seen people complaining that the process scheduler is worse. They have complained that interactive processes seem to have more latency. Con has rightly questioned whether that might be because the process scheduler has less control over CPU time allocation than in 2.4. Remember that the process scheduler only manages the CPU time not spent in I/O and other overhead.

Quoted 5 lines

If there is something in BIO chewing cycles it will wreak havoc with latency no matter what you do about process scheduling. The work on BIO to improve bandwidth and reduce latency was Herculean but the growing performance gap between CPU and I/O is a formidable challenge.

In thinking about this from the aspect of what I do here, this makes quite a bit of sense. In running 2.6.0-test3, with anticipatory scheduler, it appears the i/o intensive tasks are being pushed back in favor of interactivity, perhaps a bit too aggressively. An amanda estimate phase, which turns tar loose on the drives, had to be advanced to a -10 niceness for the whole tree of processes amanda spawns before it began to impact the setiathome use as shown by the nice display in gkrellm. Normally there is a period for maybe 20 minutes before the tape drive fires up where the machine is virtually unusable due to gzip hogging things, like the cpu, during which time seti could just as easily be swapped out. It remained at around 60%!

It did not hog/lag near as badly as usual, and the amanda run was over an hour longer than it would have been in 2.4.22-rc2.

It is my opinion that all this should have been at setiathomes expense, which is also rather cpu intensive, but it didn't seem to be without lots of forceing. This is what the original concept of niceness was all about. Or at least that was my impression. From what it feels like here, it seems the i/o stuff is whats being choked, and choked pretty badly when using the anticipatory scheduler.

Quoted 4 lines

I've read rumors that a boottime option can switch it to somethng else, so what do I do to switch it from the anticipatory scheduler to whatever the alternate is?, so that I can get a feel for the other methods and results.

Boot with "elevator=deadline" to use the more conventional elevator.

It would be good if you could get some numbers 2.4 vs 2.6, with and without seti running. Sounds like a long cycle though so you probably can't be bothered!

Unrelated question: I've applied the 2.6 patches someone pointed me at to the nvidia-linux-4496-pkg2 after figuring out how to get it to unpack and leave itself behind, so x can be run on 2.6 now. But its a 100% total crash to exit x by any method when using it that way.

Has the patch been updated in the last couple of weeks to prevent that now? It takes nearly half an hour to e2fsck a hundred gigs worth of drives, and its going to bite me if I don't let the system settle before I crash it to reboot, finishing the reboot with the hardware reset button.

Better yet, a fresh pointer to that site.

--On Wednesday, August 13, 2003 01:24:31 AM -0400 Gene Heskett

Quoted 6 lines · <[email protected]> wrote:

Unrelated question: I've applied the 2.6 patches someone pointed me at to the nvidia-linux-4496-pkg2 after figuring out how to get it to unpack and leave itself behind, so x can be run on 2.6 now. But its a 100% total crash to exit x by any method when using it that way.

Quoted 7 lines

Has the patch been updated in the last couple of weeks to prevent that now? It takes nearly half an hour to e2fsck a hundred gigs worth of drives, and its going to bite me if I don't let the system settle before I crash it to reboot, finishing the reboot with the hardware reset button.

Better yet, a fresh pointer to that site.

http://www.minion.de/

Works fine for me, as of 2.6.0-test1 (which is when I downloaded the patch). I don't get the crash on either of my systems (GeForce2Go P3 laptop and GeForce4 Athlon desktop).

Andrew

Quoted 17 lines · Andrew McGregor

--On Wednesday, August 13, 2003 01:24:31 AM -0400 Gene Heskett

Quoted 6 lines · <[email protected]> wrote:

Unrelated question: I've applied the 2.6 patches someone pointed me at to the nvidia-linux-4496-pkg2 after figuring out how to get it to unpack and leave itself behind, so x can be run on 2.6 now. But its a 100% total crash to exit x by any method when using it that way.

Quoted 7 lines

Has the patch been updated in the last couple of weeks to prevent that now? It takes nearly half an hour to e2fsck a hundred gigs worth of drives, and its going to bite me if I don't let the system settle before I crash it to reboot, finishing the reboot with the hardware reset button.

Better yet, a fresh pointer to that site.

Quoted 7 lines

http://www.minion.de/

Works fine for me, as of 2.6.0-test1 (which is when I downloaded the patch). I don't get the crash on either of my systems (GeForce2Go P3 laptop and GeForce4 Athlon desktop).

Andrew

I see some notes about patching X, which I haven't done. That might be it. I also doublechecked that I'm running the correct makefile, and get this:

[root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2]# ls -lR * |grep Makefile

(quoted diff) +0 -4

-rw-r--r--    1 root     root         3623 Jul 16 22:56 Makefile
-rw-r--r--    1 root     root         7629 Aug  5 22:24 Makefile
-rw-r--r--    1 root     root         7629 Aug  5 21:46 Makefile.kbuild
-rw-r--r--    1 root     root         4865 Aug  5 21:46 Makefile.nvidia

[root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2]# cd ../NVIDIA-Linux-x86-1.0-4496-pkg2-4-2.4/ [root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2-4-2.4]# ls -lR * |grep Makefile -rw-r--r--
1 root root 3623 Jul 16 22:56 Makefile
-rw-r--r-- 1 root root 5665 Jul 16 22:56 Makefile
[root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2-4-2.4]#

My video card, from an lspci:
01:00.0 VGA compatible controller: nVidia Corporation NV11 [GeForce2 MX DDR] (rev b2)

And the XFree86 version is:
3.2.1-21

Interesting to note that the 'nv' driver that comes with X does not do this. But it also has no openGL and such. We are instructed to remove agp support from the kernel, and use that which is in the nvidia kit, and I just checked the .config, and its off, so thats theoreticly correct. A grep for FB stuff returns this:

CONFIG_FB=y
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
CONFIG_FB_VESA=y
# CONFIG_FB_HGA is not set
CONFIG_FB_RIVA=y
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_VIRTUAL is not set

I'd assume the 'RIVA' fb is the correct one, its working in 2.4, although I can induce a crash there by switching from X to a virtual console, and then attempting to switch back to X. That will generally bring the machine down. It is perfectly ok to do that, repeatedly, when running the nv driver from X.

--
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz 512M
99.27% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.

Ah. I see you have framebuffer console on, whereas I have plain VGA console only. Try turning framebuffer off; two drivers for the same hardware may well fight over it. My X isn't patched, it just has their driver modules and the libraries installed.

Andrew

From dmesg, snippets:

--On Wednesday, August 13, 2003 08:33:43 AM -0400 Gene Heskett

Quoted 92 lines · <[email protected]> wrote:

Quoted 17 lines · Andrew McGregor

--On Wednesday, August 13, 2003 01:24:31 AM -0400 Gene Heskett

Quoted 6 lines · <[email protected]> wrote:

Unrelated question: I've applied the 2.6 patches someone pointed me at to the nvidia-linux-4496-pkg2 after figuring out how to get it to unpack and leave itself behind, so x can be run on 2.6 now. But its a 100% total crash to exit x by any method when using it that way.

Quoted 7 lines

Has the patch been updated in the last couple of weeks to prevent that now? It takes nearly half an hour to e2fsck a hundred gigs worth of drives, and its going to bite me if I don't let the system settle before I crash it to reboot, finishing the reboot with the hardware reset button.

Better yet, a fresh pointer to that site.

Quoted 7 lines

http://www.minion.de/

Works fine for me, as of 2.6.0-test1 (which is when I downloaded the patch). I don't get the crash on either of my systems (GeForce2Go P3 laptop and GeForce4 Athlon desktop).

Andrew

I see some notes about patching X, which I haven't done. That might be it. I also doublechecked that I'm running the correct makefile, and get this:

[root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2]# ls -lR * |grep

Makefile -rw-r--r--    1 root     root         3623 Jul 16 22:56
Makefile -rw-r--r--    1 root     root         7629 Aug  5 22:24
Makefile -rw-r--r--    1 root     root         7629 Aug  5 21:46

Makefile.kbuild -rw-r--r-- 1 root root 4865 Aug 5
21:46 Makefile.nvidia [root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2]# cd ../NVIDIA-Linux-x86-1.0-4496-pkg2-4-2.4/ [root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2-4-2.4]# ls -lR * |grep Makefile

(quoted diff) +0 -2

-rw-r--r-- 1 root     root         3623 Jul 16 22:56 Makefile
-rw-r--r--    1 root     root         5665 Jul 16 22:56 Makefile

[root@coyote NVIDIA-Linux-x86-1.0-4496-pkg2-4-2.4]#

My video card, from an lspci:
01:00.0 VGA compatible controller: nVidia Corporation NV11 [GeForce2 MX DDR] (rev b2)

And the XFree86 version is:
3.2.1-21

Interesting to note that the 'nv' driver that comes with X does not do this. But it also has no openGL and such. We are instructed to remove agp support from the kernel, and use that which is in the nvidia kit, and I just checked the .config, and its off, so thats theoreticly correct. A grep for FB stuff returns this:

CONFIG_FB=y
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
CONFIG_FB_VESA=y
# CONFIG_FB_HGA is not set
CONFIG_FB_RIVA=y
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_VIRTUAL is not set

I'd assume the 'RIVA' fb is the correct one, its working in 2.4, although I can induce a crash there by switching from X to a virtual console, and then attempting to switch back to X. That will generally bring the machine down. It is perfectly ok to do that, repeatedly, when running the nv driver from X.

--
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz 512M
99.27% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.

Quoted 1 line

I don't quite understand what you are getting at, but if you don't want to sleep you should be able to use a non blocking syscall. But in some cases I think there are times when you may not be able to use a non blocking call.

And if a process is a CPU hog, its a CPU hog. If its not its not. Doesn't matter how it would behave on another system.

Quoted 3 lines

In short: make the same policy for an interruptible and an uninterruptible sleep.

That's the policy that has always existed...

Interesting that I have only seen the desired effect and haven't noticed any side effect from this change so far. I'll keep experimenting as much as possible (as if I wasn't going to) and see what the testers find as well.

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.

Very early on in AS development I was playing with adding "extra boost" to the anticipated-upon task, but it did appear that the stock scheduler was sufficiently doing the right thing anyway.

Quoted 5 lines

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.

Indeed that has been on my mind. This change doesn't affect how long it takes to wake up. It simply prevents tasks from getting full interactive status during the period they are doing unint. sleep.

Am Dienstag, 5. August 2003 09:26 schrieb Con Kolivas:

Quoted 7 lines · Andrew Morton

Quoted 5 lines

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.

Quoted 3 lines

Indeed that has been on my mind. This change doesn't affect how long it takes to wake up. It simply prevents tasks from getting full interactive status during the period they are doing unint. sleep.

If you take that to its logical conclusion, such tasks should be woken immediately. Likewise, the io scheduler should be notified when you know that the task won't do io or will do other io, like waiting on character devices, go paging out or terminate.

Quoted 12 lines · Oliver Neukum

Am Dienstag, 5. August 2003 09:26 schrieb Con Kolivas:

Quoted 10 lines · Andrew Morton

Quoted 5 lines

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.

Indeed that has been on my mind. This change doesn't affect how long it takes to wake up. It simply prevents tasks from getting full interactive status during the period they are doing unint. sleep.

Quoted 4 lines

If you take that to its logical conclusion, such tasks should be woken immediately. Likewise, the io scheduler should be notified when you know that the task won't do io or will do other io, like waiting on character devices, go paging out or terminate.

Every experiment I've tried at putting tasks at the start of the queue instead of the end has resulted in some form of starvation so should not be possible for any user task and I've abandoned it.

Quoted 4 lines · Con Kolivas

Every experiment I've tried at putting tasks at the start of the queue instead
of the end has resulted in some form of starvation so should not be possible for any user task and I've abandoned it.

(ditto:)

Quoted 7 lines · Mike Galbraith

Quoted 4 lines · Con Kolivas

Every experiment I've tried at putting tasks at the start of the queue instead
of the end has resulted in some form of starvation so should not be possible for any user task and I've abandoned it.

(ditto:)

Superuser access real time tasks may be worth reconsidering though...

Quoted 7 lines · Con Kolivas

Quoted 6 lines · Mike Galbraith

Quoted 4 lines · Con Kolivas

Every experiment I've tried at putting tasks at the start of the queue instead
of the end has resulted in some form of starvation so should not be possible for any user task and I've abandoned it.

Quoted 3 lines

Quoted 1 line

(ditto:)

Superuser access real time tasks may be worth reconsidering though...

If they were guaranteed ultra-light, maybe, but userland is just not trustworthy.

Better imho would be something like Davide's SOFT_RR with an additional automatic priority adjust per cpu usage or something (cpu usage being a [very] little bit of a latency hint, and a great 'hurt me' hint). Best would be an API that allowed userland applications to describe their latency requirements explicitly, with the scheduler watching users of this API like a hawk, ever ready to sanction abusers. Anything I think about in this area gets uncomfortably close to hard rt though, and all of the wisdom I've heard on LKLM over the years wrt separation of problem spaces comes flooding back.

Am Dienstag, 5. August 2003 09:26 schrieb Con Kolivas:

Quoted 8 lines · Andrew Morton

Quoted 7 lines

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.

Quoted 2 lines

Indeed that has been on my mind. This change doesn't affect how long it takes to wake up. It simply prevents tasks from getting full interactive status during the period they are doing unint. sleep.

If you take that to its logical conclusion, such tasks should be woken immediately. Likewise, the io scheduler should be notified when you know that the task won't do io or will do other io, like waiting on character devices, go paging out or terminate.

Quoted 5 lines · Andrew Morton

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.

Indeed that has been on my mind. This change doesn't affect how long it takes to wake up. It simply prevents tasks from getting full interactive status during the period they are doing unint. sleep.

Quoted 3 lines

Very early on in AS development I was playing with adding "extra boost" to the anticipated-upon task, but it did appear that the stock scheduler was sufficiently doing the right thing anyway.

Con

It seems that there's a special case, where a task that was blocked on a read (either from a file or from swap) wants to be scheduled immediately, but with a really short timeslice. I.E. give it the ability to submit another read and block on it immediately, but if a single jiffy goes by and it hasn't done it, it should go away.
This has nothing to do with the normal priority levels or being considered interactive or not. As I said, a special case. IO_UNBLOCKED_FLAG or some such. Maybe unnecessary...
(Once again, the percentage of CPU time to devote to a task and the immediacy of scheduling that task are in opposition. The "priority" abstraction is a bit too simple at times...)

Quoted 2 lines · Con Kolivas

Quoted 2 lines

In short: make the same policy for an interruptible and an

uninterruptible

Quoted 5 lines

Quoted 1 line

sleep.

That's the policy that has always existed...

Interesting that I have only seen the desired effect and haven't

noticed any

Quoted 2 lines

side effect from this change so far. I'll keep experimenting as much as possible (as if I wasn't going to) and see what the testers find as well.

We do prefer that TASK_UNINTERRUPTIBLE processes are woken promptly so they can submit more IO and go back to sleep. Remember that we are artificially leaving the disk head idle in the expectation that the task will submit more I/O. It's pretty sad if the CPU scheduler leaves the anticipated task in the doldrums for five milliseconds.