Skip to contentSkip to conversation

O14int

PATCH In review Con Kolivas 9 participants 24 messages Aug 8, 2003 – Aug 17, 2003

lore
Share & tools
lore

Con Kolivas

author
#0 lore raw

PATCH 1/1 O14int

More duck tape interactivity tweaks

Changes

Put some bounds on the interactive_credit and specified a size beyond which a task is considered highly interactive or not.

Moved the uninterruptible sleep limiting to within recalc_task_prio removing one call of the sched_clock() and being more careful about the limits.

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Put greater limitation on the requeuing code; now it only requeues interactive tasks thereby letting cpu hogs run their full timeslice unabated which should improve cpu intensive task performance.

Made highly interactive tasks earn all their waiting time on the runqueue during requeuing as sleep_avg.

Patch applies onto 2.6.0-test2-mm5

http://kernel.kolivas.org/2.5
has the patch and a full patch against 2.6.0-test2

Patch O13.1-O14int:

kernel/sched.c +51 -19
@@ -130,6 +130,15 @@
 #define JUST_INTERACTIVE_SLEEP(p) \
 	(MAX_SLEEP_AVG - (DELTA(p) * AVG_TIMESLICE))
 
+#define HIGH_CREDIT(p) \
+	((p)->interactive_credit > MAX_SLEEP_AVG)
+
+#define LOW_CREDIT(p) \
+	((p)->interactive_credit < -MAX_SLEEP_AVG)
+
+#define VARYING_CREDIT(p) \
+	(!(HIGH_CREDIT(p) || LOW_CREDIT(p)))
+
 #define TASK_PREEMPTS_CURR(p, rq) \
 	((p)->prio < (rq)->curr->prio || \
 		((p)->prio == (rq)->curr->prio && \

@@ -364,7 +373,7 @@ static void recalc_task_prio(task_t *p, 
 	unsigned long long __sleep_time = now - p->timestamp;
 	unsigned long sleep_time;
 
-	if (!p->sleep_avg)
+	if (!p->sleep_avg && VARYING_CREDIT(p))
 		p->interactive_credit--;
 
 	if (__sleep_time > NS_MAX_SLEEP_AVG)

@@ -373,7 +382,6 @@ static void recalc_task_prio(task_t *p, 
 		sleep_time = (unsigned long)__sleep_time;
 
 	if (likely(sleep_time > 0)) {
-
 		/*
 		 * User tasks that sleep a long time are categorised as
 		 * idle and will get just interactive status to stay active &

@@ -387,20 +395,38 @@ static void recalc_task_prio(task_t *p, 
 		else {
 			/*
 			 * The lower the sleep avg a task has the more
-			 * rapidly it will rise with sleep time. Tasks
-			 * without interactive_credit are limited to
-			 * one timeslice worth of sleep avg bonus.
+			 * rapidly it will rise with sleep time.
 			 */
 			sleep_time *= (MAX_BONUS + 1 -
 					(NS_TO_JIFFIES(p->sleep_avg) *
 					MAX_BONUS / MAX_SLEEP_AVG));
 
-			if (p->interactive_credit < 0 &&
+			/*
+			 * Tasks with low interactive_credit are limited to
+			 * one timeslice worth of sleep avg bonus.
+			 */
+			if (LOW_CREDIT(p) &&
 				sleep_time > JIFFIES_TO_NS(task_timeslice(p)))
 					sleep_time =
 						JIFFIES_TO_NS(task_timeslice(p));
 
 			/*
+			 * Non high_credit tasks waking from uninterruptible
+			 * sleep are limited in their sleep_avg rise
+			 */
+			if (!HIGH_CREDIT(p) && p->activated == -1){
+				if (p->sleep_avg >=
+					JIFFIES_TO_NS(JUST_INTERACTIVE_SLEEP(p)))
+						sleep_time = 0;
+				else if (p->sleep_avg + sleep_time >=
+					JIFFIES_TO_NS(JUST_INTERACTIVE_SLEEP(p))){
+						p->sleep_avg =
+							JIFFIES_TO_NS(JUST_INTERACTIVE_SLEEP(p));
+						sleep_time = 0;
+					}
+			}
+
+			/*
 			 * This code gives a bonus to interactive tasks.
 			 *
 			 * The boost works by updating the 'average sleep time'

@@ -418,7 +444,7 @@ static void recalc_task_prio(task_t *p, 
 			 */
 			if (p->sleep_avg > NS_MAX_SLEEP_AVG){
 				p->sleep_avg = NS_MAX_SLEEP_AVG;
-				p->interactive_credit++;
+				p->interactive_credit += VARYING_CREDIT(p);
 			}
 		}
 	}

@@ -588,11 +614,7 @@ repeat_lock_task:
 				 * Tasks on involuntary sleep don't earn
 				 * sleep_avg beyond just interactive state.
 				 */
-				if (NS_TO_JIFFIES(p->sleep_avg) >=
-					JUST_INTERACTIVE_SLEEP(p)){
-						p->timestamp = sched_clock();
-						p->activated = -1;
-				}
+				p->activated = -1;
 			}
 			if (sync)
 				__activate_task(p, rq);

@@ -1156,9 +1178,9 @@ skip_queue:
 	 * 3) are cache-hot on their current CPU.
 	 */
 
-#define CAN_MIGRATE_TASK(p,rq,this_cpu)					\
-	((!idle || (((now - (p)->timestamp)>>10) > cache_decay_ticks)) &&\
-		!task_running(rq, p) &&					\
+#define CAN_MIGRATE_TASK(p,rq,this_cpu)	\
+	((!idle || (NS_TO_JIFFIES(now - (p)->timestamp) > \
+		cache_decay_ticks)) && !task_running(rq, p) && \
 			cpu_isset(this_cpu, (p)->cpus_allowed))
 
 	curr = curr->prev;

@@ -1366,13 +1388,23 @@ void scheduler_tick(int user_ticks, int 
 		 * requeue this task to the end of the list on this priority
 		 * level, which is in essence a round-robin of tasks with
 		 * equal priority.
+		 *
+		 * This only applies to user tasks in the interactive
+		 * delta range with at least MIN_TIMESLICE left.
 		 */
-		if (p->mm && !((task_timeslice(p) - p->time_slice) %
-			TIMESLICE_GRANULARITY) && (p->time_slice > MIN_TIMESLICE) &&
+		if (p->mm && TASK_INTERACTIVE(p) && !((task_timeslice(p) -
+			p->time_slice) % TIMESLICE_GRANULARITY) &&
+			(p->time_slice > MIN_TIMESLICE) &&
 			(p->array == rq->active)) {
 
 			dequeue_task(p, rq->active);
 			set_tsk_need_resched(p);
+			/*
+			 * Tasks with interactive credit get all their
+			 * time waiting on the run queue credited as sleep
+			 */
+			if (HIGH_CREDIT(p))
+				p->activated = 2;
 			enqueue_task(p, rq->active);
 		}
 	}

@@ -1426,7 +1458,7 @@ need_resched:
 	 * as their sleep_avg decreases to slow them losing their
 	 * priority bonus
 	 */
-	if (prev->interactive_credit > 0)
+	if (HIGH_CREDIT(prev))
 		run_time /= (MAX_BONUS + 1 -
 			(NS_TO_JIFFIES(prev->sleep_avg) * MAX_BONUS /
 			MAX_SLEEP_AVG));

@@ -1484,12 +1516,12 @@ pick_next_task:
 		if (next->activated == 1)
 			delta = delta * (ON_RUNQUEUE_WEIGHT * 128 / 100) / 128;
 
-		next->activated = 0;
 		array = next->array;
 		dequeue_task(next, array);
 		recalc_task_prio(next, next->timestamp + delta);
 		enqueue_task(next, array);
 	}
+	next->activated = 0;
 switch_tasks:
 	prefetch(next);
 	clear_tsk_need_resched(prev);
#1 lore raw
Quoted 2 lines · Con Kolivas

Made highly interactive tasks earn all their waiting time on the runqueue during requeuing as sleep_avg.

There are some mechanics of this that I am not familiar with, so please excuse the naive question.

Someone had suggested that a task's sleep time should be determine exclusively from the time it spends blocked by what it's waiting on, and not based on any OTHER time it sleeps. That is, the time between the I/O request being satisfied and the task actually getting the CPU doesn't count.

Is your statement above a reflection of that suggestion?

author
#2 lore raw
Quoted 7 lines · Timothy Miller
Quoted 2 lines · Con Kolivas

Made highly interactive tasks earn all their waiting time on the runqueue during requeuing as sleep_avg.

There are some mechanics of this that I am not familiar with, so please excuse the naive question.

Quoted 7 lines

Someone had suggested that a task's sleep time should be determine exclusively from the time it spends blocked by what it's waiting on, and not based on any OTHER time it sleeps. That is, the time between the I/O request being satisfied and the task actually getting the CPU doesn't count.

Is your statement above a reflection of that suggestion?

I have already been doing this in previous iterations to the extent that it was capable of being done. There's only so much information feeding into the scheduler at the moment. All the dependent subsections would have to have more code to have specific feedback to the scheduler.

Con

#3 lore raw
In reply to Con Kolivas
Quoted 13 lines · Con Kolivas

More duck tape interactivity tweaks

Changes

Put some bounds on the interactive_credit and specified a size beyond which a task is considered highly interactive or not.

Moved the uninterruptible sleep limiting to within recalc_task_prio removing one call of the sched_clock() and being more careful about the limits.

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 6 lines

Put greater limitation on the requeuing code; now it only requeues interactive tasks thereby letting cpu hogs run their full timeslice unabated which should improve cpu intensive task performance.

Made highly interactive tasks earn all their waiting time on the runqueue during requeuing as sleep_avg.

I have been testing this jewel on top of 2.6.0-test2-mm5 for a couple of hours. It behaves much like O13*int, well maybe a little bit better:
Renicing X to -20 is a total disaster (sluggish window movement, Juk skipping like mad, etc), but with X at +0 the system feels pretty good.

Evolution still feels like a mamooth when moving windows over it. XMMS doesn't skip, neither Juk does when X is at +0. This is good :-) All in all, this one feels good!

PS: May I say O10int is still a little bit smoother than this one? ;-)

author
#4 lore raw
In reply to Con Kolivas
Quoted 1 line · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 2 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Con

#5 lore raw
Quoted 11 lines · Con Kolivas
Quoted 1 line · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 2 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

I am not so sure I for one want changes to the scheduler for SMP (not UP interactivity ones anyhow).

Cheers,

Signature · 1 line
Martin Schlemmer
author
#6 lore raw
Quoted 10 lines · Martin Schlemmer
Quoted 9 lines · Con Kolivas
Quoted 1 line · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 3 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 6 lines
Quoted 2 lines

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

A UP HT box you mean? That shouldn't be capable of running multiple make -j10s without some noticable effect. Apart from looking impressive, there is no point in having 30 cpu heavy things running with only 1 and a bit processor and the machine being smooth as silk; the cpu heavy things will just be unfairly starved in the interest of appearance (I can do that easily enough). Please give details if there is a specific issue you think I've broken or else I wont know about it.

Quoted 2 lines

I am not so sure I for one want changes to the scheduler for SMP (not UP interactivity ones anyhow).

They're not; the improvements should affect fairness on SMP as well and although interactivity is what I'm addressing on the surface, fairness is the real issue.

Con.

#7 lore raw
Quoted 11 lines · Con Kolivas
Quoted 10 lines · Martin Schlemmer
Quoted 9 lines · Con Kolivas
Quoted 1 line · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 3 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

A UP HT box you mean?

Given :D

Quoted 8 lines

That shouldn't be capable of running multiple make -j10s
without some noticable effect. Apart from looking impressive, there is no point in having 30 cpu heavy things running with only 1 and a bit processor and the machine being smooth as silk; the cpu heavy things will just be unfairly starved in the interest of appearance (I can do that easily enough). Please give details if there is a specific issue you think I've broken or else I wont know about it.

My opinion when the first 3.06 with HT came out was also sceptic. They did not perform that well. Things did change though - with the 8[67]5p chipsets and dual channel ddr400 it is a vast improvement, even if only a P4 2.4C running HT.

To give a good example (right, not linux based :P), my brother have pretty much the same system running XP. With the P4T533-c (rambus) running a 2.4B, he could not do video encoding at highest priority (or even high) and be able to do much else. With 875p and 2.4C he do encoding at highest priority, get frame rates of 28-35+ (with the dual pass, etc) which is higher than the old 2.4B, while playing C&C Generals, watching movie, etc.

My system runs two make -j24's (yes, just testing), while MP3's play smooth and general moving between desktops and windows is still smooth with the default scheduler.

After all - most of what I do is compile too many things while trying to read email, browse, irc and listen to MP3's. I do not mind the obvious skip or two if really pushing the box, and after all, interactivity is 50% in the mind (ok, maybe not that much), but I do notice a degrade in 'interactivity' with your patches and HT enabled on this box.

Another question - should bogomips (or some other type of general system performance measurement) not have a bigger role in how the scheduler work ? Maybe I am on crack, but I assume a process gets more 'slices' per second/minute on a 3GHz machine than on a 300MHz ? It may already, have not checked =)

Quoted 7 lines
Quoted 2 lines

I am not so sure I for one want changes to the scheduler for SMP (not UP interactivity ones anyhow).

They're not; the improvements should affect fairness on SMP as well and although interactivity is what I'm addressing on the surface, fairness is the real issue.

Yes, but 'fairness' and 'interactivity' is not the same thing (IMHO).

'fairness' in my books means give each process enough time when it is needed so that everything gets whatever it do done without being 'forgotten' for too long, or having disk/whatever hold up too badly, or starving a smaller process to death.

'interactivity' on the other hand means (my books again) starve everything that would not be noticed by the user so that he can 'wiggle around windows' (sorry, seen this a few times, and could not help myself =) to his heats content and still not get XMMS to skip (ok, so maybe this may be once again too far fetched :-).

Just me.

NB: any chance to get you patches against vanilla/bk as well,
as I in general like rolling my own kernels more than using mm, jc, etc (no offence guys).

Thanks,

Signature · 1 line
Martin Schlemmer
#10 lore raw
In reply to Con Kolivas
Quoted 18 lines · Con Kolivas
Quoted 16 lines · Martin Schlemmer
Quoted 14 lines · Con Kolivas
Quoted 2 lines · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 4 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Quoted 7 lines
Quoted 4 lines

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

A UP HT box you mean? That shouldn't be capable of running multiple make -j10s without some noticable effect. Apart from looking impressive, there is no point in having 30 cpu heavy things running with only 1 and a bit processor and the machine being smooth as silk; the cpu heavy things will just be unfairly starved in the interest of appearance (I can do that easily enough). Please give details if there is a specific issue you think I've broken or else I wont know about it.

Yeah make -j10s won't be without impact, but I think for a lot of interactive stuff they don't need a lot of CPU, just to get it in a timely manner. And Martin did say it had been responsive. Sounds like in this case your changes are causing the interactive stuff to get less CPU or higher scheduling latency?

author
#11 lore raw
Quoted 27 lines · Nick Piggin
Quoted 18 lines · Con Kolivas
Quoted 10 lines · Martin Schlemmer
Quoted 9 lines · Con Kolivas
Quoted 1 line · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 3 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 6 lines
Quoted 2 lines

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

Quoted 7 lines

A UP HT box you mean? That shouldn't be capable of running multiple make -j10s without some noticable effect. Apart from looking impressive, there is no point in having 30 cpu heavy things running with only 1 and a bit processor and the machine being smooth as silk; the cpu heavy things will just be unfairly starved in the interest of appearance (I can do that easily enough). Please give details if there is a specific issue you think I've broken or else I wont know about it.

Quoted 5 lines

Yeah make -j10s won't be without impact, but I think for a lot of interactive stuff they don't need a lot of CPU, just to get it in a timely manner. And Martin did say it had been responsive. Sounds like in this case your changes are causing the interactive stuff to get less CPU or higher scheduling latency?

Sigh..,

No, it sounds to me like things are expiring faster than on default. He didn't say make -j10, it was multiple -j10s. This is one where you simply cannot let the scheduler keep starving the make -j10s indefinitely for X; on a server or multiuser box X will simply cause unfair starvation. I'm trying to find a workaround for this without rewriting whole sections of the scheduler code, but I'm just not sure I should be trying to optimise for a desktop that runs loads >16 per cpu. (I'll keep trying though, but if there is no workaround that remains fair it wont happen)

Con

#12 lore raw
Quoted 41 lines · Con Kolivas
Quoted 32 lines · Nick Piggin
Quoted 22 lines · Con Kolivas
Quoted 13 lines · Martin Schlemmer
Quoted 11 lines · Con Kolivas
Quoted 2 lines · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 4 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 7 lines
Quoted 3 lines

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

Quoted 8 lines

A UP HT box you mean? That shouldn't be capable of running multiple make -j10s without some noticable effect. Apart from looking impressive, there is no point in having 30 cpu heavy things running with only 1 and a bit processor and the machine being smooth as silk; the cpu heavy things will just be unfairly starved in the interest of appearance (I can do that easily enough). Please give details if there is a specific issue you think I've broken or else I wont know about it.

Quoted 6 lines

Yeah make -j10s won't be without impact, but I think for a lot of interactive stuff they don't need a lot of CPU, just to get it in a timely manner. And Martin did say it had been responsive. Sounds like in this case your changes are causing the interactive stuff to get less CPU or higher scheduling latency?

Quoted 5 lines

Sigh..,

No, it sounds to me like things are expiring faster than on default. He didn't say make -j10, it was multiple -j10s. This is one where you simply cannot let the scheduler keep starving the make -j10s indefinitely for X; on a server or multiuser box X will simply cause unfair starvation. I'm trying to find a workaround for this without rewriting whole sections of the scheduler code, but I'm just not sure I should be trying to optimise for a desktop that runs loads >16 per cpu. (I'll keep trying though, but if there is no workaround that remains fair it wont happen)

Yep, I did see the multiple j10s ;)
I wasn't aware that there was longer term starvation of gccs by X. I thought the scheduler had always been quite good at evening up the total CPU time used and a change you made had recently introduced a latency or interactiveness problem.

But Martin didn't give a very detailed description of the problem, and no I definitely don't think you should be aiming at fixing his problem if it causes starvation or harms more common loads.

#13 lore raw
Quoted 14 lines · Nick Piggin
Quoted 13 lines

Sigh..,

No, it sounds to me like things are expiring faster than on default. He didn't say make -j10, it was multiple -j10s. This is one where you simply cannot let the scheduler keep starving the make -j10s indefinitely for X; on a server or multiuser box X will simply cause unfair starvation. I'm trying to find a workaround for this without rewriting whole sections of the scheduler code, but I'm just not sure I should be trying to optimise for a desktop that runs loads >16 per cpu. (I'll keep trying though, but if there is no workaround that remains fair it wont happen)

Quoted 6 lines

Yep, I did see the multiple j10s ;)
I wasn't aware that there was longer term starvation of gccs by X. I thought the scheduler had always been quite good at evening up the total CPU time used and a change you made had recently introduced a latency or interactiveness problem.

I did not say the 'make -j10s' starved. I am saying that mouse is laggish, as well as window/desktop switching.

Also, I am not saying Con should fix it - I am asking if we really want one scheduler that should try to do the right thing for SMP *and* UP.

Signature · 1 line
Martin Schlemmer
author
#14 lore raw
Quoted 2 lines · Martin Schlemmer

I did not say the 'make -j10s' starved. I am saying that mouse is laggish, as well as window/desktop switching.

No, but I did. With the vanilla scheduler, a normal user can turn X into a cpu hog and starve cc1s for 3 seconds, and the more X like applications there are on the machine, the longer they can all sit around starving something else. Feeling ok on one machine is not enough to say it's fine.

Quoted 3 lines

Also, I am not saying Con should fix it - I am asking if we really want one scheduler that should try to do the right thing for SMP *and* UP.

No, the same issues that apply to fairness, interactivity, throughput and latency are there regardless of SMP or UP. I've had good reports from SMP in the past; your HT report is the first that it was bad, and I've said that some fairness issues have been addressed which cause those.

The current scheduler (with or without some tweak or other) will be in 2.6 and should work as much of the time, in as many settings as possible, well. Since I'm trying to work on it I hope you can report exactly what your issue is and I'll try and address it. Do you really compile jobs make -j10 each time while using your machine? (rhetoric question of course since there is absolutely no advantage to doing that without lots of cpus). If not, how does it perform under your real world conditions?

Con

#15 lore raw
Quoted 9 lines · Con Kolivas
Quoted 3 lines

Also, I am not saying Con should fix it - I am asking if we really want one scheduler that should try to do the right thing for SMP *and* UP.

No, the same issues that apply to fairness, interactivity, throughput and latency are there regardless of SMP or UP. I've had good reports from SMP in the past; your HT report is the first that it was bad, and I've said that some fairness issues have been addressed which cause those.

Quoted 8 lines

The current scheduler (with or without some tweak or other) will be in 2.6 and should work as much of the time, in as many settings as possible, well. Since I'm trying to work on it I hope you can report exactly what your issue is and I'll try and address it. Do you really compile jobs make -j10 each time while using your machine? (rhetoric question of course since there is absolutely no advantage to doing that without lots of cpus). If not, how does it perform under your real world conditions?

Normal run of things there is many times 1-3 'make -j6s' running. Yes, sure, for on of them you prob should use -j4, but hey its in the head, right =). No, it is not kernels, it is a variety of stuff - goes with the distro i guess. Yes, I have tested runs of 'make -j12' and 'make -j24' (sorry, should have been more precise, but -j{6,12,24) was used as testing, with -j6 default) running dual makes.

With vanilla the mouse pointer, XMMS, switching desktops or windows is smooth. If I really hammer the system, it does 'slow down' the general navigation of X a little, but not so that that mouse pointer is jerky, etc. With the O??int patches things starts to 'stutter' under loads that is fairly under those that vanilla handles fine. The mouse gets jerky, switching desktops is notably lagging.

Note that I am not talking about starving XMMS/the make's. I am just talking general navigation of X. Yes, even with vanilla things do start a bit slower, but the mouse goes where it should, and its not as if the vga struggles to redraw the screen on desktop switch. I do not expect the system to behave for 'interactive' processes and xmms/whatever as if there is no load - the signs of load is just way more than with vanilla.

Regards,

Signature · 1 line
Martin Schlemmer
author
#16 lore raw

[Resent because the lkml spam filter thought this was spam originally]

Thanks for detailed description.

Quoted 3 lines · Martin Schlemmer

Normal run of things there is many times 1-3 'make -j6s' running. Yes, sure, for on of them you prob should use -j4, but hey its in the head, right =). No, it is not kernels, it is a variety

Actually in benchmarking I've found no increase in speed with more than one job per cpu but it's up to you of course.

Quoted 5 lines

of stuff - goes with the distro i guess. Yes, I have tested runs of 'make -j12' and 'make -j24' (sorry, should have been more precise, but -j{6,12,24) was used as testing, with -j6 default) running dual makes.

Quoted 7 lines

With vanilla the mouse pointer, XMMS, switching desktops or windows is smooth. If I really hammer the system, it does 'slow down' the general navigation of X a little, but not so that that mouse pointer is jerky, etc. With the O??int patches things starts to 'stutter' under loads that is fairly under those that vanilla handles fine. The mouse gets jerky, switching desktops is notably lagging.

Yes what you're describing is the expiring X issue. At this moment in time with my patches with very high loads, it is easy for interactive tasks to expire if used for sustained periods. This means that they will be smoother than vanilla for a burst, then have a nasty blip when falling on the expired array. This doesn't happen at lower loads, and is representative of the hard to optimise for case - the interactive task that behaves occasionally like a cpu hog (ie X). Lots of suggestions for ways around this have been offered, but none address the fact that these will cause starvation of other loads. Note that I say the vanilla scheduler does cause starvation already in the wrong circumstances if you offer that as a solution to go back to it. Suffice to say I'm still working on it as my highest priority.

Note that tasks that are never cpu hogs (eg xmms) will never stutter or falter under these circumstances; which is why I stopped mentioning audio ages ago:
audio works without problems under normal and extreme circumstances with my patches unless you renice a cpu hog to better priority than your audio app. Note that despite all this, since people are so excited by the idea of soft RR scheduling, I actually wrote a patch that will work with my tweaks a while ago. I've not optimised or improved it at all because that is lower priority to me than getting the general interactivity correct. For those interested, it's in my experimental directory in kernel.kolivas.org/2.5

Quoted 8 lines

Note that I am not talking about starving XMMS/the make's. I am just talking general navigation of X. Yes, even with vanilla things do start a bit slower, but the mouse goes where it should, and its not as if the vga struggles to redraw the screen on desktop switch. I do not expect the system to behave for 'interactive' processes and xmms/whatever as if there is no load - the signs of load is just way more than with vanilla.

As discussed above. Thanks for your report.

Con

#17 lore raw
Quoted 3 lines · Martin Schlemmer
Quoted 3 lines

Normal run of things there is many times 1-3 'make -j6s' running. Yes, sure, for on of them you prob should use -j4, but hey its in the head, right =). No, it is not kernels, it is a variety

Quoted 2 lines · Con Kolivas

Actually in benchmarking I've found no increase in speed with more than one job per cpu but it's up to you of course.

I found some strange SMP artifacts that seemed to show a dromedary-like throughput curve with respect to tasks, with one peak at 4 tasks/cpu and another peak at 16 tasks/cpu on a 16x box (for kernel compiles).

But I don't consider that evidence of anything to do something about.

-- wli

#18 lore raw
Quoted 3 lines · William Lee Irwin III

I found some strange SMP artifacts that seemed to show a dromedary-like throughput curve with respect to tasks, with one peak at 4 tasks/cpu and another peak at 16 tasks/cpu on a 16x box (for kernel compiles).

"Dromedary-like" is a bell-shaped curve. Perhaps you meant "bactrian-like".

Sorry. I couldn't resist posting this. :-)

-Paul

#19 lore raw
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

I found some strange SMP artifacts that seemed to show a dromedary-like throughput curve with respect to tasks, with one peak at 4 tasks/cpu and another peak at 16 tasks/cpu on a 16x box (for kernel compiles).

Quoted 2 lines · Paul Dickson

"Dromedary-like" is a bell-shaped curve. Perhaps you meant "bactrian-like".
Sorry. I couldn't resist posting this. :-)

Doh. Yes.

-- wli

#20 lore raw
In reply to Con Kolivas
Quoted 5 lines · Con Kolivas

I'm trying to work on it I hope you can report exactly what your issue is and I'll try and address it. Do you really compile jobs make -j10 each time while using your machine? (rhetoric question of course since there is absolutely no advantage to doing that without lots of cpus). If not, how does it perform under your real world conditions?

Um, slight ~objection.

What's the difference between one tester running a make -j10 and 10 students compiling their assignments in a multiuser box? I test throughput with make -j30 on a 128Mb 500Mhz PIII, because I know for a very very many times measured fact that the box can handle this (heavy but _not_ extreme) load. It's not the only load in the world, but it's such a dead simple load that the kernel dare not have difficulty with it IMHO.

-Mike

#21 lore raw
In reply to Martin Schlemmer
Quoted 7 lines · Martin Schlemmer

I did not say the 'make -j10s' starved. I am saying that mouse is laggish, as well as window/desktop switching.

Also, I am not saying Con should fix it - I am asking if we really want one scheduler that should try to do the right thing for SMP *and* UP.

Putting aside the load balancer, isn't the SMP case little more than multiple UP schedulers running in parallel?

I think that was supposed to be one of the great things about the O(1) scheduler: It unlocked the CPUs from each other so there would be far fewer spinlocks.

#22 lore raw
In reply to Con Kolivas
Quoted 36 lines · Con Kolivas
Quoted 27 lines · Nick Piggin
Quoted 18 lines · Con Kolivas
Quoted 10 lines · Martin Schlemmer
Quoted 9 lines · Con Kolivas
Quoted 1 line · Con Kolivas

More duck tape interactivity tweaks

s/duck/duct

Quoted 3 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 6 lines
Quoted 2 lines

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.

Quoted 7 lines

A UP HT box you mean? That shouldn't be capable of running multiple make -j10s without some noticable effect. Apart from looking impressive, there is no point in having 30 cpu heavy things running with only 1 and a bit processor and the machine being smooth as silk; the cpu heavy things will just be unfairly starved in the interest of appearance (I can do that easily enough). Please give details if there is a specific issue you think I've broken or else I wont know about it.

Quoted 5 lines

Yeah make -j10s won't be without impact, but I think for a lot of interactive stuff they don't need a lot of CPU, just to get it in a timely manner. And Martin did say it had been responsive. Sounds like in this case your changes are causing the interactive stuff to get less CPU or higher scheduling latency?

Sigh..,

Quoted 9 lines

No, it sounds to me like things are expiring faster than on default. He didn't say make -j10, it was multiple -j10s. This is one where you simply cannot let the scheduler keep starving the make -j10s indefinitely for X; on a server or multiuser box X will simply cause unfair starvation. I'm trying to find a workaround for this without rewriting whole sections of the scheduler code, but I'm just not sure I should be trying to optimise for a desktop that runs loads >16 per cpu. (I'll keep trying though, but if there is no workaround that remains fair it wont happen)

Con, you are doing great work for UP desktop systems. All I am saying is I do not think that there will be an golden middle way. If I disable SMP, it works much as expected for the short time I tested. I guess I am just voicing what a few people have said - maybe there should be a choice for what sheduler - UP or SMP.

Cheers,

Signature · 1 line
Martin Schlemmer
#23 lore raw
In reply to Martin Schlemmer
Quoted 2 lines · Con Kolivas
Quoted 2 lines
Quoted 2 lines

Wli pointed out an error in the nanosecond to jiffy conversion which may have been causing too easy to migrate tasks on smp (? performance change).

Quoted 2 lines · Con Kolivas
Quoted 2 lines

Looks like I broke SMP build with this. Will fix soon; don't bother trying this on SMP yet.

Quoted 5 lines · Martin Schlemmer

Not to be nasty or such, but all these patches have taken a very responsive HT box to one that have issues with multiple make -j10's running and random jerkyness.
I am not so sure I for one want changes to the scheduler for SMP (not UP interactivity ones anyhow).

Please try this again with the suggested correction to the load balancer.

-- wli