RSDL v0.30 cpu scheduler for mainline kernels
There are updated patches for 2.6.20, 2.6.20.2, 2.6.21-rc3 and 2.6.21-rc3-mm2 to bring RSDL up to version 0.30 for download here:
Full patches:
http://ck.kolivas.org/patches/staircase-deadline/2.6.20-sched-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.20.2-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-mm2-rsdl-0.30.patch
incrementals:
http://ck.kolivas.org/patches/staircase-deadline/2.6.20/2.6.20.2-rsdl-0.29-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.20.2/2.6.20.2-rsdl-0.29-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3/2.6.21-rc3-rsdl-0.29-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-mm2/2.6.21-rc3-mm2-rsdl-0.29-0.30.patch
Signature · 1 line
-ck
From: Con Kolivas <[email protected]>
Date: Mon, 12 Mar 2007 10:58:11 +1100
FWIW, this boots and seems to work well on sparc64. Tested on UP SunBlade1500 and 24cpu Niagara T1000.
Quoted 8 lines · David Miller
From: Con Kolivas <[email protected]>
Date: Mon, 12 Mar 2007 10:58:11 +1100Quoted 2 lines
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.
30.patchFWIW, this boots and seems to work well on sparc64. Tested on UP SunBlade1500 and 24cpu Niagara T1000.
Very nice. Thanks for the feedback and I'm sorry you have to work with such lousy hardware.
Signature · 1 line
-ck
Quoted 4 lines · Con Kolivas
Quoted 3 lines · David Miller
From: Con Kolivas <[email protected]>
Date: Mon, 12 Mar 2007 10:58:11 +1100
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.
30.patchFWIW, this boots and seems to work well on sparc64. Tested on UP SunBlade1500 and 24cpu Niagara T1000.
Very nice. Thanks for the feedback and I'm sorry you have to work with such lousy hardware.
BTW, I don't know if you say this as a joke, but those are not necessarily lousy hardware. Sun does lousy hardware when they put Sparcs in PCs (ultra5, ultra10, blade100). But their servers generally are nice with large memory busses and very scalable SMP architectures.
Regards,
Willy
Quoted 11 lines · Willy Tarreau
Quoted 10 lines · Con Kolivas
Quoted 8 lines · David Miller
From: Con Kolivas <[email protected]>
Date: Mon, 12 Mar 2007 10:58:11 +1100Quoted 2 lines
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.
30.patchFWIW, this boots and seems to work well on sparc64. Tested on UP SunBlade1500 and 24cpu Niagara T1000.
Quoted 7 lines
Quoted 2 lines
Very nice. Thanks for the feedback and I'm sorry you have to work with such lousy hardware.
BTW, I don't know if you say this as a joke, but those are not necessarily lousy hardware. Sun does lousy hardware when they put Sparcs in PCs (ultra5, ultra10, blade100). But their servers generally are nice with large memory busses and very scalable SMP architectures.
I guess Con was kidding. A 24-CPU system can be anything but lousy hardware.
From: Willy Tarreau <[email protected]>
Date: Tue, 13 Mar 2007 05:32:07 +0100
Quoted 8 lines · Con Kolivas
Quoted 7 lines · David Miller
Quoted 6 lines
From: Con Kolivas <[email protected]>
Date: Mon, 12 Mar 2007 10:58:11 +1100Quoted 2 lines
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.
30.patch
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
FWIW, this boots and seems to work well on sparc64. Tested on UP SunBlade1500 and 24cpu Niagara T1000.
Very nice. Thanks for the feedback and I'm sorry you have to work with such lousy hardware.
BTW, I don't know if you say this as a joke,
He was definitely being sarcastic, relax :-)
Quoted 11 lines · Willy Tarreau
Quoted 10 lines · Con Kolivas
Quoted 8 lines · David Miller
From: Con Kolivas <[email protected]>
Date: Mon, 12 Mar 2007 10:58:11 +1100Quoted 2 lines
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.
30.patchFWIW, this boots and seems to work well on sparc64. Tested on UP SunBlade1500 and 24cpu Niagara T1000.
Quoted 8 lines
Quoted 2 lines
Very nice. Thanks for the feedback and I'm sorry you have to work with such lousy hardware.
BTW, I don't know if you say this as a joke, but those are not necessarily lousy hardware. Sun does lousy hardware when they put Sparcs in PCs (ultra5, ultra10, blade100). But their servers generally are nice with large memory busses and very scalable SMP architectures.
Considering that Con's general mass target for his -ck patchset is single-cored x86/PC-compatable system such as a Celeron or a single-core Athlon with 32-256 MB RAM, (disclaimer: this is an assumption) I think he is being rather sarcastic. The only other way to interpret this is that those machines are totally different from what Con uses for development, and as such, maybe Con'd find them "lousy" because if something DID go wrong, it would be less obvious he'd figure it out right away. But I think it's a joke, guys. >.>
Signature · 2 lines
~Mike - Just the crazy copy cat.
Quoted 10 lines · On Mon, 12 Mar 2007 10:58:11 +1100 Con Kolivas <[email protected]> wrote:
There are updated patches for 2.6.20, 2.6.20.2, 2.6.21-rc3 and 2.6.21-rc3-mm2 to bring RSDL up to version 0.30 for download here:
Full patches:
http://ck.kolivas.org/patches/staircase-deadline/2.6.20-sched-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.20.2-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-mm2-rsdl-0.30.patch
Quoted 13 lines
incrementals:
http://ck.kolivas.org/patches/staircase-deadline/2.6.20/2.6.20.2-rsdl-0.29-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.20.2/2.6.20.2-rsdl-0.29-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3/2.6.21-rc3-rsdl-0.29-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-mm2/2.6.21-rc3-mm2-rsdl-0.29-0.30.patchSignature · 5 lines
-ck _______________________________________________ http://ck.kolivas.org/faqs/replying-to-mailing-list.txt ck mailing list - mailto: [email protected] http://vds.kolivas.org/mailman/listinfo/ck
Here's my experience with RSDL 0.30 on 2.6.21-rc3-git6 under my normal usage scenarios...
Plain desktop use (web browsing, music, etc): no noticeable change
Desktop use during kernel compile (no -j): The compile impacts desktop use more with RSDL, but this is easily solved by nice-ing the compile (default nice of 10 seems enough). The kind of impact I am talking about is e.g. occasional delays in scrolling in the browser etc.
Desktop use whilst talking on Wengophone (run at nice -5): Under RSDL some GUI use e.g. opening a new folder in nautilus causes pops (buffer underruns) which do not occur with mainline. I suppose the changes in RSDL might require a lower nice value for equivalent performance, but (as a user) I am limited to -5.
Low-latency audio with JACK and Ardour2: Rock-solid performance with either scheduler.. realtime works nicely.
Playing Quake3 (proprietary nvidia driver): Plays smoothly for both schedulers.
So, I suppose I'd like to know what to do about the Wengophone issue, because that really is a problem for me. I guess your re-worked -ive nice values might help?
Ash
Quoted 19 lines · Ash Milsted
Here's my experience with RSDL 0.30 on 2.6.21-rc3-git6 under my normal usage scenarios...
Plain desktop use (web browsing, music, etc): no noticeable change
Desktop use during kernel compile (no -j): The compile impacts desktop use more with RSDL, but this is easily solved by nice-ing the compile (default nice of 10 seems enough). The kind of impact I am talking about is e.g. occasional delays in scrolling in the browser etc.
Desktop use whilst talking on Wengophone (run at nice -5): Under RSDL some GUI use e.g. opening a new folder in nautilus causes pops (buffer underruns) which do not occur with mainline. I suppose the changes in RSDL might require a lower nice value for equivalent performance, but (as a user) I am limited to -5.
Low-latency audio with JACK and Ardour2: Rock-solid performance with either scheduler.. realtime works nicely.
Quoted 6 lines
Playing Quake3 (proprietary nvidia driver): Plays smoothly for both schedulers.
So, I suppose I'd like to know what to do about the Wengophone issue, because that really is a problem for me. I guess your re-worked -ive nice values might help?
Well the change to -nice values was to minimise the harm they do to everything else. They will still get the lowest latencies and the most cpu but no more than previously. The difference will be to more niced tasks. I'm not sure just how much cpu you require for wengophone because at -5 it would be getting a fair chunk of cpu with the RSDL 0.30. Anyway, by sheer coincidence I just emailed out that patch I was planning so feel free to try it.
Signature · 1 line
-ck
Quoted 8 lines · Ash Milsted
Desktop use whilst talking on Wengophone (run at nice -5): Under RSDL some GUI use e.g. opening a new folder in nautilus causes pops (buffer underruns) which do not occur with mainline. I suppose the changes in RSDL might require a lower nice value for equivalent performance, but (as a user) I am limited to -5.
Low-latency audio with JACK and Ardour2: Rock-solid performance with either scheduler.. realtime works nicely.
Sounds like Wengophone is broken. It should be using RT threads for time critical work, as JACK and Ardour2 are doing.
No scheduler can help if the apps are not correctly written...
Lee
Quoted 2 lines · Lee Revell
Sounds like Wengophone is broken. It should be using RT threads for time critical work, as JACK and Ardour2 are doing.
If the app has root privileges to set RT policy, then it could also set deeply negative nice values as well.
Doesn't reallly help the regular user with no privileges.
Chris
Quoted 9 lines · Chris Friesen
Quoted 2 lines · Lee Revell
Sounds like Wengophone is broken. It should be using RT threads for time critical work, as JACK and Ardour2 are doing.
If the app has root privileges to set RT policy, then it could also set deeply negative nice values as well.
Doesn't reallly help the regular user with no privileges.
Well this was supposed to be solved by RLIMIT_NICE and RLIMIT_RTPRIO which went into mainline about a year ago but distros have been very slow to pick up the new PAM, glibc and bash packages. We don't have a clear picture yet of what defaults the distros will ship.
Lee
Quoted 11 lines · On Tue, 13 Mar 2007 11:45:50 -0600 "Chris Friesen" <[email protected]> wrote:
Quoted 2 lines · Lee Revell
Sounds like Wengophone is broken. It should be using RT threads for time critical work, as JACK and Ardour2 are doing.
If the app has root privileges to set RT policy, then it could also set deeply negative nice values as well.
Doesn't reallly help the regular user with no privileges.
Chris
Well my distro (Arch) does have the relevant PAM support for giving users realtime capabilities, but naturally Wengophone is not designed to make use of them (does any internet phone do this?). The main problem is that it worked fine on mainline and does not with RSDL, and even boosting it to nice -19 (with root privs) makes no difference (although I only tested nice -19 with the slightly dodgy patch Con posted for Mike Galbraith, that patch reportedly only affects a few positive nice values). Two possibilities occur to me:
1. RSDL just isn't giving sched_other tasks as low latency as mainline (Wengophone is not using much CPU time from as far as top can tell).
2. The problem is actually caused by increased network latency somehow caused by RSDL (this is an internet phone).
I would like to add that the problem seems to be latency *spikes* that occur when there is sudden activity in another app (say, I open a folder in nautilus) that was previously largely idle.
I suppose if Con does what he says and reduces latency overall, this problem might disappear. For now, of course, I hope his neck gets better :)
Con,
Quoted 2 lines · Con Kolivas
There are updated patches for 2.6.20, 2.6.20.2, 2.6.21-rc3 and 2.6.21-rc3-mm2 to bring RSDL up to version 0.30 for download here:
I tried this on a Core 2 Quad cpu system(system has 4 cores on a single package). When I run SPECjbb2000 with number of threads varying from 1-8, I see ~4.5% perf regression with RSDL (compared to native 2.6.21-rc3) in the 8 threads case. This I think, is coming from increased number of context switches, when we have more than one thread(at same user priority) on the same logical cpu.
Just to see the % increase in number of context switches, I ran 8 infinite loops (simple while(1); 's) and with 2.6.21-rc3 I see ~70 context switches every second, whereas with RSDL I see ~530 context switches.
thanks,
suresh
Quoted 13 lines · Suresh B
Con,
Quoted 2 lines · Con Kolivas
There are updated patches for 2.6.20, 2.6.20.2, 2.6.21-rc3 and 2.6.21-rc3-mm2 to bring RSDL up to version 0.30 for download here:
I tried this on a Core 2 Quad cpu system(system has 4 cores on a single package). When I run SPECjbb2000 with number of threads varying from 1-8, I see ~4.5% perf regression with RSDL (compared to native 2.6.21-rc3) in the 8 threads case. This I think, is coming from increased number of context switches, when we have more than one thread(at same user priority) on the same logical cpu.
Quoted 3 lines
Just to see the % increase in number of context switches, I ran 8 infinite loops (simple while(1); 's) and with 2.6.21-rc3 I see ~70 context switches every second, whereas with RSDL I see ~530 context switches.
Thanks. If it's just that then scaling rr interval with cpus somewhat would help. If you could, the following patch just to test might confirm that.
--- kernel/sched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched.c +1 -1
@@ -104,7 +104,7 @@ unsigned long long __attribute__((weak))* This is the time all tasks within the same priority round robin. * Set to a minimum of 6ms. */ -#define RR_INTERVAL ((6 * HZ / 1001) + 1) +#define RR_INTERVAL ((12 * HZ / 1001) + 1)#define DEF_TIMESLICE (RR_INTERVAL * 20)#ifdef CONFIG_SMP
Signature · 1 line
-ck
kernel/sched.c unsigned long long __attribute__((weak)) · line 107
Quoted 9 lines · Con Kolivas
Quoted 3 lines · Suresh B
Just to see the % increase in number of context switches, I ran 8 infinite loops (simple while(1); 's) and with 2.6.21-rc3 I see ~70 context switches every second, whereas with RSDL I see ~530 context switches.
Thanks. If it's just that then scaling rr interval with cpus somewhat would help. If you could, the following patch just to test might confirm that.
kernel/sched.c +1
-1-#define RR_INTERVAL ((6 * HZ / 1001) + 1) +#define RR_INTERVAL ((12 * HZ / 1001) + 1)
Context switches now are ~370 per second. Still much above the regular ~70 we see in the mainline.
why do you say the rr_interval needs to be scaled with cpus? The basic point in RSDL is, if we have more than one same priority task on a single logic cpu, context switch happens every RR_INTERVAL (6 or 12 msec) whereas in mainline it happens every 100 msec.
We need to minimize these context switches.
thanks,
suresh
Quoted 14 lines · Suresh B
kernel/sched.c unsigned long long __attribute__((weak)) · line 107
Quoted 9 lines · Con Kolivas
Quoted 3 lines · Suresh B
Just to see the % increase in number of context switches, I ran 8 infinite loops (simple while(1); 's) and with 2.6.21-rc3 I see ~70 context switches every second, whereas with RSDL I see ~530 context switches.
Thanks. If it's just that then scaling rr interval with cpus somewhat would help. If you could, the following patch just to test might confirm that.
kernel/sched.c +1
-1-#define RR_INTERVAL ((6 * HZ / 1001) + 1) +#define RR_INTERVAL ((12 * HZ / 1001) + 1)Context switches now are ~370 per second. Still much above the regular ~70 we see in the mainline.
Quoted 4 lines
why do you say the
rr_intervalneeds to be scaled with cpus? The basic point in RSDL is, if we have more than one same priority task on a single logic cpu, context switch happens everyRR_INTERVAL(6 or 12 msec) whereas in mainline it happens every 100 msec.
With more CPUs, the context switch period can be multiplied by that number of CPUs while still allowing all tasks the same frequency of access to the CPU. With 4 processors, the context switch would be 24ms, by which point we're probably reaching the point of diminishing returns for minimizing overhead and maximizing throughput.
Quoted 1 line
We need to minimize these context switches.
That's a judgement call. If a synthetic benchmark degrades but other things improve, then this, as most everything in computer science, is yet another trade-off that needs to be evaluated. (You recognize there is a tradeoff here, right? Some benchmarks would improve even further if the switch time were 500ms. But that would make the system nearly unusable in general.)
Ray
Quoted 13 lines · Ray Lee
Quoted 12 lines · Suresh B
Quoted 10 lines · Con Kolivas
Quoted 4 lines · Suresh B
Just to see the % increase in number of context switches, I ran 8 infinite loops (simple while(1); 's) and with 2.6.21-rc3 I see ~70 context switches every second, whereas with RSDL I see ~530 context switches.
Thanks. If it's just that then scaling rr interval with cpus somewhat would help. If you could, the following patch just to test might
confirm that. -#defineRR_INTERVAL((6 * HZ / 1001) + 1)
+#defineRR_INTERVAL((12 * HZ / 1001) + 1)
Quoted 7 lines
Quoted 7 lines
Context switches now are ~370 per second. Still much above the regular ~70 we see in the mainline.
why do you say the
rr_intervalneeds to be scaled with cpus? The basic point in RSDL is, if we have more than one same priority task on a single logic cpu, context switch happens everyRR_INTERVAL(6 or 12 msec) whereas in mainline it happens every 100 msec.
No. That's what we do with mainline too. The analogue of the RR_INTERVAL in mainline is the timeslice granularity. When I wrote it (TG) I assumed that any increase in latency by lengthening this value would be offset by more cpus being available which is only partially true. I also did not assume at the time that many cpus would be used on a desktop which is now, wrong.
Quoted 16 lines
With more CPUs, the context switch period can be multiplied by that number of CPUs while still allowing all tasks the same frequency of access to the CPU. With 4 processors, the context switch would be 24ms, by which point we're probably reaching the point of diminishing returns for minimizing overhead and maximizing throughput.
Quoted 1 line
We need to minimize these context switches.
That's a judgement call. If a synthetic benchmark degrades but other things improve, then this, as most everything in computer science, is yet another trade-off that needs to be evaluated. (You recognize there is a tradeoff here, right? Some benchmarks would improve even further if the switch time were 500ms. But that would make the system nearly unusable in general.)
Ray
Signature · 1 line
-ck
Quoted 3 lines · Ray Lee
With more CPUs, the context switch period can be multiplied by that number of CPUs while still allowing all tasks the same frequency of access to the CPU.
Are you assuming the other cpus might be idle?
It depends on the load of the system also, right. If all the cpus are loaded, then increasing the period will decrease the frequency. If some of the cpus are idle, then it doesn't matter what context switch rate we use(as we don't get context switched out by other task).
Quoted 3 lines
With 4 processors, the context switch would be
24ms, by which point we're probably reaching the point of diminishing returns for minimizing overhead and maximizing throughput.
BTW, the overhead is not just the context switch cost, but also the cache evictions that the incoming process will bring.
Quoted 3 lines
Quoted 1 line
We need to minimize these context switches.
That's a judgement call. If a synthetic benchmark degrades but other
I was showing the degradation with SPECjbb2000 workload. Synthetic workload was for showing/reproducing the issue quickly.
Quoted 3 lines
things improve, then this, as most everything in computer science, is yet another trade-off that needs to be evaluated. (You recognize there is a tradeoff here, right?
I am with you. But lets say, if these tasks are not interactive, then what is the need for paying this penality?
thanks,
suresh
Quoted 10 lines · Con Kolivas
There are updated patches for 2.6.20, 2.6.20.2, 2.6.21-rc3 and 2.6.21-rc3-mm2 to bring RSDL up to version 0.30 for download here:
Full patches:
http://ck.kolivas.org/patches/staircase-deadline/2.6.20-sched-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.20.2-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-sched-rsdl-0.30.patch
http://ck.kolivas.org/patches/staircase-deadline/2.6.21-rc3-mm2-rsdl-0.30.patch
Works fine here. Better than mainline. I can listen music from a video dvd while browsing with opera and amule running. In mainline (CFQ scheduler) i cannot do this because the sound skips badly. With RSDL there are very few skips, they are shorter and related on a particular event - in this case clicking a link in Opera browser.
Now system info:
CPU: AMD Duron 700 MHz, MB chipset VIAKT133 (UDMA 66), memory 384172 kB (reported by /proc/meminfo).
Kernel is 2.6.21-rc3 #3 PREEMPT with RSDL 0.30 patches.
Thanks
Bye
Calin
P.S. As a side note: in mainline the cpu was distributed almost equally between different tasks which favoured the big cpu hogs (amule and opera). Now, in RSDL the mplayer gets a bigger stake
--
"frate, trezeste-te, aici nu-i razboiul stelelor"
Radu R. pe offtopic at lug.ro