Skip to contentSkip to conversation

2.6.16-ck1

DISCUSSION Con Kolivas 7 participants 11 messages Mar 20, 2006 – Mar 25, 2006 2 messages not in the archive

lore
Share & tools
lore
#0 lore raw

These are patches designed to improve system responsiveness and interactivity. It is configurable to any workload but the default ck patch is aimed at the desktop and cks is available with more emphasis on serverspace.

Apply to 2.6.16
http://ck.kolivas.org/patches/2.6/2.6.16/2.6.16-ck1/patch-2.6.16-ck1.bz2

or server version
http://ck.kolivas.org/patches/cks/patch-2.6.16-cks1.bz2

web:
http://kernel.kolivas.org

all patches:
http://ck.kolivas.org/patches/

Split patches available.

* Take note, userspace features have changed in this version *

There are 3 special unprivileged (normal user) scheduling policies available as of 2.6.16-ck1. These can be set using the schedtools utility
(http://freshmeat.net/projects/schedtool/?topic_id=136) which has support for all of them. Additionally my toolsched scripts work as transparent wrappers for them (http://ck.kolivas.org/apps/toolsched/). Note the policies are different to earlier -ck versions:

SCHED_BATCH:
This is for tasks you explicitly want the cpu scheduler to know are never
interactive and thus should never receive low latency treatment. Their cpu usage is dependant on their nice value. This policy is also supported by mainline now which is why there is a change of the naming/numbering scheme.

SCHED_ISO:
This is for tasks you explicitly want the cpu scheduler to know are low
latency real-time like tasks but you don't have root privileges for and don't wish them to ever starve the machine. They can use up to 80% of the available cpu time (on one cpu at any time). This percentage is configurable via /proc/sys/kernel/iso_cpu

SCHED_IDLEPRIO:
This is for tasks you never want to use cpu if *anything* else wants cpu
time. That is they only ever use spare cpu cycles that would have otherwise been idle time on the machine.

Changes since 2.6.15-ck7

Added:
+sched-implement-smpnice.patch +sched-smpnice-apply-review-suggestions.patch +sched-smpnice-fix-average-load-per-run-queue-calculations.patch
New improved smpnice implementation

+sched-store-weighted-load-on-up.patch +sched-add-discrete-weighted-cpu-load-function.patch +sched-add-above-background-load-function.patch
Updates to smpnice above to be used by swap prefetch

+sched-generic_optims2.patch
Small naming cleanups / microoptimisations

+sched-idleprio-1.2.patch
The old SCHED_BATCH policy from 2.6.15-ck7 has been renamed to SCHED_IDLEPRIO and small updates to the policy were committed.

+adaptive-readahead-11.patch
Wu Fengguang's adaptive readahead for improved read throughput without thrashing. This version is configurable at build time. As some users have reported latency issues with running this code, the config option is most welcome, however those latency issues should have been mostly addressed. See Documentation/sysctl/vm.txt for more info.

Removed:
-vmsplit-config_options.patch
This is part of mainline 2.6.16 now

-2.6.15-dynticks-060227.patch -dynticks-disable_smp_config.patch
Dynticks is dead! Long live dynticks! Something killed it off in 2.6.16-rc6 and I haven't figured out what it is. I don't know if/when I'll be able to debug this so for the moment it is dropped.

-patch-2.6.15.6.bz2
Mainline

Modified:
-sched-staircase13.2.patch
-sched-staircase13.2_13.3.patch -sched-staircase13.3_13.4.patch -sched-staircase13.4_13.5.patch
+sched-staircase14.2.patch
Rolled up and updated to latest version of staircase cpu scheduler. Mostly microoptimisations, and intrinsic support for new style of SCHED_BATCH policy.

-schedrange-1.diff
+schedrange-2.diff
Resync with change in scheduler policies available

-schedbatch-2.11.diff
Replaced with new SCHED_IDLEPRIO policy

-sched-iso3.3.patch
+sched-iso-4.1.patch
Updated SCHED_ISO unprivileged soft real time policy. This now is higher priority than any SCHED_NORMAL tasks but lower priority than any true real time tasks. The default cpu limit for iso tasks was changed to 80% (Note this is set to 0 by default on the cks patchset so server admins would explicitly set this only if desired).

-isobatch_ionice2.diff
+iso_idleprio_ionice.patch
-vm-mapped.diff
+vm-mapped-1.diff
-vm-background_scan-1.diff
-mm-highmem_fix_background_scan.patch
+mm-background_scan.patch
-mm-prio_dependant_scan.patch
-mm-batch_prio.patch
+mm-prio_dependant_scan-1.patch
+mm-idleprio_prio.patch
Rename/resync/rollups

-mm-swap_prefetch-28.patch
-mm-swap_prefetch-tweaks.patch
+mm-swap_prefetch-30.patch
+sp-resume1.patch
+mm-aggresive_swap_prefetch.patch +swsusp-post_resume_aggressive_swap_prefetch.patch
Updated to the latest swap prefetch code which will perform swap prefetching if low priority tasks are running. This also adds the optional "use once" feature of aggressive swap prefetching, and makes swsusp use that setting on resume from disk which improves dramatically the immediate interactivity of a machine just after resume. See Documentation/sysctl/vm.txt for more info on the tunable

-2615ck7-version.patch
+2.6.16-ck1-version.patch
Version update

Full patchlist:

sched-implement-smpnice.patch sched-smpnice-apply-review-suggestions.patch sched-smpnice-fix-average-load-per-run-queue-calculations.patch sched-store-weighted-load-on-up.patch sched-add-discrete-weighted-cpu-load-function.patch sched-add-above-background-load-function.patch sched-staircase14.2.patch
sched-generic_optims2.patch
schedrange-2.diff
sched-iso-4.1.patch
sched-idleprio-1.2.patch
defaultcfq.diff
iso_idleprio_ionice.patch
rt_ionice.diff
pdflush-tweaks.patch
hz-default_values.patch
hz-no_default_250.patch
mm-swap_prefetch-30.patch
vm-mapped-1.diff
vm-lots_watermark.diff
mm-background_scan.patch mm-kswapd_inherit_prio-1.patch mm-prio_dependant_scan-1.patch
mm-idleprio_prio.patch
sp-resume1.patch
mm-aggresive_swap_prefetch.patch swsusp-post_resume_aggressive_swap_prefetch.patch adaptive-readahead-11.patch
2.6.16-ck1-version.patch

Cheers,
Con

#2 lore raw
In reply to a message not in the archive
Quoted 8 lines · Avuton Olrich
Quoted 5 lines · Con Kolivas

SCHED_IDLEPRIO:
This is for tasks you never want to use cpu if *anything* else wants cpu
time. That is they only ever use spare cpu cycles that would have otherwise been idle time on the machine.

Quoted 7 lines

Sounds like something I've wanted for a while, so I could compile stuff on the wife's computer, but only when she's not actively using it.
--
avuton
--

Or rather: so I could compile stuff on *this* computer while both my wife and I are on their separate monitors/kbds, when neither one of us is actively using it. ;-)

Hugo

Quoted 8 lines

Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

------------------------------------------------------------------------

_______________________________________________
http://ck.kolivas.org/faqs/replying-to-mailing-list.txt
ck mailing list - mailto: [email protected]
http://vds.kolivas.org/mailman/listinfo/ck

Signature · 2 lines
I think the death is much overrated. How could anything so ordinary become a penalty?
John J. Cox
#3 lore raw
In reply to Con Kolivas

Good job Con, on your patches.. As far as the kernel in general, I'd like to post some warnings:

Adaptive readahead: I had probs with this before, and I still do.. On a desktop if you have odd problems (nothing responding for SECONDS, very slow disk I/O during heavy I/O, etc..) disable it.

The new Yukon2 "sky2" driver: This one really pissed me off. It had me thinking apache2 AND my linksys router we're on the brink. For some unknown reason at least for me, in FF it would only half-load some pages, including ones on localhost AND my router (10.1.1.1) ... I dunno what the hell is up with this one. I have to stay with the syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am back to 2.6.15 at the moment.

nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here is a current patch that works against the current release:
http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.8178-2.diff?bug=357992;msg=15;att=1

All in all, my experience sucked for the first time on this kernel.

Good luck with this new one..
-r

Signature · 2 lines
Rodney "meff" Gordon II               -*-              [email protected]
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource
#4 lore raw
Quoted 2 lines · Rodney Gordon II

Good job Con, on your patches.. As far as the kernel in general, I'd like to post some warnings:

Thanks.

Quoted 3 lines

Adaptive readahead: I had probs with this before, and I still do.. On a desktop if you have odd problems (nothing responding for SECONDS, very slow disk I/O during heavy I/O, etc..) disable it.

I was concerned about that myself which is why the only reason I included it was because it came in a configurable form where you could choose to enable it, and the default was off, and the config option even said suitable to _servers_, not desktops.

Quoted 11 lines

The new Yukon2 "sky2" driver: This one really pissed me off. It had me thinking apache2 AND my linksys router we're on the brink. For some unknown reason at least for me, in FF it would only half-load some pages, including ones on localhost AND my router (10.1.1.1) ... I dunno what the hell is up with this one. I have to stay with the syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am back to 2.6.15 at the moment.

nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here is a current patch that works against the current release:
http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.8178-2.diff?bug=357992;msg=15;att=1

Luckily none of these are my fault.

Quoted 1 line

All in all, my experience sucked for the first time on this kernel.

/me does the "not my fault" look.

Quoted 1 line

Good luck with this new one..

Heh. No new one in the works just yet, but I'm actually not planning on changing anything. Turn adaptive readahead off, and you're left with out of kernel tree, or worse, binary driver problems.

Cheers,
Con

#5 lore raw
Quoted 7 lines · Con Kolivas
Quoted 2 lines · Rodney Gordon II

Good job Con, on your patches.. As far as the kernel in general, I'd like to post some warnings:

Thanks.

Quoted 8 lines
Quoted 3 lines

Adaptive readahead: I had probs with this before, and I still do.. On a desktop if you have odd problems (nothing responding for SECONDS, very slow disk I/O during heavy I/O, etc..) disable it.

I was concerned about that myself which is why the only reason I included it was because it came in a configurable form where you could choose to enable it, and the default was off, and the config option even said suitable to _servers_, not desktops.

Yea, just thought I would try it again sighting the improvements in the ? docs in menuconfig, was just disappointed again.

Quoted 8 lines
Quoted 8 lines

The new Yukon2 "sky2" driver: This one really pissed me off. It had me thinking apache2 AND my linksys router we're on the brink. For some unknown reason at least for me, in FF it would only half-load some pages, including ones on localhost AND my router (10.1.1.1) ... I dunno what the hell is up with this one. I have to stay with the syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am back to 2.6.15 at the moment.

Quoted 5 lines
Quoted 3 lines

nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here is a current patch that works against the current release:
http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.8178-2.diff?bug=357992;msg=15;att=1

Luckily none of these are my fault.

Yep, just warnings in general.

Quoted 3 lines
Quoted 1 line

All in all, my experience sucked for the first time on this kernel.

/me does the "not my fault" look.

LOL, I know, just warnings in general like I have said.

Quoted 5 lines
Quoted 1 line

Good luck with this new one..

Heh. No new one in the works just yet, but I'm actually not planning on changing anything. Turn adaptive readahead off, and you're left with out of kernel tree, or worse, binary driver problems.

Turned it off, disk probs are gone.

Meant to say "Good luck with this new mainline one.."

Your patches are great.. Thus, why I started this post with "Good job Con, on your patches.." ;)

-r

Signature · 2 lines
Rodney "meff" Gordon II               -*-              [email protected]
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource
#6 lore raw
In reply to Con Kolivas

Con,

I believe the author of that readahead patch stated that using the following below would help in desktop usage ::

/sbin/blockdev --setra 256 /dev/XXX

Quoted 42 lines · Con Kolivas
Quoted 2 lines · Rodney Gordon II

Good job Con, on your patches.. As far as the kernel in general, I'd like to post some warnings:

Thanks.

Quoted 3 lines

Adaptive readahead: I had probs with this before, and I still do.. On a desktop if you have odd problems (nothing responding for SECONDS, very slow disk I/O during heavy I/O, etc..) disable it.

I was concerned about that myself which is why the only reason I included it was because it came in a configurable form where you could choose to enable it, and the default was off, and the config option even said suitable to _servers_, not desktops.

Quoted 8 lines

The new Yukon2 "sky2" driver: This one really pissed me off. It had me thinking apache2 AND my linksys router we're on the brink. For some unknown reason at least for me, in FF it would only half-load some pages, including ones on localhost AND my router (10.1.1.1) ... I dunno what the hell is up with this one. I have to stay with the syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am back to 2.6.15 at the moment.

Quoted 4 lines

nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here is a current patch that works against the current release:
http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.817
8-2.diff?bug=357992;msg=15;att=1

Luckily none of these are my fault.

Quoted 1 line

All in all, my experience sucked for the first time on this kernel.

/me does the "not my fault" look.

Quoted 1 line

Good luck with this new one..

Heh. No new one in the works just yet, but I'm actually not planning on changing anything. Turn adaptive readahead off, and you're left with out of kernel tree, or worse, binary driver problems.

Cheers,
Con

#7 lore raw
In reply to Rodney Gordon II

Hi Rodney,

Thanks for your testing :)

Quoted 3 lines · Rodney Gordon II

Adaptive readahead: I had probs with this before, and I still do.. On a desktop if you have odd problems (nothing responding for SECONDS, very slow disk I/O during heavy I/O, etc..) disable it.

Your problem on I/O latency with ara can be tracked down with the help of Ingo's latency tracing patch. It goes like this:

1) download
http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.16.tar.bz2
http://www.vanheusden.com/ara/adaptive-readahead-11-2.6.16-rc6.patch.gz
http://people.redhat.com/mingo/latency-tracing-patches/latency-tracing-v2.6.16.patch

2)
tar jxf linux-2.6.16.tar.bz2
gunzip adaptive-readahead-11-2.6.16-rc6.patch.gz cd linux-2.6.16
patch -p1 < ../adaptive-readahead-11-2.6.16-rc6.patch patch -p1 < ../latency-tracing-v2.6.16.patch

3) compile kernel with Adaptive readahead support
4) boot with the new kernel, and run
echo 0 > /proc/sys/kernel/preempt_max_latency
5) feel some latency problems
6) report the content of /proc/latency_trace

Thanks,
Wu

#8 lore raw
Quoted 24 lines · Wu Fengguang

Hi Rodney,

Thanks for your testing :)

Quoted 3 lines · Rodney Gordon II

Adaptive readahead: I had probs with this before, and I still do.. On a desktop if you have odd problems (nothing responding for SECONDS, very slow disk I/O during heavy I/O, etc..) disable it.

Your problem on I/O latency with ara can be tracked down with the help of Ingo's latency tracing patch. It goes like this:

1) download
http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.16.tar.bz2
http://www.vanheusden.com/ara/adaptive-readahead-11-2.6.16-rc6.patch.gz
http://people.redhat.com/mingo/latency-tracing-patches/latency-tracing-v2.6.16.patch

2)
tar jxf linux-2.6.16.tar.bz2
gunzip adaptive-readahead-11-2.6.16-rc6.patch.gz cd linux-2.6.16
patch -p1 < ../adaptive-readahead-11-2.6.16-rc6.patch patch -p1 < ../latency-tracing-v2.6.16.patch

Quoted 8 lines

3) compile kernel with Adaptive readahead support
4) boot with the new kernel, and run
echo 0 > /proc/sys/kernel/preempt_max_latency
5) feel some latency problems
6) report the content of /proc/latency_trace

Thanks,
Wu

Will try this as soon as I can get some net drivers working under 2.6.16 ...

-r

Signature · 2 lines
Rodney "meff" Gordon II               -*-              [email protected]
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource
#10 lore raw
In reply to a message not in the archive
Quoted 5 lines · Edouard Gomez
Quoted 2 lines · Rodney Gordon II

The new Yukon2 "sky2" driver: This one really pissed me off. It had me thinking apache2 AND my linksys router we're on the brink. For some

I never had the chance to make the sk98lin one working on my box, so I backported sky2 to 2.6.15 quite a few times from netdev-2.6 git tree.

It might be worth replacing the sky2 driver with the newest bleeding edge one over here:

http://developer.osdl.org/shemminger/prototypes/sky2-1.0msi.tar.gz

You might also want to check the developer's blog entry:
http://developer.osdl.org/shemminger/blog/?p=25

---
sky2 1.0?

Looks like a found the root cause of the sky2 hangs on pci-express. I copied some code from the SysKonnect driver that reconfigured the pci-express max request size. This probably caused receive dma engine to fail in face of contention. That will teach me to stop copy/pasting in bugs.
---

It might fix some of you issues ..

later,
Steve Hassard

#11 lore raw
Quoted 6 lines · Stephen Hassard

sky2 1.0?

Looks like a found the root cause of the sky2 hangs on pci-express. I copied some code from the SysKonnect driver that reconfigured the pci-express max request size. This probably caused receive dma engine to fail in face of contention. That will teach me to stop copy/pasting in bugs.
---

It might fix some of you issues ..

I'll backport again from git. The patch i sent was equivalent to 2.6.16 sky2 module, but i see new patches that came in netdev-2.6.

[PATCH] sky2: more ethtool stats
[PATCH] sky2 version 1.1
[PATCH] sky2: handle all error irqs
[PATCH] sky2: transmit recovery
[PATCH] sky2: whitespace fixes
[PATCH] sky2: add MSI support
[PATCH] sky2: coalescing parameters
[PATCH] sky2: rework of NAPI and IRQ management [PATCH] sky2: drop broken wake on lan support [PATCH] sky2: remove support for untested Yukon EC ...
sky2: truncate oversize packets
sky2: force early transmit interruptsdiff to current
sky2: not random enough

Signature · 1 line
Edouard Gomez
#12 lore raw
Quoted 16 lines · Edouard Gomez

I'll backport again from git. The patch i sent was equivalent to 2.6.16 sky2 module, but i see new patches that came in netdev-2.6.

[PATCH] sky2: more ethtool stats
[PATCH] sky2 version 1.1
[PATCH] sky2: handle all error irqs
[PATCH] sky2: transmit recovery
[PATCH] sky2: whitespace fixes
[PATCH] sky2: add MSI support
[PATCH] sky2: coalescing parameters
[PATCH] sky2: rework of NAPI and IRQ management [PATCH] sky2: drop broken wake on lan support [PATCH] sky2: remove support for untested Yukon EC ...
sky2: truncate oversize packets
sky2: force early transmit interruptsdiff to current
sky2: not random enough

For those of us that just want something working now:
Patch up using the latest sk98lin patch generator.. then edit drivers/net/sk98lin/sky2.c and add this line:

#include "h/skdrv1st.h"
#include "h/skdrv2nd.h"
#include <linux/tcp.h>
#include <linux/ip.h> <-- ADD THIS

It will build fine and seems to work just fine until syskonnect releases a new revision

GL

-r

Signature · 2 lines
Rodney "meff" Gordon II               -*-              [email protected]
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource