updated preempt-kernel

Testers Wanted:

patches to enable a fully preemptible kernel are available at:

	http://tech9.net/rml/linux

for kernels 2.4.10, 2.4.12, 2.4.12-ac3, and 2.4.13-pre5.

What is this:

A preemptible kernel. It lowers your latency.

What is New:

* sync with new kernel releases

* if HIGHMEM_DEBUG was on the preempt counter would be incremented at times but never decremented. this resulted in preemption becoming permanently disabled.

* if HIGHMEM_DEBUG was not on, HIGHMEM would crash the system horribly due to a case where preemption was enabled without a corresponding disable.

* reapply dropped hunk to pgalloc to prevent reentrancy onto per-CPU data

The next few patches are going to work on identifying any remaining per-CPU variables that need explicit locking under preemption.

Robert Love

I just switched from 2.4.10-ac12-preempt to the following:

2.4.12 patched with 2.4.12-ac3, 2.4.12-ac3-vmpatch, 2.4.12-ac3-freeswap, preempt-kernel-rml-2.4.12-ac3-2 (where is the stats-patch for that last one?)

Compiling went without problems!

On a 850Mhz CPU with 576Mb Memory;
I did the following, all at the same time, started in this order:

X with KDE 2.1, gkrellm, licq, freeamp, 5* konqueror, kmail, bonnie++, du /home|sort -nr|head -100' (11+ Gig of files), slocate *|wc -l', `find /|wc -l', make Python and test it (117 tests went OK1 test failed: test_openpty).

During this period, everything kept very responsive, there were 2 times a little delay would occur when moving a window like crazy, or scrolling a konqueror page would delay a bit (it's only a 32Mb card and no accel.), but switching desktops went fine and more important, freeamp did not skip a single time !!!!

You should see the laugh on my face while I'm typing this ;^)

Any other testing you can think of ??

ElkOS: 2:20pm up 2:17, 3 users, load average: 2.66, 3.18, 3.60
bofhX: We've run out of licenses


This above seems to have fixed some spontaneous reboots and oopses I experiencied with 2.4.11 and 2.4.12-1 preempt-kernel patches.

Lorenzo

This is very much what I wanted to hear. Thank you. I was hoping to clear those issues up. Let me know of any other problems. Enjoy.

Robert

In article <1003597363.866.8.camel@phantasy> [email protected] wrote:

| On Sat, 2001-10-20 at 08:59, Lorenzo Allegrucci wrote:
| > At 03.27 20/10/01 -0400, Robert Love wrote:
| >
| > >* reapply dropped hunk to pgalloc to prevent reentrancy onto per-CPU
| > >data
| > 
| > This above seems to have fixed some spontaneous reboots and oopses
| > I experiencied with 2.4.11 and 2.4.12-1 preempt-kernel patches.
|  
| This is very much what I wanted to hear.  Thank you.  I was hoping to
| clear those issues up.  Let me know of any other problems.  Enjoy.

Is this safe to try on SMP again? The one-previous 2.4.12-ac3 patch
seems stable on a P5-100+48MB RAM, which I use as a test for things helping dog-slow systems, did not run well on a BP6 (crashed on first login). I didn't report it because I try to have some useful info to report and had no time.

Also, has this been tested with experimental kernel pcmcia or the real
pcmcia package? The BP6 is my only non-laptop pcmcia.

bill davidsen <[email protected]>
  His first management concern is not solving the problem, but covering
his ass. If he lived in the middle ages he'd wear his codpiece backward.

Hm, your report of failure on the BP6 is the first I have heard of that. I did (re)fix a race in a later release that may solve your problem.

I would be very interested to see if you can replicate the problem on 2.4.12-ac5 with the corresponding preempt-kernel patch from
http://tech9.net/rml/linux ... I hope not.

You will need to recompile pcmcia, but then it should work. Dave Hinds merged specific support for detecting preempt on compile into 3.1.30 ... but as long as the pcmcia-cs build can find your .config, with CONFIG_PREEMPT set, you should be OK.

Robert Love

I'm running:
Now : 5 day(s), 16:07:02 running Linux

                2.4.12-ac3+netdev_ramdom+preempt+vm_hogstop2 

on 2x366 celeron. No problems.

There was one compile bug, but that has been resolved.

Mike

[1.] NULL pointer deference in con_flush_chars with preempt patch [2.]
Ok, running with preempt-kernel-rml-2.4.12-ac3-2.patch I had a crash yesterday. It occured when the machine was under
light load, I had just exited X, and I was logging off a console - I may have hit ctrl-d twice. I did a little investigating myself, and the closest I could find in the archives was the problem mentioned in
http://www.geocrawler.com/archives/3/35/1998/11/0/217652/ , except my crash is occuring at console close instead of open. It may not be preempt related, just preempt induced :-)

Looks like a race condition on console close, driver_data gets set to NULL but a interrupt-induced con_flush_chars then tries to access it.

[3.] preempt, console
[4.] Linux version 2.4.12-ac3-preempt (root@micro) (gcc version 2.95.4 20011006 (Debian prerelease)) #2 Sat Oct 20 19:07:28 BST 2001 [5.]
ksymoops 2.4.1 on i686 2.4.12-ac3-preempt. Options used

     -V (default)
     -k /var/log/ksymoops/20011020212838.ksyms (specified)
     -l /var/log/ksymoops/20011020212838.modules (specified)
     -o /lib/modules/2.4.12-ac3-preempt/ (default)
     -m /boot/System.map-2.4.12-ac3-preempt (default)

Unable to handle kernel NULL pointer dereference at virtual address 00000000
c01878d7
*pde = 00000000
Oops: 0000
CPU:    0
EIP:    0010:[con_flush_chars+31/52]    Tainted: P 
EFLAGS: 00010246
eax: 00000000   ebx: 00000000   ecx: c0200ac0   edx: c51a4000
esi: c51a4000   edi: c51a4169   ebp: 00000000   esp: c5ff7eb0
ds: 0018   es: 0018   ss: 0018
Process keventd (pid: 2, stackpage=c5ff7000)
Stack: c51a4980 c017c0ff c51a4000 c51a4568 c51a4168 c5ff7f9c 0008e000 00000050 
       00000010 00000000 c4d55220 00000000 00000050 c51a4569 c51a4169 c0271248 
       00000001 0000001d c019d4ab c0271248 00000001 0000000d 0000001d 00000008 
Call Trace: [n_tty_receive_buf+4059/4236] [fbcon_cursor+167/456] [complete_change_console+146/152] [flush_to_ldisc+251/260] [__run_task_queue+109/124] 
Code: 8b 03 50 e8 31 c9 ff ff e8 38 be f8 ff 83 c4 04 5b c3 8d 76 

Using defaults from ksymoops -t elf32-i386 -a i386

Code; 00000000 Before first symbol
00000000 <_EIP>:
Code; 00000000 Before first symbol
0: 8b 03 mov (%ebx),%eax
Code; 00000002 Before first symbol
2: 50 push %eax
Code; 00000003 Before first symbol
3: e8 31 c9 ff ff call ffffc939 <_EIP+0xffffc939> ffffc939 <END_OF_CODE+38d6916f/????>
Code; 00000008 Before first symbol
8: e8 38 be f8 ff call fff8be45 <_EIP+0xfff8be45> fff8be45 <END_OF_CODE+38cf867b/????>
Code; 0000000d Before first symbol
d: 83 c4 04 add $0x4,%esp
Code; 00000010 Before first symbol
10: 5b pop %ebx
Code; 00000011 Before first symbol
11: c3 ret
Code; 00000012 Before first symbol
12: 8d 76 00 lea 0x0(%esi),%esi

14 warnings issued. Results may not be reliable.

[6.] Not trivially reproducible
[7.]

Gnu C                  2.95.4
Gnu make               3.79.1
util-linux             2.11h
mount                  2.11h
modutils               2.4.10
e2fsprogs              1.25
reiserfsprogs          3.x.0j
Linux C Library        2.2.4
Dynamic linker (ldd)   2.2.4
Procps                 2.0.7
Net-tools              1.60
Console-tools          0.2.3
Sh-utils               2.0.11
Modules Loaded         ide-cd cdrom rtc tulip sb sb_lib uart401 sound soundcore serial hid usb-uhci usbcore mousedev input atyfb fbcon-cfb24 fbcon-cfb8 fbcon-cfb16 fbcon-cfb32 apm unix

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 3
model name : Pentium II (Klamath)
stepping : 4
cpu MHz : 267.284
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov mmx
bogomips : 532.48

0000-001f : dma1
0020-003f : pic1
0040-005f : timer
0060-006f : keyboard
0070-007f : rtc
0080-008f : dma page reg
00a0-00bf : pic2
00c0-00df : dma2
00f0-00ff : fpu
0170-0177 : ide1
01f0-01f7 : ide0
0220-022f : soundblaster
02f8-02ff : serial(set)
0330-0333 : MPU-401 UART
0376-0376 : ide1
03c0-03df : vga+
03f6-03f6 : ide0
03f8-03ff : serial(set)
0cf8-0cff : PCI conf1
4f00-4f3f : Intel Corporation 82371AB PIIX4 ACPI
5f00-5f1f : Intel Corporation 82371AB PIIX4 ACPI
d000-dfff : PCI Bus #01
d000-d0ff : ATI Technologies Inc 3D Rage Pro AGP 1X/2X
e000-e01f : Intel Corporation 82371AB PIIX4 USB
e000-e01f : usb-uhci
e400-e47f : Digital Equipment Corporation DECchip 21041 [Tulip Pass 3]
e400-e47f : tulip
f000-f00f : Intel Corporation 82371AB PIIX4 IDE
f000-f007 : ide0
f008-f00f : ide1

00000000-0009ffff : System RAM
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM
000f0000-000fffff : System ROM
00100000-05ffffff : System RAM
00100000-001d9ef3 : Kernel code
001d9ef4-0020cb47 : Kernel data
e0000000-e3ffffff : Intel Corporation 440LX/EX - 82443LX/EX Host bridge e4000000-e5ffffff : PCI Bus #01
e5000000-e5000fff : ATI Technologies Inc 3D Rage Pro AGP 1X/2X
e6000000-e6ffffff : PCI Bus #01
e6000000-e6ffffff : ATI Technologies Inc 3D Rage Pro AGP 1X/2X

    e6000000-e6ffffff : atyfb

e8000000-e800007f : Digital Equipment Corporation DECchip 21041 [Tulip Pass 3]
e8000000-e800007f : tulip
ffff0000-ffffffff : reserved

00:00.0 Host bridge: Intel Corporation 440LX/EX - 82443LX/EX Host bridge (rev 03)
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
Latency: 64
Region 0: Memory at e0000000 (32-bit, prefetchable) [size=64M]
Capabilities: <available only to root>

00:01.0 PCI bridge: Intel Corporation 440LX/EX - 82443LX/EX AGP bridge (rev 03) (prog-if 00 [Normal decode])

	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B-
	Status: Cap- 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 64
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=64
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: e4000000-e5ffffff
	Prefetchable memory behind bridge: e6000000-e6ffffff
	BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-

00:02.0 ISA bridge: Intel Corporation 82371AB PIIX4 ISA (rev 01)
Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 0

00:02.1 IDE interface: Intel Corporation 82371AB PIIX4 IDE (rev 01) (prog-if 80 [Master])

	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 64
	Region 4: I/O ports at f000 [size=16]

00:02.2 USB Controller: Intel Corporation 82371AB PIIX4 USB (rev 01) (prog-if 00 [UHCI])
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 64
Interrupt: pin D routed to IRQ 11
Region 4: I/O ports at e000 [size=32]

00:02.3 Bridge: Intel Corporation 82371AB PIIX4 ACPI (rev 01)
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Interrupt: pin ? routed to IRQ 9

00:10.0 Ethernet controller: Digital Equipment Corporation DECchip 21041 [Tulip Pass 3] (rev 21)

	Subsystem: D-Link System Inc DE-530+
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 64
	Interrupt: pin A routed to IRQ 11
	Region 0: I/O ports at e400 [size=128]
	Region 1: Memory at e8000000 (32-bit, non-prefetchable) [size=128]
	Expansion ROM at e7000000 [disabled] [size=256K]

01:00.0 VGA compatible controller: ATI Technologies Inc 3D Rage Pro AGP 1X/2X (rev 5c) (prog-if 00 [VGA])
Subsystem: ATI Technologies Inc: Unknown device 0040
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping+ SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 64 (2000ns min), cache line size 08
Region 0: Memory at e6000000 (32-bit, prefetchable) [size=16M]
Region 1: I/O ports at d000 [size=256]
Region 2: Memory at e5000000 (32-bit, non-prefetchable) [size=4K]
Expansion ROM at <unassigned> [disabled] [size=128K]
Capabilities: <available only to root>

No SCSI.

Colin Phipps <[email protected]>   http://www.cph.demon.co.uk/

This one has been reported before.

n_tty_receive_buf() puts a character into the tty queue and then calls con_flush_chars(), which touches tty->driver_data.

Problem is, there's a window between these two operations where the device can be closed (especially if the char is "^D"!), and con_close() will zero out tty->driver_data. Hence null pointer deref.

I don't really believe this explanation, because the timing's wrong - the reader isn't woken until after the flush is called. Hence it'll be very difficult to actually trigger this race. It's probably something else. But a bit more sticking plaster should make it appear to be fixed:

@@ -2387,9 +2387,15 @@ static void con_flush_chars(struct tty_s
 		return;
 
 	pm_access(pm_con);
-	acquire_console_sem();
-	set_cursor(vt->vc_num);
-	release_console_sem();
+	if (vt) {
+		/*
+		 * If we raced with con_close(), `vt' may be null.
+		 * Hence this bandaid.   - akpm
+		 */
+		acquire_console_sem();
+		set_cursor(vt->vc_num);
+		release_console_sem();
+	}
 }
 
 /*

Colin, can you try Andrew's patch and report back? This problem has been reported before -- its a tty bug that preempt (and SMP I wager) just aggravate. I have a patch that I know fixes it, but Andrew's is _much_ simpler. I will send you that if this fails. Please let me know.

I'm using 2.4.12-xfs with the preempt-kernel-rml-1 patch. Just this morning I noticed a minute or so of the system being in "freeze". There was no significant disk activity, my open windows were working (ICQ, IPTraf under wterm, Opera), but things like opening a new wterm would work but no prompt (bash) would come out, or "ps ax" on a system with stay there.

In the IPTraf window I saw a lot of domain lookups going back and forth. Since IPTraf does reverse name lookups I quit it to hopefully bring down the load. I'm running bind 9.1.3. After the freeze everything was 100% normal. I checked the syslog and found close to a hundred lines one after the other about named complaining of a lame server.

I am under the impression that it was bind hogging system resources, although I do not know how to look at historical data of memory usage and CPU usage for such a small time.

I am curious, what does the preempt patch do for such situations? I honestly don't know how the system would have felt otherwise (if I didn't have support for preemption). And it's not so easy to reproduce since I don't cause this myself.

Thanks for your input, and I'll give your second patch a shot as soon as I can. :)

--> Jijo

--
Federico Sevilla III :: [email protected] Network Administrator :: The Leather Collection, Inc. GnuPG Key: <http://jijo.leathercollection.ph/jijo.gpg>

So I tested it :)

I tried out your patch yesterday with 2.4.13-pre6 (it applies cleanly to -pre6 although made for -pre5). I've been running with it for about a day now and I have not seen any ill effects yet.

The system does seem slightly more responsive when stressed, but I don't see (or feel) huge improvements like some other people - maybe I just run a set of apps that don't benefit much from the preempt patches, or my workload is not significant enough to notice.. I usually run things like KDE2, XMMS, Nedit, x-cd-roast, Opera, Sylpheed and a lot of console windows. This is on a 1.4Ghz Athlon Thunderbird with 512MB RAM.

Are there any tests you'd like me to try out on this box?

- Jesper Juhl - [email protected]

Hi
I'm using the preemptible kernel patch since 2.4.10 (no 2.4.11). And it makes a big difference on 486 with 12Megs of ram.
I can't send you benchmarks (i don't have time for this but if you really want one ... it can be arranged :-)).
But:

When I run a configure script I can actually see it running (without this patch it is very slow).

The coolest thing was that I could run Gnome and KDE (with loadavg of 4 and waiting 2 to 4 minutes for an application to start (because of ram I think)) something not possible without the preemtible kernel. I don't swear after them anyway.( I prefer fvwm)

Compilation is much faster (i'll make a benchmark compiling linux kernel -- i promise :-)) ).

The system is stable with high system loads. Now is kernel 2.4.12 and no problems at all.

Any chance to be in the main stable kernel ?

Bye

__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com

I am very glad to here this -- thank you :)
I'm glad it works so good.

Hopefully for 2.5.

Robert Love

This also looks a bit wrong:

Maybe should be:

acquire_console_sem();
if (vt) set_cursor(vt->vc_num);
release_console_sem();

??

Taral <[email protected]>
This message is digitally signed. Please PGP encrypt mail to me.
"Any technology, no matter how primitive, is magic to those who don't
understand it." -- Florence Ambrose

--[snip]--

Some more tests with 2.4.12-ac3-vmpatch-freeswap-preempt:

I started the following command:
$> tree -d /

The first time, this went really quick, the second time though, the system would freeze every now and then, output to the konsole (kde) stopped for a moment; I could hear /dev/hda spinning like crazy (and making some grinding sounds; very desperate; old disk?).

I let this finish, everything OK.

Now I started this command from my $HOME (3,7G; 81947 files):
$> find . | xargs slocate | sort | uniq -c | head -1

Useless I know, but it can make your system scream ;)

This is some info on the system:
--[snip]--
[elko@ElkOS elko]$ dmesg | egrep "clock |Mem"
Memory: 577440k/589824k available \
(1177k kernel code, 12000k reserved, \
347k data, 236k init, 0k highmem)
..... CPU clock speed is 852.0020 MHz.
..... host bus clock speed is 100.2353 MHz.

[elko@ElkOS elko]$ df

Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1             387M   79M  288M  22% /
/dev/hda5             387M   35M  332M  10% /tmp
/dev/hda6             387M  122M  245M  33% /var
/dev/hda8             2.7G  1.4G  1.1G  55% /usr
/dev/hdc1              19G   10G  7.7G  57% /home
/dev/hdd6             3.2G  1.2G  1.9G  39% /mnt/lfs

[elko@ElkOS elko]$ cat /proc/swaps

Filename                        Type            Size    Used    Priority
/dev/hda7                       partition       104380  104380  -1
/dev/hdd5                       partition       1465592 473372  -2

--[snip]--

First, it's a bit jumpy:
--[snip]--
[elko@ElkOS elko]$ free -m
total used free shared buffers cached

Mem:           564        561          2          0          0         14
-/+ buffers/cache:        545         18
Swap:         1533        492       1040

[elko@ElkOS elko]$ free -m
total used free shared buffers cached

Mem:           564        140        423          0          0         14
-/+ buffers/cache:        126        438

Swap: 1533 85 1448

[elko@ElkOS elko]$ free -m
total used free shared buffers cached

Mem:           564        561          2          0          0         11
-/+ buffers/cache:        549         14
Swap:         1533        500       1032

--[snip]--

There's some nice *idle* time that top reports:
--[snip]--
[elko@ElkOS elko]$ top
9:48pm up 1 day, 5:27, 3 users, load average: 4.20, 4.24, 3.46
103 processes: 96 sleeping, 7 running, 0 zombie, 0 stopped CPU states: 36.8% user, 39.8% system, 23.4% nice, 847133.3% idle
Mem: 577676K av, 574612K used, 3064K free, 0K shrd, 576K buff
Swap: 1569972K av, 1158692K used, 411280K free 12476K
cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND

23732 elko 20   0 1354M 308M   532 R    307M 40.3 54.6   1:43 slocate
  991 elko 20  15 15080  13M   672 R N     0 25.9  2.4 771:52 setiathome 
992 elko   18  16 15208  13M   772 R N     0 13.1  2.4 771:39 setiathome
    5 root 20   0     0    0     0 RW      0  9.8  0.0   0:33 kswapd
  899 elko 11   0  2752 2000  1412 S       0  6.8  0.3 117:54 gkrellm
23756 elko 15   0  1468 1468  1224 R       0  1.5  0.2   0:00 top
  800 root  9   0 48584 4132  3100 R       0  1.1  0.7  16:43 X

--[snip]--

And again:
--[snip]--
10:09pm up 1 day, 5:48, 3 users, load average: 4.04, 3.33, 3.11 103 processes: 99 sleeping, 4 running, 0 zombie, 0 stopped CPU states: 47.1% user, 17.6% system, 35.4% nice, 850488.3% idle --[snip]--

I stopped the test here:
--[snip]--
[elko@ElkOS elko]$ free -m
total used free shared buffers cached
Mem: 564 561 2 0 0 13
-/+ buffers/cache: 547 16
Swap: 1533 1177 355
--[snip]--

1 eyeblink later:
--[snip]--
[elko@ElkOS elko]$ free -m
total used free shared buffers cached

Mem:           564         74        489          0          0         12
-/+ buffers/cache:         61        503
Swap:         1533         83       1449

--[snip]--

So it seems that a lot of swap is perfectly released the instant it isn't needed anymore (not everything, since I started with (almost) no swap used in the first place).

What I can report further, is that keyboard/mouse response (^C in konsole) would sometimes not react, but 3 to 5 seconds later, the action would be taken (^C, move mouse):

I'm running the same test now, and I'm seeing the same results, system freezes (while I'm typing this), and a few seonds later, response is back, and swap drops down to ~zero, keystrokes are cached correctly though, just had another freeze, kept typing.. .. .. and here are my characters :^) and swap is freed again:
--[snip]--
[elko@ElkOS elko]$ free -m
total used free shared buffers cached

Mem:           564        462        101          0          0         12
-/+ buffers/cache:        450        113

Swap: 1533 85 1447
--[snip]--

This is nice to see happening when things slow down:
--[snip]--
[elko@ElkOS elko]$ free -m
total used free shared buffers cached
Mem: 564 561 2 0 0 12
-/+ buffers/cache: 547 16

[elko@ElkOS elko]$ free -m
total used free shared buffers cached

Mem:           564         82        481          0          0         13
-/+ buffers/cache:         68        495
Swap:         1533         81       1451

--[snip]--

My current conclusion: this combination of kernel and patches is the most responsive I've ever used, normally, when I run these command's, my systems would freeze to the point I had to give them the VNP.

I'll kick it some more though ;/

ElkOS: 10:39pm up 1 day, 6:18, 3 users, load average: 2.27, 2.67, 3.14
bofhX: Mailer-daemon is busy burning your message in hell.


Excellent. Give a lot of credit to Rik too because his VM work has relieved a lot of thrashing/storming in high-load situations such as yours.

Robert Love

Hereby, I also give *a lot of* credit to Rik, by CC'ing him ;^) (Hi Rik! No worries, I'm not from Vries <g>)

This system has not been running as smooth as now, since I switched from 2.4.0 to 2.4.10-ac12 and a few days later to 2.4.12-ac3, so I did some testing, that I could never complete with the previous kernels I used (as was asked on the list).

I've attached the results of my latest tests..

ElkOS: 4:34pm up 1:05, 3 users, load average: 2.61, 2.58, 2.83
bofhX: Network failure -  call NBC


Tests done just after a reboot:

test1: In console, play a mp3-file with mpg123
test2: In console, play a mp3-file with mpg123 and run the find command
test3: Run test1 in X, play with freeamp
test4: Run test2 in X, play with freeamp
test5: Run test2 in X, play with mpg123
no reboot after test 4, since memory was released, a bit of swap stayed behind... read the output below...

Extra apps in X: gkrellm, 2x setiathome, 4 konsoles, freeamp playing,
kmail, 2x konqueror, licq

- Fire up mpg123:
[elko@ElkOS mp3]$ while :; do mpg123 "Aphex Twin - Come to Daddy.mp3"; done

- `find' command to stress the system (~3,7G, ~81946 files):
[elko@ElkOS ~]# find .|xargs slocate|sort|uniq -c|head -1

- Make sure I'm using /dev/dsp:
[elko@ElkOS ~]# /usr/sbin/lsof|grep dsp
esd 605 elko 5w CHR 14,3 22742 /dev/dsp

- Watch the system:
[elko@ElkOS ~]# vmstat -n 60 15 |tee -a ~elko/tests
...

- Audio driver used (in kernel): ES1371

- System info:
[elko@ElkOS ~]# /sbin/swapon -s

Filename                        Type            Size    Used    Priority
/dev/hda7                       partition       104380  33648   -1
/dev/hdd5                       partition       1465592 35144   -2

[elko@ElkOS ~]# dmesg | egrep "clock |Mem"
Memory: 577440k/589824k available \
(1177k kernel code, 12000k reserved, \
347k data, 236k init, 0k highmem)
..... CPU clock speed is 852.0020 MHz.
..... host bus clock speed is 100.2353 MHz.

* Now for the tests:
-------------------------------------

test1: 2.4.12-ac3-vmpatch-freeswap-preempt

- No skips in mp3 playing, nothing special to report

- vmstat output:
procs memory swap io system cpu

 r  b  w  swpd   free   buff  cache  si  so   bi  bo   in    cs  us  sy  id
 1  1  0     0 455808  83428  13192   0   0  341   4  220   262   1   3  96
 1  0  0     0 454752  83428  14152   0   0   16   2  792  1390   2   1  97
 1  0  0     0 453768  83428  15112   0   0   16   1  790  1388   2   1  97
 1  0  0     0 452700  83428  16136   0   0   17   0  790  1387   2   1  97
 1  0  0     0 451856  83428  16948   0   0   14   0  790  1386   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   1  789  1385   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   0  789  1386   2   1  97
 1  0  0     0 451852  83428  16948   0   0    0   0  789  1386   2   1  97
 1  0  0     0 451852  83428  16948   0   0    0   0  789  1386   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   1  789  1386   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   0  789  1386   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   0  789  1386   2   1  97
 1  0  0     0 451852  83428  16948   0   0    0   0  789  1386   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   0  789  1387   2   1  97
 1  0  0     0 451856  83428  16948   0   0    0   1  789  1386   2   1  97

-------------------------------------

test2: 2.4.12-ac3-vmpatch-freeswap-preempt

- This stressed my system a little, she couldn't handle the load perfect, but she managed to stay alive ;^)

- Top1 when vmstat was finished, but the find command still running (top):
658 elko 19 0 766M 321M 536 R 321M 98.0 57.0 1:01 slocate

- Uptimes shows no big load (uptime):

  2:18pm  up  9 min,  4 users,  load average: 1.45, 0.85, 0.38
  2:19pm  up  9 min,  4 users,  load average: 1.47, 0.91, 0.41
  2:21pm  up 12 min,  4 users,  load average: 1.56, 1.14, 0.57
  2:27pm  up 18 min,  4 users,  load average: 1.45, 1.34, 0.84

- Memory suddenly released (free):
total used free shared buffers cached

Mem:        577676     451124     126552          0        168       7124
-/+ buffers/cache:     443832     133844

Swap: 1569972 6696 1563276

- Skips of mp3-file during test:
1st 5 mins.: 5 skips ~1,5 second each
2nd 5 mins.: 14 skips, same length
3rd 5 mins.: 15 skips, same length

- vmstat output:
procs memory swap io system cpu

r  b  w   swpd   free  buff cache  si   so   bi    bo   in    cs  us  sy  id
1  0  0      0 445268 83860 17560   0    0  308     4  186   199   2   3  95
2  0  0 356740   3064   228  6372   3  3950  18  4054  820  1473  74  19   8
2  0  0   6904 266992   244  6196  15  4692  39  4692  806  1295  77  17   6
2  0  0   6908 366392   280  6116  15  4190  39  4191  810  1291  76  19   6
2  0  0 557888   3064   140  5816  25  4617  61  4620  783  1433  70  18  12
2  0  0  14200 454988   224  6704  31  9375  60  9377  771  1116  66  24  10
2  0  0 451308   2860   120  5856   9  3477  26  3477  769  1409  75  16   9
2  0  0 946388   3900   124  5900  45 10954  75 10963  797  1170  62  21  18
2  0  0 492616   2860   100  5520  16  4142  39  4142  795  1267  75  18   7
2  0  0 979576   3060   104  5172   8  9130  26  9130  815  1174  70  25   5
2  0  0 450700   3064   108  5352  27  1696  72  1697  799  1313  73  15  12
2  0  0 860220   2844   104  5212   6 10968  22 10968  790  1386  64  22  14
2  0  0 445292   2588    96  5448  13  3835  43  3835  869  1399  71  16  12
2  0  0 895300   3060   104  4988   6  7457  23  7457  787  1187  70  21   9
2  0  0 443772   3064   164  4752  37  4832  86  4834  808  1277  68  17  15

-------------------------------------

test3: 2.4.12-ac3-vmpatch-freeswap-preempt

- No skips in mp3 playing, nothing special to report

- Using /dev/dsp with freeamp:
[elko@ElkOS ~]$ /usr/sbin/lsof | grep dsp

freeamp   716 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   717 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   718 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   719 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   720 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   721 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   723 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   852 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   853 elko   17w   CHR       14,3            22742 /dev/dsp
freeamp   854 elko   17w   CHR       14,3            22742 /dev/dsp

- vmstat output:
procs memory swap io system cpu

 r  b  w  swpd   free   buff  cache  si  so   bi bo   in   cs  us  sy  id
 4  0  0     0 301868  86072  62260   0   0  114  5  138  280  73   8  19
 2  0  0     0 305256  86072  62680   0   0    8  3  303  752  90  10   0
 2  0  0     0 302248  86084  63716   0   0   17  4  316  575  88  12   0
 7  0  0     0 298572  86104  64876   0   0   17  9  296  507  90  10   0
 4  0  0     0 297036  86108  65976   0   0   18  5  311  453  91   9   0
 2  0  0     0 295916  86112  66916   0   0   16  3  300  523  88  12   0
 4  0  0     0 295220  86112  67812   0   0   15  1  274  322  92   8   0
 2  0  0     0 293344  86112  68728   0   0   15  1  276  344  93   7   0
 3  0  0     0 291392  86184  70644   0   0   33  5  286  403  90  10   0
 3  0  0     0 290636  86196  71588   0   0   16  1  296  551  89  11   0
 2  0  0     0 291744  86196  72484   0   0   15  1  278  371  92   8   0
 2  0  0     0 290828  86196  73380   0   0   15  1  282  389  93   7   0
 2  0  0     0 290024  86200  74168   0   0   13  0  279  385  89  11   0
 2  0  0     0 287168  86212  75412   0   0   21  2  291  509  91   9   0
 2  0  0     0 286228  86212  76308   0   0   15  2  274  332  92   8   0

-------------------------------------

test4: 2.4.12-ac3-vmpatch-freeswap-preempt

- This is a strange one, since I've loaded more apps. in X, I would think the system would be even more stressed then in test 2 (same test in console); maybe it's freeamp that's doing so well, since I only had 2 skips; therefore, I'll run this test again with mpg123, like in test2

- Skips of mp3-file during test (4 skips total):
1st 5 mins.: none
2nd 5 mins.: none
3rd 5 mins.: 1 at 12 mins. and 1 at ~14,9678 mins.

- Note: 3 times, when memory began to get full, swap took over without slowing anything down at all, you couldn't notice it...

- vmstat output:
procs memory swap io system cpu

 r  b  w   swpd   free   buff cache  si  so    bi   bo   in   cs  us  sy  id
 3  2  0      0 281596 100628 61968   0   0   218    6  176  309  26   5  69
 4  0  0      0 225064 108032 81312   0   0   403   26  368  516  85  15   0
 3  0  0      0 116020 113436 96648   0   0   303   62  347  484  88  12   0
 4  0  0 212492   3068 103368 32608   0   0    42   20  280  333  86  14   0
 3  3  0 327916   3060    776 21440  13 4180   47 4183  322  331  83  17   0
 3  0  0 487860   3064    636 14908  89 3045  121 3054  302  309  86  14   0
 3  0  0 750580   3060    384 12788  36 3023   52 3024  301  315  82  18   0
 3  0  0  68756 292060    704 18208 131   0   226    3  277  347  86  14   0
 3  0  0  68756  31500    704 19232   3   0    20    1  273  309  88  12   0
 3  0  0 472564   3064    424 13860  34 3912   68 3913  314  377  77  17   7
 3  0  0  68992 318612    812 20552 415 1911  555 1913  291  357  86  14   0
 3  0  0  68960  63564    988 22088  15   0    43    1  277  318  88  12   0
 3  0  0 480228   2552    476 13688  37 3446   67 3447  292  325  76  20   4
 4  0  0 551760   3120    584 14076 281 3791  333 3792  324  357  84  15   1
 3  0  0 863540   2896    428  8616  14 4522   32 4523  305  309  76  17   7

-------------------------------------

test5: 2.4.12-ac3-vmpatch-freeswap-preempt

- Some skips, but not as much as I would have thought:
1st skip at 7 mins. (during a desktop-switch) 2nd skip at 9,5 mins.
3rd skip at 13 mins.
4th skip at 14 mins. (just before the end of vmstat)

- I would have expected much more skips during this test, since it's the same as test2, only much more programs running (X etc.)...

- When running these tests in X, the machine would freeze when trying to switch kde-desktops or switch apps., but when nothing is touched, she just (about) keeps on playing...

- vmstat output:
procs memory swap io system cpu

 r  b  w   swpd   free  buff  cache  si  so    bi   bo  in    cs  us  sy  id
 5  1  0  68120 465408  1188  24364  45 766   161  773 216   317  66  10  24
 4  0  0  68120 412296  8892  41464   5   0   384   27 884  1508  88  12   0
 4  0  0  68120 316464 14652  58128   1   0   323   53 868  1477  88  12   0
 4  0  0  68120  63152 15636  59784   0   0    44   28 798  1364  85  15   0
 5  1  0 440360   3064  1132  13772  15 3964   38 3964 847  1335  81  19   0
 4  0  0 487248   3064   916  21280 381 1927  552 1938 817  1358  85  15   0
 4  0  0 815376   3064   684  19176 274 3886  386 3889 837  1448  81  19   0
 4  0  0  70248 259996   924  22368 134   0   190    4 801  1372  86  14   0
 6  0  0  70208   3064   988  24160  18   0    57    0 796  1421  87  13   0
 4  0  0 467696   2804   508  15660  30 4383   68 4384 784  1246  74  21   5
 4  0  0  70356 322744   648  14424   8 1546   29 1549 807  1345  85  15   0
 4  0  0  70352  60732   872  15580  11   0    34    1 792  1372  87  13   0
 4  0  0 505900   3064   392  11020  33 4972   95 4973 793  1237  79  18   3
 4  0  0 568404   3064   520  10104  13 2317   35 2317 785  1298  85  15   0
 4  0  0 858932   3064   564  10796  43 3068   93 3069 767  1235  79  21   0

-------------------------------------

BTW: the `find' command never got to finish...
should I let it and `time -v' it for fun ????

~~
End of testing 2.4.12-ac3-vmpatch-freeswap-preempt (23-10-2001 16:51) ~~

--
elko

Hello,

Does anyone have information when (and where) a patch from Andre Hedrick will be available containing the Promise PDC20269 support he mentioned earlier this month?

There is nothing there at
http://www.kernel.org/pub/linux/kernel/people/hedrick/

Thanx

Johan.
---

Microsoft... is that somekind of toiletpaper?