update re: fork() failures in 2.1.101

[CC-ed to linux-mm, and it should stay that way...]

Damn, this looks near-perfect for normal system load... I really don't understand what's wrong.

I wonder what kind of software / networking app you are using, and what memory usage those programs have...

Rik.

+-------------------------------------------------------------------+
| Linux memory management tour guide.        [email protected] |
| Scouting Vries cubscout leader.      http://www.phys.uu.nl/~riel/ |
+-------------------------------------------------------------------+

It's a mixed libc5/libc6 system.
Here is a snapshot of the Top 20 in RSS:

%CPU %MEM SIZE RSS

 1.3 18.9 13552  5876 Xwrapper        XFree 3.3.2.2
 1.0 18.4 10612  5716 netscape        3.01
 0.0  5.6  4508  1740 kermitbeta      6.1.193 Beta.05
 1.2  5.1  4072  1584 rvplayer        5.0.0.35
 0.0  3.7  4372  1176 kermitbeta
 0.0  3.7  4372  1168 kermitbeta
 0.0  2.9  1824   908 named           8.1.2
 0.0  2.9   960   908 xntpd           3-5.91 (locked into memory)
 0.0  2.8  2584   876 xterm
 0.0  2.4  2420   748 xterm
 0.0  2.3  1448   716 zsh             3.1.4
 0.0  2.1  1380   676 zsh
 0.0  2.1  1380   676 zsh
 0.0  2.1  1404   668 perl            5.004_04
 0.0  1.9  1512   592 gnuplot_x11     3.5 (3.50.1.17)
 0.0  1.9  2164   592 xload
 0.0  1.6   932   520 pppd            2.3.5
95.7  1.6  9364   520 mprime          15.4.2 (internet Mersenne prime search)
 0.0  1.5  1756   496 gnuplot
 0.0  1.5   836   488 ps              1.2.4

-Paul <[email protected]>
[please cc: relevant messages to me]

Shouldn't be much of a problem... But 'eh, does the Mersenne program regularly do memory I/O?
It could be that it loads large chunks of memory and frees small portions from the middle of it. The Linux MM system could have a problem with that...

Of course we should be able to handle such stuff, but with the current buddy allocator things might just get a little bit tricky :(

The reason I picked this process, is that it's RSS is only one 18th of it's total size, which is somewhat weird for a 'normal' Unix process.

Rik.

+-------------------------------------------------------------------+
| Linux memory management tour guide.        [email protected] |
| Scouting Vries cubscout leader.      http://www.phys.uu.nl/~riel/ |
+-------------------------------------------------------------------+

I *think* that it allocates a huge amount of memory, then uses only a small portion of it.

The above shows an inconsistency between "ps" and "top":
according to "ps", SIZE=9364, RSS=404;
but according to "top", SIZE= 500, RSS=404, SWAP=96.

"grep '^Vm' /proc/<pid>/status" says

-Paul <[email protected]>

This is indeed the case. I know it's a sloppy programming practice, but it was the easiest way for me to interface with all my assembly code that assumes the FFT data is at a fixed address. Mprime actually has 16MB of global variables. Unless you are testing an exponent above 20,000,000 then you are only using a small fraction of the 16MB.

Best regards,
George

RECAP: In 2.1.99, 2.1.101, 2.1.103, and 2.1.104-pre1, my system has been usable for only ~1 day with 32 MB of memory, or ~2.5 days with 48 MB. Then my system has trouble forking, typically with EAGAIN. The situation can be alleviated temporarily by killing off a few processes, but the errors always reappear soon thereafter. I have sent in the results of Shift-ScrollLock, which Rik thinks are not typical of excessive memory fragmentation.

Now, I have scripts that run "ifconfig ppp0" hourly (to check whether PPP is "UP"). Recently I joined the modern era by changing from net-tools 1.432 to 1.45. The forking errors have gone away (at least for uptimes twice the above). When I changed these scripts to run "/sbin/ifconfig.old ppp0" instead, they came back.

Running the old ifconfig (when the problem arises) would put "kmod: fork failed, errno 11" messages in the logfiles. The new ifconfig doesn't. Running strace on "ifconfig ppp0" shows that the old version makes the following system calls that the new one doesn't:

(I am not sure whether these system calls have been taken out of the new ifconfig, or whether I merely configured net-tools to be ignorant of appletalk, etc.)

Something about my old ifconfig must be triggering a bug (or hardware error?) somewhere. I am willing to take further suggestions for experiments to try, if anyone is still interested.

-Paul <[email protected]>
(please cc: relevant messages to me)