swapping,any updates ?? Just wasted money on mem upgrade performance still suck :-(

Hi,all

You might remember I had issues with massive swapping and wanted to know whether I can control the amount of cache and buffers and so on.Well I thought a mem upgrade would do the trick ,but no :-( Not easy to explain to my boss that it still crawls with 512 MB mem and that's the max limit in this laptop..Anyone found any solutions ?? Check this out:

top -bn 1|head -n 30

16:25:46 up 2:56, 4 users, load average: 0.50, 0.46, 0.43
79 processes: 77 sleeping, 2 running, 0 zombie, 0 stopped

CPU states:   9.8% user,   8.5% system,   0.0% nice,  81.7% idle
Mem:    513692K total,   512072K used,     1620K free,    21564K buffers
Swap:   248968K total,    60180K used,   188788K free,   323668K cached

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
  950 ce        14   0  150M 150M  149M R    18.2 29.9  13:01 vmware
 1146 ce        17   0  1000  996   772 R     7.3  0.1   0:00 top
  952 ce         9   0 42572 9528  8268 S     0.9  1.8   0:12 vmware-mks
    1 root       8   0   524  480   460 S     0.0  0.0   0:03 init
    2 root       9   0     0    0     0 SW    0.0  0.0   0:00 keventd
    3 root       9   0     0    0     0 SW    0.0  0.0   0:00 kapm-idled
    4 root      19  19     0    0     0 SWN   0.0  0.0   0:00 
ksoftirqd_CPU0
    5 root       9   0     0    0     0 SW    0.0  0.0   0:00 kswapd
    6 root       9   0     0    0     0 SW    0.0  0.0   0:00 bdflush
    7 root       9   0     0    0     0 SW    0.0  0.0   0:00 kupdated
   69 root       9   0     0    0     0 SW    0.0  0.0   0:00 kreiserfsd
   92 daemon     9   0   464  380   380 S     0.0  0.0   0:00 portmap
  218 root       9   0   648  596   528 S     0.0  0.1   0:00 syslogd
  221 root       9   0  1192  460   460 S     0.0  0.0   0:00 klogd
  242 root       8   0   676  520   520 S     0.0  0.1   0:00 cardmgr
  248 root       9   0   728  620   620 S     0.0  0.1   0:00 rpc.statd
  251 root       8   0   512  508   496 S     0.0  0.0   0:00 apmd
  262 root       9   0   556  484   484 S     0.0  0.0   0:00 inetd
  307 root       9   0  1192  968   856 S     0.0  0.1   0:00 nmbd
  309 root       9   0  1168  812   812 S     0.0  0.1   0:00 smbd
  316 root       9   0  1240 1028  1028 S     0.0  0.2   0:00 sshd

it just caches like crazy and things start to crawl cause its swapping.More than 300 MB of cache what on earth is being cached ?? I can't stand ths anymore I guess I'll have to back down to 2.2 again,but that'll have other downsides :-( *sigh* I'm willing to help as much as I can with this I don't want to give up on Linux just like that.

best regards

Christian

Besides the fact I couldn't identify the kernel version from your mail, I would try:

1) Turn off swap, then
2) Use 2.4.17 with patch I send you off LKML.

Then give us a hint if things got better.

Regards,
Stephan

Simply try to disable swap (swapoff -a or comment out swap partitions in /etc/fstab and reboot).
I did this on a 384 MiB SMP box as I had bad interactive performance with some desktop apps and heavy background disk I/O (video grabing) around 2.4.10-2.4.13 (I did not reactivate swap since) and it solved my interactive perf woes.

You might also try Andrea's patches:
http://www.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.17rc2aa2.bz2

or/and Rik's ones:
http://www.surriel.com/patches/2.4/2.4.17-rmap-10c

LB.

Sorry forgot that.Currently running 2.4.17-mjc1.
Turning off swap is apparently not an option 'cause now VMware won't run anymore (really a swap happy app if I ever saw one :o) I get this error when starting to log on to my virtual Windows XP Pro:

AIO: unexpected loss of channel ide0:0 (thread ide 0:0)

and turning swap back on it runs with no problems..*sigh*
Can I make a RAM drive and then use that for swap ??Will the patch you sent me work with swap turned on ??

For info my system:

Dell latitude cpx j650GT,650 MHz P3
512 MB mem
12 GB hdd
3com 3c575ct pcmcia NIC
Debian woody
kernel 2.4.17-mjc1

running in a dock station with Nvidia TNT2 32 MB graphic card

that should be all..

best regards

Christian

You have to face it: Linus will never accept VM subsystem with lots on tuning knobs. It just won't happen. Fixing VM behavior is the only way. It has to work satisfactorily _without_ tuning.

Can you send full top output along with /proc/meminfo and /proc/slabinfo? Last time you had mighty X (>100MB) too listed there...

Well, vmware uses binary-only kernel module, can you confirm bad behaviour without vmware?

You may try -aa and Rik's VM.
I'm CC'ing them, maybe they are interested...

I use a proggy below to flush caches. It triggers OOM condition. But before it gets OOM-killed, it forces kernel to discard excessive caches. At least it has to work that way, but it isn't (on stock kernel), which confirms your observation that kernel is over-reluctant to flush caches.

#include <stdlib.h>

int main() {
    void *p;
    unsigned size = 1<<20;
    unsigned long total=0;
    while(size) {
        p = malloc(size);
        if(!p) size>>=1;
        else {
            memset(p, 0x77, size);
            total+=size;
            printf("Allocated %9u bytes, %12lu total\n",size,total);
        }
    }
    return 0;
}

How2use:
* do repeatedly:
* top c b n 1 >topN (N=1,2,3,4,...)
* same for 'cat /proc/meminfo' and 'cat /proc/slabinfo'
* ./oom_trigger (will(should be) oom killed)
* try to figure out what cache isn't freed as it should :-)

To get clearer picture I do
* killall5 -15; sleep 3; killall5 -9 several times first, but that will ruin your test case it seems.

Anyway, if you'll spot something unusual in those debug printouts, please inform lkml
--
vda

Thats something you will never achieve. Virtual memory is about heuristics, crystal ball gazing and guesswork. There are always some workloads where you want little caching and some where you want lots of caching - such as a fileserver.

You can make it right for most people but the last few percent you will always get by tuning knobs - either directly or via GUI tools like powertweak

Alan

Thank you, Alan!! Now if the *other* kernel developers would just buy into this. :)

M. Edward "Give me tuning knobs or give me death" Borasky

[email protected]
http://www.borasky-research.net "The Few! The Proud! The Tweakers!"

The American people are tired of being told what the American people
are tired of.

non sequitur: Linus would like an adaptive VM, which recognizes apps with the properties you describe. there's no theoretical or practical reason this cannot be achieved.

you have the source. whinging about knobs is just whinging. all serious knobs require recompilation anyway.

Exactly.Please think about what Alan just said.You just can't satisfy everyone in every situation without tweaking.That's the way it is.

best regards

Christian

You can get close though. The fact that we aren't close is no proof of impossibility. If we do give up and decide to ship only 'manual transmissions', we can be quite sure we'll never get there.

Except, as you and others have pointed out, we are far from knowing what the knobs should be.

--
Daniel

Oh there is. To compute the correct VM behaviour requires knowledge of what the workload will do in the future. Now if you can solve the halting problem and/or invent time travel I'm waiting to hear.

There are heuristics, and Linus goal is the right one. Its just useful to recognize someone will always have a load you get wrong. Most users don't understand tweaking vm configurations, yet even windows NT boxes let you do so. For the critical jobs there will be someone who is willing to make the effort to learn how to tune it.

Now that is definitely not the case. I'm not talking

Virtual memory system (Andrea, Marcelo, Rik, Rik rmap, Linus) CONFIG_VM

would> > try:

Please try a stock 2.4.17 (with the patch), otherwise we will have no idea what is going on.

Uh, this is no good. How much mem does your XP need? I can't really believe you need more than the 512 MB you have to get this config running. Really: please try stock kernel with patch.

Yes, it will work of course. But we would like to see this config without swap running too, won't we?

Nice box. Must work somehow.

Regards,
Stephan

There are a few things to keep in mind, however:

1) the default settings shouldn't result in any system falling apart, it really should work everywhere (though possibly slower than after tweaking)

2) the tunables should make sense and be easy enough to tune without first needing to understand all of the VM internals

regards,

Rik

Shortwave goes a long way:  irc.starchat.net  #swl

http://www.surriel.com/		http://distro.conectiva.com/

Just patched the kernel and booted it up..To begin with it looked OK and there wasn't any swapping.Even firing up VMware didn't cause it to swap..

Then all of a sudden the mouse started moving all over the screen and left and right clicking on everything on it's own. ?? Really weird.. Ran like that for 5 minutes then the machine crashed hard.
Now I've booted up with the std 2.4.17 again..

I assigned 192 MB for XP that should be more than enough..And as long as Linux doesn't swap it is..

I really hope so..*sigh* :-(

best regards

christian

what> > is going on.

That is fine.

This is for sure no VM issue. This is strange, is this with VMware started, inside XP? Does this happen under Linux-only, too (with no vmware running)? It doesn't even look quite like a linux issue at all. Just guessing: do you have a virus-checker for XP at hand? Only to make sure...

This can be driven without swap for sure.

Regards,
Stephan

VMware modules are far from GPL but they are provided with source. Quite handy when some kernel structs change...

LB.

Hauling out my Gilbert and Sullivan: "I've got a little list" :-)

1. The time slice. Currently this is a #define. Make it a variable and give me a sysctl to set it.

2. The processor change penalty in the scheduler. Again a #define. Make it a variable and give me a sysctl to set it.

3. The "memory map" change penalty in the scheduler. Currently hard-coded to 1 in "sched.c". Make it a variable and give me a sysctl to set it.

4. The DMA, lowmem and highmem balance ratios and maxima in "page_alloc.c". Currently these are hard coded. From the code, it looks like the values I *really* want to change, the watermarks for each zone, are computed at boot time and can't be changed once the system boots. But I'd like to be able to change the watermarks at run time if at all possible.

5. The minimum and maximum fraction of memory that can be allocated to page cache for each zone.
--
M. Edward "Lord High Executioner" Borasky

[email protected]
http://www.borasky-research.net
http://groups.yahoo.com/group/meta-trading-coach

How to Stop A Folksinger Cold # 5
"Where have all the flowers gone..."
Beats me.