Skip to contentSkip to conversation

vm in 2.4.5

DISCUSSION J . A . Magallon 3 participants 4 messages May 26, 2001 – May 27, 2001

lore
Share & tools
lore
#0 lore raw

Hi.

This is a little experiment to smash 2.4 vm, and there is something I do not understand.

Experiment: compile a C file with, say, 100k lines of puts("test"), auto generated. Box is running vanilla 2.4.5, on 256Mb of ram.

State before gcc tst.c (just logged in a Gnome session with a couple rxvt's open):

werewolf:~> free
total used free shared buffers cached

Mem:        255688      99572     156116          0       6788      42876
-/+ buffers/cache:      49908     205780
Swap:       152576          0     152576

Start gcc tst.c, and used memory begins to grow linearly in time, until:

werewolf:~> free
total used free shared buffers cached

Mem:        255688     213856      41832          0       6788      42876
-/+ buffers/cache:     164192      91496

Swap: 152576 0 152576

Stays some moments this way, and suddenly both memory and swap are full:

werewolf:~> free

             total       used       free     shared    buffers     cached
Mem:        255688     251268       4420          0       3752     179948

-/+ buffers/cache: 67568 188120
Swap: 152576 152576 0

It does not begin to use swap in a growing fashion, it just appears full in a moment.

When compiler (cc1) ends, and assembler starts, some mem is freed:

werewolf:~> free
total used free shared buffers cached
Mem: 255688 195784 59904 0 1708 178512
-/+ buffers/cache: 15564 240124
Swap: 152576 152576 0

And when all the gcc process ends, my mem ends up like:

werewolf:~> free
total used free shared buffers cached
Mem: 255688 191232 64456 0 1760 174628
-/+ buffers/cache: 14844 240844
Swap: 152576 152576 0

What process do belong the 150Mb of swap ???!!!! Shouldn't that pages have been freed when gcc ends ?

In short, I begin in one state, gcc something, and I do not return to the same state (I know it could not be the same, something has been paged out, and still has not get in core again, but should not totals be roughly the same ???). It looks like the files gcc has used are still cached and also swapped out and reserved, or the like...

Perhaps this helps someone.

BTW, I do not understand what the hell for needs gcc 256 Mb to compile 100k sequential sentences...

Signature · 4 lines
J.A. Magallon                           #  Let the source be with you...        
mailto:[email protected]
Linux Mandrake release 8.1 (Cooker) for i586
Linux werewolf 2.4.4-ac15 #1 SMP Wed May 23 21:55:23 CEST 2001 i686
#1 lore raw
Quoted 2 lines · J . A . Magallon

It does not begin to use swap in a growing fashion, it just appears full in a moment.

It gets _allocated_ in a moment, but things don't actually get swapped out. This isn't a problem.

The real problem is that we don't actively reclaim swap space when it gets full. We just assign swap to parts of processes, but we never reclaim it when we need swap space for something else; at least, not until the process exit()s or exec()s.

Quoted 7 lines

And when all the gcc process ends, my mem ends up like:

total used free shared buffers cached
Swap: 152576 152576 0

What process do belong the 150Mb of swap ???!!!! Shouldn't that pages have been freed when gcc ends ?

Linux reclaims swap cache (and swap space) when it encounters them in its scan of memory. It doesn't take the trouble of freeing the swap on exit() but the swap space will be freed later.

regards,

Rik
--
Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

http://www.surriel.com/ http://distro.conectiva.com/

Send all your spam to [email protected] (spam digging piggy)

#2 lore raw
Quoted 11 lines · J . A . Magallon
Quoted 2 lines

It does not begin to use swap in a growing fashion, it just appears full in a moment.

It gets _allocated_ in a moment, but things don't actually get swapped out. This isn't a problem.

The real problem is that we don't actively reclaim swap space when it gets full. We just assign swap to parts of processes, but we never reclaim it when we need swap space for something else; at least, not until the process exit()s or exec()s.

As I understand the original message, before the compilation and after the system is in a "similar" state ("the same" processes should be running; even less of them if some were killed by OOM).

: total used free shared buffers cached
: -/+ buffers/cache: 49908 205780
: Swap: 152576 0 152576

~49 MB in use befere the test,

: -/+ buffers/cache: 14844 240844
: Swap: 152576 152576 0

~149 MB allocated swap. By processes of total size of 49 MB ? Strange...

Either the test shows some leak in running userspace processes (they allocate a lot of memory) as an effect of memory shortage (strange) or there is some leak in the kernel. Or the test is bad (something else is running when it finishes).

Am I right ?

Andrzej

#3 lore raw
In reply to J . A . Magallon
Quoted 18 lines · On 05.26 Rik van Riel wrote:
Quoted 2 lines · J . A . Magallon

It does not begin to use swap in a growing fashion, it just appears full in a moment.

It gets _allocated_ in a moment, but things don't actually get swapped out. This isn't a problem.

The real problem is that we don't actively reclaim swap space when it gets full. We just assign swap to parts of processes, but we never reclaim it when we need swap space for something else; at least, not until the process exit()s or exec()s.

Quoted 5 lines

And when all the gcc process ends, my mem ends up like:

total used free shared buffers cached
Swap: 152576 152576 0

Quoted 8 lines
Quoted 2 lines

What process do belong the 150Mb of swap ???!!!! Shouldn't that pages have been freed when gcc ends ?

Linux reclaims swap cache (and swap space) when it encounters them in its scan of memory. It doesn't take the trouble of freeing the swap on exit() but the swap space will be freed later.

That seems to be partially true, if I start just the same comilation when the first finishes, it behaves like preivous, in the moment the new gcc needs swap the old swap gets freed, but not the in-core cache, so if I choose carefully the number of 'puts' lines to stress my system just to the border, I can't do two times the same 'gcc tst.c'.

Signature · 4 lines
J.A. Magallon                           #  Let the source be with you...        
mailto:[email protected]
Linux Mandrake release 8.1 (Cooker) for i586
Linux werewolf 2.4.4-ac15 #1 SMP Wed May 23 21:55:23 CEST 2001 i686