LINUX-MM

DISCUSSION Rik van Riel 4 participants 9 messages Mar 23, 1999 – Mar 24, 1999 4 messages not in the archive Memory management

lore

Share & tools

lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Rik van Riel

author

1999-03-23 15:12 UTC #1 lore raw

Quoted 3 lines · Matthias Arnold

Thanks for your reply.Unfortunately the memory is not (at least not completely) returned to the system after the program has finished (as the OS comand 'free' tells me).

Hmm, what version of the kernel are you using?

IIRC there's a slight bug in some of the newer kernels where the swap cache isn't being freed when you exit your program, but only later on when the system tries to reclaim memory...

Rik -- Open Source: you deserve to be in control of your data.

+-------------------------------------------------------------------+
| Linux Memory Management site:  http://humbolt.geo.uu.nl/Linux-MM/ |
| Nederlandse Linux documentatie:          http://www.nl.linux.org/ |
+-------------------------------------------------------------------+

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Kev

1999-03-23 15:49 UTC #2 lore raw

Quoted 10 lines

Quoted 3 lines
Thanks for your reply.Unfortunately the memory is not (at least not completely) returned to the system after the program has finished (as the OS comand 'free' tells me).
Hmm, what version of the kernel are you using?
IIRC there's a slight bug in some of the newer kernels where the swap cache isn't being freed when you exit your program, but only later on when the system tries to reclaim memory...

I believe the problem lies in the fact that there is not enough SysV shared memory available.

Signature · 11 lines

Kevin L. Mitchell <[email protected]>
-------------------------  -. .---- --.. ..- -..-  --------------------------
http://web.mit.edu/klmitch/www/               (PGP keys availiable from here)
    RSA AE87D37D/1024:  DE EA 1E 99 3F 2B F9 23  A0 D8 05 E0 6F BA B9 D2
    DSS ED0DB34E/1024: D9BF 0E74 FDCB 43F5 C597  878F 9455 EC24 ED0D B34E
    DH  2A2C31D4/2048: 1A77 4BA5 9E32 14AE 87DA  9FEC 7106 FC62 2A2C 31D4

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to [email protected].  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Kev

1999-03-23 16:31 UTC #3 lore raw

Quoted 2 lines

I believe the problem lies in the fact that there is not enough SysV shared memory available.

blah, forget it; I'm paying attention to too many email conversations. Sorry,

Signature · 11 lines

Kevin L. Mitchell <[email protected]>
-------------------------  -. .---- --.. ..- -..-  --------------------------
http://web.mit.edu/klmitch/www/               (PGP keys availiable from here)
    RSA AE87D37D/1024:  DE EA 1E 99 3F 2B F9 23  A0 D8 05 E0 6F BA B9 D2
    DSS ED0DB34E/1024: D9BF 0E74 FDCB 43F5 C597  878F 9455 EC24 ED0D B34E
    DH  2A2C31D4/2048: 1A77 4BA5 9E32 14AE 87DA  9FEC 7106 FC62 2A2C 31D4

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to [email protected].  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Stephen C. Tweedie

1999-03-23 17:32 UTC #4 lore raw

In reply to Kev

Hi,

Quoted 7 lines · Kev

Quoted 4 lines
IIRC there's a slight bug in some of the newer kernels where the swap cache isn't being freed when you exit your program, but only later on when the system tries to reclaim memory...
I believe the problem lies in the fact that there is not enough SysV shared memory available.

It's nothing to do with SysV shared memory.

The behaviour is there, but the only impact on the normal user will be that "free" lies a little. No big deal: it just shows up as cache. The effect is only a matter of when we recover the memory, not whether we recover it.

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

A message that is not in the archive (sent off-list or not yet received): [email protected]

Stephen C. Tweedie

1999-03-24 14:39 UTC #8 lore raw

Hi,

Quoted 4 lines · Matthias Arnold

The system lost a remarkable amount of memory after each run of my programs.After several runs the performance of the machine slowes down due to swapping (in other words the system hangs) and I have to reboot.

Which kernel precisely? What does "vmstat 1" look like? If you swapoff/swapon between application runs does the effect persist? What does the application do? What does /proc/sys/fs/inode-nr contain?

This does not sound like a result of the swap caching behaviour. Once you start swapping that memory _is_ returned (or something is not working as it should).

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Andrea Arcangeli

1999-03-24 00:24 UTC #9 lore raw

In reply to Rik van Riel

Quoted 9 lines · Rik van Riel

Quoted 3 lines · Matthias Arnold
Thanks for your reply.Unfortunately the memory is not (at least not completely) returned to the system after the program has finished (as the OS comand 'free' tells me).
Hmm, what version of the kernel are you using?
IIRC there's a slight bug in some of the newer kernels

I think at it as a feature and not a bug ;).

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Rik van Riel

author

1999-03-24 00:39 UTC #10 lore raw

Quoted 7 lines · Andrea Arcangeli

Quoted 6 lines · Rik van Riel
Quoted 3 lines · Matthias Arnold
Thanks for your reply.Unfortunately the memory is not (at least not completely) returned to the system after the program has finished (as the OS comand 'free' tells me).

Quoted 5 lines

Quoted 3 lines
Hmm, what version of the kernel are you using?
IIRC there's a slight bug in some of the newer kernels
I think at it as a feature and not a bug ;).

It is a bug when it causes other programs to fail
miserably...

Rik -- Open Source: you deserve to be in control of your data.

+-------------------------------------------------------------------+
| Linux Memory Management site:  http://humbolt.geo.uu.nl/Linux-MM/ |
| Nederlandse Linux documentatie:          http://www.nl.linux.org/ |
+-------------------------------------------------------------------+

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Stephen C. Tweedie

1999-03-24 13:57 UTC #11 lore raw

Hi,

Quoted 2 lines · Rik van Riel

It is a bug when it causes other programs to fail
miserably...

Does this happen, and if so, in what way?

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Andrea Arcangeli

1999-03-24 17:14 UTC #12 lore raw

Quoted 9 lines · Stephen C. Tweedie

Hi,
Quoted 2 lines · Rik van Riel
It is a bug when it causes other programs to fail
miserably...
Does this happen, and if so, in what way?

I don't think that can happens. Anyway I just have a patch to return to the old behavior, but I can't see any difference here (except that we won't do the work on demand).

Just for the record my patch is this, probably it won't apply cleanly but upporting it, it's trivial.

Index: linux/include/linux/swap.h
diff -c linux/include/linux/swap.h:1.1.1.1.14.3 linux/include/linux/swap.h:1.1.1.1.14.4
*** linux/include/linux/swap.h:1.1.1.1.14.3 Thu Oct 29 19:42:56 1998
--- linux/include/linux/swap.h Sun Nov 1 18:20:48 1998
***************
*** 90,95 ****
--- 90,96 ----
extern struct page * read_swap_cache_async(unsigned long, int);
#define read_swap_cache(entry) read_swap_cache_async(entry, 1);
extern int FASTCALL(swap_count(unsigned long));
+ extern void FASTCALL(try_to_free_last_swap_entry(unsigned long));
/*
* Make these inline later once they are working properly.
*/
Index: linux/mm/swap_state.c
diff -c linux/mm/swap_state.c:1.1.1.1.14.4 linux/mm/swap_state.c:1.1.1.1.14.5
*** linux/mm/swap_state.c:1.1.1.1.14.4 Fri Oct 30 19:11:15 1998
--- linux/mm/swap_state.c Sun Nov 1 18:20:50 1998
***************
*** 287,292 ****
--- 287,316 ----
return 0;
}

+ void try_to_free_last_swap_entry(unsigned long entry)
+ {
+ struct page * page = lookup_swap_cache(entry);
+ if (page)
+ {
+ /*
+ * The last reference in the swap_map[entry] is caused
+ * by this swap cache page.
+ *
+ * Decrease the page->count increased by __find_page().
+ * -arca
+ */
+ __free_page(page);
+ if (atomic_read(&page->count) == 1)
+ /*
+ * The page is resident in memory only because
+ * it' s in the swap cache so we can remove it
+ * because it can' t be useful anymore.
+ * -arca
+ */
+ delete_from_swap_cache(page);
+ }
+ }
+
/*
* Locate a page of swap in physical memory, reserving swap cache space
* and reading the disk if it is not already cached. If wait==0, we are
Index: linux/mm/swapfile.c
diff -c linux/mm/swapfile.c:1.1.1.1 linux/mm/swapfile.c:1.1.1.1.16.1
*** linux/mm/swapfile.c:1.1.1.1 Fri Oct 2 19:22:39 1998
--- linux/mm/swapfile.c Sun Nov 1 18:20:50 1998
***************
*** 144,153 ****
p->highest_bit = offset;
if (!p->swap_map[offset])
goto bad_free;
! if (p->swap_map[offset] < SWAP_MAP_MAX) {
! if (!--p->swap_map[offset])
nr_swap_pages++;
! }
#ifdef DEBUG_SWAP
printk("DebugVM: swap_free(entry %08lx, count now %d)\n",
entry, p->swap_map[offset]);
--- 144,158 ----
p->highest_bit = offset;
if (!p->swap_map[offset])
goto bad_free;
! if (p->swap_map[offset] < SWAP_MAP_MAX)
! switch(--p->swap_map[offset])
! {
! case 0:
nr_swap_pages++;
! break;
! case 1:
! try_to_free_last_swap_entry(entry);
! }
#ifdef DEBUG_SWAP
printk("DebugVM: swap_free(entry %08lx, count now %d)\n",
entry, p->swap_map[offset]);

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/