Cached memory never gets released

In the thread this is made clear somewhat, but when I post new emails I don't summarize all that is known about the problem.

I can kill all user processes, including mozilla,
XFree86, and everything else except an sshd to
log into the system, and the cache is still not
released.

The cached memory is just *gone*, no recovery, bye-bye, history.

I am experimenting with strace to see what mozilla is doing. It does repeatedly
Open file X with O_EXCL and result is -1 EEXIST Open file Y with O_EXCL and it is created Close that file
Open file Y without O_EXCL
Allocate some memory
write to Y
close file Y
later unlink file Y

I tried duplicating this sequence of events thinking it could be something related to the create/write/close/delete sequence of a file, perhaps the file's inode isn't getting freed. However my test program didn't cause the same trouble.

Specifically the discussed strace output includes this:
mkdir("/root/mynfsarea/mozilla/plugtmp", 0777) = -1 EEXIST (File exists) open("/root/mynfsarea/mozilla/plugtmp/ipg_template.swf", O_WRONLY|O_CREAT|O_TRUNC|O_EXCL|0x8000, 0600) = -1 EEXIST (File exists) open("/root/mynfsarea/mozilla/plugtmp/ipg_template-1.swf", O_WRONLY|O_CREAT|O_TRUNC|O_EXCL|0x8000, 0600) = 30
close(30) = 0
open("/root/mynfsarea/mozilla/plugtmp/ipg_template-1.swf", O_WRONLY|O_CREAT|O_TRUNC|0x8000, 0600) = 30
brk(0x82d8000) = 0x82d8000
gettimeofday({1088521806, 310960}, NULL) = 0

brk(0x82d9000)                          = 0x82d9000
brk(0x82db000)                          = 0x82db000
brk(0x82dc000)                          = 0x82dc000
brk(0x82dd000)                          = 0x82dd000
brk(0x82de000)                          = 0x82de000
brk(0x82df000)                          = 0x82df000
brk(0x82e0000)                          = 0x82e0000
brk(0x82e1000)                          = 0x82e1000
brk(0x82e2000)                          = 0x82e2000

write(30, "FWS\6o\211\0\0x\0\7\320\0\0\27p\0\0\36\n\0C\2\377\377\377"..., 35183) = 35183
brk(0x82ea000) = 0x82ea000
close(30) = 0
... very much later in log unlink("/root/mynfsarea/mozilla/plugtmp/ipg_template-1.swf") = 0

My thinking here is that every reload of the page steals about 200K of cache memory that can never be recovered, and that the total size of these files in the plugtmp directory (mozilla's cache) is on the order of 200k.

-Dave

Sorry, I missed that part of the thread.

It does sounds like you may have a real problem, so the next step I would do is instrument kswapd to explain why it's not freeing cache when it's under demand.

The first step is to add something to kernel/sysctrl.c to create a variable to turn the debugging code on or off.

Then add a bunch of printk's to mm/vmscan.c explaining why every page is not being freed, but only when the sysctrl variable is set.

Then get the machine into the bad state and turn on the printks.

Run your program that can't allocate memory

Turn off the printks and analyze the logs.

I'll be happy to help you figure out where to put printks and go over the logs, but you have to make sure you are really getting an OOM kill with that much cache for the output to be useful.