Cached memory never gets released

Not so easy, we've got some custom modules and the kernel is modified a little to suit our needs (not related to buffer caches though).

It seems the problem is *not* brought on by the kernel killing XFree86 like I had posted before. Just normal use of the system seems to cause the Cached value in /proc/meminfo to go up, and it seems it can't go back down as needed when memory runs low.

What would be helpful is any advice as to where to look in the kernel source to try and track this down. I followed the path:
fs/proc/proc_misc.c handles /proc/meminfo, the cached value is based on
page_cache_size
mm/swap.c is what changes page_cache_size in delta_nr_cache_pages() function linux/swap.h has macros dec_nr_cache_pages inc_nr_cache_pages which call this mm/filemap.c is the only place that calls dec_nr_cache_pages in function
remove_page_from_hash_queue
mm/filemap.c function __remove_inode_page and remove_inode_page call that mm/filemap.c function invalidate_inode_pages and truncate_complete_page
call those
mm/filemap.c invalide_this_page2 and truncate_list_pages call
truncate_complete_page
invalidate_inode_pages is called all over the place...

So we're snowballing but I don't know what mechanism is supposed to actually free the cached pages when the system is low on memory. Any advice would be welcome.

Thanks--
Dave

You may want to examine /proc/meminfo, /proc/slabinfo, and the output of sysrq-m.

mm/vmscan.c (kswapd) is responsible for freeing most memory. The routine you are probably most interested in is shrink_cache.

I would check to make sure that the pages in the icache are backed by
a mapping and if so, that they are clean. If either of those two
conditions are not met, then the page cannot be thrown away.