Skip to contentSkip to conversation

Question about memory usage in 2.4 vs 2.2

DISCUSSION Manfred Spraul 4 participants 4 messages Mar 21, 2001

lore
Share & tools
lore
#0 lore raw
Quoted 2 lines

inode_cache 189974 243512 480 30439 30439 1 : 124 62 dentry_cache 201179 341940 128 11398 11398 1 : 252 126

1) number of used objects
2) number of allocated objects
3) size of each object
4) number of slabs that are at least partially in use
5) number of slabs that are allocated for the cache
i.e. 5)-4) are the number of freeable slabs in the cache
6) size in pages for a slab
:
7) length of the per-cpu list. Each cpu some objects in a local list it can use without acquiring a spinlock
8) batch count. If the per-cpu list overflows multiple objects are freed/allocated in one block.

7 and 8 are only present if your kernel is compiled for SMP, root can tune them with

#echo "<slab name> <length> <batchcount>" > /proc/slabinfo

It seems that the dentry cache is severely fragmented, nearly 20 MB (or 30%) are
unfreeable due to fragmentation.

--
Manfred

#1 lore raw

This is what I'm afraid of, in my case we have millions of files that are dealt with in no real order, and if cache fragmentation will keep the memory from being freed, we're in for problems. This reading was taken with the machine having been up for only 5 days. Currently, I show:

inode_cache       194565 234696    480 29337 29337    1 :  124   62
dentry_cache      207257 327300    128 10910 10910    1 :  252  126

However, memory usage is still at 44% according to /proc/meminfo.

I've had to put my SMTP box back to 2.2 as it was up to 90% memory used, where the others were around 18%. I'm keeping the pop/imap server at 2.4 as 44% is standable, while not exactly desirable. I'm

---
Josh Grebe
Senior Unix Systems Administrator
Primary Network, an MPower Company
http://www.primary.net

Quoted 34 lines · Manfred Spraul
Quoted 2 lines

inode_cache 189974 243512 480 30439 30439 1 : 124 62 dentry_cache 201179 341940 128 11398 11398 1 : 252 126

1) number of used objects
2) number of allocated objects
3) size of each object
4) number of slabs that are at least partially in use
5) number of slabs that are allocated for the cache
i.e. 5)-4) are the number of freeable slabs in the cache
6) size in pages for a slab
:
7) length of the per-cpu list. Each cpu some objects in a local list it can use without acquiring a spinlock
8) batch count. If the per-cpu list overflows multiple objects are freed/allocated in one block.

7 and 8 are only present if your kernel is compiled for SMP, root can tune them with

#echo "<slab name> <length> <batchcount>" > /proc/slabinfo

It seems that the dentry cache is severely fragmented, nearly 20 MB (or 30%) are
unfreeable due to fragmentation.

--
Manfred

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
In reply to Manfred Spraul
Quoted 4 lines · Josh Grebe

This is what I'm afraid of, in my case we have millions of files that are dealt with in no real order, and if cache fragmentation will keep the memory from being freed, we're in for problems. This reading was taken with the machine having been up for only 5 days. Currently, I show:

You shouldn't worry too much about fragmentation within the slabcache. It actually is designed to limit the amount of fragmentation. And the next 100000 dentry/inode allocations will come at no cost ;)

The fact that your icache/dcache slabs are not 100% used indicates that shrink_i/dcache_memory actually was triggered a couple of times due to free memory shortage. In contrast, on my machine, the available and allocated numbers are pretty much always identical. I'm probably not putting enough VM pressure on the machine so the normal aging/pageout mechanisms manage to free up just enough memory to avoid dentry/inode pruning.

Quoted 3 lines

I've had to put my SMTP box back to 2.2 as it was up to 90% memory used, where the others were around 18%. I'm keeping the pop/imap server at 2.4 as 44% is standable, while not exactly desirable. I'm

Did the SMTP box become unusable? Although the VM ends up working with only the memory that is not locked up in slabs, it does a pretty good job, as you reported yourself.

Quoted 10 lines · Josh Grebe

As far as performance goes, I can only say that my max load is slightly lower on the 2.4 box then on the 2.2 boxes. Our average load for yesterday on 2.4 was .23, with a max of 1.11. In comparison, my averages for the other machines are .27, .27, .23, and .23. The maxes are 1.85, 1.33, 2.06, 1.47.

As far as speed goes, I am not able to measure any real difference (only testing pop3) between 2.2 and 2.4. I would blame this on the NAS device, a NetApp Filer F760 being only able to push about 110mbit sustained on the gig-e network.

Doing an (intuitive) cost/benefit analysis, throwing out an inode does not guarantee to free up any memory. While throwing out clean pages from the pagecache will free up some memory and they do not necessarily have to be read back in at all (otherwise they would have been 'active').

I've been thinking about this a bit and one possible solution would be to significantly lower the cost of prune_icache by removing the sync_all_inodes and only let it prune inodes that do not have any mappings associated with them. Then it might become possible to call it more frequently, like every time we hit do_try_free_pages.

If there is enough memory available in the system, the inodes will have mappings and won't be removed. Once memory pressure builds up the mappings are cleaned and pruned, by the VM system and removing the remains of the inode should be relatively inexpensive. There is probably something that I missed, so I should just shut up now and write the code.

Jan

#3 lore raw
Quoted 5 lines · Jan Harkes

I've been thinking about this a bit and one possible solution would be to significantly lower the cost of prune_icache by removing the sync_all_inodes and only let it prune inodes that do not have any mappings associated with them. Then it might become possible to call it more frequently, like every time we hit do_try_free_pages.

Marcelo and me have been looking at this issue too, and have come to almost the same conclusion as you, with one small change.

We -need- to have a way to trigger the writeout of dirty inodes under memory pressure. Imagine doing 'chown -R' on a huge tree on a low-memory box; you'd end up with zillions of dirty inodes in memory with no way to free them.

Now if prune_icache would write all the dirty inodes without data pages to disk automatically, we'd have this issue fixed and we'll be able to make a much more efficient prune_icache.

Anybody willing to give it a shot ?

regards,

Rik
--
Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com.br/