slabnow

Hi,

Here is a rewritten slablru - this time its not using the lru... If changes long standing slab behavior. Now slab.c releases pages as soon as possible. This was done since we noticed that slablru was taking a long time to release the pages it freed - from other vm experiences this is not a good thing.

In this patch I have tried to make as few changes as possible.

With this in mind I am using
the percentage of the active+inactive pages reclaimed to recover the same percentage of the pruneable caches. In slablru the affect was to age the pruneable caches by percentage of the active+inactive pages scanned - this could be done but required more code so I went used pages reclaimed. The same choise was made about accounting of pages freed by the shrink_<something>_memory calls.

There is also a question as to if we should only use the ZONE_DMA and ZONE_NORMAL to drive the cache shrinking. Talk with Rik on irc convinced me to go with the choise that required less code, so we use all zones.

To apply the patch to mm5 use the follow procedure:
copy the two slablru patch and discard all but the vmscan changes.
replace the slablru patch with the just created patches that just hit vmscan after applying the mm5 patches apply the following patch to adjust vmscan and add slabasap.

This passes the normal group of tests I apply to my patches (mm4 stalled force watchdog to
reboot). The varient for bk linus also survives these tests.

I have seen some unexpected messages from the kde artsd daemon when I left kde running all night. This may imply we want to have slab be a little less aggressive freeing high order slabs. Would like to see if other have problems though - it could just be debian and kde 3.0.3 (which is not offical yet).

Please let me know if you want any changes or the addition of any of the options mentioned.

From a quick test, the shrinking rate seems quite reasonable to

...
If we have the pruner callbacks then vmscan can just do:

        kmem_shrink_stuff(ratio);

and then kmem_shrink_stuff() can do:

        cachep->nr_to_prune += cacheb->inuse / ratio;
        if (cachep->nr_to_prune > cachep->prune_batch) {
                int prune = cachep->nr_to_prune;

                cachep->nr_to_prune = 0;
                (*cachep->pruner)(nr_to_prune);
        }

Quoted 7 lines · Ed Tomlinson

Hi,

Here is a rewritten slablru - this time its not using the lru... If changes long standing slab behavior. Now slab.c releases pages as soon as possible. This was done since we noticed that slablru was taking a long time to release the pages it freed - from other vm experiences this is not a good thing.

Right. There remains the issue that we're ripping away constructed objects from slabs which have constructors, as Stephen points out.

I doubt if that matters. slab constructors just initialise stuff. If the memory is in cache then the initialisation is negligible. If the memory is not in cache then the initialisation will pull it into cache, which is something which we needed to do anyway. And unless the slab's access pattern is extremely LIFO, chances are that most allocations will come in from part-filled slab pages anyway.

And other such waffly words ;) I'll do the global LIFO page hotlists soonl; that'll fix it up.

Quoted 1 line

In this patch I have tried to make as few changes as possible.

Thanks. I've shuffled the patching sequence (painful), and diddled a few things. We actually do have the "number of scanned pages" in there, so we can use that. I agree that the ratio should be
nr_scanned/total rather than nr_reclaimed/total. This way, if
nr_reclaimed < nr_scanned (page reclaim is in trouble) then we put more pressure on slabs.

Quoted 8 lines

With this in mind I am using
the percentage of the active+inactive pages reclaimed to recover the same percentage of the pruneable caches. In slablru the affect was to age the pruneable caches by percentage of the active+inactive pages scanned - this could be done but required more code so I went used pages reclaimed.
The same choise was made about accounting of pages freed by the
shrink_<something>_memory calls.

Quoted 3 lines

There is also a question as to if we should only use the ZONE_DMA and ZONE_NORMAL to drive the cache shrinking. Talk with Rik on irc convinced me to go with the choise that required less code, so we use all zones.

OK. We could do with a `gimme_the_direct_addressed_classzone' utility anyway. It is currently open-coded in fs/buffer.c:free_more_memory(). We can just pull that out of there and use memclass() on it for this.

Quoted 10 lines

To apply the patch to mm5 use the follow procedure:
copy the two slablru patch and discard all but the vmscan changes. replace the slablru patch with the just created patches that just hit vmscan after applying the mm5 patches apply the following patch to adjust vmscan and add slabasap.

This passes the normal group of tests I apply to my patches (mm4 stalled
force watchdog to reboot). The varient for bk linus also survives these
tests.

Quoted 5 lines

I have seen some unexpected messages from the kde artsd daemon when I left kde running all night. This may imply we want to have slab be a little less aggressive freeing high order slabs. Would like to see if other have problems though - it could just be debian and kde 3.0.3 (which is not offical yet).

hm.

Quoted 2 lines

Please let me know if you want any changes or the addition of any of the options mentioned.

In here:

        int entries = inodes_stat.nr_inodes / ratio + 1;

what is the "+ 1" for? If it is to avoid divide-by-zero then it needs parentheses. I added the "+ 1" to the call site to cover that.

From a quick test, the shrinking rate seems quite reasonable to me. mem=512m, with twenty megs of ext2 inodes in core, a `dd' of one gigabyte (twice the size of memory) steadily pushed the ext2 inodes down to 2.5 megs (although total memory was still 9 megs - internal fragmentation of the slab).

A second 1G dd pushed it down to 1M/3M.

A third 1G dd pushed it down to .25M/1.25M

Seems OK.

A few things we should do later:

- We're calling prune_icache with a teeny number of inodes, many times. Would be better to batch that up a bit.

- It would be nice to bring back the pruner callbacks. The post-facto hook registration thing will be fine. Hit me with a stick for making you change the kmem_cache_create() prototype. Sorry about that.

If we have the pruner callbacks then vmscan can just do:

	kmem_shrink_stuff(ratio);

and then kmem_shrink_stuff() can do:

	cachep->nr_to_prune += cacheb->inuse / ratio;
	if (cachep->nr_to_prune > cachep->prune_batch) {
		int prune = cachep->nr_to_prune;

		cachep->nr_to_prune = 0;
		(*cachep->pruner)(nr_to_prune);
	}

But let's get the current code settled in before doing these refinements.

There are some usage patterns in which the dentry/inode aging might be going wrong. Try, with mem=512m

cp -a linux a
cp -a linux b
cp -a linux c

etc.

Possibly the inode/dentry cache is just being FIFO here and is doing exactly the wrong thing. But the dcache referenced-bit logic should cause the inodes in `linux' to be pinned with this test, so that should be OK. Dunno.

The above test will be hurt a bit by the aggressively lowered (10%) background writeback threshold - more reads competing with writes. Maybe I should not kick off background writeback until the dirty threshold reaches 30% if there are reads queued against the device. That's easy enough to do.

drop-behind should help here too.

Quoted 9 lines · Andrew Morton

Quoted 7 lines · Ed Tomlinson

Hi,

Here is a rewritten slablru - this time its not using the lru... If changes long standing slab behavior. Now slab.c releases pages as soon as possible. This was done since we noticed that slablru was taking a long time to release the pages it freed - from other vm experiences this is not a good thing.

Quoted 2 lines

Right. There remains the issue that we're ripping away constructed objects from slabs which have constructors, as Stephen points out.

I have a small optimization coded in slab. If there are not any free
slab objects I do not free the page. If we have problems with high
order slabs we can change this to be if we do not have <n> objects do not free it.

Quoted 10 lines

I doubt if that matters. slab constructors just initialise stuff. If the memory is in cache then the initialisation is negligible. If the memory is not in cache then the initialisation will pull it into cache, which is something which we needed to do anyway. And unless the slab's access pattern is extremely LIFO, chances are that most allocations will come in from part-filled slab pages anyway.

And other such waffly words ;) I'll do the global LIFO page hotlists soonl; that'll fix it up.

Quoted 8 lines

Quoted 1 line

In this patch I have tried to make as few changes as possible.

Thanks. I've shuffled the patching sequence (painful), and diddled a few things. We actually do have the "number of scanned pages" in there, so we can use that. I agree that the ratio should be
nr_scanned/total rather than nr_reclaimed/total. This way, if
nr_reclaimed < nr_scanned (page reclaim is in trouble) then we put more pressure on slabs.

OK will change this. This also means the changes to prune functions made for slablru will come back - they convert these fuctions so they age <n> object rather than purge <n>.

Quoted 12 lines

Quoted 11 lines

With this in mind I am using
the percentage of the active+inactive pages reclaimed to recover the same percentage of the pruneable caches. In slablru the affect was to age the pruneable caches by percentage of the active+inactive pages scanned - this could be done but required more code so I went used pages reclaimed.
The same choise was made about accounting of pages freed by the
shrink_<something>_memory calls.

There is also a question as to if we should only use the ZONE_DMA and ZONE_NORMAL to drive the cache shrinking. Talk with Rik on irc convinced me to go with the choise that required less code, so we use all zones.

Quoted 3 lines

OK. We could do with a `gimme_the_direct_addressed_classzone' utility anyway. It is currently open-coded in fs/buffer.c:free_more_memory(). We can just pull that out of there and use memclass() on it for this.

Ah thanks. Was wondering the best way to do this. Will read the code.

...

Quoted 12 lines

From a quick test, the shrinking rate seems quite reasonable to me. mem=512m, with twenty megs of ext2 inodes in core, a `dd' of one gigabyte (twice the size of memory) steadily pushed the ext2 inodes down to 2.5 megs (although total memory was still 9 megs - internal fragmentation of the slab).

A second 1G dd pushed it down to 1M/3M.

A third 1G dd pushed it down to .25M/1.25M

Seems OK.

Quoted 4 lines

A few things we should do later:

- We're calling prune_icache with a teeny number of inodes, many times. Would be better to batch that up a bit.

Why not move the prunes to try_to_free_pages? The should help a little to get
bigger batches of pages as will using the number of scanned pages.

...

Quoted 2 lines

But let's get the current code settled in before doing these refinements.

I can get the aging changes to you real fast if you want them. I initially coded it this way then pull the changes to reduce the code... see below

The other thing we want to be careful with is to make sure the lack of free page accounting is detected by oom - we definitly do not want to oom when slab has freed memory by try_to_free_pages does not realize it..

Quoted 20 lines

There are some usage patterns in which the dentry/inode aging might be going wrong. Try, with mem=512m

cp -a linux a
cp -a linux b
cp -a linux c

etc.

Possibly the inode/dentry cache is just being FIFO here and is doing exactly the wrong thing. But the dcache referenced-bit logic should cause the inodes in `linux' to be pinned with this test, so that should be OK. Dunno.

The above test will be hurt a bit by the aggressively lowered (10%) background writeback threshold - more reads competing with writes. Maybe I should not kick off background writeback until the dirty threshold reaches 30% if there are reads queued against the device. That's easy enough to do.

Quoted 1 line

drop-behind should help here too.

This converts the prunes in inode and dcache to age <n> entries rather than purge them. Think this is the more correct behavior. Code is from slablru.

Ed Tomlinson wrote:

Quoted 4 lines

I have a small optimization coded in slab. If there are not any free
slab objects I do not free the page. If we have problems with high
order slabs we can change this to be if we do not have <n> objects do not free it.

OK.

Quoted 3 lines

OK will change this. This also means the changes to prune functions made for slablru will come back - they convert these fuctions so they age <n> object rather than purge <n>.

That would make the slab pruning less aggressive than the code I'm testing now. I'm not sure it needs that change. Not sure...

Quoted 1 line

Ah thanks. Was wondering the best way to do this. Will read the code.

Then again, shrinking slab harder for big highmem machines is good ;)

But the prunes are miles too small at present. We go into try_to_free_pages() and reclaim 32 pages. And we also call into prune_cache() and free about 0.3 pages. It's out of whack. I'd suggest not calling out to the pruner until we want at least several pages' worth of objects.

Quoted 4 lines

The other thing we want to be careful with is to make sure the lack of free page accounting is detected by oom - we definitly do not want to oom when slab has freed memory by try_to_free_pages does not realize it..

How much memory are we talking about here? Not much I think?

Quoted 3 lines

This converts the prunes in inode and dcache to age <n> entries rather than purge them. Think this is the more correct behavior. Code is from slablru.

Makes sense (I think).

...

Quoted 2 lines

That would make the slab pruning less aggressive than the code I'm testing now. I'm not sure it needs that change. Not sure...

Well without this change slab sometimes really get hurt. It took me a while to figure out what was happening in when I coded slablru. In any case you do have the code to fix it.

Quoted 3 lines

Quoted 1 line

Ah thanks. Was wondering the best way to do this. Will read the code.

Then again, shrinking slab harder for big highmem machines is good ;)

That was Rik's comment too... Just figured it best to mention the options.

Quoted 5 lines

But the prunes are miles too small at present. We go into try_to_free_pages() and reclaim 32 pages. And we also call into prune_cache() and free about 0.3 pages. It's out of whack. I'd suggest not calling out to the pruner until we want at least several pages' worth of objects.

Agreed. I had not quite digested your last comments when I wrote this. Once we are happy I will readd the callbacks (using a second call to set the callback - btw I have some nice oak hiking sticks here...) and fix this as you sugested.

Quoted 6 lines

Quoted 4 lines

The other thing we want to be careful with is to make sure the lack of free page accounting is detected by oom - we definitly do not want to oom when slab has freed memory by try_to_free_pages does not realize it..

How much memory are we talking about here? Not much I think?

Usually not much. I do know that when Rik added my slab accounting to rmap the number of oom reports dropped. We just need to be aware there is a hole and there might be a small problem.

Quoted 4 lines

Quoted 3 lines

This converts the prunes in inode and dcache to age <n> entries rather than purge them. Think this is the more correct behavior. Code is from slablru.

Quoted 1 line

Makes sense (I think).

As I mentioned above I needed this to make slablru stable... Might be since you now limit the number of pages scanned to 2*nr_pages we can get away without this - not at all sure though. Going back the basics. Without this are we not devaluating seeks required to rebuild slab objects vs lru pages?

Right. There remains the issue that we're ripping away constructed objects from slabs which have constructors, as Stephen points out.

I doubt if that matters. slab constructors just initialise stuff. If the memory is in cache then the initialisation is negligible. If the memory is not in cache then the initialisation will pull it into cache, which is something which we needed to do anyway. And unless the slab's access pattern is extremely LIFO, chances are that most allocations will come in from part-filled slab pages anyway.

Hi Andrew,

Found three oops when checking this afternoon's log. Looks like *total_scanned can be zero...

how about;

ratio = pages > *total_scanned ? pages / (*total_scanned | 1) : 1;

Quoted 8 lines · Ed Tomlinson

Hi Andrew,

Found three oops when checking this afternoon's log. Looks like *total_scanned can be zero...

how about;

ratio = pages > *total_scanned ? pages / (*total_scanned | 1) : 1;

Yup, thanks. I went the "+ 1" route ;)

mm/vmscan.c line 502

Quoted 6 lines · Ed Tomlinson

mm/vmscan.c +6 -0

+     /*
+      * Here we assume it costs one seek to replace a lru page and that
+      * it also takes a seek to recreate a cache object.  With this in
+      * mind we age equal percentages of the lru and ageable caches.
+      * This should balance the seeks generated by these structures.
+      */

It's not a reliable assumption, for example, what is the relation between an skbuff and a disk seek?

Quoted 9 lines · Daniel Phillips

Quoted 6 lines · Ed Tomlinson

mm/vmscan.c +6 -0

+     /*
+      * Here we assume it costs one seek to replace a lru page and that
+      * it also takes a seek to recreate a cache object.  With this in
+      * mind we age equal percentages of the lru and ageable caches.
+      * This should balance the seeks generated by these structures.
+      */

Quoted 2 lines

It's not a reliable assumption, for example, what is the relation between an skbuff and a disk seek?

Of course. But all the caches in question happen to be filesystem ones. I can live with that...