Autoregulate vm swappiness 2.6.0-test8
Subsystems (4)
PATCH 1/1 Autoregulate vm swappiness 2.6.0-test8
The vm_swappiness dial in 2.6 was never quite the right setting without me constantly changing it depending on the workload. If I was copying large files or encoding video it was best at 0. If I was using lots of applications it was best much higher. Furthermore it depended on the amount of ram in the machine I was using. This patch was done just for fun a while back but it turned out to be quite effectual so I thought I'd make it available for the wider community to play with. Do whatever you like with it.
This patch autoregulates the vm_swappiness dial in 2.6 by making it equal to the percentage of physical ram consumed by application pages.
This has the effect of preventing applications from being swapped out if the ram is filling up with cached data.
Conversely, if many applications are in ram the swappiness increases which means the application currently in use gets to stay in physical ram while other less used applications are swapped out.
For desktop enthusiasts this means if you are copying large files around like ISO images or leave your machine unattended for a while it will not swap out your applications. Conversely if the machine has a lot of applications currently loaded it will give the currently running applications preference and swap out the less used ones.
The performance effect on larger boxes seems to be either unchanged or slight improvement (1%) in database benchmarks.
The value in vm_swappiness is updated only when the vm is under pressure to swap and you can check the last vm_swappiness value under pressure by cat /proc/sys/vm/swappiness
Manually setting the swappiness with this patch in situ has no effect. This patch has been heavily tested without noticable harm. Note I am not sure of the best way to do this so it may look rather crude.
Patch against 2.6.0-test8
Con
mm/vmscan.c +13 -1
@@ -47,7 +47,7 @@/* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness =60 ; +int vm_swappiness =0 ;static long total_memory;#ifdef ARCH_HAS_PREFETCH
@@ -595,11 +595,13 @@ refill_inactive_zone(struct zone *zone,int pgmoved;int pgdeactivate =0 ;int nr_pages = nr_pages_in; +int pg_size;LIST_HEAD (l_hold);/* The pages which were snipped off */ LIST_HEAD (l_inactive);/* Pages to go onto the inactive_list */ LIST_HEAD (l_active);/* Pages to go onto the active_list */ struct page *page;struct pagevec pvec; +struct sysinfo i;int reclaim_mapped =0 ;long mapped_ratio;long distress;
@@ -642,6 +644,16 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped *100 ) / total_memory;/* +* Autoregulate vm_swappiness to be application pages % -ck. +*/ +si_meminfo (&i); +si_swapinfo (&i); + pg_size =get_page_cache_size () - i.bufferram ; + vm_swappiness =100 - (((i.freeram + i.bufferram + + (pg_size - swapper_space.nrpages)) *100 ) / + (i.totalram ? i.totalram :1 )); + +/* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding.
mm/vmscan.c refill_inactive_zone(struct zone *zone, · line 656
Quoted 11 lines
mm/vmscan.c +10
-0+* Autoregulate vm_swappiness to be application pages % -ck. +*/ +si_meminfo (&i); +si_swapinfo (&i); + pg_size =get_page_cache_size () - i.bufferram ; + vm_swappiness =100 - (((i.freeram + i.bufferram + + (pg_size - swapper_space.nrpages)) *100 ) / + (i.totalram ? i.totalram :1 )); + +/*
It seems that you don't need si_swapinfo here, do you? i.freeram, i.bufferram, and i.totalram all come from meminfo, as far as I can see? Maybe I'm missing a bit ...
M.
Quoted 6 lines · Martin J. Bligh
Quoted 6 lines
(quoted diff) +6
-0+ * Autoregulate vm_swappiness to be application pages % -ck. + */ + si_meminfo(&i); + si_swapinfo(&i); + pg_size = get_page_cache_size() - i.bufferram ; + vm_swappiness = 100 - (((i.freeram + i.bufferram +
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ (pg_size - swapper_space.nrpages)) * 100) / + (i.totalram ? i.totalram : 1)); + + /*It seems that you don't need
si_swapinfohere, do you? i.freeram, i.bufferram, and i.totalram all come from meminfo, as far as I can see? Maybe I'm missing a bit ...
Well I did do it a while ago and it seems I got carried away adding and subtracting info indeed. :-) Here's a simpler patch that does the same thing.
Con
mm/vmscan.c +9 -1
@@ -47,7 +47,7 @@/* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness =60 ; +int vm_swappiness =0 ;static long total_memory;#ifdef ARCH_HAS_PREFETCH
@@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone,LIST_HEAD (l_active);/* Pages to go onto the active_list */ struct page *page;struct pagevec pvec; +struct sysinfo i;int reclaim_mapped =0 ;long mapped_ratio;long distress;
@@ -642,6 +643,13 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped *100 ) / total_memory;/* +* Autoregulate vm_swappiness to be application pages% -ck +*/ +si_meminfo (&i); + vm_swappiness =100 - (((i.freeram +get_page_cache_size () - + swapper_space.nrpages) *100 ) / (i.totalram ? i.totalram :1 )); + +/* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding.
PATCH 1/1 Autoregulate vm swappiness cleanup
Quoted 8 lines · Con Kolivas
Quoted 3 lines · Martin J. Bligh
It seems that you don't need
si_swapinfohere, do you? i.freeram, i.bufferram, and i.totalram all come from meminfo, as far as I can see? Maybe I'm missing a bit ...Well I did do it a while ago and it seems I got carried away adding and subtracting info indeed. :-) Here's a simpler patch that does the same thing.
The off-list enthusiasm has been rather strong so here is a patch done the right way (tm). There is no need for the check of totalram being zero (the original version of this patch modified the swappiness every tick which was wasteful and had a divide by zero on init). Adjusting vm_swappiness only when there is pressure to swap means totalram shouldn't be ever be zero. The sysctl is made read only since writing to it would be ignored now.
Con
kernel/sysctl.c +2 -5
@@ -664,11 +664,8 @@ static ctl_table vm_table[] = { .procname = "swappiness", .data = &vm_swappiness, .maxlen = sizeof(vm_swappiness), - .mode = 0644, - .proc_handler = &proc_dointvec_minmax, - .strategy = &sysctl_intvec, - .extra1 = &zero, - .extra2 = &one_hundred, + .mode = 0444 /* read-only*/, + .proc_handler = &proc_dointvec, },#ifdef CONFIG_HUGETLB_PAGE {
mm/vmscan.c +10 -1
@@ -47,7 +47,7 @@/* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness =60 ; +int vm_swappiness =0 ;static long total_memory;#ifdef ARCH_HAS_PREFETCH
@@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone,LIST_HEAD (l_active);/* Pages to go onto the active_list */ struct page *page;struct pagevec pvec; +struct sysinfo i;int reclaim_mapped =0 ;long mapped_ratio;long distress;
@@ -642,6 +643,14 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped *100 ) / total_memory;/* +* Autoregulate vm_swappiness to be equal to the percentage of +* pages in physical ram that are application pages. -ck +*/ +si_meminfo (&i); + vm_swappiness =100 - (((i.freeram +get_page_cache_size () - + swapper_space.nrpages) *100 ) / i.totalram); + +/* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding.
Re: [PATCH] Autoregulate vm swappiness cleanup
Quoted 62 lines · Con Kolivas
Quoted 10 lines · Con Kolivas
Quoted 4 lines · Martin J. Bligh
It seems that you don't need
si_swapinfohere, do you? i.freeram, i.bufferram, and i.totalram all come from meminfo, as far as I can see? Maybe I'm missing a bit ...Well I did do it a while ago and it seems I got carried away adding and subtracting info indeed. :-) Here's a simpler patch that does the same thing.
The off-list enthusiasm has been rather strong so here is a patch done the right way (tm). There is no need for the check of totalram being zero (the original version of this patch modified the swappiness every tick which was wasteful and had a divide by zero on init). Adjusting
vm_swappinessonly when there is pressure to swap means totalram shouldn't be ever be zero. The sysctl is made read only since writing to it would be ignored now.Con
------------------------------------------------------------------------
kernel/sysctl.c +2
-5@@ -664,11 +664,8 @@ static ctl_table vm_table[] = { .procname = "swappiness", .data = &vm_swappiness, .maxlen = sizeof(vm_swappiness), - .mode = 0644, - .proc_handler = &proc_dointvec_minmax, - .strategy = &sysctl_intvec, - .extra1 = &zero, - .extra2 = &one_hundred, + .mode = 0444 /* read-only*/, + .proc_handler = &proc_dointvec, },#ifdef CONFIG_HUGETLB_PAGE {mm/vmscan.c +3
-1@@ -47,7 +47,7 @@/* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness =60 ; +int vm_swappiness =0 ;static long total_memory;#ifdef ARCH_HAS_PREFETCH @@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone,LIST_HEAD (l_active);/* Pages to go onto the active_list */ struct page *page;struct pagevec pvec; +struct sysinfo i;int reclaim_mapped =0 ;long mapped_ratio;long distress;@@ -642,6 +643,14 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped *100 ) / total_memory;/* +* Autoregulate vm_swappiness to be equal to the percentage of
mm/vmscan.c refill_inactive_zone(struct zone *zone, · line 656
Quoted 10 lines
mm/vmscan.c +7
-0+* pages in physical ram that are application pages. -ck +*/ +si_meminfo (&i); + vm_swappiness =100 - (((i.freeram +get_page_cache_size () - + swapper_space.nrpages) *100 ) / i.totalram); + +/* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding.
Hi Con,
If this indeed makes VM behaviour better, why not just merge the calculation with the swap_tendancy calculation and leave vm_swappiness there as a tunable?
Re: [PATCH] Autoregulate vm swappiness cleanup
Quoted 4 lines · Nick Piggin
Hi Con,
If this indeed makes VM behaviour better, why not just merge the calculation with theswap_tendancycalculation and leavevm_swappinessthere as a tunable?
Because the whole point of it is to remove the tunable and make it auto tuning. We could do away with the vm_swappiness variable altogether too (which I would actually prefer to do) but this leaves it intact to see what the vm is doing.
Con
Re: [PATCH] Autoregulate vm swappiness cleanup
Quoted 9 lines · Con Kolivas
Quoted 4 lines · Nick Piggin
Hi Con,
If this indeed makes VM behaviour better, why not just merge the calculation with theswap_tendancycalculation and leavevm_swappinessthere as a tunable?Because the whole point of it is to remove the tunable and make it auto tuning. We could do away with the
vm_swappinessvariable altogether too (which I would actually prefer to do) but this leaves it intact to see what the vm is doing.
Right. This just had me a bit confused. No worries.
Re: [PATCH] Autoregulate vm swappiness cleanup
In reply to Con KolivasHi!
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
It seems that you don't need
si_swapinfohere, do you? i.freeram, i.bufferram, and i.totalram all come from meminfo, as far as I can see? Maybe I'm missing a bit ...Well I did do it a while ago and it seems I got carried away adding and subtracting info indeed. :-) Here's a simpler patch that does the same thing.
Quoted 6 lines
The off-list enthusiasm has been rather strong so here is a patch done the right way (tm). There is no need for the check of totalram being zero (the original version of this patch modified the swappiness every tick which was wasteful and had a divide by zero on init). Adjusting
vm_swappinessonly when there is pressure to swap means totalram shouldn't be ever be zero. The sysctl is made read only since writing to it would be ignored now.
I believe swappiness == 100 was "I want max throughput, I don't care about latency going through roof", while swappiness == 0 was "I don't want you to swap too much, behave reasonably".
As you don't know if user cares about latency or not, I don't see how you can autotune this.
Signature · 2 lines
When do you have a heart between your knees? [Johanka's followup: and *two* hearts?]
Re: [PATCH] Autoregulate vm swappiness cleanup
Quoted 1 line · Pavel Machek
Hi!
Hello!
Quoted 6 lines
I believe swappiness == 100 was "I want max throughput, I don't care about latency going through roof", while swappiness == 0 was "I don't want you to swap too much, behave reasonably".
As you don't know if user cares about latency or not, I don't see how you can autotune this.
Well I guess you either see merit in what my patch does based on what I said, or you don't... so I guess you don't. That's fine; I just offered why I felt this helped in my varied workloads more than a static value did.
Con