2.6.16-ck3

PATCH In review Con Kolivas 4 participants 33 messages Apr 2, 2006 – May 18, 2006

Subsystems (2)

Memory management Page allocator

lore

Share & tools

lore

Con Kolivas

author

2006-04-02 04:01 UTC #0 lore raw

These are patches designed to improve system responsiveness and interactivity. It is configurable to any workload but the default ck patch is aimed at the desktop and cks is available with more emphasis on serverspace.

THESE INCLUDE THE PATCHES FROM 2.6.16.1 SO START WITH 2.6.16 AS YOUR BASE

Apply to 2.6.16
http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.16/2.6.16-ck3/patch-2.6.16-ck3.bz2

or server version
http://www.kernel.org/pub/linux/kernel/people/ck/patches/cks/patch-2.6.16-cks3.bz2

web:
http://kernel.kolivas.org

all patches:
http://www.kernel.org/pub/linux/kernel/people/ck/patches/

Split patches available.

Changes:

Added:
+sched-staircase14.2_15.patch
A major improvement in the staircase code fixes low I/O under heavy cpu load, makes it starvation resistant, improves behaviour under heavy "system" loads, fixes a slowdown under "compute" mode and has numerous other microoptimisations.

Modified:
-swsusp-post_resume_aggressive_swap_prefetch.patch +swsusp-post_resume_aggressive_swap_prefetch-1.patch
A minor change in the code to perform aggressive swap prefetching on swsusp resume slightly later should speed up resume time.

-2.6.16-ck2-version.patch
+2.6.16-ck3-version.patch
Version update

Cheers,
Con

Nick Piggin

2006-04-02 08:40 UTC #1 lore raw

Quoted 7 lines · Con Kolivas

These are patches designed to improve system responsiveness and interactivity. It is configurable to any workload but the default ck patch is aimed at the desktop and cks is available with more emphasis on serverspace.
THESE INCLUDE THE PATCHES FROM 2.6.16.1 SO START WITH 2.6.16 AS YOUR BASE
Apply to 2.6.16
http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.16/2.6.16-ck3/patch-2.6.16-ck3.bz2

The swap prefetching here, and the one in -mm AFAIKS still do not follow the lowmem reserve ratio correctly. This might explain why prefetching appears to help some people after updatedb swaps stuff out to make room for pagecache -- it may actually be dipping into lower zones when it shouldn't.

Signature · 2 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com

Con Kolivas

author

2006-04-02 08:51 UTC #2 lore raw

Quoted 5 lines · Nick Piggin

The swap prefetching here, and the one in -mm AFAIKS still do not follow the lowmem reserve ratio correctly. This might explain why prefetching appears to help some people after updatedb swaps stuff out to make room for pagecache -- it may actually be dipping into lower zones when it shouldn't.

Curious. I was under the impression lowmem reserve only did anything if you manually set it, and the users reporting on swap prefetch behaviour are not the sort of users likely to do so. I'm happy to fix whatever the lowmem reserve bug is but I doubt this bug is making swap prefetch behave better for ordinary users. Well, whatever the case is I'll have another look at lowmem reserve of course.

Cheers,
Con

Con Kolivas

author

2006-04-02 09:39 UTC #3 lore raw

Quoted 7 lines · Con Kolivas

Quoted 5 lines · Nick Piggin
The swap prefetching here, and the one in -mm AFAIKS still do not follow the lowmem reserve ratio correctly. This might explain why prefetching appears to help some people after updatedb swaps stuff out to make room for pagecache -- it may actually be dipping into lower zones when it shouldn't.

Quoted 6 lines

Curious. I was under the impression lowmem reserve only did anything if you manually set it, and the users reporting on swap prefetch behaviour are not the sort of users likely to do so. I'm happy to fix whatever the lowmem reserve bug is but I doubt this bug is making swap prefetch behave better for ordinary users. Well, whatever the case is I'll have another look at lowmem reserve of course.

Ok I can't see what I'm doing wrong.

here are my watermarks

idx = zone_idx(z);
ns->lowfree[idx] = z->pages_high * 3 + z->lowmem_reserve[idx]; ns->highfree[idx] = ns->lowfree[idx] + z->pages_high;

It's (3 * pages_high) +lowmem_reserve which is well in excess of the reserve so I can't see any problem. Am I missing something?

Cheers,
Con

Nick Piggin

2006-04-02 14:29 UTC #4 lore raw

Quoted 10 lines · Con Kolivas

Ok I can't see what I'm doing wrong.
here are my watermarks
idx = zone_idx(z);
ns->lowfree[idx] = z->pages_high * 3 + z->lowmem_reserve[idx]; ns->highfree[idx] = ns->lowfree[idx] + z->pages_high;
It's (3 * pages_high) +lowmem_reserve which is well in excess of the reserve so I can't see any problem. Am I missing something?

That zone->lowmem_reserve[zone_idx(zone)] == 0 ?

;)

lowmem_reserve could be much bigger than zone->high*3, when higher zones are much larger.

Signature · 2 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com

Con Kolivas

author

2006-04-03 02:48 UTC #5 lore raw

lowmem_reserve question

Quoted 1 line · Nick Piggin

That zone->lowmem_reserve[zone_idx(zone)] == 0 ?

I haven't figured out how to tackle the swap prefetch issue with lowmem reserve just yet. While trying to digest just what the lowmem_reserve does and how it's utilised I looked at some of the numbers

int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1] = { 256, 256, 32 };

lower_zone->lowmem_reserve[j] = present_pages /
sysctl_lowmem_reserve_ratio[idx];

This is interesting because there are no bounds on this value and it seems possible to set the sysctl to have a lowmem_reserve that is larger than the zone size. Ok that's a sysctl so if a user is setting it wrongly that's their fault... or should there be some upper bound?

Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.

On a basically idle 1GB lowmem box that I have it looks like this:

Node 0, zone DMA

  pages free     1025
        min      15
        low      18
        high     22
        active   2185
        inactive 0
        scanned  555 (a: 21 i: 6)
        spanned  4096
        present  4096
        protection: (0, 0, 1007, 1007)

With 3GB lowmem the default settings seem too tight to me. The way I see it, there should be some upper bounds on the lowmem reserves. Or perhaps I'm just missing something again... I'm feeling even thicker than usual.

Cheers,
Con

Mike Galbraith

2006-04-03 04:42 UTC #6 lore raw

Re: lowmem_reserve question

Quoted 3 lines · Con Kolivas

Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.

I haven't crawled around in the vm for ages, but I think that's only 16MB if you support antique cards.

-Mike

Con Kolivas

author

2006-04-03 04:48 UTC #7 lore raw

Re: lowmem_reserve question

Quoted 7 lines · Mike Galbraith

Quoted 3 lines · Con Kolivas
Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.
I haven't crawled around in the vm for ages, but I think that's only 16MB if you support antique cards.

That's what the ram is used for, but that is all the ZONE_DMA 32bit machines get, whether you use it for that purpose or not. My concern is that this will have all sorts of effects on the balancing since it will always appear almost full.

Cheers,
Con

Con Kolivas

author

2006-04-03 04:50 UTC #8 lore raw

Re: [ck] Re: lowmem_reserve question

Quoted 11 lines · Con Kolivas

Quoted 7 lines · Mike Galbraith
Quoted 3 lines · Con Kolivas
Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.
I haven't crawled around in the vm for ages, but I think that's only 16MB if you support antique cards.
That's what the ram is used for, but that is all the ZONE_DMA 32bit machines get

32bit i386 architecture I mean.

Quoted 3 lines

, whether you use it for that purpose or not. My concern is that this will have all sorts of effects on the balancing since it will always appear almost full.

Cheers,
Con

Mike Galbraith

2006-04-03 05:13 UTC #9 lore raw

Re: lowmem_reserve question

In reply to Con Kolivas

Quoted 6 lines · Con Kolivas

Quoted 5 lines · Mike Galbraith
Quoted 3 lines · Con Kolivas
Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.

Quoted 7 lines

Quoted 2 lines
I haven't crawled around in the vm for ages, but I think that's only 16MB if you support antique cards.
That's what the ram is used for, but that is all the ZONE_DMA 32bit machines get, whether you use it for that purpose or not. My concern is that this will have all sorts of effects on the balancing since it will always appear almost full.

If that dinky 16MB zone still exists, and always appears nearly full, be happy. It used to be a real PITA.

-Mike

Con Kolivas

author

2006-04-03 05:19 UTC #10 lore raw

Re: lowmem_reserve question

Quoted 10 lines · Mike Galbraith

Quoted 9 lines · Con Kolivas
Quoted 7 lines · Mike Galbraith
Quoted 3 lines · Con Kolivas
Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.
I haven't crawled around in the vm for ages, but I think that's only 16MB if you support antique cards.

Quoted 7 lines

Quoted 4 lines
That's what the ram is used for, but that is all the ZONE_DMA 32bit machines get, whether you use it for that purpose or not. My concern is that this will have all sorts of effects on the balancing since it will always appear almost full.
If that dinky 16MB zone still exists, and always appears nearly full, be happy. It used to be a real PITA.

That's not the point. If you try to do any allocation anywhere else it also checks that zone, and it will find it full (always) leading to reclaim all over the place for no good reason. This has nothing to do with actually wanting to use that space or otherwise.

Cheers,
Con

Mike Galbraith

2006-04-03 05:30 UTC #11 lore raw

Re: lowmem_reserve question

Quoted 8 lines · Con Kolivas

Quoted 2 lines · Mike Galbraith
If that dinky 16MB zone still exists, and always appears nearly full, be happy. It used to be a real PITA.
That's not the point. If you try to do any allocation anywhere else it also checks that zone, and it will find it full (always) leading to reclaim all over the place for no good reason. This has nothing to do with actually wanting to use that space or otherwise.

That doesn't make any sense. Why would you scan/reclaim if the zone is not depleted?

Like I said, I'm _way_ out of date. The problem scenario used to be that you run low on memory, dip into dinky dma zone, pin it, then grind to powder trying to find a reclaimable dma page.

-Mike

Con Kolivas

author

2006-04-04 02:35 UTC #12 lore raw

Re: [ck] lowmem_reserve question

In reply to Con Kolivas

Quoted 31 lines · Con Kolivas

While trying to digest just what the lowmem_reserve does and how it's utilised I looked at some of the numbers
int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1] = { 256, 256, 32 };
lower_zone->lowmem_reserve[j] = present_pages /
sysctl_lowmem_reserve_ratio[idx];
This is interesting because there are no bounds on this value and it seems possible to set the sysctl to have a lowmem_reserve that is larger than the zone size. Ok that's a sysctl so if a user is setting it wrongly that's their fault... or should there be some upper bound?
Furthermore, now that we have the option of up to 3GB lowmem split on 32bit we can have a default lowmem_reserve of almost 12MB (if I'm reading it right) which seems very tight with only 16MB of ZONE_DMA.
On a basically idle 1GB lowmem box that I have it looks like this:
Node 0, zone DMA
  pages free     1025
        min      15
        low      18
        high     22
        active   2185
        inactive 0
        scanned  555 (a: 21 i: 6)
        spanned  4096
        present  4096
        protection: (0, 0, 1007, 1007)

Quoted 3 lines

With 3GB lowmem the default settings seem too tight to me. The way I see it, there should be some upper bounds on the lowmem reserves. Or perhaps I'm just missing something again... I'm feeling even thicker than usual.

Silence. Low priority I guess.

If I propose a patch that might get some response. /me threatens to post a patch.

Cheers,
Con

Con Kolivas

author

2006-04-06 01:10 UTC #13 lore raw

PATCH 1/1 mm: limit lowmem_reserve

It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok always fail if the lower_zone is small enough. Impose a lower limit on the ratio to only allow 1/4 of the lower_zone size to be set as lowmem_reserve. This limit is hit in ZONE_DMA by changing the default vmsplit on i386 even without changing the default sysctl values.

Signed-off-by Con Kolivas <[email protected]>

---
 mm/page_alloc.c |   24 +++++++++++++++++++++---
 1 files changed, 21 insertions(+), 3 deletions(-)

mm/page_alloc.c +21 -3

Index: linux-2.6.17-rc1-mm1/mm/page_alloc.c
===================================================================
@@ -2566,14 +2566,32 @@ static void setup_per_zone_lowmem_reserv
 			zone->lowmem_reserve[j] = 0;
 
 			for (idx = j-1; idx >= 0; idx--) {
+				unsigned long max_reserve;
+				unsigned long reserve;
 				struct zone *lower_zone;
 
+				lower_zone = pgdat->node_zones + idx;
+				/*
+				 * Put an upper limit on the reserve at 1/4
+				 * the lower_zone size. This prevents large
+				 * zone size differences such as 3G VMSPLIT
+				 * or low sysctl values from making
+				 * zone_watermark_ok always fail. This
+				 * enforces a lower limit on the reserve_ratio
+				 */
+				max_reserve = lower_zone->present_pages / 4;
+
 				if (sysctl_lowmem_reserve_ratio[idx] < 1)
 					sysctl_lowmem_reserve_ratio[idx] = 1;
-
-				lower_zone = pgdat->node_zones + idx;
-				lower_zone->lowmem_reserve[j] = present_pages /
+				reserve = present_pages /
 					sysctl_lowmem_reserve_ratio[idx];
+				if (reserve > max_reserve) {
+					reserve = max_reserve;
+					sysctl_lowmem_reserve_ratio[idx] =
+						present_pages / max_reserve;
+				}
+
+				lower_zone->lowmem_reserve[j] = reserve;
 				present_pages += lower_zone->present_pages;
 			}
 		}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Con Kolivas

author

2006-04-06 01:29 UTC #14 lore raw

Respin: [PATCH] mm: limit lowmem_reserve

Err zone needs to have some pages too sorry.

Respin
---
It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok fail repeatedly if the lower_zone is small enough. Impose a lower limit on the ratio to only allow 1/4 of the lower_zone size to be set as lowmem_reserve. This limit is hit in ZONE_DMA by changing the default vmsplit on i386 even without changing the default sysctl values.

Signed-off-by Con Kolivas <[email protected]>

---
 mm/page_alloc.c |   24 +++++++++++++++++++++---
 1 files changed, 21 insertions(+), 3 deletions(-)

mm/page_alloc.c +21 -3

Index: linux-2.6.17-rc1-mm1/mm/page_alloc.c
===================================================================
@@ -2566,14 +2566,32 @@ static void setup_per_zone_lowmem_reserv
 			zone->lowmem_reserve[j] = 0;
 
 			for (idx = j-1; idx >= 0; idx--) {
+				unsigned long max_reserve;
+				unsigned long reserve;
 				struct zone *lower_zone;
 
+				lower_zone = pgdat->node_zones + idx;
+				/*
+				 * Put an upper limit on the reserve at 1/4
+				 * the lower_zone size. This prevents large
+				 * zone size differences such as 3G VMSPLIT
+				 * or low sysctl values from making
+				 * zone_watermark_ok always fail. This
+				 * enforces a lower limit on the reserve_ratio
+				 */
+				max_reserve = lower_zone->present_pages / 4;
+
 				if (sysctl_lowmem_reserve_ratio[idx] < 1)
 					sysctl_lowmem_reserve_ratio[idx] = 1;
-
-				lower_zone = pgdat->node_zones + idx;
-				lower_zone->lowmem_reserve[j] = present_pages /
+				reserve = present_pages /
 					sysctl_lowmem_reserve_ratio[idx];
+				if (max_reserve && reserve > max_reserve) {
+					reserve = max_reserve;
+					sysctl_lowmem_reserve_ratio[idx] =
+						present_pages / max_reserve;
+				}
+
+				lower_zone->lowmem_reserve[j] = reserve;
 				present_pages += lower_zone->present_pages;
 			}
 		}

Andrew Morton

2006-04-06 02:43 UTC #15 lore raw

Re: Respin: [PATCH] mm: limit lowmem_reserve

Quoted 3 lines · Con Kolivas

It is possible with a low enough lowmem_reserve ratio to make
zone_watermark_ok fail repeatedly if the lower_zone is small enough.

Is that actually a problem?

Con Kolivas

author

2006-04-06 02:55 UTC #16 lore raw

Re: Respin: [PATCH] mm: limit lowmem_reserve

Quoted 5 lines · Andrew Morton

Quoted 2 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make
zone_watermark_ok fail repeatedly if the lower_zone is small enough.
Is that actually a problem?

Every single call to get_page_from_freelist will call on zone reclaim. It seems a problem to me if every call to __alloc_pages will do that?

Cheers,
Con

Con Kolivas

author

2006-04-06 02:58 UTC #17 lore raw

Re: Respin: [PATCH] mm: limit lowmem_reserve

Quoted 9 lines · Con Kolivas

Quoted 5 lines · Andrew Morton
Quoted 2 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make
zone_watermark_ok fail repeatedly if the lower_zone is small enough.
Is that actually a problem?
Every single call to get_page_from_freelist will call on zone reclaim. It seems a problem to me if every call to __alloc_pages will do that?

every call to __alloc_pages of that zone I mean

Cheers,
Con

Andrew Morton

2006-04-06 03:40 UTC #18 lore raw

Re: Respin: [PATCH] mm: limit lowmem_reserve

Quoted 7 lines · Con Kolivas

Quoted 5 lines · Con Kolivas
Quoted 4 lines · Andrew Morton
Quoted 2 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make
zone_watermark_ok fail repeatedly if the lower_zone is small enough.

Quoted 7 lines

Quoted 4 lines
Quoted 1 line
Is that actually a problem?
Every single call to get_page_from_freelist will call on zone reclaim. It seems a problem to me if every call to __alloc_pages will do that?
every call to __alloc_pages of that zone I mean

One would need to check with the NUMA guys. zone_reclaim() has a (lame-looking) timer in there to prevent it from doing too much work.

That, or I'm missing something. This problem wasn't particularly well described, sorry.

Con Kolivas

author

2006-04-06 04:36 UTC #19 lore raw

Re: Respin: [PATCH] mm: limit lowmem_reserve

Quoted 13 lines · Andrew Morton

Quoted 12 lines · Con Kolivas
Quoted 10 lines · Con Kolivas
Quoted 6 lines · Andrew Morton
Quoted 3 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make
zone_watermark_ok fail repeatedly if the lower_zone is small
enough.
Is that actually a problem?
Every single call to get_page_from_freelist will call on zone reclaim. It seems a problem to me if every call to __alloc_pages will do that?

Quoted 7 lines

Quoted 1 line
every call to __alloc_pages of that zone I mean
One would need to check with the NUMA guys. zone_reclaim() has a (lame-looking) timer in there to prevent it from doing too much work.
That, or I'm missing something. This problem wasn't particularly well described, sorry.

Ah ok. This all came about because I'm trying to honour the lowmem_reserve better in swap_prefetch at Nick's request. It's hard to honour a watermark that on some configurations is never reached.

Cheers,
Con

Con Kolivas

author

2006-04-06 04:52 UTC #20 lore raw

Re: Respin: [PATCH] mm: limit lowmem_reserve

Quoted 20 lines · Con Kolivas

Quoted 19 lines · Andrew Morton
Quoted 7 lines · Con Kolivas
Quoted 6 lines · Con Kolivas
Quoted 5 lines · Andrew Morton
Quoted 3 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make
zone_watermark_ok fail repeatedly if the lower_zone is small
enough.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Is that actually a problem?
Every single call to get_page_from_freelist will call on zone reclaim. It seems a problem to me if every call to __alloc_pages will do that?
every call to __alloc_pages of that zone I mean
One would need to check with the NUMA guys. zone_reclaim() has a (lame-looking) timer in there to prevent it from doing too much work.

Quoted 6 lines

Quoted 2 lines
That, or I'm missing something. This problem wasn't particularly well described, sorry.
Ah ok. This all came about because I'm trying to honour the lowmem_reserve better in swap_prefetch at Nick's request. It's hard to honour a watermark that on some configurations is never reached.

Forget that. If the numa people don't care about it I shouldn't touch it. I thought I was doing something helpful at the source but got no response from Nick or the the other numa_ids out there so they obviously don't care. I'll tackle it differently in swap prefetch.

Cheers,
Con

Nick Piggin

2006-04-07 06:25 UTC #21 lore raw

Re: [PATCH] mm: limit lowmem_reserve

In reply to Con Kolivas

Quoted 2 lines · Con Kolivas

It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok always fail if the lower_zone is small enough.

I don't see how this would happen?

Signature · 8 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Con Kolivas

author

2006-04-07 09:02 UTC #22 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 5 lines · Nick Piggin

Quoted 2 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok always fail if the lower_zone is small enough.
I don't see how this would happen?

3GB lowmem and a reserve ratio of 180 is enough to do it.

Cheers,
Con

Nick Piggin

2006-04-07 12:40 UTC #23 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 4 lines · Con Kolivas

Quoted 2 lines · Nick Piggin
Con Kolivas wrote:

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok always fail if the lower_zone is small enough.
I don't see how this would happen?
3GB lowmem and a reserve ratio of 180 is enough to do it.

How would zone_watermark_ok always fail though?

Signature · 8 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Con Kolivas

author

2006-04-08 00:15 UTC #24 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 11 lines · Nick Piggin

Quoted 8 lines · Con Kolivas
Quoted 5 lines · Nick Piggin
Quoted 2 lines · Con Kolivas
It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok always fail if the lower_zone is small enough.
I don't see how this would happen?
3GB lowmem and a reserve ratio of 180 is enough to do it.
How would zone_watermark_ok always fail though?

Withdrew this patch a while back; ignore

Signature · 7 lines

-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nick Piggin

2006-04-08 00:55 UTC #25 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 8 lines · Con Kolivas

Quoted 1 line · Nick Piggin
How would zone_watermark_ok always fail though?
Withdrew this patch a while back; ignore

Well, whether or not that particular patch isa good idea, it is definitely a bug if zone_watermark_ok could ever always fail due to lowmem reserve and we should fix it.

Signature · 8 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Con Kolivas

author

2006-04-08 01:01 UTC #26 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 9 lines · Nick Piggin

Quoted 4 lines · Con Kolivas
Quoted 1 line · Nick Piggin
How would zone_watermark_ok always fail though?
Withdrew this patch a while back; ignore
Well, whether or not that particular patch isa good idea, it is definitely a bug if zone_watermark_ok could ever always fail due to lowmem reserve and we should fix it.

Ok. I think I presented enough information for why I thought zone_watermark_ok would fail (for ZONE_DMA). With 16MB ZONE_DMA and a vmsplit of 3GB we have a lowmem_reserve of 12MB. It's pretty hard to keep that much ZONE_DMA free, I don't think I've ever seen that much free on my ZONE_DMA on an ordinary desktop without any particular ZONE_DMA users. Changing the tunable can make the lowmem_reserve larger than ZONE_DMA is on any vmsplit too as far as I understand the ratio.

Signature · 7 lines

-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nick Piggin

2006-04-08 01:25 UTC #27 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 8 lines · Con Kolivas

Quoted 6 lines · Nick Piggin
Quoted 4 lines · Con Kolivas
Quoted 1 line · Nick Piggin
How would zone_watermark_ok always fail though?

Quoted 9 lines

Quoted 5 lines
Quoted 1 line
Withdrew this patch a while back; ignore
Well, whether or not that particular patch isa good idea, it is definitely a bug if zone_watermark_ok could ever always fail due to lowmem reserve and we should fix it.
Ok. I think I presented enough information for why I thought zone_watermark_ok would fail (for ZONE_DMA). With 16MB ZONE_DMA and a vmsplit of 3GB we have a lowmem_reserve of 12MB. It's pretty hard to keep that much ZONE_DMA free, I don't think I've ever seen that much free on my ZONE_DMA on an ordinary desktop without any particular ZONE_DMA users. Changing the tunable can make the lowmem_reserve larger than ZONE_DMA is on any vmsplit too as far as I understand the ratio.

Umm, for ZONE_DMA allocations, ZONE_DMA isn't a lower zone. So that 12MB protection should never come into it (unless it is buggy?).

Signature · 8 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Con Kolivas

author

2006-05-17 14:11 UTC #28 lore raw

Re: [PATCH] mm: limit lowmem_reserve

I hate to resuscitate this old thread, sorry but I'm still not sure we resolved it and I want to make sure this issue isn't here as I see it.

Quoted 11 lines · Nick Piggin

Quoted 7 lines · Con Kolivas
Ok. I think I presented enough information for why I thought zone_watermark_ok would fail (for ZONE_DMA). With 16MB ZONE_DMA and a vmsplit of 3GB we have a lowmem_reserve of 12MB. It's pretty hard to keep that much ZONE_DMA free, I don't think I've ever seen that much free on my ZONE_DMA on an ordinary desktop without any particular ZONE_DMA users. Changing the tunable can make the lowmem_reserve larger than ZONE_DMA is on any vmsplit too as far as I understand the ratio.
Umm, for ZONE_DMA allocations, ZONE_DMA isn't a lower zone. So that 12MB protection should never come into it (unless it is buggy?).

An i386 pc with a 3GB split will have approx

4000 pages ZONE_DMA

and lowmem reserve will set lowmem reserve to approx

0 0 3000 3000

So if we call zone_watermark_ok with zone of ZONE_DMA and a classzone_idx of a ZONE_NORMAL we will fail a zone_watermark_ok test almost always since it's almost impossible to have 3000 free ZONE_DMA pages. I believe it can happen like this:

In balance_pgdat (vmscan.c:1116) if we end up with end_zone being a ZONE_NORMAL zone, then during the scan below we (vmscan.c:1137) iterate over all zones from 0 to end_zone and (vmscan.c:1147) we end up calling

if (!zone_watermark_ok(zone, order, zone->pages_high, end_zone, 0))

which would now call zone_watermark_ok with zone being a ZONE_DMA, and end_zone being the idx of a ZONE_NORMAL.

So in summary if I'm not mistaken (and I'm good at being mistaken), if we balance pgdat and find that ZONE_NORMAL or higher needs scanning, we'll end up trying to flush the crap out of ZONE_DMA.

On my test case this indeed happens and my ZONE_DMA never goes below 3000 pages free. If I lower the reserve even further my pages free gets stuck at 3208 and can't free any more, and doesn't ever drop below that either.

Here is the patch I was proposing

---
It is possible with a low enough lowmem_reserve ratio to make zone_watermark_ok fail repeatedly if the lower_zone is small enough. Impose a lower limit on the ratio to only allow 1/4 of the lower_zone size to be set as lowmem_reserve. This limit is hit in ZONE_DMA by changing the default vmsplit on i386 even without changing the default sysctl values.

Signed-off-by Con Kolivas <[email protected]>

---
 mm/page_alloc.c |   24 +++++++++++++++++++++---
 1 files changed, 21 insertions(+), 3 deletions(-)

mm/page_alloc.c +21 -3

Index: linux-2.6.17-rc1-mm1/mm/page_alloc.c
===================================================================
@@ -2566,14 +2566,32 @@ static void setup_per_zone_lowmem_reserv
 			zone->lowmem_reserve[j] = 0;
 
 			for (idx = j-1; idx >= 0; idx--) {
+				unsigned long max_reserve;
+				unsigned long reserve;
 				struct zone *lower_zone;
 
+				lower_zone = pgdat->node_zones + idx;
+				/*
+				 * Put an upper limit on the reserve at 1/4
+				 * the lower_zone size. This prevents large
+				 * zone size differences such as 3G VMSPLIT
+				 * or low sysctl values from making
+				 * zone_watermark_ok always fail. This
+				 * enforces a lower limit on the reserve_ratio
+				 */
+				max_reserve = lower_zone->present_pages / 4;
+
 				if (sysctl_lowmem_reserve_ratio[idx] < 1)
 					sysctl_lowmem_reserve_ratio[idx] = 1;
-
-				lower_zone = pgdat->node_zones + idx;
-				lower_zone->lowmem_reserve[j] = present_pages /
+				reserve = present_pages /
 					sysctl_lowmem_reserve_ratio[idx];
+				if (max_reserve && reserve > max_reserve) {
+					reserve = max_reserve;
+					sysctl_lowmem_reserve_ratio[idx] =
+						present_pages / max_reserve;
+				}
+
+				lower_zone->lowmem_reserve[j] = reserve;
 				present_pages += lower_zone->present_pages;
 			}
 		}

Signature · 7 lines

-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nick Piggin

2006-05-18 07:11 UTC #29 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 2 lines · Con Kolivas

I hate to resuscitate this old thread, sorry but I'm still not sure we resolved it and I want to make sure this issue isn't here as I see it.

OK, reclaim is slightly different.

Quoted 26 lines · Nick Piggin

Quoted 11 lines
Quoted 7 lines · Con Kolivas
Ok. I think I presented enough information for why I thought zone_watermark_ok would fail (for ZONE_DMA). With 16MB ZONE_DMA and a vmsplit of 3GB we have a lowmem_reserve of 12MB. It's pretty hard to keep that much ZONE_DMA free, I don't think I've ever seen that much free on my ZONE_DMA on an ordinary desktop without any particular ZONE_DMA users. Changing the tunable can make the lowmem_reserve larger than ZONE_DMA is on any vmsplit too as far as I understand the ratio.
Quoted 2 lines
Umm, for ZONE_DMA allocations, ZONE_DMA isn't a lower zone. So that 12MB protection should never come into it (unless it is buggy?).
An i386 pc with a 3GB split will have approx
4000 pages ZONE_DMA
and lowmem reserve will set lowmem reserve to approx
0 0 3000 3000
So if we call zone_watermark_ok with zone of ZONE_DMA and a classzone_idx of a ZONE_NORMAL we will fail a zone_watermark_ok test almost always since it's almost impossible to have 3000 free ZONE_DMA pages. I believe it can happen like this:

Quoted 7 lines

In balance_pgdat (vmscan.c:1116) if we end up with end_zone being a ZONE_NORMAL zone, then during the scan below we (vmscan.c:1137) iterate over all zones from 0 to end_zone and (vmscan.c:1147) we end up calling
if (!zone_watermark_ok(zone, order, zone->pages_high, end_zone, 0))
which would now call zone_watermark_ok with zone being a ZONE_DMA, and end_zone being the idx of a ZONE_NORMAL.
So in summary if I'm not mistaken (and I'm good at being mistaken), if we balance pgdat and find that ZONE_NORMAL or higher needs scanning, we'll end up trying to flush the crap out of ZONE_DMA.

If we're under memory pressure, kswapd will try to free up any candidate zone, yes.

Quoted 6 lines

On my test case this indeed happens and my ZONE_DMA never goes below 3000 pages free. If I lower the reserve even further my pages free gets stuck at 3208 and can't free any more, and doesn't ever drop below that either.
Here is the patch I was proposing

What problem does that fix though?

Signature · 8 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Con Kolivas

author

2006-05-18 07:21 UTC #30 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 10 lines · Nick Piggin

If we're under memory pressure, kswapd will try to free up any candidate zone, yes.
Quoted 5 lines
On my test case this indeed happens and my ZONE_DMA never goes below 3000 pages free. If I lower the reserve even further my pages free gets stuck at 3208 and can't free any more, and doesn't ever drop below that either.
Here is the patch I was proposing
What problem does that fix though?

It's a generic concern and I honestly don't know how significant it is which is why I'm asking if it needs attention. That concern being that any time we're under any sort of memory pressure, ZONE_DMA will undergo intense reclaim even though there may not really be anything specifically going on in ZONE_DMA. It just seems a waste of cycles doing that.

Signature · 7 lines

-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nick Piggin

2006-05-18 07:26 UTC #31 lore raw

Re: [PATCH] mm: limit lowmem_reserve

Quoted 10 lines · Con Kolivas

Quoted 8 lines · Nick Piggin
If we're under memory pressure, kswapd will try to free up any candidate zone, yes.
Quoted 4 lines
On my test case this indeed happens and my ZONE_DMA never goes below 3000 pages free. If I lower the reserve even further my pages free gets stuck at 3208 and can't free any more, and doesn't ever drop below that either.

Quoted 7 lines

Quoted 3 lines
Quoted 1 line
Here is the patch I was proposing
What problem does that fix though?
It's a generic concern and I honestly don't know how significant it is which is why I'm asking if it needs attention. That concern being that any time we're under any sort of memory pressure, ZONE_DMA will undergo intense reclaim even though there may not really be anything specifically going on in ZONE_DMA. It just seems a waste of cycles doing that.

If it doesn't have any/much pagecache or slab cache in it, there won't be intense reclaim; if it does then it can be reclaimed and the memory used.

reclaim / allocation could be slightly smarter about scaling watermarks, however I don't think it is much of an issue at the moment.

Signature · 8 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nick Piggin

2006-04-02 14:09 UTC #32 lore raw

In reply to Con Kolivas

Quoted 3 lines · Con Kolivas

On Sunday 02 April 2006 14:46, Nick Piggin wrote:
Curious. I was under the impression lowmem reserve only did anything if you manually set it, and the users reporting on swap prefetch behaviour are not the sort of users likely to do so. I'm happy to fix whatever the lowmem

It is enabled by default for over a year.

Quoted 2 lines

reserve bug is but I doubt this bug is making swap prefetch behave better for ordinary users. Well, whatever the case is I'll have another look at lowmem reserve of course.

It would potentially make swap prefetch very happy to swap pages into the dma zone and the normal zone on highmem systems when the system is otherwise full of pagecache. So it might easily change behaviour on those systems.

Signature · 2 lines

SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com