Skip to contentSkip to conversation

Reset the high water marks in CPUs pcp list

PATCH In review Seth, Rohit 4 participants 11 messages Sep 28, 2005 – Sep 29, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Reset the high water marks in CPUs pcp list

Recent changes in page allocations for pcps has increased the high watermark for these lists. This has resulted in scenarios where pcp lists could be having bigger number of free pages even under low memory conditions.

[PATCH]: Reduce the high mark in cpu's pcp lists.

mm/page_alloc.c +2 -2
@@ -1859,15 +1859,15 @@
 	pcp = &p->pcp[0];		/* hot */
 	pcp->count = 0;
 	pcp->low = 0;
-	pcp->high = 6 * batch;
+	pcp->high = 4 * batch;
 	pcp->batch = max(1UL, 1 * batch);
 	INIT_LIST_HEAD(&pcp->list);
 
 	pcp = &p->pcp[1];		/* cold*/
 	pcp->count = 0;
 	pcp->low = 0;
-	pcp->high = 2 * batch;
 	pcp->batch = max(1UL, batch/2);
+	pcp->high = pcp->batch + 1;
 	INIT_LIST_HEAD(&pcp->list);
 }
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw
Quoted 3 lines · Rohit

Recent changes in page allocations for pcps has increased the high watermark for these lists. This has resulted in scenarios where pcp lists could be having bigger number of free pages even under low memory conditions.

[PATCH]: Reduce the high mark in cpu's pcp lists.

There is no need for such a patch. The pcp lists are regularly flushed. See drain_remote_pages.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#2 lore raw
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · Rohit

Recent changes in page allocations for pcps has increased the high watermark for these lists. This has resulted in scenarios where pcp lists could be having bigger number of free pages even under low memory conditions.

[PATCH]: Reduce the high mark in cpu's pcp lists.

There is no need for such a patch. The pcp lists are regularly flushed. See drain_remote_pages.

CONFIG_NUMA needs to be defined for that. And then too for flushing the remote pages. Also, when are you flushing the local pcps. Also note that this patch is just bringing the free pages on the pcp list closer to what used to be the number earlier.

-rohit

#3 lore raw
Quoted 4 lines · Rohit Seth

CONFIG_NUMA needs to be defined for that. And then too for flushing the remote pages. Also, when are you flushing the local pcps. Also note that this patch is just bringing the free pages on the pcp list closer to what used to be the number earlier.

What was the reason for the increase of those numbers?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#4 lore raw
Quoted 9 lines · Christoph Lameter
Quoted 4 lines · Rohit Seth

CONFIG_NUMA needs to be defined for that. And then too for flushing the remote pages. Also, when are you flushing the local pcps. Also note that this patch is just bringing the free pages on the pcp list closer to what used to be the number earlier.

What was the reason for the increase of those numbers?

Bugger batch size to possibly get more physical contiguous pages. That indirectly increased the high water marks for the pcps.

#5 lore raw
Quoted 11 lines · Rohit Seth
Quoted 8 lines · Christoph Lameter
Quoted 4 lines · Rohit Seth

CONFIG_NUMA needs to be defined for that. And then too for flushing the remote pages. Also, when are you flushing the local pcps. Also note that this patch is just bringing the free pages on the pcp list closer to what used to be the number earlier.

What was the reason for the increase of those numbers?

Bugger batch size to possibly get more physical contiguous pages. That indirectly increased the high water marks for the pcps.

I know that Jack and Nick did something with those counts to insure that page coloring effects are avoided. Would you comment?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#6 lore raw
Quoted 10 lines · Christoph Lameter
Quoted 8 lines · Rohit Seth
Quoted 7 lines · Christoph Lameter
Quoted 4 lines · Rohit Seth

CONFIG_NUMA needs to be defined for that. And then too for flushing the remote pages. Also, when are you flushing the local pcps. Also note that this patch is just bringing the free pages on the pcp list closer to what used to be the number earlier.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

What was the reason for the increase of those numbers?

Bugger batch size to possibly get more physical contiguous pages. That indirectly increased the high water marks for the pcps.

I know that Jack and Nick did something with those counts to insure that page coloring effects are avoided. Would you comment?

About 10% performance variation was seen from run to run with original setting with certain workloads on x86 and IA-64 platforms. And this variation came down to about 2% with new settings.

-rohit

#7 lore raw
In reply to Christoph Lameter
Quoted 4 lines · Christoph Lameter

I know that Jack and Nick did something with those counts to insure that page coloring effects are avoided. Would you comment?

The 'batch' argument to setup_pageset should be clamped to a power of 2 minus 1 (ie. 15, 31, etc), which was found to avoid the worst of the colouring problems.

pcp->high of the hotlist IMO should have been reduced to 4 anyway after its pcp->low was reduced from 2 to 0.

I don't see that there would be any problems with playing with the ->high and ->low numbers so long as they are a reasonable multiple of batch, however I would question the merit of setting the high watermark of the cold queue to ->batch + 1 (should really stay at 2*batch IMO).

Nick

Send instant messages to your online friends http://au.messenger.yahoo.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#8 lore raw
Quoted 6 lines · Nick Piggin

I don't see that there would be any problems with playing with the ->high and ->low numbers so long as they are a reasonable multiple of batch, however I would question the merit of setting the high watermark of the cold queue to ->batch + 1 (should really stay at 2*batch IMO).

I agree that this watermark is little low at this point. But that is mainly because currently we don't have a way to drain the pcps for low memory conditions. Once I add that support, I will bump up the high water marks.

Can you share a list of specific workloads that you ran earlier while fixing these numbers.

-rohit

#9 lore raw
In reply to Christoph Lameter
Quoted 8 lines · --On Wednesday, September 28, 2005 13:01:23 -0700 Christoph Lameter…
Quoted 3 lines · Rohit

Recent changes in page allocations for pcps has increased the high watermark for these lists. This has resulted in scenarios where pcp lists could be having bigger number of free pages even under low memory conditions.

[PATCH]: Reduce the high mark in cpu's pcp lists.

There is no need for such a patch. The pcp lists are regularly flushed. See drain_remote_pages.

That's only retrieving pages which have migrated off-node, is it not?

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#10 lore raw
Quoted 8 lines · Martin J. Bligh
Quoted 6 lines
Quoted 3 lines

Recent changes in page allocations for pcps has increased the high watermark for these lists. This has resulted in scenarios where pcp lists could be having bigger number of free pages even under low memory conditions.

[PATCH]: Reduce the high mark in cpu's pcp lists.

There is no need for such a patch. The pcp lists are regularly flushed. See drain_remote_pages.

That's only retrieving pages which have migrated off-node, is it not?

Its freeing all pages in off node pcps. There is no page migration in the current kernels.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>