mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

2. numactl -N +0 memhog 0.5*Mg
This start a memory-using application in node 0.

The expected behaviour is that the clean pages get reclaimed and the application uses node 0 for its memory. The observed behaviour was that the memory for the memhog application was allocated off-node since commits cd38b11 (mm: page allocator: initialise ZLC for first zone eligible for zone_reclaim) and commit 76d3fbf (mm: page allocator: reconsider zones for allocation after direct reclaim).

The assumption of those patches was that it was always preferable to allocate quickly than stall for long periods of time and they were meant to take care that the zone was only marked full when necessary but an important case was missed.

In the allocator fast path, only the low watermarks are checked. If the zones free pages are between the low and min watermark then allocations from the allocators slow path will succeed. However, zone_reclaim will only reclaim SWAP_CLUSTER_MAX or 1<<order pages. There is no guarantee that this will meet the low watermark causing the zone to be marked full prematurely.

This patch will only mark the zone full after zone_reclaim if it the min watermarks are checked or if page reclaim failed to make sufficient progress.

Reported-and-tested-by Hedi Berriche <[email protected]>

Signed-off-by Mel Gorman <[email protected]>

---
 mm/page_alloc.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

mm/page_alloc.c +16 -1

index 8fcced7..adce823 100644
@@ -1940,9 +1940,24 @@ zonelist_scan:
 				continue;
 			default:
 				/* did we reclaim enough */
-				if (!zone_watermark_ok(zone, order, mark,
+				if (zone_watermark_ok(zone, order, mark,
 						classzone_idx, alloc_flags))
+					goto try_this_zone;
+
+				/*
+				 * Failed to reclaim enough to meet watermark.
+				 * Only mark the zone full if checking the min
+				 * watermark or if we failed to reclaim just
+				 * 1<<order pages or else the page allocator
+				 * fastpath will prematurely mark zones full
+				 * when the watermark is between the low and
+				 * min watermarks.
+				 */
+				if ((alloc_flags & ALLOC_WMARK_MIN) ||
+				    ret == ZONE_RECLAIM_SOME)
 					goto this_zone_full;
+
+				continue;
 			}
 		}
 

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

2. numactl -N +0 memhog 0.5*Mg
This start a memory-using application in node 0.

The expected behaviour is that the clean pages get reclaimed and the application uses node 0 for its memory. The observed behaviour was that the memory for the memhog application was allocated off-node since commits cd38b11 (mm: page allocator: initialise ZLC for first zone eligible for zone_reclaim) and commit 76d3fbf (mm: page allocator: reconsider zones for allocation after direct reclaim).

The assumption of those patches was that it was always preferable to allocate quickly than stall for long periods of time and they were meant to take care that the zone was only marked full when necessary but an important case was missed.

In the allocator fast path, only the low watermarks are checked. If the zones free pages are between the low and min watermark then allocations from the allocators slow path will succeed. However, zone_reclaim will only reclaim SWAP_CLUSTER_MAX or 1<<order pages. There is no guarantee that this will meet the low watermark causing the zone to be marked full prematurely.

This patch will only mark the zone full after zone_reclaim if it the min watermarks are checked or if page reclaim failed to make sufficient progress.

Reported-and-tested-by Hedi Berriche <[email protected]>

Signed-off-by Mel Gorman <[email protected]>

---
mm/page_alloc.c | 17 ++++++++++++++++-

1 file changed, 16 insertions(+), 1 deletion(-)

mm/page_alloc.c +16 -2

index 8fcced7..adce823 100644
@@ -1940,9 +1940,24 @@ zonelist_scan:
 				continue;
 			default:
 				/* did we reclaim enough */
-				if (!zone_watermark_ok(zone, order, mark,
+				if (zone_watermark_ok(zone, order, mark,
 						classzone_idx, alloc_flags))
+					goto try_this_zone;
+
+				/*
+				 * Failed to reclaim enough to meet watermark.
+				 * Only mark the zone full if checking the min
+				 * watermark or if we failed to reclaim just
+				 * 1<<order pages or else the page allocator
+				 * fastpath will prematurely mark zones full
+				 * when the watermark is between the low and
+				 * min watermarks.
+				 */
+				if ((alloc_flags & ALLOC_WMARK_MIN) ||
+				    ret == ZONE_RECLAIM_SOME)
 					goto this_zone_full;
+
+				continue;
 			}
 		}
 
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

2. numactl -N +0 memhog 0.5*Mg
This start a memory-using application in node 0.

The expected behaviour is that the clean pages get reclaimed and the application uses node 0 for its memory. The observed behaviour was that the memory for the memhog application was allocated off-node since commits cd38b11 (mm: page allocator: initialise ZLC for first zone eligible for zone_reclaim) and commit 76d3fbf (mm: page allocator: reconsider zones for allocation after direct reclaim).

The assumption of those patches was that it was always preferable to allocate quickly than stall for long periods of time and they were meant to take care that the zone was only marked full when necessary but an important case was missed.

In the allocator fast path, only the low watermarks are checked. If the zones free pages are between the low and min watermark then allocations from the allocators slow path will succeed. However, zone_reclaim will only reclaim SWAP_CLUSTER_MAX or 1<<order pages. There is no guarantee that this will meet the low watermark causing the zone to be marked full prematurely.

This patch will only mark the zone full after zone_reclaim if it the min watermarks are checked or if page reclaim failed to make sufficient progress.

Reported-and-tested-by Hedi Berriche <[email protected]>

Signed-off-by Mel Gorman <[email protected]>

---
mm/page_alloc.c | 17 ++++++++++++++++-

1 file changed, 16 insertions(+), 1 deletion(-)

mm/page_alloc.c +16 -2

index 8fcced7..adce823 100644
@@ -1940,9 +1940,24 @@ zonelist_scan:
 				continue;
 			default:
 				/* did we reclaim enough */
-				if (!zone_watermark_ok(zone, order, mark,
+				if (zone_watermark_ok(zone, order, mark,
 						classzone_idx, alloc_flags))
+					goto try_this_zone;
+
+				/*
+				 * Failed to reclaim enough to meet watermark.
+				 * Only mark the zone full if checking the min
+				 * watermark or if we failed to reclaim just
+				 * 1<<order pages or else the page allocator
+				 * fastpath will prematurely mark zones full
+				 * when the watermark is between the low and
+				 * min watermarks.
+				 */
+				if ((alloc_flags & ALLOC_WMARK_MIN) ||
+				    ret == ZONE_RECLAIM_SOME)
 					goto this_zone_full;
+
+				continue;
 			}
 		}
 
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

2. numactl -N +0 memhog 0.5*Mg
This start a memory-using application in node 0.

The expected behaviour is that the clean pages get reclaimed and the application uses node 0 for its memory. The observed behaviour was that the memory for the memhog application was allocated off-node since commits cd38b11 (mm: page allocator: initialise ZLC for first zone eligible for zone_reclaim) and commit 76d3fbf (mm: page allocator: reconsider zones for allocation after direct reclaim).

The assumption of those patches was that it was always preferable to allocate quickly than stall for long periods of time and they were meant to take care that the zone was only marked full when necessary but an important case was missed.

In the allocator fast path, only the low watermarks are checked. If the zones free pages are between the low and min watermark then allocations from the allocators slow path will succeed. However, zone_reclaim will only reclaim SWAP_CLUSTER_MAX or 1<<order pages. There is no guarantee that this will meet the low watermark causing the zone to be marked full prematurely.

This patch will only mark the zone full after zone_reclaim if it the min watermarks are checked or if page reclaim failed to make sufficient progress.

Reported-and-tested-by Hedi Berriche <[email protected]>

Signed-off-by Mel Gorman <[email protected]>

---
  mm/page_alloc.c | 17 ++++++++++++++++-
  1 file changed, 16 insertions(+), 1 deletion(-)

mm/page_alloc.c +16 -2

index 8fcced7..adce823 100644
@@ -1940,9 +1940,24 @@ zonelist_scan:
  				continue;
  			default:
  				/* did we reclaim enough */
-				if (!zone_watermark_ok(zone, order, mark,
+				if (zone_watermark_ok(zone, order, mark,
  						classzone_idx, alloc_flags))
+					goto try_this_zone;
+
+				/*
+				 * Failed to reclaim enough to meet watermark.
+				 * Only mark the zone full if checking the min
+				 * watermark or if we failed to reclaim just
+				 * 1<<order pages or else the page allocator
+				 * fastpath will prematurely mark zones full
+				 * when the watermark is between the low and
+				 * min watermarks.
+				 */
+				if ((alloc_flags & ALLOC_WMARK_MIN) ||
+				    ret == ZONE_RECLAIM_SOME)
  					goto this_zone_full;
+
+				continue;
  			}
  		}
 
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hi Mel,

Quoted 6 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

Quoted 6 lines · Simon Jeons

Hi Mel,

Quoted 4 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

Quoted 3 lines

Quoted 2 lines

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

Hi Michal,

Quoted 9 lines · Michal Hocko

Quoted 8 lines · Simon Jeons

Hi Mel,

Quoted 6 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

Quoted 5 lines

Quoted 1 line

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

Thanks for your clarify Michal.

dd will use page cache instead of direct IO?

Where can I got dd source codes?

One offline question, when should use page cache and when should use direct IO?

Hi Michal,

Quoted 7 lines · Michal Hocko

Quoted 6 lines · Simon Jeons

Hi Mel,

Quoted 4 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

Quoted 7 lines

Quoted 3 lines

Quoted 2 lines

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

Thanks for your clarify Michal.
dd will use page cache instead of direct IO? Where can I got dd source codes?
One offline question, when should use page cache and when should use direct IO?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hi Michal,

Quoted 7 lines · Michal Hocko

Quoted 6 lines · Simon Jeons

Hi Mel,

Quoted 4 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

Quoted 7 lines

Quoted 3 lines

Quoted 2 lines

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

Thanks for your clarify Michal.
dd will use page cache instead of direct IO? Where can I got dd source codes?
One offline question, when should use page cache and when should use direct IO?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 6 lines · Simon Jeons

Hi Mel,

Quoted 4 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

Quoted 3 lines

Quoted 2 lines

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

Hi Michal,

Quoted 7 lines · Michal Hocko

Quoted 6 lines · Simon Jeons

Hi Mel,

Quoted 4 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

Quoted 7 lines

Quoted 3 lines

Quoted 2 lines

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

I try this in v3.9-rc5:
dd if=/dev/sda of=/dev/null bs=1MB
14813+0 records in
14812+0 records out
14812000000 bytes (15 GB) copied, 105.988 s, 140 MB/s

free -m -s 1

total used free shared buffers
cached

Mem:          7912       1181       6731          0 663        239
-/+ buffers/cache:        277       7634
Swap:         8011          0       8011

It seems that almost 15GB copied before I stop dd, but the used pages which I monitor during dd always around 1200MB. Weird, why?

Hi Michal,

Quoted 7 lines · Michal Hocko

Quoted 6 lines · Simon Jeons

Hi Mel,

Quoted 4 lines · Mel Gorman

The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows

Quoted 7 lines

Quoted 3 lines

Quoted 2 lines

1. Run numactl -m +0 dd if=largefile of=/dev/null This allocates a large number of clean pages in node 0

I confuse why this need allocate a large number of clean pages?

It reads from file and puts pages into the page cache. The pages are not modified so they are clean. Output file is /dev/null so no pages are written. dd doesn't call fadvise(POSIX_FADV_DONTNEED) on the input file by default so pages from the file stay in the page cache

I try this in v3.9-rc5:
dd if=/dev/sda of=/dev/null bs=1MB
14813+0 records in
14812+0 records out
14812000000 bytes (15 GB) copied, 105.988 s, 140 MB/s

free -m -s 1

total used free shared buffers
cached

Mem:          7912       1181       6731          0 663        239
-/+ buffers/cache:        277       7634
Swap:         8011          0       8011

It seems that almost 15GB copied before I stop dd, but the used pages which I monitor during dd always around 1200MB. Weird, why?

Quoted 14 lines

I try this in v3.9-rc5:
dd if=/dev/sda of=/dev/null bs=1MB
14813+0 records in
14812+0 records out
14812000000 bytes (15 GB) copied, 105.988 s, 140 MB/s

free -m -s 1

total used free shared buffers
cached

Mem:          7912       1181       6731          0 663        239
-/+ buffers/cache:        277       7634
Swap:         8011          0       8011

Quoted 3 lines

It seems that almost 15GB copied before I stop dd, but the used pages which I monitor during dd always around 1200MB. Weird, why?

Sorry for waste your time, but the test result is weird, is it?

On Tue 09-04-13 18:05:30, Simon Jeons wrote:
[...]

Quoted 14 lines

Quoted 14 lines

I try this in v3.9-rc5:
dd if=/dev/sda of=/dev/null bs=1MB
14813+0 records in
14812+0 records out
14812000000 bytes (15 GB) copied, 105.988 s, 140 MB/s

free -m -s 1

total used free shared buffers
cached

Mem:          7912       1181       6731          0 663        239
-/+ buffers/cache:        277       7634
Swap:         8011          0       8011

Quoted 4 lines

Quoted 3 lines

It seems that almost 15GB copied before I stop dd, but the used pages which I monitor during dd always around 1200MB. Weird, why?

Sorry for waste your time, but the test result is weird, is it?

I am not sure which values you have been watching but you have to realize that you are reading a _partition_ not a file and those pages go into buffers rather than the page chache.

On Tue 09-04-13 18:05:30, Simon Jeons wrote:
[...]

Quoted 14 lines

Quoted 14 lines

I try this in v3.9-rc5:
dd if=/dev/sda of=/dev/null bs=1MB
14813+0 records in
14812+0 records out
14812000000 bytes (15 GB) copied, 105.988 s, 140 MB/s

free -m -s 1

total used free shared buffers
cached

Mem:          7912       1181       6731          0 663        239
-/+ buffers/cache:        277       7634
Swap:         8011          0       8011

Quoted 4 lines

Quoted 3 lines

It seems that almost 15GB copied before I stop dd, but the used pages which I monitor during dd always around 1200MB. Weird, why?

Sorry for waste your time, but the test result is weird, is it?

I am not sure which values you have been watching but you have to realize that you are reading a _partition_ not a file and those pages go into buffers rather than the page chache.