vmscan: promote shared file mapped pages

Logic added in commit v2.6.30-5507-g8cab475
(vmscan: make mapped executable pages the first class citizen) was noticeably weakened in commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)

Currently these pages can become "first class citizens" only after second usage.

After this patch page_check_references() will activate they after first usage, and executable code gets yet better chance to stay in memory.

TODO:
run some cool tests like in v2.6.30-5507-g8cab475 =)

Signed-off-by Konstantin Khlebnikov <[email protected]>

---

Quoted 7 lines · Konstantin Khlebnikov

Logic added in commit v2.6.30-5507-g8cab475
(vmscan: make mapped executable pages the first class citizen) was noticeably weakened in commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)

Currently these pages can become "first class citizens" only after second usage.

Quoted 8 lines

After this patch page_check_references() will activate they after first usage, and executable code gets yet better chance to stay in memory.

TODO:
run some cool tests like in v2.6.30-5507-g8cab475 =)

Signed-off-by Konstantin Khlebnikov <[email protected]>

---

It might be a very controversial topic.
AFAIR, at least, we did when vmscan: make mapped executable pages the first class citizen was merged. :)

You try to change behavior.

Old : protect *working set* executable page
New: protect executable page *unconditionally*.

Quoted 18 lines · On Tue, 9 Aug 2011 09:02:28 +0900 Minchan Kim <[email protected]> wrote:

Quoted 7 lines · Konstantin Khlebnikov

Logic added in commit v2.6.30-5507-g8cab475
(vmscan: make mapped executable pages the first class citizen) was noticeably weakened in commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)

Currently these pages can become "first class citizens" only after second usage.

Quoted 8 lines

After this patch page_check_references() will activate they after first usage, and executable code gets yet better chance to stay in memory.

TODO:
run some cool tests like in v2.6.30-5507-g8cab475 =)

Signed-off-by Konstantin Khlebnikov <[email protected]>

---

Quoted 9 lines

It might be a very controversial topic.
AFAIR, at least, we did when vmscan: make mapped executable pages the first class citizen was merged. :)

You try to change behavior.

Old : protect *working set* executable page
New: protect executable page *unconditionally*.

Hmm ? I thought
Old: protect pages if referenced twice
New: protect executable page if referenced once.

IIUC, ANON is proteced if it's referenced once.

So, this patch changes EXECUTABLE file to the same class as ANON pages.

Anyway, I agree test/measurement is required.

Logic added in commit v2.6.30-5507-g8cab475
(vmscan: make mapped executable pages the first class citizen) was noticeably weakened in commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)

Currently these pages can become "first class citizens" only after second usage.

After this patch page_check_references() will activate they after first usage, and executable code gets yet better chance to stay in memory.

TODO:
run some cool tests like in v2.6.30-5507-g8cab475 =)

Signed-off-by Konstantin Khlebnikov <[email protected]>

---
 mm/vmscan.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

mm/vmscan.c +6 -0

index 3cd766d..29b3612 100644
@@ -727,6 +727,12 @@ static enum page_references page_check_references(struct page *page,
 		if (referenced_page || referenced_ptes > 1)
 			return PAGEREF_ACTIVATE;
 
+		/*
+		 * Activate file-backed executable pages after first usage.
+		 */
+		if (vm_flags & VM_EXEC)
+			return PAGEREF_ACTIVATE;
+
 		return PAGEREF_KEEP;
 	}
 

Logic added in commit v2.6.30-5507-g8cab475
(vmscan: make mapped executable pages the first class citizen) was noticeably weakened in commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)

Currently these pages can become "first class citizens" only after second usage.

After this patch page_check_references() will activate they after first usage, and executable code gets yet better chance to stay in memory.

TODO:
run some cool tests like in v2.6.30-5507-g8cab475 =)

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.

Signed-off-by Konstantin Khlebnikov <[email protected]>

Hi Konstantin,

Quoted 11 lines · Konstantin Khlebnikov

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

Quoted 4 lines

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.

Signed-off-by Konstantin Khlebnikov<[email protected]>

Both patches seem reasonable but the changelogs don't really explain why you're doing the changes. How did you find out about the problem? Is there some workload that's affected? How did you test your changes?

Pekka

Quoted 27 lines · Pekka Enberg

Hi Konstantin,

Quoted 22 lines · Konstantin Khlebnikov

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)
greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages.
Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path)
page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while
it stays in inactive list, meanwhile it activates anon pages after first access.
Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

After this patch page_check_references() also activate file mapped page at first
inactive list scan if this page is already used multiple times via several ptes.

Quoted 6 lines

Quoted 1 line

Signed-off-by Konstantin Khlebnikov<[email protected]>

Both patches seem reasonable but the changelogs don't really explain why you're doing the changes. How did you find out about the problem? Is there some workload that's affected? How did you test your changes?

I found this while trying to fix degragation in rhel6 (~2.6.32) from rhel5 (~2.6.18).
There a complete mess with >100 web/mail/spam/ftp containers, they share all their files but there a lot of anonymous pages:
~500mb shared file mapped memory and 15-20Gb non-shared anonymous memory. In this situation major-pagefaults are very costly, because all containers share the same page.
In my load kernel created a disproportionate pressure on the file memory, compared with the anonymous,
they equaled only if I raise swappiness up to 150 =)

These patches actually wasn't helped a lot in my problem, but I saw noticable (10-20 times) reduce in count and average time of major-pagefault in file-mapped areas.

Actually both patches are fixes for commit v2.6.33-5448-g6457474, because it was aimed at one scenario (singly used pages),
but it breaks the logic in other scenarios (shared and/or executable pages)

Quoted 50 lines · Konstantin Khlebnikov

Quoted 27 lines · Pekka Enberg

Hi Konstantin,

Quoted 22 lines · Konstantin Khlebnikov

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once)
greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages.
Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path)
page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while
it stays in inactive list, meanwhile it activates anon pages after first access.
Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

After this patch page_check_references() also activate file mapped page at first
inactive list scan if this page is already used multiple times via several ptes.

Quoted 6 lines

Quoted 1 line

Signed-off-by Konstantin Khlebnikov<[email protected]>

Both patches seem reasonable but the changelogs don't really explain why you're doing the changes. How did you find out about the problem? Is there some workload that's affected? How did you test your changes?

I found this while trying to fix degragation in rhel6 (~2.6.32) from rhel5 (~2.6.18).
There a complete mess with>100 web/mail/spam/ftp containers, they share all their files but there a lot of anonymous pages:
~500mb shared file mapped memory and 15-20Gb non-shared anonymous memory. In this situation major-pagefaults are very costly, because all containers share the same page.
In my load kernel created a disproportionate pressure on the file memory, compared with the anonymous,
they equaled only if I raise swappiness up to 150 =)

These patches actually wasn't helped a lot in my problem, but I saw noticable (10-20 times) reduce in count and average time of major-pagefault in file-mapped areas.

Quoted 3 lines

Actually both patches are fixes for commit v2.6.33-5448-g6457474, because it was aimed at one scenario (singly used pages),
but it breaks the logic in other scenarios (shared and/or executable pages)

It'd be nice to have such details in the changelogs. FWIW,

Acked-by Pekka Enberg<[email protected]>

Quoted 15 lines · Pekka Enberg

Hi Konstantin,

Quoted 11 lines · Konstantin Khlebnikov

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

Quoted 9 lines

Quoted 4 lines

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.

Signed-off-by Konstantin Khlebnikov<[email protected]>

Both patches seem reasonable but the changelogs don't really explain why you're doing the changes. How did you find out about the problem? Is there some workload that's affected? How did you test your changes?

I found this while trying to fix degragation in rhel6 (~2.6.32) from rhel5 (~2.6.18). There a complete mess with >100 web/mail/spam/ftp containers,
they share all their files but there a lot of anonymous pages:
~500mb shared file mapped memory and 15-20Gb non-shared anonymous memory.

In this situation major-pagefaults are very costly, because all containers share the same page. In my load kernel created a disproportionate pressure on the file memory, compared with the anonymous, they equaled only if I raise swappiness up to 150 =)

These patches actually wasn't helped a lot in my problem,
but I saw noticable (10-20 times) reduce in count and average time of major-pagefault in file-mapped areas.

Quoted 15 lines · Pekka Enberg

Hi Konstantin,

Quoted 11 lines · Konstantin Khlebnikov

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

Quoted 9 lines

Quoted 4 lines

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.

Signed-off-by Konstantin Khlebnikov<[email protected]>

Both patches seem reasonable but the changelogs don't really explain why you're doing the changes. How did you find out about the problem? Is there some workload that's affected? How did you test your changes?

I found this while trying to fix degragation in rhel6 (~2.6.32) from rhel5 (~2.6.18). There a complete mess with >100 web/mail/spam/ftp containers,
they share all their files but there a lot of anonymous pages:
~500mb shared file mapped memory and 15-20Gb non-shared anonymous memory.
In this situation major-pagefaults are very costly, because all containers share the same page. In my load kernel created a disproportionate pressure on the file memory, compared with the anonymous, they equaled only if I raise swappiness up to 150 =)

These patches actually wasn't helped a lot in my problem,
but I saw noticable (10-20 times) reduce in count and average time of major-pagefault in file-mapped areas.

Actually both patches are fixes for commit v2.6.33-5448-g6457474, because it was aimed at one scenario (singly used pages),
but it breaks the logic in other scenarios (shared and/or executable pages)

---
 mm/vmscan.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

mm/vmscan.c +0 -0

index a90c603..39d3da3 100644

Quoted 13 lines

---
 mm/vmscan.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

mm/vmscan.c +1 -0

index a90c603..39d3da3 100644
@@ -831,6 +831,8 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		 * Try to allocate it some swap space here.l
 		 */
 		if (PageAnon(page) && !PageSwapCache(page)) {
+			if (priority >= DEF_PRIORITY - 2)

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 838

Quoted 4 lines

mm/vmscan.c +1 -0

+				goto keep_locked;
 			if (!(sc->gfp_mask & __GFP_IO))
 				goto keep_locked;
 			if (!add_to_swap(page))

Hehe, i tried very similar way very long time ago. Unfortunately, it doesn't work. "DEF_PRIORITY - 2" is really poor indicator for reclaim pressure. example, if the machine have 1TB memory, DEF_PRIORITY-2 mean 1TB>>10 = 1GB. It't too big.

Quoted 13 lines

Quoted 13 lines

---
 mm/vmscan.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

mm/vmscan.c +1 -0

index a90c603..39d3da3 100644
@@ -831,6 +831,8 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		 * Try to allocate it some swap space here.l
 		 */
 		if (PageAnon(page) && !PageSwapCache(page)) {
+			if (priority >= DEF_PRIORITY - 2)

Quoted 8 lines

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 838

Quoted 4 lines

mm/vmscan.c +1 -0

+				goto keep_locked;
 			if (!(sc->gfp_mask & __GFP_IO))
 				goto keep_locked;for
 			if (!add_to_swap(page))

Hehe, i tried very similar way very long time ago. Unfortunately, it doesn't work. "DEF_PRIORITY - 2" is really poor indicator for reclaim pressure. example, if the machine have 1TB memory, DEF_PRIORITY-2 mean 1TB>>10 = 1GB. It't too big.

Do you remember what kind of tests you ran that demonstrated misbehaviour?

We can not reclaim anonymous pages without swapping, so the priority cutoff applies only to inactive file pages. If you had 1TB of inactive file pages, the scanner would have to go through

	((1 << (40 - 12)) >> 12) +
	((1 << (40 - 12)) >> 11) +
	((1 << (40 - 12)) >> 10) = 1792MB

without reclaiming SWAP_CLUSTER_MAX before it considers swapping. That's a lot of scanning but how likely is it that you have a TB of unreclaimable inactive cache pages?

Put into proportion, with a priority threshold of 10 a reclaimer will look at 0.17% ((n >> 12) + (n >> 11) + (n >> 10) (excluding the list balance bias) of inactive file pages without reclaiming SWAP_CLUSTER_MAX before it considers swapping.

Currently, the list balance biasing with each newly-added file page has much higher resistance to scan anonymous pages initially. But once it shifted toward anon pages, all reclaimers will start swapping, unlike the priority threshold that each reclaimer has to reach individually. Could this have been what was causing problems for you?

We want to prevent floods of used-once file cache pushing us to swap out anonymous pages. Never swap under a certain priority level. The availability of used-once cache pages should prevent us from reaching that threshold.

This is needed because subsequent patches will revert some of the mechanisms that tried to prefer file over anon, and this should not result in more eager swapping again.

It might also be better to keep the aging machinery going and just not swap, rather than staying away from anonymous pages in the first place and having less useful age information at the time of swapout.

Signed-off-by Johannes Weiner <[email protected]>

---
 mm/vmscan.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

mm/vmscan.c +0 -0

index a90c603..39d3da3 100644

Quoted 27 lines · On Wed, 2 Nov 2011 17:31:41 +0100 Johannes Weiner <[email protected]> wrote:

We want to prevent floods of used-once file cache pushing us to swap out anonymous pages. Never swap under a certain priority level. The availability of used-once cache pages should prevent us from reaching that threshold.

This is needed because subsequent patches will revert some of the mechanisms that tried to prefer file over anon, and this should not result in more eager swapping again.

It might also be better to keep the aging machinery going and just not swap, rather than staying away from anonymous pages in the first place and having less useful age information at the time of swapout.

Signed-off-by Johannes Weiner <[email protected]>

---
 mm/vmscan.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

mm/vmscan.c +1 -0

index a90c603..39d3da3 100644
@@ -831,6 +831,8 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		 * Try to allocate it some swap space here.
 		 */
 		if (PageAnon(page) && !PageSwapCache(page)) {
+			if (priority >= DEF_PRIORITY - 2)

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 838

Quoted 4 lines

mm/vmscan.c +1 -0

+				goto keep_locked;
 			if (!(sc->gfp_mask & __GFP_IO))
 				goto keep_locked;
 			if (!add_to_swap(page))

Hm, how about not scanning LRU_ANON rather than checking here ? Add some bias to get_scan_count() or some..
If you think to need rotation of LRU, only kswapd should do that..

Each page that is scanned but put back to the inactive list is counted as a successful reclaim, which tips the balance between file and anon lists more towards the cycling list.

This does - in my opinion - not make too much sense, but at the same time it was not much of a problem, as the conditions that lead to an inactive list cycle were mostly temporary - locked page, concurrent page table changes, backing device congested - or at least limited to a single reclaimer that was not allowed to unmap or meddle with IO. More important than being moderately rare, those conditions should apply to both anon and mapped file pages equally and balance out in the end.

Recently, we started cycling file pages in particular on the inactive list much more aggressively, for used-once detection of mapped pages, and when avoiding writeback from direct reclaim.

Those rotated pages do not exactly speak for the reclaimability of the list they sit on and we risk putting immense pressure on file list for no good reason.

Instead, count each page not reclaimed and put back to any list, active or inactive, as rotated, so they are neutral with respect to the scan/rotate ratio of the list class, as they should be.

Signed-off-by Johannes Weiner <[email protected]>

---
 mm/vmscan.c |    9 ++++-----
 1 files changed, 4 insertions(+), 5 deletions(-)

mm/vmscan.c +2 -0

index 39d3da3..6da66a7 100644
@@ -1360,7 +1360,9 @@ putback_lru_pages(struct zone *zone, struct scan_control *sc,
 	 */
 	spin_lock(&zone->lru_lock);
 	while (!list_empty(page_list)) {
+		int file;
 		int lru;
+
 		page = lru_to_page(page_list);
 		VM_BUG_ON(PageLRU(page));
 		list_del(&page->lru);

@@ -1373,11 +1375,8 @@ putback_lru_pages(struct zone *zone, struct scan_control *sc,
 		SetPageLRU(page);
 		lru = page_lru(page);
 		add_page_to_lru_list(zone, page, lru);

Quoted 54 lines · Johannes Weiner

Each page that is scanned but put back to the inactive list is counted as a successful reclaim, which tips the balance between file and anon lists more towards the cycling list.

This does - in my opinion - not make too much sense, but at the same time it was not much of a problem, as the conditions that lead to an inactive list cycle were mostly temporary - locked page, concurrent page table changes, backing device congested - or at least limited to a single reclaimer that was not allowed to unmap or meddle with IO. More important than being moderately rare, those conditions should apply to both anon and mapped file pages equally and balance out in the end.

Recently, we started cycling file pages in particular on the inactive list much more aggressively, for used-once detection of mapped pages, and when avoiding writeback from direct reclaim.

Those rotated pages do not exactly speak for the reclaimability of the list they sit on and we risk putting immense pressure on file list for no good reason.

Instead, count each page not reclaimed and put back to any list, active or inactive, as rotated, so they are neutral with respect to the scan/rotate ratio of the list class, as they should be.

Signed-off-by Johannes Weiner <[email protected]>

---
 mm/vmscan.c |    9 ++++-----
 1 files changed, 4 insertions(+), 5 deletions(-)

mm/vmscan.c +3 -5

index 39d3da3..6da66a7 100644
@@ -1360,7 +1360,9 @@ putback_lru_pages(struct zone *zone, struct scan_control *sc,
 	 */
 	spin_lock(&zone->lru_lock);
 	while (!list_empty(page_list)) {
+		int file;
 		int lru;
+
 		page = lru_to_page(page_list);
 		VM_BUG_ON(PageLRU(page));
 		list_del(&page->lru);

@@ -1373,11 +1375,8 @@ putback_lru_pages(struct zone *zone, struct scan_control *sc,
 		SetPageLRU(page);
 		lru = page_lru(page);
 		add_page_to_lru_list(zone, page, lru);
-		if (is_active_lru(lru)) {
-			int file = is_file_lru(lru);
-			int numpages = hpage_nr_pages(page);
-			reclaim_stat->recent_rotated[file] += numpages;
-		}
+		file = is_file_lru(lru);

mm/vmscan.c putback_lru_pages(struct zone *zone, struct scan_control *sc, · line 1382

Quoted 4 lines

mm/vmscan.c +1 -0

+		reclaim_stat->recent_rotated[file] += hpage_nr_pages(page);
 		if (!pagevec_add(&pvec, page)) {
 			spin_unlock_irq(&zone->lru_lock);
 			__pagevec_release(&pvec);

When avoiding writeback from direct reclaim case, I think we shouldn't increase recent_rotated because VM decided "the page should be eviceted, but also it should be delayed". i'm not sure it's minor factor or not.

Each page that is scanned but put back to the inactive list is counted as a successful reclaim, which tips the balance between file and anon lists more towards the cycling list.

This does - in my opinion - not make too much sense, but at the same time it was not much of a problem, as the conditions that lead to an inactive list cycle were mostly temporary - locked page, concurrent page table changes, backing device congested - or at least limited to a single reclaimer that was not allowed to unmap or meddle with IO. More important than being moderately rare, those conditions should apply to both anon and mapped file pages equally and balance out in the end.

Recently, we started cycling file pages in particular on the inactive list much more aggressively, for used-once detection of mapped pages, and when avoiding writeback from direct reclaim.

Those rotated pages do not exactly speak for the reclaimability of the list they sit on and we risk putting immense pressure on file list for no good reason.

Instead, count each page not reclaimed and put back to any list, active or inactive, as rotated, so they are neutral with respect to the scan/rotate ratio of the list class, as they should be.

Signed-off-by Johannes Weiner <[email protected]>

Quoted 25 lines · On Wed, 2 Nov 2011 17:32:13 +0100 Johannes Weiner <[email protected]> wrote:

Each page that is scanned but put back to the inactive list is counted as a successful reclaim, which tips the balance between file and anon lists more towards the cycling list.

This does - in my opinion - not make too much sense, but at the same time it was not much of a problem, as the conditions that lead to an inactive list cycle were mostly temporary - locked page, concurrent page table changes, backing device congested - or at least limited to a single reclaimer that was not allowed to unmap or meddle with IO. More important than being moderately rare, those conditions should apply to both anon and mapped file pages equally and balance out in the end.

Recently, we started cycling file pages in particular on the inactive list much more aggressively, for used-once detection of mapped pages, and when avoiding writeback from direct reclaim.

Those rotated pages do not exactly speak for the reclaimability of the list they sit on and we risk putting immense pressure on file list for no good reason.

Instead, count each page not reclaimed and put back to any list, active or inactive, as rotated, so they are neutral with respect to the scan/rotate ratio of the list class, as they should be.

Quoted 1 line

Signed-off-by Johannes Weiner <[email protected]>

I think this makes sense.

Acked-by KAMEZAWA Hiroyuki <[email protected]>

I wonder it may be better to have victim list for written-backed pages..

Quoted 30 lines · KAMEZAWA Hiroyuki

Quoted 26 lines · On Wed, 2 Nov 2011 17:32:13 +0100 Johannes Weiner <[email protected]> wrote:

Each page that is scanned but put back to the inactive list is counted as a successful reclaim, which tips the balance between file and anon lists more towards the cycling list.

This does - in my opinion - not make too much sense, but at the same time it was not much of a problem, as the conditions that lead to an inactive list cycle were mostly temporary - locked page, concurrent page table changes, backing device congested - or at least limited to a single reclaimer that was not allowed to unmap or meddle with IO. More important than being moderately rare, those conditions should apply to both anon and mapped file pages equally and balance out in the end.

Recently, we started cycling file pages in particular on the inactive list much more aggressively, for used-once detection of mapped pages, and when avoiding writeback from direct reclaim.

Those rotated pages do not exactly speak for the reclaimability of the list they sit on and we risk putting immense pressure on file list for no good reason.

Instead, count each page not reclaimed and put back to any list, active or inactive, as rotated, so they are neutral with respect to the scan/rotate ratio of the list class, as they should be.

Signed-off-by Johannes Weiner <[email protected]>

Quoted 5 lines

I think this makes sense.

Acked-by KAMEZAWA Hiroyuki <[email protected]>

I wonder it may be better to have victim list for written-backed pages..

Do you mean an extra LRU list that holds dirty pages?

The idea in 9ff473b 'vmscan: evict streaming IO first' was to steer reclaim focus onto file pages with every new file page that hits the lru list, so that an influx of used-once file pages does not lead to swapping of anonymous pages.

The problem is that nobody is fixing up the balance if the pages in fact become part of the resident set.

Anonymous page creation is neutral to the inter-lru balance, so even a comparably tiny number of heavily used file pages tip the balance in favor of the file list.

In addition, there is no refault detection, and every refault will bias the balance even more. A thrashing file working set will be mistaken for a very lucrative source of reclaimable pages.

As anonymous pages are no longer swapped above a certain priority level, this mechanism is no longer needed. Used-once file pages should get reclaimed before the VM even considers swapping.

Signed-off-by Johannes Weiner <[email protected]>

Quoted 16 lines · On Wed, 2 Nov 2011 17:32:47 +0100 Johannes Weiner <[email protected]> wrote:

The idea in 9ff473b 'vmscan: evict streaming IO first' was to steer reclaim focus onto file pages with every new file page that hits the lru list, so that an influx of used-once file pages does not lead to swapping of anonymous pages.

The problem is that nobody is fixing up the balance if the pages in fact become part of the resident set.

Anonymous page creation is neutral to the inter-lru balance, so even a comparably tiny number of heavily used file pages tip the balance in favor of the file list.

In addition, there is no refault detection, and every refault will bias the balance even more. A thrashing file working set will be mistaken for a very lucrative source of reclaimable pages.

Quoted 5 lines

As anonymous pages are no longer swapped above a certain priority level, this mechanism is no longer needed. Used-once file pages should get reclaimed before the VM even considers swapping.

Signed-off-by Johannes Weiner <[email protected]>

Do you have some results ?

Tipping the balance for inactive list rotation has been there from the beginning, but I don't quite understand why. It probably was not a problem as the conditions for inactive cycling applied to both file and anon equally, but with used-once detection for file and deferred file writeback from direct reclaim, we tend to cycle more file pages on the inactive list than anonymous ones. Those rotated pages should be a signal to favor file reclaim, though.

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.

Signed-off-by Konstantin Khlebnikov <[email protected]>

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.

Signed-off-by Konstantin Khlebnikov <[email protected]>

---
 mm/vmscan.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

mm/vmscan.c +0 -0

index 47403c9..3cd766d 100644

Commit v2.6.33-5448-g6457474 (vmscan: detect mapped file pages used only once) greatly decreases lifetime of single-used mapped file pages. Unfortunately it also decreases life time of all shared mapped file pages. Because after commit v2.6.28-6130-gbf3f3bc (mm: don't mark_page_accessed in fault path) page-fault handler does not mark page active or even referenced.

Thus page_check_references() activates file page only if it was used twice while it stays in inactive list, meanwhile it activates anon pages after first access. Inactive list can be small enough, this way reclaimer can accidentally throw away any widely used page if it wasn't used twice in short period.

After this patch page_check_references() also activate file mapped page at first inactive list scan if this page is already used multiple times via several ptes.