Skip to contentSkip to conversation

mm: vmscan: correct nr_reclaimed for THP

PATCH In review Yang Shi 4 participants 11 messages May 10, 2019 – May 21, 2019

Subsystems (3)
lore
Share & tools
lore

Yang Shi

author
#0 lore raw

PATCH v2 1/1 mm: vmscan: correct nr_reclaimed for THP

Since commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after swapped out"), THP can be swapped out in a whole. But, nr_reclaimed still gets inc'ed by one even though a whole THP (512 pages) gets swapped out.

This doesn't make too much sense to memory reclaim. For example, direct reclaim may just need reclaim SWAP_CLUSTER_MAX pages, reclaiming one THP could fulfill it. But, if nr_reclaimed is not increased correctly, direct reclaim may just waste time to reclaim more pages, SWAP_CLUSTER_MAX * 512 pages in worst case.

This change may result in more reclaimed pages than scanned pages showed by /proc/vmstat since scanning one head page would reclaim 512 base pages.

Cc "Huang, Ying" <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Mel Gorman <[email protected]>
Cc "Kirill A . Shutemov" <[email protected]>
Cc Hugh Dickins <[email protected]>

---
v2: Added Shakeel's Reviewed-by
Use hpage_nr_pages instead of compound_order per Huang Ying and William Kucharski

 mm/vmscan.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
mm/vmscan.c +5 -1
index fd9de50..4226d6b 100644
@@ -1446,7 +1446,11 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
 		unlock_page(page);
 free_it:
-		nr_reclaimed++;
+		/* 
+		 * THP may get swapped out in a whole, need account
+		 * all base pages.
+		 */
+		nr_reclaimed += hpage_nr_pages(page);
 
 		/*
 		 * Is there need to periodically free_page_list? It would
Signature · 1 line
1.8.3.1
reviewer
#1 lore raw
Quoted 10 lines · Yang Shi

Since commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after swapped out"), THP can be swapped out in a whole. But, nr_reclaimed still gets inc'ed by one even though a whole THP (512 pages) gets swapped out.

This doesn't make too much sense to memory reclaim. For example, direct reclaim may just need reclaim SWAP_CLUSTER_MAX pages, reclaiming one THP could fulfill it. But, if nr_reclaimed is not increased correctly, direct reclaim may just waste time to reclaim more pages, SWAP_CLUSTER_MAX * 512 pages in worst case.

You are technically right here. This has been a known issue for a while. I am wondering whether somebody actually noticed some misbehavior. Swapping out is a rare event and you have to have a considerable number of THPs to notice.

Quoted 2 lines

This change may result in more reclaimed pages than scanned pages showed by /proc/vmstat since scanning one head page would reclaim 512 base pages.

This is quite nasty and confusing. I am worried that having those two unsynced begs for subtle issues. Can we account THP as scanning 512 base pages as well?

Quoted 35 lines
Cc "Huang, Ying" <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Mel Gorman <[email protected]>
Cc "Kirill A . Shutemov" <[email protected]>
Cc Hugh Dickins <[email protected]>

---
v2: Added Shakeel's Reviewed-by
Use hpage_nr_pages instead of compound_order per Huang Ying and William Kucharski

 mm/vmscan.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
mm/vmscan.c +5 -1
index fd9de50..4226d6b 100644
@@ -1446,7 +1446,11 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
 		unlock_page(page);
 free_it:
-		nr_reclaimed++;
+		/* 
+		 * THP may get swapped out in a whole, need account
+		 * all base pages.
+		 */
+		nr_reclaimed += hpage_nr_pages(page);
 
 		/*
 		 * Is there need to periodically free_page_list? It would
Signature · 1 line
1.8.3.1
Signature · 2 lines
Michal Hocko
SUSE Labs
author
#2 lore raw
Quoted 11 lines · Michal Hocko
Quoted 10 lines · Yang Shi

Since commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after swapped out"), THP can be swapped out in a whole. But, nr_reclaimed still gets inc'ed by one even though a whole THP (512 pages) gets swapped out.

This doesn't make too much sense to memory reclaim. For example, direct reclaim may just need reclaim SWAP_CLUSTER_MAX pages, reclaiming one THP could fulfill it. But, if nr_reclaimed is not increased correctly, direct reclaim may just waste time to reclaim more pages, SWAP_CLUSTER_MAX * 512 pages in worst case.

Quoted 4 lines

You are technically right here. This has been a known issue for a while. I am wondering whether somebody actually noticed some misbehavior. Swapping out is a rare event and you have to have a considerable number of THPs to notice.

The commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after swapped out") was added in 4.14, it might be not used widely yet. And we need swap + THPs to trigger it, furthermore the misbehavior might be accumulative over time. I don't expect it will have any obvious misbehavior for a single shot.

Quoted 6 lines
Quoted 3 lines

This change may result in more reclaimed pages than scanned pages showed by /proc/vmstat since scanning one head page would reclaim 512 base pages.

This is quite nasty and confusing. I am worried that having those two unsynced begs for subtle issues. Can we account THP as scanning 512 base pages as well?

Actually, such unsync has been there for a while. The isolate_lru_pages() returns nr_taken which has THP accounted as 512 base pages, but nr_scanned just shows 1. nr_taken is not showed via /proc/vmstat, but it is showed via trace point.

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Quoted 36 lines
Quoted 36 lines
Cc "Huang, Ying" <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Mel Gorman <[email protected]>
Cc "Kirill A . Shutemov" <[email protected]>
Cc Hugh Dickins <[email protected]>

---
v2: Added Shakeel's Reviewed-by
Use hpage_nr_pages instead of compound_order per Huang Ying and William Kucharski

  mm/vmscan.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
mm/vmscan.c +5 -1
index fd9de50..4226d6b 100644
@@ -1446,7 +1446,11 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
  		unlock_page(page);
  free_it:
-		nr_reclaimed++;
+		/*
+		 * THP may get swapped out in a whole, need account
+		 * all base pages.
+		 */
+		nr_reclaimed += hpage_nr_pages(page);
 
  		/*
  		 * Is there need to periodically free_page_list? It would
Signature · 1 line
1.8.3.1
reviewer
#3 lore raw

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 7 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining than "it should be fine". This should have happened when THP swap out was implemented...

Signature · 2 lines
Michal Hocko
SUSE Labs
#4 lore raw
Quoted 7 lines · Michal Hocko

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 4 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

Quoted 6 lines
Quoted 3 lines

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining

I don't know why my company mailbox didn't receive this email, so I replied with my personal email.

It is not used to double slab pressure any more since commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets"). It uses sc->priority to determine the pressure for slab shrinking now.

So, I think we can just remove that "double slab pressure" code. It is not used actually and looks confusing now. Actually, the "double slab pressure" does something opposite. The extra inc to sc->nr_scanned just prevents from raising sc->priority.

Quoted 7 lines

than "it should be fine". This should have happened when THP swap out was implemented...

--
Michal Hocko
SUSE Labs

reviewer
#5 lore raw
Quoted 18 lines · Yang Shi
Quoted 7 lines · Michal Hocko

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 4 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

Quoted 6 lines
Quoted 3 lines

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining

I don't know why my company mailbox didn't receive this email, so I replied with my personal email.

Quoted 8 lines

It is not used to double slab pressure any more since commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets"). It uses sc->priority to determine the pressure for slab shrinking now.

So, I think we can just remove that "double slab pressure" code. It is not used actually and looks confusing now. Actually, the "double slab pressure" does something opposite. The extra inc to sc->nr_scanned just prevents from raising sc->priority.

I have to get in sync with the recent changes. I am aware there were some patches floating around but I didn't get to review them. I was trying to point out that nr_scanned used to have a side effect to be careful about. If it doesn't have anymore then this is getting much more easier of course. Please document everything in the changelog.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#6 lore raw
Quoted 24 lines · Michal Hocko
Quoted 15 lines · Yang Shi
Quoted 11 lines · Michal Hocko

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 7 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining

I don't know why my company mailbox didn't receive this email, so I replied with my personal email.

Quoted 8 lines

It is not used to double slab pressure any more since commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets"). It uses sc->priority to determine the pressure for slab shrinking now.

So, I think we can just remove that "double slab pressure" code. It is not used actually and looks confusing now. Actually, the "double slab pressure" does something opposite. The extra inc to sc->nr_scanned just prevents from raising sc->priority.

Quoted 5 lines

I have to get in sync with the recent changes. I am aware there were some patches floating around but I didn't get to review them. I was trying to point out that nr_scanned used to have a side effect to be careful about. If it doesn't have anymore then this is getting much more easier of course. Please document everything in the changelog.

Thanks for reminding. Yes, I remembered nr_scanned would double slab pressure. But, when I inspected into the code yesterday, it turns out it is not true anymore. I will run some test to make sure it doesn't introduce regression.

BTW, I noticed the counter of memory reclaim is not correct with THP swap on vanilla kernel, please see the below:

pgsteal_kswapd 21435
pgsteal_direct 26573329
pgscan_kswapd 3514
pgscan_direct 14417775

pgsteal is always greater than pgscan, my patch could fix the problem.

Anyway, I will elaborate these in the commit log.

maintainer
#7 lore raw
Quoted 31 lines · Yang Shi
Quoted 24 lines · Michal Hocko
Quoted 15 lines · Yang Shi
Quoted 11 lines · Michal Hocko

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 7 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining

I don't know why my company mailbox didn't receive this email, so I replied with my personal email.

Quoted 8 lines

It is not used to double slab pressure any more since commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets"). It uses sc->priority to determine the pressure for slab shrinking now.

So, I think we can just remove that "double slab pressure" code. It is not used actually and looks confusing now. Actually, the "double slab pressure" does something opposite. The extra inc to sc->nr_scanned just prevents from raising sc->priority.

Quoted 5 lines

I have to get in sync with the recent changes. I am aware there were some patches floating around but I didn't get to review them. I was trying to point out that nr_scanned used to have a side effect to be careful about. If it doesn't have anymore then this is getting much more easier of course. Please document everything in the changelog.

Quoted 4 lines

Thanks for reminding. Yes, I remembered nr_scanned would double slab pressure. But, when I inspected into the code yesterday, it turns out it is not true anymore. I will run some test to make sure it doesn't introduce regression.

Yeah, sc->nr_scanned is used for three things right now:

1. vmpressure - this looks at the scanned/reclaimed ratio so it won't change semantics as long as scanned & reclaimed are fixed in parallel

2. compaction/reclaim - this is broken. Compaction wants a certain number of physical pages freed up before going back to compacting. Without Yang Shi's fix, we can overreclaim by a factor of 512.

3. kswapd priority raising - this is broken. kswapd raises priority if we scan fewer pages than the reclaim target (which itself is obviously expressed in order-0 pages). As a result, kswapd can falsely raise its aggressiveness even when it's making great progress.

Both sc->nr_scanned & sc->nr_reclaimed should be fixed.

Quoted 9 lines

BTW, I noticed the counter of memory reclaim is not correct with THP swap on vanilla kernel, please see the below:

pgsteal_kswapd 21435
pgsteal_direct 26573329
pgscan_kswapd 3514
pgscan_direct 14417775

pgsteal is always greater than pgscan, my patch could fix the problem.

Ouch, how is that possible with the current code?

I think it happens when isolate_lru_pages() counts 1 nr_scanned for a THP, then shrink_page_list() splits the THP and we reclaim tail pages one by one. This goes all the way back to the initial THP patch!

isolate_lru_pages() needs to be fixed. Its return value, nr_taken, is correct, but its *nr_scanned parameter is wrong, which causes issues:

1. The trace point, as Yang Shi pointed out, will underreport the number of pages scanned, as it reports it along with nr_to_scan (base pages) and nr_taken (base pages)

2. vmstat and memory.stat count 'struct page' operations rather than base pages, which makes zero sense to neither user nor kernel developers (I routinely multiply these counters by 4096 to get a sense of work performed).

All of isolate_lru_pages()'s accounting should be in base pages, which includes nr_scanned and PGSCAN_SKIPPED.

That should also simplify the code; e.g.:

	for (total_scan = 0;
	     scan < nr_to_scan && nr_taken < nr_to_scan && !list_empty(src);
	     total_scan++) {

scan < nr_to_scan && nr_taken >= nr_to_scan is a weird condition that does not make sense in page reclaim imo. Reclaim cares about physical memory - freeing one THP is as much progress for reclaim as freeing 512 order-0 pages.

IMO *all* '++' in vmscan.c are suspicious and should be reviewed:
nr_scanned, nr_reclaimed, nr_dirty, nr_unqueued_dirty, nr_congested, nr_immediate, nr_writeback, nr_ref_keep, nr_unmap_fail, pgactivate, total_scan & scan, nr_skipped.

Yang Shi, it would be nice if you could convert all of these to base page accounting in one patch, as it's a single logical fix for the initial introduction of THP that had huge pages show up on the LRUs.

[ check_move_unevictable_pages() seems weird. It gets a pagevec from
find_get_entries(), which, if I understand the THP page cache code correctly, might contain the same compound page over and over. It'll be !unevictable after the first iteration, so will only run once. So it produces incorrect numbers now, but it is probably best to ignore it until we figure out THP cache. Maybe add an XXX comment. ]

author
#8 lore raw
Quoted 44 lines · Johannes Weiner
Quoted 30 lines · Yang Shi
Quoted 24 lines · Michal Hocko
Quoted 15 lines · Yang Shi
Quoted 11 lines · Michal Hocko

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 7 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining

I don't know why my company mailbox didn't receive this email, so I replied with my personal email.

Quoted 8 lines

It is not used to double slab pressure any more since commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets"). It uses sc->priority to determine the pressure for slab shrinking now.

So, I think we can just remove that "double slab pressure" code. It is not used actually and looks confusing now. Actually, the "double slab pressure" does something opposite. The extra inc to sc->nr_scanned just prevents from raising sc->priority.

Quoted 5 lines

I have to get in sync with the recent changes. I am aware there were some patches floating around but I didn't get to review them. I was trying to point out that nr_scanned used to have a side effect to be careful about. If it doesn't have anymore then this is getting much more easier of course. Please document everything in the changelog.

Quoted 4 lines

Thanks for reminding. Yes, I remembered nr_scanned would double slab pressure. But, when I inspected into the code yesterday, it turns out it is not true anymore. I will run some test to make sure it doesn't introduce regression.

Yeah, sc->nr_scanned is used for three things right now:

1. vmpressure - this looks at the scanned/reclaimed ratio so it won't change semantics as long as scanned & reclaimed are fixed in parallel

2. compaction/reclaim - this is broken. Compaction wants a certain number of physical pages freed up before going back to compacting. Without Yang Shi's fix, we can overreclaim by a factor of 512.

Quoted 6 lines

3. kswapd priority raising - this is broken. kswapd raises priority if we scan fewer pages than the reclaim target (which itself is obviously expressed in order-0 pages). As a result, kswapd can falsely raise its aggressiveness even when it's making great progress.

Both sc->nr_scanned & sc->nr_reclaimed should be fixed.

Yes, v3 patch (sit in my local repo now) did fix both.

Quoted 9 lines
Quoted 9 lines

BTW, I noticed the counter of memory reclaim is not correct with THP swap on vanilla kernel, please see the below:

pgsteal_kswapd 21435
pgsteal_direct 26573329
pgscan_kswapd 3514
pgscan_direct 14417775

Quoted 6 lines
Quoted 1 line

pgsteal is always greater than pgscan, my patch could fix the problem.

Ouch, how is that possible with the current code?

I think it happens when isolate_lru_pages() counts 1 nr_scanned for a THP, then shrink_page_list() splits the THP and we reclaim tail pages one by one. This goes all the way back to the initial THP patch!

I think so. It does make sense. But, the weird thing is I just see this with synchronous swap device (some THPs got swapped out in a whole, some got split), but I've never seen this with rotate swap device (all THPs got split).

I haven't figured out why.

Quoted 18 lines

isolate_lru_pages() needs to be fixed. Its return value, nr_taken, is correct, but its *nr_scanned parameter is wrong, which causes issues:

1. The trace point, as Yang Shi pointed out, will underreport the number of pages scanned, as it reports it along with nr_to_scan (base pages) and nr_taken (base pages)

2. vmstat and memory.stat count 'struct page' operations rather than base pages, which makes zero sense to neither user nor kernel developers (I routinely multiply these counters by 4096 to get a sense of work performed).

All of isolate_lru_pages()'s accounting should be in base pages, which includes nr_scanned and PGSCAN_SKIPPED.

That should also simplify the code; e.g.:

Quoted 8 lines
	for (total_scan = 0;
	     scan < nr_to_scan && nr_taken < nr_to_scan && !list_empty(src);
	     total_scan++) {

scan < nr_to_scan && nr_taken >= nr_to_scan is a weird condition that does not make sense in page reclaim imo. Reclaim cares about physical memory - freeing one THP is as much progress for reclaim as freeing 512 order-0 pages.

Yes, I do agree. The v3 patch did this.

Quoted 5 lines

IMO *all* '++' in vmscan.c are suspicious and should be reviewed:
nr_scanned, nr_reclaimed, nr_dirty, nr_unqueued_dirty, nr_congested, nr_immediate, nr_writeback, nr_ref_keep, nr_unmap_fail, pgactivate, total_scan & scan, nr_skipped.

Some of them should be fine but I'm not sure the side effect. IMHO, let's fix the most obvious problem first.

Quoted 4 lines

Yang Shi, it would be nice if you could convert all of these to base page accounting in one patch, as it's a single logical fix for the initial introduction of THP that had huge pages show up on the LRUs.\

Yes, sure.

Quoted 7 lines

[ check_move_unevictable_pages() seems weird. It gets a pagevec from
find_get_entries(), which, if I understand the THP page cache code correctly, might contain the same compound page over and over. It'll be !unevictable after the first iteration, so will only run once. So it produces incorrect numbers now, but it is probably best to ignore it until we figure out THP cache. Maybe add an XXX comment. ]

author
#9 lore raw
Quoted 106 lines · Yang Shi
Quoted 44 lines · Johannes Weiner
Quoted 30 lines · Yang Shi
Quoted 24 lines · Michal Hocko
Quoted 15 lines · Yang Shi
Quoted 11 lines · Michal Hocko

On Mon 13-05-19 14:09:59, Yang Shi wrote:
[...]

Quoted 7 lines

I think we can just account 512 base pages for nr_scanned for isolate_lru_pages() to make the counters sane since PGSCAN_KSWAPD/DIRECT just use it.

And, sc->nr_scanned should be accounted as 512 base pages too otherwise we may have nr_scanned < nr_to_reclaim all the time to result in false-negative for priority raise and something else wrong (e.g. wrong vmpressure).

Be careful. nr_scanned is used as a pressure indicator to slab shrinking AFAIR. Maybe this is ok but it really begs for much more explaining

I don't know why my company mailbox didn't receive this email, so I replied with my personal email.

Quoted 8 lines

It is not used to double slab pressure any more since commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets"). It uses sc->priority to determine the pressure for slab shrinking now.

So, I think we can just remove that "double slab pressure" code. It is not used actually and looks confusing now. Actually, the "double slab pressure" does something opposite. The extra inc to sc->nr_scanned just prevents from raising sc->priority.

Quoted 5 lines

I have to get in sync with the recent changes. I am aware there were some patches floating around but I didn't get to review them. I was trying to point out that nr_scanned used to have a side effect to be careful about. If it doesn't have anymore then this is getting much more easier of course. Please document everything in the changelog.

Quoted 4 lines

Thanks for reminding. Yes, I remembered nr_scanned would double slab pressure. But, when I inspected into the code yesterday, it turns out it is not true anymore. I will run some test to make sure it doesn't introduce regression.

Yeah, sc->nr_scanned is used for three things right now:

1. vmpressure - this looks at the scanned/reclaimed ratio so it won't change semantics as long as scanned & reclaimed are fixed in parallel

2. compaction/reclaim - this is broken. Compaction wants a certain number of physical pages freed up before going back to compacting. Without Yang Shi's fix, we can overreclaim by a factor of 512.

Quoted 6 lines

3. kswapd priority raising - this is broken. kswapd raises priority if we scan fewer pages than the reclaim target (which itself is obviously expressed in order-0 pages). As a result, kswapd can falsely raise its aggressiveness even when it's making great progress.

Both sc->nr_scanned & sc->nr_reclaimed should be fixed.

Yes, v3 patch (sit in my local repo now) did fix both.

Quoted 9 lines
Quoted 9 lines

BTW, I noticed the counter of memory reclaim is not correct with THP swap on vanilla kernel, please see the below:

pgsteal_kswapd 21435
pgsteal_direct 26573329
pgscan_kswapd 3514
pgscan_direct 14417775

Quoted 6 lines
Quoted 1 line

pgsteal is always greater than pgscan, my patch could fix the problem.

Ouch, how is that possible with the current code?

I think it happens when isolate_lru_pages() counts 1 nr_scanned for a THP, then shrink_page_list() splits the THP and we reclaim tail pages one by one. This goes all the way back to the initial THP patch!

I think so. It does make sense. But, the weird thing is I just see this with synchronous swap device (some THPs got swapped out in a whole, some got split), but I've never seen this with rotate swap device (all THPs got split).

I haven't figured out why.

Quoted 18 lines

isolate_lru_pages() needs to be fixed. Its return value, nr_taken, is correct, but its *nr_scanned parameter is wrong, which causes issues:

1. The trace point, as Yang Shi pointed out, will underreport the number of pages scanned, as it reports it along with nr_to_scan (base pages) and nr_taken (base pages)

2. vmstat and memory.stat count 'struct page' operations rather than base pages, which makes zero sense to neither user nor kernel developers (I routinely multiply these counters by 4096 to get a sense of work performed).

All of isolate_lru_pages()'s accounting should be in base pages, which includes nr_scanned and PGSCAN_SKIPPED.

That should also simplify the code; e.g.:

Quoted 8 lines

    for (total_scan = 0;
         scan < nr_to_scan && nr_taken < nr_to_scan && !list_empty(src); total_scan++) {

scan < nr_to_scan && nr_taken >= nr_to_scan is a weird condition that does not make sense in page reclaim imo. Reclaim cares about physical memory - freeing one THP is as much progress for reclaim as freeing 512 order-0 pages.

Yes, I do agree. The v3 patch did this.

Quoted 1 line
Quoted 6 lines
Quoted 4 lines

IMO *all* '++' in vmscan.c are suspicious and should be reviewed:
nr_scanned, nr_reclaimed, nr_dirty, nr_unqueued_dirty, nr_congested, nr_immediate, nr_writeback, nr_ref_keep, nr_unmap_fail, pgactivate, total_scan & scan, nr_skipped.

Some of them should be fine but I'm not sure the side effect. IMHO, let's fix the most obvious problem first.

A quick review shows we should correct nr_scanned, nr_reclaimed, pgactivate, nr_skipped, nr_ref_keep and nr_unmap_fail since they are user visible (via cgroup, /proc/vmstat or trace point) and the wrong number may confuse and mislead the users.

nr_dirty, nr_unqueued_dirty, nr_congested and nr_writeback are used by file cache, so they are not impacted by THP swap.

Quoted 16 lines
Quoted 5 lines

Yang Shi, it would be nice if you could convert all of these to base page accounting in one patch, as it's a single logical fix for the initial introduction of THP that had huge pages show up on the LRUs.\

Yes, sure.

Quoted 7 lines

[ check_move_unevictable_pages() seems weird. It gets a pagevec from find_get_entries(), which, if I understand the THP page cache code correctly, might contain the same compound page over and over. It'll be !unevictable after the first iteration, so will only run once. So it produces incorrect numbers now, but it is probably best to ignore it until we figure out THP cache. Maybe add an XXX comment. ]

author
#10 lore raw
In reply to Johannes Weiner
Quoted 7 lines

[ check_move_unevictable_pages() seems weird. It gets a pagevec from
find_get_entries(), which, if I understand the THP page cache code correctly, might contain the same compound page over and over. It'll be !unevictable after the first iteration, so will only run once. So it produces incorrect numbers now, but it is probably best to ignore it until we figure out THP cache. Maybe add an XXX comment. ]

The commit 5fd4ca2d84b2 ("mm: page cache: store only head pages in i_pages") changed how THP is stored in page cache, but find_get_entries() would return base page by calling find_subpage(), so check_move_unevictable_pages() should just returns the number of base pages.