Skip to contentSkip to conversation

mm: add lazyfree folio to lru tail

PATCH In review gaoxu 9 participants 23 messages Aug 16, 2024 – Sep 10, 2024 Memory management

lore
Share & tools
lore

gaoxu

author
#0 lore raw

PATCH v2 1/1 mm: add lazyfree folio to lru tail

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

---
V1 -> V2: Based on the latest mm-unstable, recreate the patch.

 mm/swap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/swap.c +1 -1
index 6b838986d..e0dbfc983 100644
@@ -641,7 +641,7 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio)
 	 * anonymous folios
 	 */
 	folio_clear_swapbacked(folio);
-	lruvec_add_folio(lruvec, folio);
+	lruvec_add_folio_tail(lruvec, folio);
 
 	__count_vm_events(PGLAZYFREE, nr_pages);
 	__count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, nr_pages);
Signature · 1 line
2.17.1
#1 lore raw
Quoted 11 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

The patch subject does not match what you do in the patch -- at all.

"mm/swap: use lruvec_add_folio_tail() in lru_lazyfree()" ?

Signature · 3 lines
Cheers,

David / dhildenb
#2 lore raw
Quoted 9 lines · David Hildenbrand
Quoted 8 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

Quoted 7 lines
Quoted 3 lines

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

The patch subject does not match what you do in the patch -- at all.

"mm/swap: use lruvec_add_folio_tail() in lru_lazyfree()" ?

Ah, sorry, I read it too fast. Yours does make sense :)

(mm/swap might make sense, though)

Signature · 3 lines
Cheers,

David / dhildenb
#3 lore raw
Quoted 11 lines · David Hildenbrand
Quoted 9 lines · David Hildenbrand
Quoted 8 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

Was this improvement recorded with MGLRU enabled or disabled? CC'ing Yu Zhao as well.

Quoted 15 lines
Quoted 5 lines
Quoted 1 line

The patch subject does not match what you do in the patch -- at all.

"mm/swap: use lruvec_add_folio_tail() in lru_lazyfree()" ?

Ah, sorry, I read it too fast. Yours does make sense :)

(mm/swap might make sense, though)

--
Cheers,

David / dhildenb

#4 lore raw
In reply to gaoxu
Quoted 9 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

Quoted 3 lines

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

Hi Gao,

Just curious, in which scenario are we frequently calling MADV_FREE but not MADV_DONTNEED?

Quoted 22 lines

---
V1 -> V2: Based on the latest mm-unstable, recreate the patch.

 mm/swap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/swap.c +1 -2
index 6b838986d..e0dbfc983 100644
@@ -641,7 +641,7 @@ static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio)
         * anonymous folios
         */
        folio_clear_swapbacked(folio);
-       lruvec_add_folio(lruvec, folio);
+       lruvec_add_folio_tail(lruvec, folio);
 
        __count_vm_events(PGLAZYFREE, nr_pages);
        __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, nr_pages);
--

2.17.1

author
#5 lore raw
Quoted 11 lines · gaoxu
Quoted 9 lines

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

Quoted 8 lines
Quoted 3 lines

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

Hi Gao,

Just curious, in which scenario are we frequently calling MADV_FREE but not MADV_DONTNEED?

Hi Song,
Android ART use MADV_FREE, please refer to the following link.
https://android-review.googlesource.com/c/platform/art/+/2633132

Quoted 25 lines

mm/swap.c static void lru_lazyfree(struct lruvec *lruvec, struct

Quoted 13 lines

---
V1 -> V2: Based on the latest mm-unstable, recreate the patch.

 mm/swap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/swap.c +0 -0
index 6b838986d..e0dbfc983 100644
@@ -641,7 +641,7 @@ static void lru_lazyfree(struct lruvec *lruvec, struct

folio *folio)

Quoted 11 lines
mm/swap.c +1 -1
         * anonymous folios
         */
        folio_clear_swapbacked(folio);
-       lruvec_add_folio(lruvec, folio);
+       lruvec_add_folio_tail(lruvec, folio);
 
        __count_vm_events(PGLAZYFREE, nr_pages);
        __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE,

nr_pages);
--
2.17.1

#6 lore raw
Quoted 16 lines · gaoxu
Quoted 16 lines · gaoxu
Quoted 12 lines

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.
2. When mglru is enabled, the lazy-free folio is reclaimabled and should be added using lru_gen_add_folio(lruvec, folio, true) instead of lru_gen_add_folio(lruvec, folio, false) for adding to gen.

With the change in place, workingset_refault_file is reduced by 33% in the continuous startup testing of the applications in the Android system.

Quoted 7 lines
Quoted 4 lines

Hi Gao,

Just curious, in which scenario are we frequently calling MADV_FREE but not MADV_DONTNEED?

Hi Song,
Android ART use MADV_FREE, please refer to the following link.
https://android-review.googlesource.com/c/platform/art/+/2633132

thanks! It seems like the art guys owe us some changelogs, with no data and no reason :-)

Your change seems reasonable to me. Since users plan to free those anon folios, we should avoid placing them in hard-to-reclaim areas, which could lead to unnecessarily reclaiming file folios instead.

Quoted 20 lines
Quoted 20 lines

mm/swap.c static void lru_lazyfree(struct lruvec *lruvec, struct

Quoted 13 lines

---
V1 -> V2: Based on the latest mm-unstable, recreate the patch.

 mm/swap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/swap.c +0 -0
index 6b838986d..e0dbfc983 100644
@@ -641,7 +641,7 @@ static void lru_lazyfree(struct lruvec *lruvec, struct

folio *folio)

Quoted 6 lines
mm/swap.c +1 -1
         * anonymous folios
         */
        folio_clear_swapbacked(folio);
-       lruvec_add_folio(lruvec, folio);
+       lruvec_add_folio_tail(lruvec, folio);
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
mm/swap.c +0 -0
        __count_vm_events(PGLAZYFREE, nr_pages);
        __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE,

nr_pages);
--
2.17.1

Thanks
Barry

#7 lore raw
In reply to gaoxu
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

Signature · 2 lines
Michal Hocko
SUSE Labs
#8 lore raw
Quoted 11 lines · Michal Hocko
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

The priority of this memory compared to other inactive memory that has been marked for a longer time likely depends on the user's expectations - How soon do users expect MADV_FREE to be reclaimed compared with old file folios.

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while helping anonymous folio persist longer in memory.

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Quoted 5 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#9 lore raw
Quoted 22 lines · Barry Song
Quoted 11 lines · Michal Hocko
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

The priority of this memory compared to other inactive memory that has been marked for a longer time likely depends on the user's expectations - How soon do users expect MADV_FREE to be reclaimed compared with old file folios.

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Quoted 7 lines

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while helping anonymous folio persist longer in memory.

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Quoted 9 lines
Quoted 6 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#10 lore raw

Thanks Suren for looping in

Quoted 11 lines · Suren Baghdasaryan
Quoted 9 lines · Barry Song
Quoted 7 lines · Michal Hocko
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 5 lines
Quoted 5 lines

The priority of this memory compared to other inactive memory that has been marked for a longer time likely depends on the user's expectations - How soon do users expect MADV_FREE to be reclaimed compared with old file folios.

Quoted 8 lines
Quoted 8 lines

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while helping anonymous folio persist longer in memory.

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

Quoted 3 lines
Quoted 3 lines

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

Quoted 10 lines
Quoted 10 lines
Quoted 7 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#11 lore raw
Quoted 20 lines · Lokesh Gidra

Thanks Suren for looping in

Quoted 11 lines · Suren Baghdasaryan
Quoted 9 lines · Barry Song
Quoted 7 lines · Michal Hocko
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

Quoted 5 lines

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line

Hi Lokesh,
Thanks!

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

The priority of this memory compared to other inactive memory that has been marked for a longer time likely depends on the user's expectations - How soon do users expect MADV_FREE to be reclaimed compared with old file folios.

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while helping anonymous folio persist longer in memory.

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the usual total_size for a phone with memory pressure by running multiple apps and how much for each app?

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree anon folios are moved to file but *NOT* anon LRU.

Quoted 7 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

Quoted 14 lines
Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#12 lore raw
Quoted 44 lines · Barry Song
Quoted 20 lines · Lokesh Gidra

Thanks Suren for looping in

Quoted 11 lines · Suren Baghdasaryan
Quoted 9 lines · Barry Song
Quoted 7 lines · Michal Hocko
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

Quoted 5 lines

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line

Hi Lokesh,
Thanks!

Quoted 13 lines
Quoted 12 lines
Quoted 12 lines

The priority of this memory compared to other inactive memory that has been marked for a longer time likely depends on the user's expectations - How soon do users expect MADV_FREE to be reclaimed compared with old file folios.

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while helping anonymous folio persist longer in memory.

Quoted 7 lines
Quoted 2 lines

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the usual total_size for a phone with memory pressure by running multiple apps and how much for each app?

Every app is a separate process and therefore has its own private ART heap. Those numbers that you are asking vary drastically. But here's what I can tell you:

Max heap size for an app is 512MB typically. But it is rarely entirely used. Typical heap usage is 50MB to 250MB. But as I said, not all of it is MADV_FREE'ed. Only those pages which are freed after GC compaction are.

Quoted 17 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree anon folios are moved to file but *NOT* anon LRU.

Quoted 5 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

Quoted 9 lines
Quoted 2 lines

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

That's definitely an important question to get answered. But putting my users hat on again, by explicitly MADV_FREE'ing we ask for that behavior. IMHO, MADV_FREE'ed pages should be the first ones to be reclaimed on memory pressure.

Quoted 17 lines
Quoted 14 lines
Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#13 lore raw
Quoted 53 lines · Lokesh Gidra
Quoted 44 lines · Barry Song
Quoted 20 lines · Lokesh Gidra

Thanks Suren for looping in

Quoted 11 lines · Suren Baghdasaryan
Quoted 9 lines · Barry Song
Quoted 7 lines · Michal Hocko
Quoted 4 lines · gaoxu

Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio and reduces the impact on file refault.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This has been discussed when MADV_FREE was introduced. The question was whether this memory has a lower priority than other inactive memory that has been marked that way longer ago. Also consider several MADV_FREE users should they be LIFO from the reclaim POV?

Quoted 5 lines

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line

Hi Lokesh,
Thanks!

Quoted 13 lines
Quoted 12 lines
Quoted 12 lines

The priority of this memory compared to other inactive memory that has been marked for a longer time likely depends on the user's expectations - How soon do users expect MADV_FREE to be reclaimed compared with old file folios.

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while helping anonymous folio persist longer in memory.

Quoted 7 lines
Quoted 2 lines

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the usual total_size for a phone with memory pressure by running multiple apps and how much for each app?

Quoted 5 lines

Every app is a separate process and therefore has its own private ART heap. Those numbers that you are asking vary drastically. But here's what I can tell you:

Max heap size for an app is 512MB typically. But it is rarely entirely

On my phone, I am seeing a VMA named "dalvik-main space", its virtual address is 0x14000000-0x34000000 and its size is 0x20000000 (512MB). I guess this is exactly the ART heap we are talking about?

Quoted 1 line

used. Typical heap usage is 50MB to 250MB. But as I said, not all of

Thank you! Considering we might have dozens of apps running in the background and foreground, will the total size of the ART heaps on a phone still be large?

Quoted 2 lines

it is MADV_FREE'ed. Only those pages which are freed after GC compaction are.

Are you saying that some memory in the ART heap is marked with MADV_DONTNEED instead of MADV_FREE? If so, when did this happen? Alternatively, is my understanding incorrect and you are referring to memory that is neither MADV_DONTNEED nor MADV_FREE?

Quoted 26 lines
Quoted 17 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree anon folios are moved to file but *NOT* anon LRU.

Quoted 5 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

Quoted 9 lines
Quoted 2 lines

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

Quoted 4 lines

That's definitely an important question to get answered. But putting my users hat on again, by explicitly MADV_FREE'ing we ask for that behavior. IMHO, MADV_FREE'ed pages should be the first ones to be reclaimed on memory pressure.

Thanks for clarification! I'd also like to collect some performance data with this patch.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

author
#14 lore raw
In reply to Lokesh Gidra
Quoted 114 lines

-----邮件原件-----
发件人: Lokesh Gidra <[email protected]>
发送时间: 2024年8月27日 8:12
收件人: Barry Song <[email protected]>
抄送: Suren Baghdasaryan <[email protected]>; Nicolas Geoffray <[email protected]>; Michal Hocko <[email protected]>; gaoxu <[email protected]>; Andrew Morton <[email protected]>; [email protected]; [email protected]; Shaohua Li <[email protected]>; yipengxiang <[email protected]>; fengbaopeng <[email protected]>; Kalesh Singh <[email protected]> 主题: Re: [PATCH v2] mm: add lazyfree folio to lru tail

Quoted 2 lines · Barry Song

On Tue, Aug 27, 2024 at 4:37 AM Lokesh Gidra <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Thanks Suren for looping in

On Fri, Aug 23, 2024 at 4:39 PM Suren Baghdasaryan <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 2:47 PM Barry Song <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 8:46 PM Michal Hocko <[email protected]>

Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines · gaoxu
Quoted 1 line

Replace lruvec_add_folio with lruvec_add_folio_tail in the

lru_lazyfree_fn:

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The

question was

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

whether this memory has a lower priority than other inactive memory

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

has been marked that way longer ago. Also consider several

MADV_FREE

Quoted 12 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

users should they be LIFO from the reclaim POV?

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line

Hi Lokesh,
Thanks!

Quoted 1 line
Quoted 1 line
Quoted 1 line

The priority of this memory compared to other inactive memory that has

been

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

marked for a longer time likely depends on the user's expectations - How

soon

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

do users expect MADV_FREE to be reclaimed compared with old file

folios.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while

helping

Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

anonymous folio persist longer in memory.

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the usual total_size for a phone with memory pressure by running multiple apps and

how

Quoted 2 lines

much for each app?

Every app is a separate process and therefore has its own private ART heap. Those numbers that you are asking vary drastically. But here's what I can tell you:

Max heap size for an app is 512MB typically. But it is rarely entirely used. Typical heap usage is 50MB to 250MB. But as I said, not all of it is MADV_FREE'ed. Only those pages which are freed after GC compaction are.

Quoted 17 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree anon folios are moved to file but *NOT* anon LRU.

Quoted 5 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

Quoted 9 lines
Quoted 2 lines

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

Quoted 4 lines

That's definitely an important question to get answered. But putting my users hat on again, by explicitly MADV_FREE'ing we ask for that behavior. IMHO, MADV_FREE'ed pages should be the first ones to be reclaimed on memory pressure.

For non-Android systems, perhaps the author of MADV_FREE can provide a more reasonable opinion;

Add Minchan Kim.
Please forgive me for forgetting to add you when sending the patch.

Quoted 17 lines
Quoted 17 lines
Quoted 14 lines
Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#15 lore raw
Quoted 118 lines · gaoxu
Quoted 114 lines

-----邮件原件-----
发件人: Lokesh Gidra <[email protected]>
发送时间: 2024年8月27日 8:12
收件人: Barry Song <[email protected]>
抄送: Suren Baghdasaryan <[email protected]>; Nicolas Geoffray <[email protected]>; Michal Hocko <[email protected]>; gaoxu <[email protected]>; Andrew Morton <[email protected]>; [email protected]; [email protected]; Shaohua Li <[email protected]>; yipengxiang <[email protected]>; fengbaopeng <[email protected]>; Kalesh Singh <[email protected]> 主题: Re: [PATCH v2] mm: add lazyfree folio to lru tail

Quoted 2 lines · Barry Song

On Tue, Aug 27, 2024 at 4:37 AM Lokesh Gidra <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Thanks Suren for looping in

On Fri, Aug 23, 2024 at 4:39 PM Suren Baghdasaryan <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 2:47 PM Barry Song <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 8:46 PM Michal Hocko <[email protected]>

Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines · gaoxu
Quoted 1 line

Replace lruvec_add_folio with lruvec_add_folio_tail in the

lru_lazyfree_fn:

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's moved to the LRU tail, it allows for faster release lazy-free folio

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The

question was

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

whether this memory has a lower priority than other inactive memory

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

has been marked that way longer ago. Also consider several

MADV_FREE

Quoted 12 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

users should they be LIFO from the reclaim POV?

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line

Hi Lokesh,
Thanks!

Quoted 1 line
Quoted 1 line
Quoted 1 line

The priority of this memory compared to other inactive memory that has

been

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

marked for a longer time likely depends on the user's expectations - How

soon

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

do users expect MADV_FREE to be reclaimed compared with old file

folios.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases the likelihood of packing the file LRU and reduces the chances of reclaiming anonymous memory, which could result in more file re-faults while

helping

Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

anonymous folio persist longer in memory.

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the usual total_size for a phone with memory pressure by running multiple apps and

how

Quoted 2 lines

much for each app?

Every app is a separate process and therefore has its own private ART heap. Those numbers that you are asking vary drastically. But here's what I can tell you:

Max heap size for an app is 512MB typically. But it is rarely entirely used. Typical heap usage is 50MB to 250MB. But as I said, not all of it is MADV_FREE'ed. Only those pages which are freed after GC compaction are.

Quoted 17 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

I am really curious why art guys have moved to MADV_FREE if we have an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree anon folios are moved to file but *NOT* anon LRU.

Quoted 5 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

Quoted 9 lines
Quoted 2 lines

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

Quoted 4 lines

That's definitely an important question to get answered. But putting my users hat on again, by explicitly MADV_FREE'ing we ask for that behavior. IMHO, MADV_FREE'ed pages should be the first ones to be reclaimed on memory pressure.

Quoted 5 lines

For non-Android systems, perhaps the author of MADV_FREE can provide a more reasonable opinion;

Add Minchan Kim.
Please forgive me for forgetting to add you when sending the patch.

AFAIR, there were two concerns:

1. The file LRU would contain pages used only once.

While MADV_FREE allows discarding pages under memory pressure, the system would still have non-working set pages within the file LRU (e.g., those used only once).

2. LRU inversion among MADV_FREE users.

Consider this time order:

1. A process: MADV_FREE
2. B process: MADV_FREE
3. C process: MADV_FREE

The moving tail approach would discard the most recent pages from Process C first, instead of those from Process A.

Of course, this isn't universally true for all workloads, but it's the reality.

At the time, I proposed introducing an additional "ez_reclaimable" LRU list to store MADV_FREE pages (and potentially other hinted pages in the future).
This would allow differentiating priority among LRU lists based on knobs or heuristics. However, this idea wasn't well-received.

author
#16 lore raw
Quoted 143 lines · gaoxu
Quoted 2 lines
Quoted 2 lines

On Mon, Aug 26, 2024 at 12:55 PM Barry Song <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 2 lines

On Tue, Aug 27, 2024 at 4:37 AM Lokesh Gidra <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Thanks Suren for looping in

On Fri, Aug 23, 2024 at 4:39 PM Suren Baghdasaryan

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 2:47 PM Barry Song <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 8:46 PM Michal Hocko

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines · gaoxu
Quoted 1 line

Replace lruvec_add_folio with lruvec_add_folio_tail in the

lru_lazyfree_fn:

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If

it's

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

moved to the LRU tail, it allows for faster release lazy-free

folio

Quoted 6 lines
Quoted 6 lines

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The

question was

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

whether this memory has a lower priority than other inactive

memory

Quoted 11 lines
Quoted 11 lines

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

has been marked that way longer ago. Also consider several

MADV_FREE

Quoted 8 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

users should they be LIFO from the reclaim POV?

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Hi Lokesh,
Thanks!

Quoted 1 line
Quoted 1 line
Quoted 1 line

The priority of this memory compared to other inactive memory that

has

Quoted 2 lines
Quoted 2 lines

been

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

marked for a longer time likely depends on the user's expectations -

How

Quoted 9 lines
Quoted 9 lines

soon

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

do users expect MADV_FREE to be reclaimed compared with old file

folios.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases

the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

likelihood of packing the file LRU and reduces the chances of

reclaiming

Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

anonymous memory, which could result in more file re-faults while

helping

Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

anonymous folio persist longer in memory.

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the

usual

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

total_size for a phone with memory pressure by running multiple apps and

how

Quoted 2 lines

much for each app?

Every app is a separate process and therefore has its own private ART heap. Those numbers that you are asking vary drastically. But here's what I can tell you:

Quoted 6 lines
Quoted 6 lines

Max heap size for an app is 512MB typically. But it is rarely entirely used. Typical heap usage is 50MB to 250MB. But as I said, not all of it is MADV_FREE'ed. Only those pages which are freed after GC compaction are.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I am really curious why art guys have moved to MADV_FREE if we

have

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line

an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of

lazyfree

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

anon folios are moved to file but *NOT* anon LRU.

Quoted 7 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

Quoted 5 lines
Quoted 4 lines

That's definitely an important question to get answered. But putting my users hat on again, by explicitly MADV_FREE'ing we ask for that behavior. IMHO, MADV_FREE'ed pages should be the first ones to be reclaimed on memory pressure.

For non-Android systems, perhaps the author of MADV_FREE can provide a

more

Quoted 4 lines

reasonable opinion;

Add Minchan Kim.
Please forgive me for forgetting to add you when sending the patch.

AFAIR, there were two concerns:

1. The file LRU would contain pages used only once.

While MADV_FREE allows discarding pages under memory pressure, the system would
still have non-working set pages within the file LRU (e.g., those used only once).

2. LRU inversion among MADV_FREE users.

Consider this time order:

1. A process: MADV_FREE
2. B process: MADV_FREE
3. C process: MADV_FREE

Quoted 5 lines

The moving tail approach would discard the most recent pages from Process C first,
instead of those from Process A.

Of course, this isn't universally true for all workloads, but it's the reality.

After enabling MGLRU, the implementation of age and evict based on gen dilutes the FIFO mechanism. Although the joining time points are different, they are all reclaimed based on the same gen. Android has always been plagued by performance issues caused by high IO. Many engineers adjust strategies to prefer reclaiming anon when the system is low on memory. For the same reason, we believe lazy free folio should prioritize file reclamation.(If I misunderstood, please correct me.)

For other discussions that lean towards reclaiming anon folio, please refer to:
https://patchwork.kernel.org/project/linux-mm/cover/[email protected]/

Adding lazyfree folio to the LRU tail has no impact on the Android system, allowing the system to normally utilize the reuse of MADV_FREE when not in a low mem state. If added to the file LRU head, the Android system will encounter various issues such as high IO and heavy kswapd load, forcing us to prohibit Android ART from continuing to use MADV_FREE. Adding lazyfree folio to the LRU tail is not the best approach, but it is more acceptable compared to adding it to the LRU head.

Quoted 6 lines

At the time, I proposed introducing an additional "ez_reclaimable" LRU list to store MADV_FREE pages
(and potentially other hinted pages in the future).
This would allow differentiating priority among LRU lists based on knobs or heuristics.

This solution looks good, might need to think about how to adapt it to mglru.

Quoted 1 line

However, this idea wasn't well-received.

#17 lore raw
Quoted 161 lines · gaoxu
Quoted 144 lines · gaoxu
Quoted 2 lines
Quoted 2 lines

On Mon, Aug 26, 2024 at 12:55 PM Barry Song <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 2 lines

On Tue, Aug 27, 2024 at 4:37 AM Lokesh Gidra <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Thanks Suren for looping in

On Fri, Aug 23, 2024 at 4:39 PM Suren Baghdasaryan

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 2:47 PM Barry Song <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

On Wed, Aug 21, 2024 at 8:46 PM Michal Hocko

<[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines · gaoxu
Quoted 1 line

Replace lruvec_add_folio with lruvec_add_folio_tail in the

lru_lazyfree_fn:

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If

it's

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

moved to the LRU tail, it allows for faster release lazy-free

folio

Quoted 6 lines
Quoted 6 lines

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

reduces the impact on file refault.

This has been discussed when MADV_FREE was introduced. The

question was

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

whether this memory has a lower priority than other inactive

memory

Quoted 11 lines
Quoted 11 lines

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

has been marked that way longer ago. Also consider several

MADV_FREE

Quoted 8 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

users should they be LIFO from the reclaim POV?

Thinking from the user's perspective, it seems to me that FIFO within MADV_FREE'ed pages makes more sense. As a user I expect the longer a MADV_FREE'ed page hasn't been touched, the chances are higher that it may not be around anymore.

Quoted 1 line
Quoted 1 line
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Hi Lokesh,
Thanks!

Quoted 1 line
Quoted 1 line
Quoted 1 line

The priority of this memory compared to other inactive memory that

has

Quoted 2 lines
Quoted 2 lines

been

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

marked for a longer time likely depends on the user's expectations -

How

Quoted 9 lines
Quoted 9 lines

soon

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

do users expect MADV_FREE to be reclaimed compared with old file

folios.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

art guys moved to MADV_FREE from MADV_DONTNEED without any useful performance data and reason in the changelog:
https://android-review.googlesource.com/c/platform/art/+/2633132

Since art is the Android Java heap, it can be quite large. This increases

the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

likelihood of packing the file LRU and reduces the chances of

reclaiming

Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

anonymous memory, which could result in more file re-faults while

helping

Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

anonymous folio persist longer in memory.

Individual heaps of android apps are not big, and even in there we don't call MADV_FREE on the entire heap.

How do you define "Individual heaps of android apps", do you know the

usual

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

total_size for a phone with memory pressure by running multiple apps and

how

Quoted 2 lines

much for each app?

Every app is a separate process and therefore has its own private ART heap. Those numbers that you are asking vary drastically. But here's what I can tell you:

Quoted 6 lines
Quoted 6 lines

Max heap size for an app is 512MB typically. But it is rarely entirely used. Typical heap usage is 50MB to 250MB. But as I said, not all of it is MADV_FREE'ed. Only those pages which are freed after GC compaction are.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I am really curious why art guys have moved to MADV_FREE if we

have

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line

an approach to reach them.

Honestly, it makes little sense as a user that calling MADV_FREE on an anonymous mapping will impact file LRU. That was never the intention with our ART change.

This is just how MADV_FREE is implemented in the kernel, this kind of

lazyfree

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

anon folios are moved to file but *NOT* anon LRU.

Quoted 7 lines

From our perspective, once a set of pages are MADV_FREE'ed, they are like a page-cache. It gives an opportunity, without hurting memory use, to avoid overhead of page-faults, which happen frequently after GC is done on running apps.

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

This is exactly what this patch is doing, putting lazyfree anon folios to the tail of file LRU so that they can be reclaimed earlier than file folios. But the question is: is the requirement "MADV_FREE'ed pages should be prioritized for reclamation over file ones" universally true for all other non-Android users?

Quoted 5 lines
Quoted 4 lines

That's definitely an important question to get answered. But putting my users hat on again, by explicitly MADV_FREE'ing we ask for that behavior. IMHO, MADV_FREE'ed pages should be the first ones to be reclaimed on memory pressure.

For non-Android systems, perhaps the author of MADV_FREE can provide a

more

Quoted 4 lines

reasonable opinion;

Add Minchan Kim.
Please forgive me for forgetting to add you when sending the patch.

AFAIR, there were two concerns:

1. The file LRU would contain pages used only once.

While MADV_FREE allows discarding pages under memory pressure, the system would
still have non-working set pages within the file LRU (e.g., those used only once).

2. LRU inversion among MADV_FREE users.

Consider this time order:

1. A process: MADV_FREE
2. B process: MADV_FREE
3. C process: MADV_FREE

Quoted 5 lines

The moving tail approach would discard the most recent pages from Process C first,
instead of those from Process A.

Of course, this isn't universally true for all workloads, but it's the reality.

After enabling MGLRU, the implementation of age and evict based on gen dilutes the FIFO mechanism. Although the joining time points are different, they are all reclaimed based on the same gen. Android has always been plagued by performance issues caused by high IO. Many engineers adjust strategies to prefer reclaiming anon when the system is low on memory. For the same reason, we believe lazy free folio should prioritize file reclamation.(If I misunderstood, please correct me.)

For other discussions that lean towards reclaiming anon folio, please refer to:
https://patchwork.kernel.org/project/linux-mm/cover/[email protected]/

Adding lazyfree folio to the LRU tail has no impact on the Android system, allowing the system to normally utilize the reuse of MADV_FREE when not in a low mem state. If added to the file LRU head, the Android system will encounter various issues such as high IO and heavy kswapd load, forcing us to prohibit Android ART from continuing to use MADV_FREE. Adding lazyfree folio to the LRU tail is not the best approach, but it is more acceptable compared to adding it to the LRU head.

Quoted 1 line
Quoted 6 lines
Quoted 5 lines

At the time, I proposed introducing an additional "ez_reclaimable" LRU list to store MADV_FREE pages
(and potentially other hinted pages in the future).
This would allow differentiating priority among LRU lists based on knobs or heuristics.

This solution looks good, might need to think about how to adapt it to mglru.

That's right. Adapting to MGLRU isn't straightforward. We might need a separate generation smaller than min_seq for this, or alternatively, it could be handled by
a separate LRU list that isn't tied to any MGLRU generation. both seem hard.

Quoted 1 line
Quoted 1 line

However, this idea wasn't well-received.

#18 lore raw
In reply to Lokesh Gidra
Quoted 11 lines · Lokesh Gidra

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

IMHO, lruvec_add_folio is enough. if lruvec_add_folio_tail why not use MADV_DONTNEED instead? In MM the reclaim policy prefer to reclaim file cache, if MADV_FREE'd pages directly add to the tail, they might be reclaimed instantly. Also the benefit of workingset_refault_file cannot be convinced for me.

So we should know the reasons and the benefits of the changes. page faults or ?

Quoted 11 lines
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

--

Help you, Help me,
Hailong.

#19 lore raw
Quoted 12 lines · Hailong Liu
Quoted 10 lines · Lokesh Gidra

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Quoted 5 lines
Quoted 1 line

Adding Nicolas as well, in case he wants to add something.

IMHO, lruvec_add_folio is enough. if lruvec_add_folio_tail why not use MADV_DONTNEED instead? In MM the reclaim policy prefer to reclaim file cache, if MADV_FREE'd pages directly add to the tail, they might be reclaimed instantly. Also the benefit of workingset_refault_file cannot be convinced for me.

My understanding is that MADV_DONTNEED will immediately free the memory, whereas MADV_FREE will release memory only under memory pressure. If memory pressure is low, the anonymous memory may still be gotten back without causing page faults. This might be what Lokesh is aiming to achieve.

Quoted 10 lines

So we should know the reasons and the benefits of the changes. page faults or ?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

#20 lore raw
Quoted 19 lines · Barry Song
Quoted 12 lines · Hailong Liu
Quoted 10 lines · Lokesh Gidra

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Quoted 5 lines
Quoted 1 line

Adding Nicolas as well, in case he wants to add something.

IMHO, lruvec_add_folio is enough. if lruvec_add_folio_tail why not use MADV_DONTNEED instead? In MM the reclaim policy prefer to reclaim file cache, if MADV_FREE'd pages directly add to the tail, they might be reclaimed instantly. Also the benefit of workingset_refault_file cannot be convinced for me.

Quoted 5 lines

My understanding is that MADV_DONTNEED will immediately free the memory, whereas MADV_FREE will release memory only under memory pressure. If memory pressure is low, the anonymous memory may still be gotten back without causing page faults. This might be what Lokesh is aiming to achieve.

Hmm, IIUC, for the reason of watermark, the kswapd would do reclamation without memory pressure. I worried here is that the madv_free'd pages reclaimed too fast if add to tail.

Quoted 5 lines
Quoted 5 lines

So we should know the reasons and the benefits of the changes. page faults or ?

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

--
Michal Hocko
SUSE Labs

Thanks
Barry

--

Help you, Help me,
Hailong.

maintainer
#21 lore raw
Quoted 20 lines · On Tue, 27 Aug 2024 10:29:11 +0800 Hailong Liu <[email protected]> wrote:
Quoted 19 lines · Barry Song
Quoted 17 lines · Hailong Liu
Quoted 11 lines · Lokesh Gidra

IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be prioritized for reclamation over file ones.

Quoted 4 lines

Adding Lokesh.
Lokesh, could you please comment on the reasoning behind the above mentioned change?

Adding Nicolas as well, in case he wants to add something.

IMHO, lruvec_add_folio is enough. if lruvec_add_folio_tail why not use MADV_DONTNEED instead? In MM the reclaim policy prefer to reclaim file cache, if MADV_FREE'd pages directly add to the tail, they might be reclaimed instantly. Also the benefit of workingset_refault_file cannot be convinced for me.

Quoted 8 lines
Quoted 5 lines

My understanding is that MADV_DONTNEED will immediately free the memory, whereas MADV_FREE will release memory only under memory pressure. If memory pressure is low, the anonymous memory may still be gotten back without causing page faults. This might be what Lokesh is aiming to achieve.

Hmm, IIUC, for the reason of watermark, the kswapd would do reclamation without memory pressure. I worried here is that the madv_free'd pages reclaimed too fast if add to tail.

I'm not seeing much clarity on whether we should merge this change. I think I'll drop this version - let's please revisit after -rc1.

#22 lore raw

On Mon 09-09-24 15:22:15, Andrew Morton wrote:
[...]

Quoted 2 lines

I'm not seeing much clarity on whether we should merge this change. I think I'll drop this version - let's please revisit after -rc1.

I think the biggest thing to focus on is the change of the behavior and whether LIFO aging strategy for MADV_FREE pages is really desirable. It is a noticeable LRU antipattern.

Signature · 2 lines
Michal Hocko
SUSE Labs