Skip to contentSkip to conversation

mm/khugepaged: skip copying lazyfree pages on collapse

PATCH In review Lance Yang 5 participants 23 messages Feb 1, 2024 – Feb 20, 2024

Subsystems (2)
lore
Share & tools
lore

Lance Yang

author
#0 lore raw

PATCH 1/1 mm/khugepaged: skip copying lazyfree pages on collapse

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

After a successful MADV_FREE operation, if there is no subsequent write, the kernel can free the pages at any time. Therefore, in my opinion, counting lazyfree pages in max_pte_none seems reasonable.

Perhaps treating MADV_FREE like MADV_DONTNEED, not copying lazyfree pages when khugepaged collapses huge pages in the background better aligns with user expectations.

---
 mm/khugepaged.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
mm/khugepaged.c +9 -1
index 2b219acb528e..6cbf46d42c6a 100644
@@ -777,6 +777,7 @@ static int __collapse_huge_page_copy(pte_t *pte,
 				     pmd_t orig_pmd,
 				     struct vm_area_struct *vma,
 				     unsigned long address,
+				     struct collapse_control *cc,
 				     spinlock_t *ptl,
 				     struct list_head *compound_pagelist)
 {
@@ -797,6 +798,13 @@ static int __collapse_huge_page_copy(pte_t *pte,
 			continue;
 		}
 		src_page = pte_page(pteval);
+
+		if (cc->is_khugepaged
+				&& !folio_test_swapbacked(page_folio(src_page))) {
+			clear_user_highpage(page, _address);
+			continue;
+		}
+
 		if (copy_mc_user_highpage(page, src_page, _address, vma) > 0) {
 			result = SCAN_COPY_MC;
 			break;
@@ -1205,7 +1213,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,
 	anon_vma_unlock_write(vma->anon_vma);
 
 	result = __collapse_huge_page_copy(pte, hpage, pmd, _pmd,
-					   vma, address, pte_ptl,
+					   vma, address, cc, pte_ptl,
 					   &compound_pagelist);
 	pte_unmap(pte);
 	if (unlikely(result != SCAN_SUCCEED))
Signature · 1 line
2.33.1
author
#1 lore raw

THP:
enable=madivse
defrag=defer
max_ptes_none=511
scan_sleep_millisecs=1000
alloc_sleep_millisecs=1000

Test code:

// allocate a 2MB chunk using mmap and later
// release it by MADV_FRE

root@x:/tmp# ./a.out | grep -B 23 hg
7f762a200000-7f762a400000 rw-p 00000000 00:00 0
Size:                     2048 kB
Anonymous:         2048 kB
LazyFree:             2048 kB
AnonHugePages:       0 kB
THPeligible:    1
VmFlags: rd wr mr mw me ac sd hg

// allocate a 2MB chunk using mmap and later
// some pages marked as lazyfree with MADV_FREE

root@x:/tmp# ./a.out | grep -B 23 hg
7f762a200000-7f762a400000 rw-p 00000000 00:00 0
Size:                     2048 kB
Anonymous:         2048 kB
LazyFree:                   0 kB
AnonHugePages: 2048 kB
THPeligible:    1
VmFlags: rd wr mr mw me ac sd hg
root@x:/tmp# ./a.out
[...]
root@x:/tmp# echo $?
2
Quoted 67 lines · Lance Yang

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

After a successful MADV_FREE operation, if there is no subsequent write, the kernel can free the pages at any time. Therefore, in my opinion, counting lazyfree pages in max_pte_none seems reasonable.

Perhaps treating MADV_FREE like MADV_DONTNEED, not copying lazyfree pages when khugepaged collapses huge pages in the background better aligns with user expectations.

---
 mm/khugepaged.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
mm/khugepaged.c +9 -2
index 2b219acb528e..6cbf46d42c6a 100644
@@ -777,6 +777,7 @@ static int __collapse_huge_page_copy(pte_t *pte,
                                     pmd_t orig_pmd,
                                     struct vm_area_struct *vma,
                                     unsigned long address,
+                                    struct collapse_control *cc,
                                     spinlock_t *ptl,
                                     struct list_head *compound_pagelist)
 {
@@ -797,6 +798,13 @@ static int __collapse_huge_page_copy(pte_t *pte,
                        continue;
                }
                src_page = pte_page(pteval);
+
+               if (cc->is_khugepaged
+                               && !folio_test_swapbacked(page_folio(src_page))) {
+                       clear_user_highpage(page, _address);
+                       continue;
+               }
+
                if (copy_mc_user_highpage(page, src_page, _address, vma) > 0) {
                        result = SCAN_COPY_MC;
                        break;
@@ -1205,7 +1213,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,
        anon_vma_unlock_write(vma->anon_vma);
 
        result = __collapse_huge_page_copy(pte, hpage, pmd, _pmd,
-                                          vma, address, pte_ptl,
+                                          vma, address, cc, pte_ptl,
                                           &compound_pagelist);
        pte_unmap(pte);
        if (unlikely(result != SCAN_SUCCEED))
--

2.33.1

#2 lore raw
In reply to Lance Yang
Quoted 42 lines · Lance Yang

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

After a successful MADV_FREE operation, if there is no subsequent write, the kernel can free the pages at any time. Therefore, in my opinion, counting lazyfree pages in max_pte_none seems reasonable.

Perhaps treating MADV_FREE like MADV_DONTNEED, not copying lazyfree pages when khugepaged collapses huge pages in the background better aligns with user expectations.

---
 mm/khugepaged.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
mm/khugepaged.c +1 -0
index 2b219acb528e..6cbf46d42c6a 100644
@@ -777,6 +777,7 @@ static int __collapse_huge_page_copy(pte_t *pte,
                                     pmd_t orig_pmd,
                                     struct vm_area_struct *vma,
                                     unsigned long address,
+                                    struct collapse_control *cc,
                                     spinlock_t *ptl,
                                     struct list_head *compound_pagelist)
 {

mm/khugepaged.c static int __collapse_huge_page_copy(pte_t *pte, · line 805

Quoted 8 lines
mm/khugepaged.c +5 -0
@@ -797,6 +798,13 @@ static int __collapse_huge_page_copy(pte_t *pte,
                        continue;
                }
                src_page = pte_page(pteval);
+
+               if (cc->is_khugepaged
+                               && !folio_test_swapbacked(page_folio(src_page))) {
+                       clear_user_highpage(page, _address);
+                       continue;

If the page was written before khugepaged collapsed it, and khugepaged collapsed the page before memory reclaim kicked in, didn't this somehow cause data corruption?

Quoted 17 lines
mm/khugepaged.c +3 -2
+               }
+
                if (copy_mc_user_highpage(page, src_page, _address, vma) > 0) {
                        result = SCAN_COPY_MC;
                        break;
@@ -1205,7 +1213,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,
        anon_vma_unlock_write(vma->anon_vma);
 
        result = __collapse_huge_page_copy(pte, hpage, pmd, _pmd,
-                                          vma, address, pte_ptl,
+                                          vma, address, cc, pte_ptl,
                                           &compound_pagelist);
        pte_unmap(pte);
        if (unlikely(result != SCAN_SUCCEED))
--

2.33.1

author
#3 lore raw
Quoted 48 lines · Yang Shi
Quoted 46 lines · Lance Yang

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

After a successful MADV_FREE operation, if there is no subsequent write, the kernel can free the pages at any time. Therefore, in my opinion, counting lazyfree pages in max_pte_none seems reasonable.

Perhaps treating MADV_FREE like MADV_DONTNEED, not copying lazyfree pages when khugepaged collapses huge pages in the background better aligns with user expectations.

---
 mm/khugepaged.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
mm/khugepaged.c +2 -0
index 2b219acb528e..6cbf46d42c6a 100644
@@ -777,6 +777,7 @@ static int __collapse_huge_page_copy(pte_t *pte,
                                     pmd_t orig_pmd,
                                     struct vm_area_struct *vma,
                                     unsigned long address,
+                                    struct collapse_control *cc,
                                     spinlock_t *ptl,
                                     struct list_head *compound_pagelist)
 {
@@ -797,6 +798,13 @@ static int __collapse_huge_page_copy(pte_t *pte,
                        continue;
                }
                src_page = pte_page(pteval);
+
Quoted 9 lines

mm/khugepaged.c static int __collapse_huge_page_copy(pte_t *pte, · line 805

Quoted 4 lines
mm/khugepaged.c +4 -0
+               if (cc->is_khugepaged
+                               && !folio_test_swapbacked(page_folio(src_page))) {
+                       clear_user_highpage(page, _address);
+                       continue;

If the page was written before khugepaged collapsed it, and khugepaged collapsed the page before memory reclaim kicked in, didn't this somehow cause data corruption?

Thanks a lot! Yang, you're correct; indeed, there is a potential issue with data corruption.

I took a look at the check for lazyfree pages in
smaps_pte_entry.

Here's the modification:
if (cc->is_khugepaged && !PageSwapBacked(src_page)

        && !pte_dirty(pteval) && !PageDirty(src_page)) {
        clear_user_highpage(page, _address);
        continue;
}

Could you please take a look?

Thanks,
Lance

Quoted 17 lines
Quoted 17 lines
mm/khugepaged.c +3 -2
+               }
+
                if (copy_mc_user_highpage(page, src_page, _address, vma) > 0) {
                        result = SCAN_COPY_MC;
                        break;
@@ -1205,7 +1213,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,
        anon_vma_unlock_write(vma->anon_vma);
 
        result = __collapse_huge_page_copy(pte, hpage, pmd, _pmd,
-                                          vma, address, pte_ptl,
+                                          vma, address, cc, pte_ptl,
                                           &compound_pagelist);
        pte_unmap(pte);
        if (unlikely(result != SCAN_SUCCEED))
--

2.33.1

#4 lore raw
Quoted 66 lines · Lance Yang
Quoted 48 lines · Yang Shi
Quoted 46 lines · Lance Yang

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

After a successful MADV_FREE operation, if there is no subsequent write, the kernel can free the pages at any time. Therefore, in my opinion, counting lazyfree pages in max_pte_none seems reasonable.

Perhaps treating MADV_FREE like MADV_DONTNEED, not copying lazyfree pages when khugepaged collapses huge pages in the background better aligns with user expectations.

---
 mm/khugepaged.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
mm/khugepaged.c +2 -0
index 2b219acb528e..6cbf46d42c6a 100644
@@ -777,6 +777,7 @@ static int __collapse_huge_page_copy(pte_t *pte,
                                     pmd_t orig_pmd,
                                     struct vm_area_struct *vma,
                                     unsigned long address,
+                                    struct collapse_control *cc,
                                     spinlock_t *ptl,
                                     struct list_head *compound_pagelist)
 {
@@ -797,6 +798,13 @@ static int __collapse_huge_page_copy(pte_t *pte,
                        continue;
                }
                src_page = pte_page(pteval);
+
Quoted 9 lines

mm/khugepaged.c static int __collapse_huge_page_copy(pte_t *pte, · line 805

Quoted 4 lines
mm/khugepaged.c +4 -0
+               if (cc->is_khugepaged
+                               && !folio_test_swapbacked(page_folio(src_page))) {
+                       clear_user_highpage(page, _address);
+                       continue;

If the page was written before khugepaged collapsed it, and khugepaged collapsed the page before memory reclaim kicked in, didn't this somehow cause data corruption?

Thanks a lot! Yang, you're correct; indeed, there is a potential issue with data corruption.

I took a look at the check for lazyfree pages in
smaps_pte_entry.

Quoted 6 lines

Here's the modification:
if (cc->is_khugepaged && !PageSwapBacked(src_page)

        && !pte_dirty(pteval) && !PageDirty(src_page)) {
        clear_user_highpage(page, _address);
        continue;
}

This may be ok. But as I said in another reply, this may still incur data corruption.

Quoted 23 lines

Could you please take a look?

Thanks,
Lance

Quoted 17 lines
Quoted 17 lines
mm/khugepaged.c +3 -2
+               }
+
                if (copy_mc_user_highpage(page, src_page, _address, vma) > 0) {
                        result = SCAN_COPY_MC;
                        break;
@@ -1205,7 +1213,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,
        anon_vma_unlock_write(vma->anon_vma);
 
        result = __collapse_huge_page_copy(pte, hpage, pmd, _pmd,
-                                          vma, address, pte_ptl,
+                                          vma, address, cc, pte_ptl,
                                           &compound_pagelist);
        pte_unmap(pte);
        if (unlikely(result != SCAN_SUCCEED))
--

2.33.1

#5 lore raw
In reply to Lance Yang
Quoted 4 lines · Lance Yang

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

Quoted 8 lines

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

Is that any more confusing than collapsing pte_none pages?

TBH I do not really see why this is a problem. MADV_FREE are correctly recognized same as pte_none so the user defined trashold applies.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#6 lore raw
Quoted 15 lines · Michal Hocko
Quoted 4 lines · Lance Yang

The collapsing behavior of khugepaged with pages marked using MADV_FREE might cause confusion among users.

Quoted 8 lines

For instance, allocate a 2MB chunk using mmap and later release it by MADV_FREE. Khugepaged will not collapse this chunk. From the user's perspective, it treats lazyfree pages as pte_none. However, for some pages marked as lazyfree with MADV_FREE, khugepaged might collapse this chunk and copy these pages to a new huge page. This inconsistency in behavior could be confusing for users.

Quoted 6 lines

Is that any more confusing than collapsing pte_none pages?

TBH I do not really see why this is a problem. MADV_FREE are correctly recognized same as pte_none so the user defined trashold applies.

Sorry, I might not have expressed it clearly. MADV_FREE is correctly recognized, just like pte_none. This is also reasonable and not an issue.

IMO, since it's treated the same as pte_none, perhaps lazyfree pages shouldn't be copied to the new huge page.

Thanks,
Lance

Quoted 3 lines

--
Michal Hocko
SUSE Labs

#7 lore raw
Quoted 3 lines · Lance Yang

IMO, since it's treated the same as pte_none,
perhaps lazyfree pages shouldn't be copied to
the new huge page.

Why? The content of MADV_FREE page is valid until it is reclaimed.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#8 lore raw
Quoted 7 lines · Michal Hocko
Quoted 3 lines · Lance Yang

IMO, since it's treated the same as pte_none,
perhaps lazyfree pages shouldn't be copied to
the new huge page.

Why? The content of MADV_FREE page is valid until it is reclaimed.

IMO, if MADV_FREE pages are considered valid until reclaimed, treating them the same as pte_none might pose a conflict.

Thanks,
Lance

Quoted 3 lines

--
Michal Hocko
SUSE Labs

#9 lore raw
Quoted 7 lines · Lance Yang
Quoted 6 lines · Michal Hocko
Quoted 3 lines · Lance Yang

IMO, since it's treated the same as pte_none,
perhaps lazyfree pages shouldn't be copied to
the new huge page.

Quoted 5 lines
Quoted 1 line

Why? The content of MADV_FREE page is valid until it is reclaimed.

IMO, if MADV_FREE pages are considered valid until reclaimed, treating them the same as pte_none might pose a conflict.

What kind of conflict?

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#10 lore raw

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed. khugepaged treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Thanks,
Lance

Quoted 19 lines · Michal Hocko
Quoted 7 lines · Lance Yang
Quoted 6 lines · Michal Hocko
Quoted 3 lines · Lance Yang

IMO, since it's treated the same as pte_none,
perhaps lazyfree pages shouldn't be copied to
the new huge page.

Quoted 5 lines
Quoted 1 line

Why? The content of MADV_FREE page is valid until it is reclaimed.

IMO, if MADV_FREE pages are considered valid until reclaimed, treating them the same as pte_none might pose a conflict.

What kind of conflict?
--
Michal Hocko
SUSE Labs

author
#11 lore raw

I'd like to add one more point.

Users mark pages as lazyfree with MADV_FREE, expecting these pages to be reclaimed until memory pressure occurs. Currently, khugepaged treats lazyfree pages the same as pte_none, which seems reasonable. IMO, avoiding the copying of these pages to the new huge page during khugepaged collapse can better keep the semantics of MADV_FREE. It appears that lazyfree pages are reclaimed before khugepaged collapses them.

Thanks,
Lance

Quoted 48 lines · Lance Yang

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed. khugepaged treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Thanks,
Lance

Quoted 19 lines · Michal Hocko
Quoted 7 lines · Lance Yang
Quoted 6 lines · Michal Hocko
Quoted 3 lines · Lance Yang

IMO, since it's treated the same as pte_none,
perhaps lazyfree pages shouldn't be copied to
the new huge page.

Quoted 5 lines
Quoted 1 line

Why? The content of MADV_FREE page is valid until it is reclaimed.

IMO, if MADV_FREE pages are considered valid until reclaimed, treating them the same as pte_none might pose a conflict.

What kind of conflict?
--
Michal Hocko
SUSE Labs

#12 lore raw
In reply to Lance Yang
Quoted 11 lines · Lance Yang

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

Quoted 2 lines

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed.

If there is no memory pressure then this might not ever happen. User cannot make any assumption about their content once madvise call has been done. The content has to be considered lost. Sure the userspace might have means to tell those pages from zero pages and recheck after the write but that is about it.

Quoted 11 lines

khugepaged
treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Why?

Unless I am missing something (which is possible of course) I do not really see why dropping the content of those pages and replacing them with a THP is any difference from reclaiming those pages and then faulting in a non-THP zero page.

Now, if khugepaged reused the original content of MADV_FREE pages that would be a slightly different story. I can see why users would expect zero pages to back madvised area.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#13 lore raw

How about blocking khugepaged from
collapsing lazyfree pages? This way,
is it not better to keep the semantics
of MADV_FREE?

What do you think?

Thanks,
Lance

Quoted 50 lines · Michal Hocko
Quoted 11 lines · Lance Yang

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

Quoted 2 lines

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed.

If there is no memory pressure then this might not ever happen. User cannot make any assumption about their content once madvise call has been done. The content has to be considered lost. Sure the userspace might have means to tell those pages from zero pages and recheck after the write but that is about it.

Quoted 11 lines

khugepaged
treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Why?

Unless I am missing something (which is possible of course) I do not really see why dropping the content of those pages and replacing them with a THP is any difference from reclaiming those pages and then faulting in a non-THP zero page.

Now, if khugepaged reused the original content of MADV_FREE pages that would be a slightly different story. I can see why users would expect zero pages to back madvised area.
--
Michal Hocko
SUSE Labs

#14 lore raw
Quoted 6 lines · Lance Yang

How about blocking khugepaged from
collapsing lazyfree pages? This way,
is it not better to keep the semantics
of MADV_FREE?

What do you think?

Why do we even want to change any behavior here? A lot of things "might cause confusion among users". Even worse, a lot of things do cause confusion ;)

Signature · 3 lines
Cheers,

David / dhildenb
#15 lore raw
In reply to Lance Yang
Quoted 4 lines · Lance Yang

How about blocking khugepaged from
collapsing lazyfree pages? This way,
is it not better to keep the semantics
of MADV_FREE?

I do not see any reason why. And you are not providing any actual reasoning either. Unless you have a very specific example in mind I do not think we want to change the current behavior.

Signature · 2 lines
Michal Hocko
SUSE Labs
#16 lore raw
In reply to Lance Yang
Quoted 7 lines · Lance Yang

How about blocking khugepaged from
collapsing lazyfree pages? This way,
is it not better to keep the semantics
of MADV_FREE?

What do you think?

First of all, khugepaged doesn't treat MADV_FREE pages as pte_none IIUC. The khugepaged does skip the 2M block if all the pages are old and unreferenced pages in the range in hpage_collapse_scan_pmd(), then repeat the check in collapse_huge_page() again.

And MADV_FREE pages are just old and unreferenced. This is actually what your first test case does. The whole 2M range is MADV_FREE range, so they are skipped by khugepaged.

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

The current behavior is copying the page.

Quoted 55 lines

Thanks,
Lance

Quoted 50 lines · Michal Hocko
Quoted 11 lines · Lance Yang

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

Quoted 2 lines

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed.

If there is no memory pressure then this might not ever happen. User cannot make any assumption about their content once madvise call has been done. The content has to be considered lost. Sure the userspace might have means to tell those pages from zero pages and recheck after the write but that is about it.

Quoted 11 lines

khugepaged
treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Why?

Unless I am missing something (which is possible of course) I do not really see why dropping the content of those pages and replacing them with a THP is any difference from reclaiming those pages and then faulting in a non-THP zero page.

Now, if khugepaged reused the original content of MADV_FREE pages that would be a slightly different story. I can see why users would expect zero pages to back madvised area.
--
Michal Hocko
SUSE Labs

author
#17 lore raw

Hey Michal, David, Yang,

I sincerely appreciate your time!

I still have two questions that are perplexing me.

First question:
Given that khugepaged doesn't treat MADV_FREE pages as pte_none, why skip the 2M block when all the pages within the range are old and unreferenced, but won't skip if the partial range is MADV_FREE, even if it's not redirtied? Why make this distinction? Would it not be more straightforward to maintain if either all were skipped or not?

Second question:
Does copying lazyfree pages (not redirtied) to the new huge page during khugepaged collapse undermine the semantics of MADV_FREE? Users mark pages as lazyfree with MADV_FREE, expecting these pages to be eventually reclaimed. Even without subsequent writes, these pages will no longer be reclaimed, even if memory pressure occurs.

BR,
Lance

Quoted 92 lines · Yang Shi
Quoted 7 lines · Lance Yang

How about blocking khugepaged from
collapsing lazyfree pages? This way,
is it not better to keep the semantics
of MADV_FREE?

What do you think?

First of all, khugepaged doesn't treat MADV_FREE pages as pte_none IIUC. The khugepaged does skip the 2M block if all the pages are old and unreferenced pages in the range in hpage_collapse_scan_pmd(), then repeat the check in collapse_huge_page() again.

And MADV_FREE pages are just old and unreferenced. This is actually what your first test case does. The whole 2M range is MADV_FREE range, so they are skipped by khugepaged.

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

The current behavior is copying the page.

Quoted 55 lines

Thanks,
Lance

Quoted 50 lines · Michal Hocko
Quoted 11 lines · Lance Yang

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

Quoted 2 lines

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed.

If there is no memory pressure then this might not ever happen. User cannot make any assumption about their content once madvise call has been done. The content has to be considered lost. Sure the userspace might have means to tell those pages from zero pages and recheck after the write but that is about it.

Quoted 11 lines

khugepaged
treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Why?

Unless I am missing something (which is possible of course) I do not really see why dropping the content of those pages and replacing them with a THP is any difference from reclaiming those pages and then faulting in a non-THP zero page.

Now, if khugepaged reused the original content of MADV_FREE pages that would be a slightly different story. I can see why users would expect zero pages to back madvised area.
--
Michal Hocko
SUSE Labs

#18 lore raw
Quoted 7 lines · Lance Yang

Hey Michal, David, Yang,

I sincerely appreciate your time!

I still have two questions that are perplexing me.

Quoted 8 lines

First question:
Given that khugepaged doesn't treat MADV_FREE pages as pte_none, why skip the 2M block when all the pages within the range are old and unreferenced, but won't skip if the partial range is MADV_FREE, even if it's not redirtied? Why make this distinction? Would it not be more straightforward to maintain if either all were skipped or not?

It is just some heuristic in the code and may be some arbitrary choice. It could controlled in a more fine-grained way if we really see some workloads get benefit.

Quoted 10 lines

Second question:
Does copying lazyfree pages (not redirtied) to the new huge page during khugepaged collapse undermine the semantics of MADV_FREE? Users mark pages as lazyfree with MADV_FREE, expecting these pages to be eventually reclaimed. Even without subsequent writes, these pages will no longer be reclaimed, even if memory pressure occurs.

Yeah, it just means khugepaged wins the race against page reclaim. I'm supposed the delayed free is one of the design goals of MADV_FREE, and the risk is the pages may not be freed eventually. If you want immediate free or more deterministic behavior, you should use MADV_DONTNEED or munmap IIUC.

Quoted 97 lines

BR,
Lance

Quoted 92 lines · Yang Shi
Quoted 7 lines · Lance Yang

How about blocking khugepaged from
collapsing lazyfree pages? This way,
is it not better to keep the semantics
of MADV_FREE?

What do you think?

First of all, khugepaged doesn't treat MADV_FREE pages as pte_none IIUC. The khugepaged does skip the 2M block if all the pages are old and unreferenced pages in the range in hpage_collapse_scan_pmd(), then repeat the check in collapse_huge_page() again.

And MADV_FREE pages are just old and unreferenced. This is actually what your first test case does. The whole 2M range is MADV_FREE range, so they are skipped by khugepaged.

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

The current behavior is copying the page.

Quoted 55 lines

Thanks,
Lance

Quoted 50 lines · Michal Hocko
Quoted 11 lines · Lance Yang

Here is a part from the man page explaining the MADV_FREE semantics:

The kernel can thus free thesepages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page. If there is no subsequent write, the kernel can free the pages at any time.

Quoted 2 lines

IIUC, if there is no subsequent write, lazyfree pages will eventually be reclaimed.

If there is no memory pressure then this might not ever happen. User cannot make any assumption about their content once madvise call has been done. The content has to be considered lost. Sure the userspace might have means to tell those pages from zero pages and recheck after the write but that is about it.

Quoted 11 lines

khugepaged
treats lazyfree pages the same as pte_none, avoiding copying them to the new huge page during collapse. It seems that lazyfree pages are reclaimed before khugepaged collapses them. This aligns with user expectations.

However, IMO, if the content of MADV_FREE pages remains valid during collapse, then khugepaged treating lazyfree pages the same as pte_none might not be suitable.

Why?

Unless I am missing something (which is possible of course) I do not really see why dropping the content of those pages and replacing them with a THP is any difference from reclaiming those pages and then faulting in a non-THP zero page.

Now, if khugepaged reused the original content of MADV_FREE pages that would be a slightly different story. I can see why users would expect zero pages to back madvised area.
--
Michal Hocko
SUSE Labs

#19 lore raw
In reply to Yang Shi
Quoted 8 lines · Yang Shi

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

Correct. You cannot assume the content of the first page as it could have been reclaimed at any time.

Quoted 6 lines

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

Right. As pointed above this is a valid outcome if the page has been dropped. User has means to tell that from /proc/vmstat though. Not in a great precision but I think it would be really surprising to not see any pglazyfreed yet the content is gone. I think it would be legit to call it a bug. One could argue the bug would be in the accounting rather than the khugepaged implementation because madvised pages could be dropped at any time. But I think it makes more sense to copy the existing content.

Signature · 2 lines
Michal Hocko
SUSE Labs
#20 lore raw
Quoted 21 lines · Michal Hocko
Quoted 8 lines · Yang Shi

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

Correct. You cannot assume the content of the first page as it could have been reclaimed at any time.

Quoted 6 lines

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

Quoted 7 lines

Right. As pointed above this is a valid outcome if the page has been dropped. User has means to tell that from /proc/vmstat though. Not in a great precision but I think it would be really surprising to not see any pglazyfreed yet the content is gone. I think it would be legit to call it a bug. One could argue the bug would be in the accounting rather than the khugepaged implementation because madvised pages could be dropped at any time. But I think it makes more sense to copy the existing content.

Yeah, as long as khugepaged sees the MADV_FREE pages, it means they have "NOT" been dropped yet. It may be dropped later if memory pressure occurs, but anyway khugepaged wins the race and khugepaged can't assume the pages will be dropped before they get redirtied. So copying the content does make sense.

Quoted 3 lines

--
Michal Hocko
SUSE Labs

#21 lore raw
Quoted 23 lines · Yang Shi
Quoted 21 lines · Michal Hocko
Quoted 8 lines · Yang Shi

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

Correct. You cannot assume the content of the first page as it could have been reclaimed at any time.

Quoted 6 lines

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

Quoted 7 lines
Quoted 7 lines

Right. As pointed above this is a valid outcome if the page has been dropped. User has means to tell that from /proc/vmstat though. Not in a great precision but I think it would be really surprising to not see any pglazyfreed yet the content is gone. I think it would be legit to call it a bug. One could argue the bug would be in the accounting rather than the khugepaged implementation because madvised pages could be dropped at any time. But I think it makes more sense to copy the existing content.

+1. I agree that the content should be dropped iff pglazyfreed is incremented. Of course, we could do that here, but I don't think there is a good reason to, and we should just copy the contents.

Quoted 5 lines

Yeah, as long as khugepaged sees the MADV_FREE pages, it means they have "NOT" been dropped yet. It may be dropped later if memory pressure occurs, but anyway khugepaged wins the race and khugepaged can't assume the pages will be dropped before they get redirtied. So copying the content does make sense.

Per Lance, I kinda get that this "undermines" MADV_FREE, insofar that, from the user's perspective, that memory which was intended as a buffer against OOM kill scenarios, is no longer there to reclaim trivially. I don't have a real world example where this is an issue, however. Also, not copying the contents doesn't change that fact.

The proper alternative, if you want to make the "undermining" argument, is for khugepaged to stay away from hugepage regions with any MADV_FREE pages. I think it's fair to assume MADV_FREE'd memory is likely cold memory, and therefore not a good hugepage target anyways. However, it'd be unfortunate if there were a couple MADV_FREE pages in the middle of an otherwise hot / highly-utilized hugepage region that would prevent it from being pmd-mapped via khugepaged. But.. this is exactly-ish what you get when hugepage-ware system/runtime allocators split THPs to free up internal caches.

Best,
Zach

Quoted 3 lines
Quoted 3 lines

--
Michal Hocko
SUSE Labs

author
#22 lore raw

Hey Zach, Yang, Michal, and David,

Please accept my sincerest apologies for the delayed response.

Thanks for the replies; it‘s been very helpful to me! I also appreciate the valuable information you’ve shared!

I agree that it’s not a good idea to let khugepaged avoid any pages marked with MADV_FREE.

Thanks again for your time!

Best,
Lance

Quoted 66 lines · Zach O'Keefe
Quoted 23 lines · Yang Shi
Quoted 21 lines · Michal Hocko
Quoted 8 lines · Yang Shi

But if the partial range is MADV_FREE, khugepaged won't skip them. This is what your second test case does.

Secondly, I think it depends on the semantics of MADV_FREE, particularly how to treat the redirtied pages. TBH I'm always confused by the semantics. For example, the page contained "abcd", then it was MADV_FREE'ed, then it was written again with "1234" after "abcd". So the user should expect to see "abcd1234" or "00001234".

Correct. You cannot assume the content of the first page as it could have been reclaimed at any time.

Quoted 6 lines

I'm supposed it should be "abcd1234" since MADV_FREE pages are still valid and available, if I'm wrong please feel free to correct me. If so we should always copy MADV_FREE pages in khugepaged regardless of whether it is redirtied or not otherwise it may incur data corruption. If we don't copy, then the follow up redirty after collapse to the hugepage may return "00001234", right?

Quoted 7 lines
Quoted 7 lines

Right. As pointed above this is a valid outcome if the page has been dropped. User has means to tell that from /proc/vmstat though. Not in a great precision but I think it would be really surprising to not see any pglazyfreed yet the content is gone. I think it would be legit to call it a bug. One could argue the bug would be in the accounting rather than the khugepaged implementation because madvised pages could be dropped at any time. But I think it makes more sense to copy the existing content.

+1. I agree that the content should be dropped iff pglazyfreed is incremented. Of course, we could do that here, but I don't think there is a good reason to, and we should just copy the contents.

Quoted 5 lines

Yeah, as long as khugepaged sees the MADV_FREE pages, it means they have "NOT" been dropped yet. It may be dropped later if memory pressure occurs, but anyway khugepaged wins the race and khugepaged can't assume the pages will be dropped before they get redirtied. So copying the content does make sense.

Per Lance, I kinda get that this "undermines" MADV_FREE, insofar that, from the user's perspective, that memory which was intended as a buffer against OOM kill scenarios, is no longer there to reclaim trivially. I don't have a real world example where this is an issue, however. Also, not copying the contents doesn't change that fact.

The proper alternative, if you want to make the "undermining" argument, is for khugepaged to stay away from hugepage regions with any MADV_FREE pages. I think it's fair to assume MADV_FREE'd memory is likely cold memory, and therefore not a good hugepage target anyways. However, it'd be unfortunate if there were a couple MADV_FREE pages in the middle of an otherwise hot / highly-utilized hugepage region that would prevent it from being pmd-mapped via khugepaged. But.. this is exactly-ish what you get when hugepage-ware system/runtime allocators split THPs to free up internal caches.

Best,
Zach

Quoted 3 lines
Quoted 3 lines

--
Michal Hocko
SUSE Labs