Skip to contentSkip to conversation

add mTHP support for anonymous share pages

RFC PATCH In review Baolin Wang 6 participants 46 messages Apr 22, 2024 – Apr 25, 2024

Subsystems (4)
lore
Share & tools
lore

Baolin Wang

author reviewer
#0 lore raw

cover add mTHP support for anonymous share pages

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled. Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation for these anonymous shared pages.

TODO:
- More testing and provide some performance data.
- Need more discussion about the large folio allocation strategy for a 'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify the 'file' size, which need to follow the anonymous mTHP rule too?
- Do not split the large folio when share memory swap out.
- Can swap in a large folio for share memory.

Baolin Wang (5):
mm: memory: extend finish_fault() to support large folio
mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
mm: shmem: add THP validation for PMD-mapped THP related statistics
mm: shmem: add mTHP support for anonymous share pages
mm: shmem: add anonymous share mTHP counters

 include/linux/huge_mm.h |   4 +-
 mm/huge_memory.c        |   8 ++-
 mm/memory.c             |  25 +++++++---
 mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------
 4 files changed, 108 insertions(+), 36 deletions(-)
Signature · 1 line
2.39.3
author reviewer
#1 lore raw

RFC PATCH 1/5 mm: memory: extend finish_fault() to support large folio

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shared pages in the following patches.

---
 mm/memory.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)
mm/memory.c +18 -7
index b6fa5146b260..094a76730776 100644
@@ -4766,7 +4766,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct page *page;
+	struct folio *folio;
 	vm_fault_t ret;
+	int nr_pages, i;
+	unsigned long addr;
 
 	/* Did we COW the page? */
 	if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED))
@@ -4797,22 +4800,30 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
 			return VM_FAULT_OOM;
 	}
 
+	folio = page_folio(page);
+	nr_pages = folio_nr_pages(folio);
+	addr = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
 	vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-				      vmf->address, &vmf->ptl);
+				       addr, &vmf->ptl);
 	if (!vmf->pte)
 		return VM_FAULT_NOPAGE;
 
 	/* Re-check under ptl */
-	if (likely(!vmf_pte_changed(vmf))) {
-		struct folio *folio = page_folio(page);
-
-		set_pte_range(vmf, folio, page, 1, vmf->address);
-		ret = 0;
-	} else {
+	if (nr_pages == 1 && vmf_pte_changed(vmf)) {
 		update_mmu_tlb(vma, vmf->address, vmf->pte);
 		ret = VM_FAULT_NOPAGE;
+		goto unlock;
+	} else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
+		for (i = 0; i < nr_pages; i++)
+			update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
+		ret = VM_FAULT_NOPAGE;
+		goto unlock;
 	}
 
+	set_pte_range(vmf, folio, &folio->page, nr_pages, addr);
+	ret = 0;
+
+unlock:
 	pte_unmap_unlock(vmf->pte, vmf->ptl);
 	return ret;
 }
Signature · 1 line
2.39.3
#2 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

Hey Baolin,

[...]
@@ -4727,9 +4725,11 @@ vm_fault_t finish_fault(struct vm_fault *vmf) [...]

The mm counters have been moved from updating out of set_pte_range(), so we we may need to rebase against Kefeng's patch[1].

+	int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
	set_pte_range(vmf, folio, &folio->page, nr_pages, addr);

+ add_mm_counter(vma->vm_mm, type, nr_pages);
[...]

[1] https://lore.kernel.org/all/[email protected]/T/#me2827c26ff453c0fa86f2af9f68f245978b08774

Thanks,
Lance

author reviewer
#3 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

Quoted 8 lines · Lance Yang

Hey Baolin,

[...]

(quoted diff) +0 -0
@@ -4727,9 +4725,11 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
[...]

The mm counters have been moved from updating out of set_pte_range(), so we we may need to rebase against Kefeng's patch[1].

Yes, I did not rebase on the new mm-unstable branch, and I will do so for the next formal patch set. Thanks.

reviewer
#4 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

In reply to Baolin Wang
Quoted 25 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shared pages in the following patches.

---
 mm/memory.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)
mm/memory.c +3 -0
index b6fa5146b260..094a76730776 100644
@@ -4766,7 +4766,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct page *page;
+	struct folio *folio;
 	vm_fault_t ret;
+	int nr_pages, i;
+	unsigned long addr;
 
 	/* Did we COW the page? */
 	if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED))

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4805

Quoted 6 lines
mm/memory.c +3 -0
@@ -4797,22 +4800,30 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
 			return VM_FAULT_OOM;
 	}
 
+	folio = page_folio(page);
+	nr_pages = folio_nr_pages(folio);
+	addr = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);

I'm not sure this is safe. IIUC, finish_fault() is called for any file-backed mapping. So you could have a situation where part of a (regular) file is mapped in the process, faults and hits in the pagecache. But the folio returned by the pagecache is bigger than the portion that the process has mapped. So you now end up mapping beyond the VMA limits? In the pagecache case, you also can't assume that the folio is naturally aligned in virtual address space.

Quoted 8 lines
mm/memory.c +1 -2
 	vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-				      vmf->address, &vmf->ptl);
+				       addr, &vmf->ptl);
 	if (!vmf->pte)
 		return VM_FAULT_NOPAGE;
 
 	/* Re-check under ptl */
-	if (likely(!vmf_pte_changed(vmf))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4816

Quoted 10 lines
mm/memory.c +3 -5
-		struct folio *folio = page_folio(page);
-
-		set_pte_range(vmf, folio, page, 1, vmf->address);
-		ret = 0;
-	} else {
+	if (nr_pages == 1 && vmf_pte_changed(vmf)) {
 		update_mmu_tlb(vma, vmf->address, vmf->pte);
 		ret = VM_FAULT_NOPAGE;
+		goto unlock;
+	} else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {

I think you have grabbed this from do_anonymous_page()? But I'm not sure it works in the same way here as it does there. For the anon case, if userfaultfd is armed, alloc_anon_folio() will only ever allocate order-0. So we end up in the vmf_pte_changed() path, which will allow overwriting a uffd entry. But here, there is nothing stopping nr_pages being greater than 1 when there could be a uffd entry present, and you will fail due to the pte_range_none() check. (see pte_marker_handle_uffd_wp()).

Thanks,
Ryan

Quoted 13 lines
mm/memory.c +8 -0
+		for (i = 0; i < nr_pages; i++)
+			update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
+		ret = VM_FAULT_NOPAGE;
+		goto unlock;
 	}
 
+	set_pte_range(vmf, folio, &folio->page, nr_pages, addr);
+	ret = 0;
+
+unlock:
 	pte_unmap_unlock(vmf->pte, vmf->ptl);
 	return ret;
 }
author reviewer
#5 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

Quoted 29 lines · Ryan Roberts
Quoted 28 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shared pages in the following patches.

---
  mm/memory.c | 25 ++++++++++++++++++-------
  1 file changed, 18 insertions(+), 7 deletions(-)
mm/memory.c +3 -0
index b6fa5146b260..094a76730776 100644
@@ -4766,7 +4766,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
  {
  	struct vm_area_struct *vma = vmf->vma;
  	struct page *page;
+	struct folio *folio;
  	vm_fault_t ret;
+	int nr_pages, i;
+	unsigned long addr;
 
  	/* Did we COW the page? */
  	if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED))
@@ -4797,22 +4800,30 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
  			return VM_FAULT_OOM;
  	}
 
Quoted 10 lines

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4805

Quoted 3 lines
mm/memory.c +3 -0
+	folio = page_folio(page);
+	nr_pages = folio_nr_pages(folio);
+	addr = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);

I'm not sure this is safe. IIUC, finish_fault() is called for any file-backed mapping. So you could have a situation where part of a (regular) file is mapped in the process, faults and hits in the pagecache. But the folio returned by the pagecache is bigger than the portion that the process has mapped. So you now end up mapping beyond the VMA limits? In the pagecache case, you also can't assume that the folio is naturally aligned in virtual address space.

Good point. Yes, I think you are right, I need consider the VMA limits, and I should refer to the calculations of the start pte and end pte in do_fault_around().

Quoted 6 lines
Quoted 6 lines
mm/memory.c +1 -1
  	vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-				      vmf->address, &vmf->ptl);
+				       addr, &vmf->ptl);
  	if (!vmf->pte)
  		return VM_FAULT_NOPAGE;
 
Quoted 16 lines

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4816

Quoted 12 lines
mm/memory.c +3 -6
  	/* Re-check under ptl */
-	if (likely(!vmf_pte_changed(vmf))) {
-		struct folio *folio = page_folio(page);
-
-		set_pte_range(vmf, folio, page, 1, vmf->address);
-		ret = 0;
-	} else {
+	if (nr_pages == 1 && vmf_pte_changed(vmf)) {
  		update_mmu_tlb(vma, vmf->address, vmf->pte);
  		ret = VM_FAULT_NOPAGE;
+		goto unlock;
+	} else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {

I think you have grabbed this from do_anonymous_page()? But I'm not sure it works in the same way here as it does there. For the anon case, if userfaultfd is armed, alloc_anon_folio() will only ever allocate order-0. So we end up in

IMO, the userfaultfd validation should do in the vma->vm_ops->fault() callback, to make sure the nr_pages is always 1 if userfaultfd is armed.

Quoted 4 lines

the vmf_pte_changed() path, which will allow overwriting a uffd entry. But here, there is nothing stopping nr_pages being greater than 1 when there could be a uffd entry present, and you will fail due to the pte_range_none() check. (see pte_marker_handle_uffd_wp()).

So if we do the userfaultfd validation in ->fault() callback, then here we can use the same logic as with anonymous case.

reviewer
#6 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

Quoted 42 lines · Baolin Wang
Quoted 30 lines · Ryan Roberts
Quoted 29 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shared pages in the following patches.

---
  mm/memory.c | 25 ++++++++++++++++++-------

  1 file changed, 18 insertions(+), 7 deletions(-)
mm/memory.c +4 -0
index b6fa5146b260..094a76730776 100644
@@ -4766,7 +4766,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
   {
       struct vm_area_struct *vma = vmf->vma;
       struct page *page;
+    struct folio *folio;
       vm_fault_t ret;
+    int nr_pages, i;
+    unsigned long addr;
         /* Did we COW the page? */
       if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED))
@@ -4797,22 +4800,30 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
               return VM_FAULT_OOM;
       }
   +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
Quoted 8 lines

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4805

Quoted 1 line
mm/memory.c +1 -0
+    addr = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);

I'm not sure this is safe. IIUC, finish_fault() is called for any file-backed mapping. So you could have a situation where part of a (regular) file is mapped in the process, faults and hits in the pagecache. But the folio returned by the pagecache is bigger than the portion that the process has mapped. So you now end up mapping beyond the VMA limits? In the pagecache case, you also can't assume that the folio is naturally aligned in virtual address space.

Quoted 2 lines

Good point. Yes, I think you are right, I need consider the VMA limits, and I should refer to the calculations of the start pte and end pte in do_fault_around().

You might also need to be careful not to increase reported RSS. I have a vague recollection that David once mentioned a problem with fault-around because it causes the reported RSS to increase for the process and this could lead to different decisions in other places. IIRC Redhat had an advisory somewhere with suggested workaround being to disable fault-around. For the anon-shared memory case, it shouldn't be a problem because the user has opted into allocating bigger blocks, but there may be a need to ensure we don't also start eagerly mapping regular files beyond what fault-around is configured for.

Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
mm/memory.c +4 -7
       vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-                      vmf->address, &vmf->ptl);
+                       addr, &vmf->ptl);
       if (!vmf->pte)
           return VM_FAULT_NOPAGE;
         /* Re-check under ptl */
-    if (likely(!vmf_pte_changed(vmf))) {
-        struct folio *folio = page_folio(page);
-
-        set_pte_range(vmf, folio, page, 1, vmf->address);
-        ret = 0;
-    } else {
+    if (nr_pages == 1 && vmf_pte_changed(vmf)) {
           update_mmu_tlb(vma, vmf->address, vmf->pte);
           ret = VM_FAULT_NOPAGE;
+        goto unlock;
+    } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
Quoted 6 lines
Quoted 3 lines

I think you have grabbed this from do_anonymous_page()? But I'm not sure it works in the same way here as it does there. For the anon case, if userfaultfd is armed, alloc_anon_folio() will only ever allocate order-0. So we end up in

IMO, the userfaultfd validation should do in the vma->vm_ops->fault() callback, to make sure the nr_pages is always 1 if userfaultfd is armed.

OK. Are you saying there is already logic to do that today? Great!

Quoted 8 lines
Quoted 5 lines

the vmf_pte_changed() path, which will allow overwriting a uffd entry. But here, there is nothing stopping nr_pages being greater than 1 when there could be a uffd entry present, and you will fail due to the pte_range_none() check. (see pte_marker_handle_uffd_wp()).

So if we do the userfaultfd validation in ->fault() callback, then here we can use the same logic as with anonymous case.

author reviewer
#7 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

Quoted 46 lines · Ryan Roberts
Quoted 42 lines · Baolin Wang
Quoted 30 lines · Ryan Roberts
Quoted 29 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shared pages in the following patches.

---
   mm/memory.c | 25 ++++++++++++++++++-------
   1 file changed, 18 insertions(+), 7 deletions(-)
mm/memory.c +4 -0
index b6fa5146b260..094a76730776 100644
@@ -4766,7 +4766,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
   {
       struct vm_area_struct *vma = vmf->vma;
       struct page *page;
+    struct folio *folio;
       vm_fault_t ret;
+    int nr_pages, i;
+    unsigned long addr;
         /* Did we COW the page? */
       if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED))
@@ -4797,22 +4800,30 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
               return VM_FAULT_OOM;
       }
   +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
Quoted 8 lines

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4805

Quoted 1 line
mm/memory.c +1 -0
+    addr = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);

I'm not sure this is safe. IIUC, finish_fault() is called for any file-backed mapping. So you could have a situation where part of a (regular) file is mapped in the process, faults and hits in the pagecache. But the folio returned by the pagecache is bigger than the portion that the process has mapped. So you now end up mapping beyond the VMA limits? In the pagecache case, you also can't assume that the folio is naturally aligned in virtual address space.

Quoted 2 lines

Good point. Yes, I think you are right, I need consider the VMA limits, and I should refer to the calculations of the start pte and end pte in do_fault_around().

Quoted 8 lines

You might also need to be careful not to increase reported RSS. I have a vague recollection that David once mentioned a problem with fault-around because it causes the reported RSS to increase for the process and this could lead to different decisions in other places. IIRC Redhat had an advisory somewhere with suggested workaround being to disable fault-around. For the anon-shared memory case, it shouldn't be a problem because the user has opted into allocating bigger blocks, but there may be a need to ensure we don't also start eagerly mapping regular files beyond what fault-around is configured for.

Thanks for reminding. And I also agree with you that this should not be a problem since user has selected the larger folio, which is not the same as fault-around.

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
mm/memory.c +4 -7
       vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-                      vmf->address, &vmf->ptl);
+                       addr, &vmf->ptl);
       if (!vmf->pte)
           return VM_FAULT_NOPAGE;
         /* Re-check under ptl */
-    if (likely(!vmf_pte_changed(vmf))) {
-        struct folio *folio = page_folio(page);
-
-        set_pte_range(vmf, folio, page, 1, vmf->address);
-        ret = 0;
-    } else {
+    if (nr_pages == 1 && vmf_pte_changed(vmf)) {
           update_mmu_tlb(vma, vmf->address, vmf->pte);
           ret = VM_FAULT_NOPAGE;
+        goto unlock;
+    } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

I think you have grabbed this from do_anonymous_page()? But I'm not sure it works in the same way here as it does there. For the anon case, if userfaultfd is armed, alloc_anon_folio() will only ever allocate order-0. So we end up in

IMO, the userfaultfd validation should do in the vma->vm_ops->fault() callback, to make sure the nr_pages is always 1 if userfaultfd is armed.

OK. Are you saying there is already logic to do that today? Great!

I mean I should add the userfaultfd validation in shmem_fault(), and may be need add a warning in finish_fault() to catch this issue if other vma->vm_ops->fault() will support large folio allocation?

WARN_ON(nr_pages > 1 && userfaultfd_armed(vma));

reviewer
#8 lore raw

Re: [RFC PATCH 1/5] mm: memory: extend finish_fault() to support large folio

Quoted 90 lines · Baolin Wang
Quoted 49 lines · Ryan Roberts
Quoted 44 lines · Baolin Wang
Quoted 33 lines · Ryan Roberts
Quoted 26 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shared pages in the following
patches.

---
   mm/memory.c | 25 ++++++++++++++++++-------

   1 file changed, 18 insertions(+), 7 deletions(-)
mm/memory.c +3 -0
index b6fa5146b260..094a76730776 100644
@@ -4766,7 +4766,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
    {
        struct vm_area_struct *vma = vmf->vma;
        struct page *page;
+    struct folio *folio;
        vm_fault_t ret;
+    int nr_pages, i;
+    unsigned long addr;
          /* Did we COW the page? */
        if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED))

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4804

Quoted 5 lines
mm/memory.c +2 -0
@@ -4797,22 +4800,30 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
                return VM_FAULT_OOM;
        }
    +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
+    addr = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
Quoted 7 lines

I'm not sure this is safe. IIUC, finish_fault() is called for any file-backed mapping. So you could have a situation where part of a (regular) file is mapped in the process, faults and hits in the pagecache. But the folio returned by the pagecache is bigger than the portion that the process has mapped. So you now end
up mapping beyond the VMA limits? In the pagecache case, you also can't assume that the folio is naturally aligned in virtual address space.

Quoted 3 lines

Good point. Yes, I think you are right, I need consider the VMA limits, and I should refer to the calculations of the start pte and end pte in do_fault_around().

Quoted 8 lines

You might also need to be careful not to increase reported RSS. I have a vague recollection that David once mentioned a problem with fault-around because it causes the reported RSS to increase for the process and this could lead to different decisions in other places. IIRC Redhat had an advisory somewhere with suggested workaround being to disable fault-around. For the anon-shared memory case, it shouldn't be a problem because the user has opted into allocating bigger blocks, but there may be a need to ensure we don't also start eagerly mapping regular files beyond what fault-around is configured for.

Thanks for reminding. And I also agree with you that this should not be a problem since user has selected the larger folio, which is not the same as fault-around.

Quoted 25 lines
Quoted 24 lines
Quoted 18 lines
Quoted 7 lines
mm/memory.c +1 -2
        vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-                      vmf->address, &vmf->ptl);
+                       addr, &vmf->ptl);
        if (!vmf->pte)
            return VM_FAULT_NOPAGE;
          /* Re-check under ptl */
-    if (likely(!vmf_pte_changed(vmf))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4816

Quoted 10 lines
mm/memory.c +3 -5
-        struct folio *folio = page_folio(page);
-
-        set_pte_range(vmf, folio, page, 1, vmf->address);
-        ret = 0;
-    } else {
+    if (nr_pages == 1 && vmf_pte_changed(vmf)) {
            update_mmu_tlb(vma, vmf->address, vmf->pte);
            ret = VM_FAULT_NOPAGE;
+        goto unlock;
+    } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
Quoted 3 lines

I think you have grabbed this from do_anonymous_page()? But I'm not sure it works in the same way here as it does there. For the anon case, if userfaultfd is armed, alloc_anon_folio() will only ever allocate order-0. So we end up in

IMO, the userfaultfd validation should do in the vma->vm_ops->fault() callback, to make sure the nr_pages is always 1 if userfaultfd is armed.

Quoted 7 lines
Quoted 1 line

OK. Are you saying there is already logic to do that today? Great!

I mean I should add the userfaultfd validation in shmem_fault(), and may be need add a warning in finish_fault() to catch this issue if other vma->vm_ops->fault() will support large folio allocation?

WARN_ON(nr_pages > 1 && userfaultfd_armed(vma));

That adds quite a subtle requirement to vm_ops::fault() though, which I guess is implemented in a lot of places. It would be better if it could be handled centrally - i.e. that all the ptes are either none or a uffd marker? I'm sure there would be corner cases to think about if taking that route.

author reviewer
#9 lore raw

RFC PATCH 2/5 mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()

In reply to Baolin Wang

Add a new parameter to specify the huge page order for shmem_alloc_hugefolio(), as a preparation to supoort mTHP.

---
 mm/shmem.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
mm/shmem.c +6 -5
index fa2a0ed97507..893c88efc45f 100644
@@ -1604,14 +1604,14 @@ static gfp_t limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp)
 }
 
 static struct folio *shmem_alloc_hugefolio(gfp_t gfp,
-		struct shmem_inode_info *info, pgoff_t index)
+		struct shmem_inode_info *info, pgoff_t index, int order)
 {
 	struct mempolicy *mpol;
 	pgoff_t ilx;
 	struct page *page;
 
-	mpol = shmem_get_pgoff_policy(info, index, HPAGE_PMD_ORDER, &ilx);
-	page = alloc_pages_mpol(gfp, HPAGE_PMD_ORDER, mpol, ilx, numa_node_id());
+	mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
+	page = alloc_pages_mpol(gfp, order, mpol, ilx, numa_node_id());
 	mpol_cond_put(mpol);
 
 	return page_rmappable_folio(page);
@@ -1639,13 +1639,14 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
 	struct shmem_inode_info *info = SHMEM_I(inode);
 	struct folio *folio;
 	long pages;
-	int error;
+	int error, order;
 
 	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
 		huge = false;
 
 	if (huge) {
 		pages = HPAGE_PMD_NR;
+		order = HPAGE_PMD_ORDER;
 		index = round_down(index, HPAGE_PMD_NR);
 
 		/*
@@ -1660,7 +1661,7 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
 				index + HPAGE_PMD_NR - 1, XA_PRESENT))
 			return ERR_PTR(-E2BIG);
 
-		folio = shmem_alloc_hugefolio(gfp, info, index);
+		folio = shmem_alloc_hugefolio(gfp, info, index, order);
 		if (!folio)
 			count_vm_event(THP_FILE_FALLBACK);
 	} else {
Signature · 1 line
2.39.3
#10 lore raw

Re: [RFC PATCH 2/5] mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()

Quoted 47 lines · Baolin Wang

Add a new parameter to specify the huge page order for shmem_alloc_hugefolio(), as a preparation to supoort mTHP.

---
  mm/shmem.c | 11 ++++++-----
  1 file changed, 6 insertions(+), 5 deletions(-)
mm/shmem.c +5 -4
index fa2a0ed97507..893c88efc45f 100644
@@ -1604,14 +1604,14 @@ static gfp_t limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp)
  }
 
  static struct folio *shmem_alloc_hugefolio(gfp_t gfp,
-		struct shmem_inode_info *info, pgoff_t index)
+		struct shmem_inode_info *info, pgoff_t index, int order)
  {
  	struct mempolicy *mpol;
  	pgoff_t ilx;
  	struct page *page;
 
-	mpol = shmem_get_pgoff_policy(info, index, HPAGE_PMD_ORDER, &ilx);
-	page = alloc_pages_mpol(gfp, HPAGE_PMD_ORDER, mpol, ilx, numa_node_id());
+	mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
+	page = alloc_pages_mpol(gfp, order, mpol, ilx, numa_node_id());
  	mpol_cond_put(mpol);
 
  	return page_rmappable_folio(page);
@@ -1639,13 +1639,14 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
  	struct shmem_inode_info *info = SHMEM_I(inode);
  	struct folio *folio;
  	long pages;
-	int error;
+	int error, order;
 
  	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
  		huge = false;
 
  	if (huge) {
  		pages = HPAGE_PMD_NR;
+		order = HPAGE_PMD_ORDER;
  		index = round_down(index, HPAGE_PMD_NR);
 
  		/*

mm/shmem.c static struct folio *shmem_alloc_and_add_folio(gfp_t gfp, · line 1664

Quoted 5 lines
mm/shmem.c +1 -1
@@ -1660,7 +1661,7 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
  				index + HPAGE_PMD_NR - 1, XA_PRESENT))
  			return ERR_PTR(-E2BIG);
 
-		folio = shmem_alloc_hugefolio(gfp, info, index);
+		folio = shmem_alloc_hugefolio(gfp, info, index, order);

Avoid order variable, we can directly use HPAGE_PMD_NR here.

Quoted 3 lines
mm/shmem.c +0 -0
  		if (!folio)
  			count_vm_event(THP_FILE_FALLBACK);
  	} else {
author reviewer
#11 lore raw

Re: [RFC PATCH 2/5] mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()

Quoted 46 lines · Kefeng Wang
Quoted 43 lines · Baolin Wang

Add a new parameter to specify the huge page order for shmem_alloc_hugefolio(), as a preparation to supoort mTHP.

---
  mm/shmem.c | 11 ++++++-----

  1 file changed, 6 insertions(+), 5 deletions(-)
mm/shmem.c +5 -4
index fa2a0ed97507..893c88efc45f 100644
@@ -1604,14 +1604,14 @@ static gfp_t limit_gfp_mask(gfp_t huge_gfp, 
 gfp_t limit_gfp)
   }
   static struct folio *shmem_alloc_hugefolio(gfp_t gfp,
-        struct shmem_inode_info *info, pgoff_t index)
+        struct shmem_inode_info *info, pgoff_t index, int order)
   {
       struct mempolicy *mpol;
       pgoff_t ilx;
       struct page *page;
-    mpol = shmem_get_pgoff_policy(info, index, HPAGE_PMD_ORDER, &ilx);
-    page = alloc_pages_mpol(gfp, HPAGE_PMD_ORDER, mpol, ilx, 
 numa_node_id());
+    mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
+    page = alloc_pages_mpol(gfp, order, mpol, ilx, numa_node_id());
       mpol_cond_put(mpol);
       return page_rmappable_folio(page);
@@ -1639,13 +1639,14 @@ static struct folio 
 *shmem_alloc_and_add_folio(gfp_t gfp,
       struct shmem_inode_info *info = SHMEM_I(inode);
       struct folio *folio;
       long pages;
-    int error;
+    int error, order;
       if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
           huge = false;
       if (huge) {
           pages = HPAGE_PMD_NR;
+        order = HPAGE_PMD_ORDER;
           index = round_down(index, HPAGE_PMD_NR);
           /*
Quoted 8 lines

mm/shmem.c static struct folio · line 1664

Quoted 6 lines
mm/shmem.c +1 -1
@@ -1660,7 +1661,7 @@ static struct folio 
 *shmem_alloc_and_add_folio(gfp_t gfp,
                   index + HPAGE_PMD_NR - 1, XA_PRESENT))
               return ERR_PTR(-E2BIG);
-        folio = shmem_alloc_hugefolio(gfp, info, index);
+        folio = shmem_alloc_hugefolio(gfp, info, index, order);

Avoid order variable, we can directly use HPAGE_PMD_NR here.

Yes, sure. Thanks.

author reviewer
#12 lore raw

RFC PATCH 3/5 mm: shmem: add THP validation for PMD-mapped THP related statistics

In reply to Baolin Wang

In order to extend support for mTHP, add THP validation for PMD-mapped THP related statistics to avoid statistical confusion.

---
 mm/shmem.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
mm/shmem.c +4 -3
index 893c88efc45f..b4afda71a3f0 100644
@@ -1662,7 +1662,7 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
 			return ERR_PTR(-E2BIG);
 
 		folio = shmem_alloc_hugefolio(gfp, info, index, order);
-		if (!folio)
+		if (!folio && pages == HPAGE_PMD_NR)
 			count_vm_event(THP_FILE_FALLBACK);
 	} else {
 		pages = 1;
@@ -1680,7 +1680,7 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
 		if (xa_find(&mapping->i_pages, &index,
 				index + pages - 1, XA_PRESENT)) {
 			error = -EEXIST;
-		} else if (huge) {
+		} else if (pages == HPAGE_PMD_NR) {
 			count_vm_event(THP_FILE_FALLBACK);
 			count_vm_event(THP_FILE_FALLBACK_CHARGE);
 		}
@@ -2046,7 +2046,8 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index,
 		folio = shmem_alloc_and_add_folio(huge_gfp,
 				inode, index, fault_mm, true);
 		if (!IS_ERR(folio)) {
-			count_vm_event(THP_FILE_ALLOC);
+			if (folio_test_pmd_mappable(folio))
+				count_vm_event(THP_FILE_ALLOC);
 			goto alloced;
 		}
 		if (PTR_ERR(folio) == -EEXIST)
Signature · 1 line
2.39.3
Reviewed-by Barry Song on v1 3/5
#13 lore raw

Re: [RFC PATCH 3/5] mm: shmem: add THP validation for PMD-mapped THP related statistics

Quoted 6 lines · Baolin Wang

In order to extend support for mTHP, add THP validation for PMD-mapped THP related statistics to avoid statistical confusion.

---

Quoted 38 lines
 mm/shmem.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
mm/shmem.c +4 -4
index 893c88efc45f..b4afda71a3f0 100644
@@ -1662,7 +1662,7 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
                        return ERR_PTR(-E2BIG);
 
                folio = shmem_alloc_hugefolio(gfp, info, index, order);
-               if (!folio)
+               if (!folio && pages == HPAGE_PMD_NR)
                        count_vm_event(THP_FILE_FALLBACK);
        } else {
                pages = 1;
@@ -1680,7 +1680,7 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp,
                if (xa_find(&mapping->i_pages, &index,
                                index + pages - 1, XA_PRESENT)) {
                        error = -EEXIST;
-               } else if (huge) {
+               } else if (pages == HPAGE_PMD_NR) {
                        count_vm_event(THP_FILE_FALLBACK);
                        count_vm_event(THP_FILE_FALLBACK_CHARGE);
                }
@@ -2046,7 +2046,8 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index,
                folio = shmem_alloc_and_add_folio(huge_gfp,
                                inode, index, fault_mm, true);
                if (!IS_ERR(folio)) {
-                       count_vm_event(THP_FILE_ALLOC);
+                       if (folio_test_pmd_mappable(folio))
+                               count_vm_event(THP_FILE_ALLOC);
                        goto alloced;
                }
                if (PTR_ERR(folio) == -EEXIST)
--

2.39.3

author reviewer
#14 lore raw

RFC PATCH 4/5 mm: shmem: add mTHP support for anonymous share pages

In reply to Baolin Wang

Commit 19eaf44954df adds multi-size THP (mTHP) for anonymous pages, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous share pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Users expect to apply the mTHP rule for all anonymous pages, including the anonymous share pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled. Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation for these anonymous shared pages.

---
 include/linux/huge_mm.h |  2 +-
 mm/huge_memory.c        |  4 +-
 mm/shmem.c              | 92 +++++++++++++++++++++++++++++++----------
 3 files changed, 74 insertions(+), 24 deletions(-)
include/linux/huge_mm.h +1 -1
index b67294d5814f..26b6fa98d8ac 100644
@@ -246,7 +246,7 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma,
 				       unsigned long orders)
 {
 	/* Optimization to check if required orders are enabled early. */
-	if (enforce_sysfs && vma_is_anonymous(vma)) {
+	if (enforce_sysfs && (vma_is_anonymous(vma) || vma_is_anon_shmem(vma))) {
 		unsigned long mask = READ_ONCE(huge_anon_orders_always);
 
 		if (vm_flags & VM_HUGEPAGE)
mm/huge_memory.c +2 -2
index 9a1b57ef9c60..9e52c0db7580 100644
@@ -86,7 +86,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
 					 unsigned long orders)
 {
 	/* Check the intersection of requested and supported orders. */
-	orders &= vma_is_anonymous(vma) ?
+	orders &= (vma_is_anonymous(vma) || vma_is_anon_shmem(vma)) ?
 			THP_ORDERS_ALL_ANON : THP_ORDERS_ALL_FILE;
 	if (!orders)
 		return 0;
@@ -152,7 +152,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
 				     !enforce_sysfs, vma->vm_mm, vm_flags)
 			? orders : 0;
 
-	if (!vma_is_anonymous(vma)) {
+	if (!vma_is_anonymous(vma) && !vma_is_anon_shmem(vma)) {
 		/*
 		 * Enforce sysfs THP requirements as necessary. Anonymous vmas
 		 * were already handled in thp_vma_allowable_orders().
mm/shmem.c +71 -21

143 lines not on this page · raw message

Signature · 1 line
2.39.3
author reviewer
#15 lore raw

RFC PATCH 5/5 mm: shmem: add anonymous share mTHP counters

In reply to Baolin Wang
---
 include/linux/huge_mm.h | 2 ++
 mm/huge_memory.c        | 4 ++++
 mm/shmem.c              | 5 ++++-
 3 files changed, 10 insertions(+), 1 deletion(-)
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
 	MTHP_STAT_ANON_SWPOUT,
 	MTHP_STAT_ANON_SWPOUT_FALLBACK,
 	MTHP_STAT_ANON_SWPIN_REFAULT,
+	MTHP_STAT_SHMEM_ANON_ALLOC,
+	MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,
 	__MTHP_STAT_COUNT
 };
 
mm/huge_memory.c +4 -0
index 9e52c0db7580..dc15240c1ab3 100644
@@ -557,6 +557,8 @@ DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback, MTHP_STAT_ANON_ALLOC_FALLBACK);
 DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT);
 DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK);
 DEFINE_MTHP_STAT_ATTR(anon_swpin_refault, MTHP_STAT_ANON_SWPIN_REFAULT);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc, MTHP_STAT_SHMEM_ANON_ALLOC);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc_fallback, MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
 
 static struct attribute *stats_attrs[] = {
 	&anon_alloc_attr.attr,
@@ -564,6 +566,8 @@ static struct attribute *stats_attrs[] = {
 	&anon_swpout_attr.attr,
 	&anon_swpout_fallback_attr.attr,
 	&anon_swpin_refault_attr.attr,
+	&shmem_anon_alloc_attr.attr,
+	&shmem_anon_alloc_fallback_attr.attr,
 	NULL,
 };
 
mm/shmem.c +4 -1
index 8b009e7040b2..4a0aa75ab29c 100644
@@ -1706,11 +1706,14 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
 			pages = 1 << order;
 			index = round_down(index, pages);
 			folio = shmem_alloc_hugefolio(gfp, info, index, order);
-			if (folio)
+			if (folio) {
+				count_mthp_stat(order, MTHP_STAT_SHMEM_ANON_ALLOC);
 				goto allocated;
+			}
 
 			if (pages == HPAGE_PMD_NR)
 				count_vm_event(THP_FILE_FALLBACK);
+			count_mthp_stat(order, MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
 			order = next_order(&orders, order);
 		}
 	} else {
Signature · 1 line
2.39.3
#16 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 13 lines · Baolin Wang
---
 include/linux/huge_mm.h | 2 ++
 mm/huge_memory.c        | 4 ++++
 mm/shmem.c              | 5 ++++-
 3 files changed, 10 insertions(+), 1 deletion(-)
include/linux/huge_mm.h +0 -0
index 26b6fa98d8ac..67b9c1acad31 100644

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 5 lines
include/linux/huge_mm.h +2 -0
@@ -270,6 +270,8 @@ enum mthp_stat_item {
        MTHP_STAT_ANON_SWPOUT,
        MTHP_STAT_ANON_SWPOUT_FALLBACK,
        MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Quoted 46 lines
include/linux/huge_mm.h +0 -0
        __MTHP_STAT_COUNT
 };
 
mm/huge_memory.c +4 -0
index 9e52c0db7580..dc15240c1ab3 100644
@@ -557,6 +557,8 @@ DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback, MTHP_STAT_ANON_ALLOC_FALLBACK);
 DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT);
 DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK);
 DEFINE_MTHP_STAT_ATTR(anon_swpin_refault, MTHP_STAT_ANON_SWPIN_REFAULT);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc, MTHP_STAT_SHMEM_ANON_ALLOC);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc_fallback, MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
 
 static struct attribute *stats_attrs[] = {
        &anon_alloc_attr.attr,
@@ -564,6 +566,8 @@ static struct attribute *stats_attrs[] = {
        &anon_swpout_attr.attr,
        &anon_swpout_fallback_attr.attr,
        &anon_swpin_refault_attr.attr,
+       &shmem_anon_alloc_attr.attr,
+       &shmem_anon_alloc_fallback_attr.attr,
        NULL,
 };
 
mm/shmem.c +4 -2
index 8b009e7040b2..4a0aa75ab29c 100644
@@ -1706,11 +1706,14 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
                        pages = 1 << order;
                        index = round_down(index, pages);
                        folio = shmem_alloc_hugefolio(gfp, info, index, order);
-                       if (folio)
+                       if (folio) {
+                               count_mthp_stat(order, MTHP_STAT_SHMEM_ANON_ALLOC);
                                goto allocated;
+                       }
 
                        if (pages == HPAGE_PMD_NR)
                                count_vm_event(THP_FILE_FALLBACK);
+                       count_mthp_stat(order, MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
                        order = next_order(&orders, order);
                }
        } else {
--
Quoted 2 lines

2.39.3

Thanks
Barry

author reviewer
#17 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang
---
  include/linux/huge_mm.h | 2 ++
  mm/huge_memory.c        | 4 ++++
  mm/shmem.c              | 5 ++++-
  3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
         MTHP_STAT_ANON_SWPOUT,
         MTHP_STAT_ANON_SWPOUT_FALLBACK,
         MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Thanks for reviewing.

IMO, I think both approaches are acceptable, which also reflects the dual nature of anonymous shared pages: on the one hand they are anonymous pages, and on the other hand, they are backed by a pseudo file. From the user's perspective, I prefer to use the term "anonymous shmem", which can be distinguished from the real file-backed THP.

Anyway, let's see what others think.

Quoted 51 lines
Quoted 46 lines
include/linux/huge_mm.h +0 -0
         __MTHP_STAT_COUNT
  };
 
mm/huge_memory.c +4 -0
index 9e52c0db7580..dc15240c1ab3 100644
@@ -557,6 +557,8 @@ DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback, MTHP_STAT_ANON_ALLOC_FALLBACK);
  DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT);
  DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK);
  DEFINE_MTHP_STAT_ATTR(anon_swpin_refault, MTHP_STAT_ANON_SWPIN_REFAULT);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc, MTHP_STAT_SHMEM_ANON_ALLOC);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc_fallback, MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
 
  static struct attribute *stats_attrs[] = {
         &anon_alloc_attr.attr,
@@ -564,6 +566,8 @@ static struct attribute *stats_attrs[] = {
         &anon_swpout_attr.attr,
         &anon_swpout_fallback_attr.attr,
         &anon_swpin_refault_attr.attr,
+       &shmem_anon_alloc_attr.attr,
+       &shmem_anon_alloc_fallback_attr.attr,
         NULL,
  };
 
mm/shmem.c +4 -2
index 8b009e7040b2..4a0aa75ab29c 100644
@@ -1706,11 +1706,14 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
                         pages = 1 << order;
                         index = round_down(index, pages);
                         folio = shmem_alloc_hugefolio(gfp, info, index, order);
-                       if (folio)
+                       if (folio) {
+                               count_mthp_stat(order, MTHP_STAT_SHMEM_ANON_ALLOC);
                                 goto allocated;
+                       }
 
                         if (pages == HPAGE_PMD_NR)
                                 count_vm_event(THP_FILE_FALLBACK);
+                       count_mthp_stat(order, MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
                         order = next_order(&orders, order);
                 }
         } else {
--
Quoted 2 lines

2.39.3

Thanks
Barry

reviewer
#18 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 31 lines · Baolin Wang
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang

---
  include/linux/huge_mm.h | 2 ++
  mm/huge_memory.c        | 4 ++++
  mm/shmem.c              | 5 ++++-

  3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
          MTHP_STAT_ANON_SWPOUT,
          MTHP_STAT_ANON_SWPOUT_FALLBACK,
          MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Thanks for reviewing.

Quoted 7 lines

IMO, I think both approaches are acceptable, which also reflects the dual nature of anonymous shared pages: on the one hand they are anonymous pages, and on the other hand, they are backed by a pseudo file. From the user's perspective, I prefer to use the term "anonymous shmem", which can be distinguished from the real file-backed THP.

Anyway, let's see what others think.

From a quick look at the code, it looks like the shmem alloc/fallback/charge events are all lumped in with FILE_THP. But the instantaneous "how many are allocated" and "how many are mapped" have their own NR_SHMEM_THPS and NR_SHMEM_PMDMAPPED counters? So its a bit inconsistent today.

My preference would be to add these to be consistent with the anon stats:

MTHP_STAT_SHMEM_FAULT_ALLOC,
MTHP_STAT_SHMEM_FAULT_FALLBACK,
MTHP_STAT_SHMEM_FAULT_FALLBACK_CHARGE,

But it looks like these aren't always allocated due to faults? So perhaps:

MTHP_STAT_SHMEM_ALLOC,
MTHP_STAT_SHMEM_FALLBACK,
MTHP_STAT_SHMEM_FALLBACK_CHARGE,

If I've understood the code correctly (I know nothing about shmem), the allocation can be for both mmap(SHARED|ANON) and for tmpfs? So "SHMEM_ANON" probably isn't quite right?

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines

         __MTHP_STAT_COUNT
  };

mm/huge_memory.c +2 -0
index 9e52c0db7580..dc15240c1ab3 100644
@@ -557,6 +557,8 @@ DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback,
 MTHP_STAT_ANON_ALLOC_FALLBACK);
   DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT);
   DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK);
   DEFINE_MTHP_STAT_ATTR(anon_swpin_refault, MTHP_STAT_ANON_SWPIN_REFAULT);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc, MTHP_STAT_SHMEM_ANON_ALLOC);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc_fallback,
 MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
 

  static struct attribute *stats_attrs[] = {
         &anon_alloc_attr.attr,

mm/huge_memory.c +2 -0
@@ -564,6 +566,8 @@ static struct attribute *stats_attrs[] = {
          &anon_swpout_attr.attr,
          &anon_swpout_fallback_attr.attr,
          &anon_swpin_refault_attr.attr,
+       &shmem_anon_alloc_attr.attr,
+       &shmem_anon_alloc_fallback_attr.attr,
          NULL,
   };
 
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
mm/shmem.c +0 -0
index 8b009e7040b2..4a0aa75ab29c 100644
@@ -1706,11 +1706,14 @@ static struct folio *shmem_alloc_and_add_folio(struct
Quoted 8 lines

mm/shmem.c static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,

Quoted 8 lines
mm/shmem.c +2 -1
 vm_fault *vmf,
                          pages = 1 << order;
                          index = round_down(index, pages);
                          folio = shmem_alloc_hugefolio(gfp, info, index, order);
-                       if (folio)
+                       if (folio) {
+                               count_mthp_stat(order,
 MTHP_STAT_SHMEM_ANON_ALLOC);

is there any reason why this can't go next to the existing PMD-size stat?

Quoted 16 lines
Quoted 16 lines
Quoted 7 lines
mm/shmem.c +2 -0
                                  goto allocated;
+                       }
 
                          if (pages == HPAGE_PMD_NR)
                                  count_vm_event(THP_FILE_FALLBACK);
+                       count_mthp_stat(order,
 MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
Quoted 6 lines

                         order = next_order(&orders, order); }
         } else {

Signature · 1 line
2.39.3

Thanks
Barry

author reviewer
#19 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 51 lines · Ryan Roberts
Quoted 31 lines · Baolin Wang
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang
---
   include/linux/huge_mm.h | 2 ++
   mm/huge_memory.c        | 4 ++++
   mm/shmem.c              | 5 ++++-
   3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
          MTHP_STAT_ANON_SWPOUT,
          MTHP_STAT_ANON_SWPOUT_FALLBACK,
          MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Thanks for reviewing.

Quoted 7 lines

IMO, I think both approaches are acceptable, which also reflects the dual nature of anonymous shared pages: on the one hand they are anonymous pages, and on the other hand, they are backed by a pseudo file. From the user's perspective, I prefer to use the term "anonymous shmem", which can be distinguished from the real file-backed THP.

Anyway, let's see what others think.

From a quick look at the code, it looks like the shmem alloc/fallback/charge
events are all lumped in with FILE_THP. But the instantaneous "how many are allocated" and "how many are mapped" have their own NR_SHMEM_THPS and NR_SHMEM_PMDMAPPED counters? So its a bit inconsistent today.

My preference would be to add these to be consistent with the anon stats:

MTHP_STAT_SHMEM_FAULT_ALLOC,
MTHP_STAT_SHMEM_FAULT_FALLBACK,
MTHP_STAT_SHMEM_FAULT_FALLBACK_CHARGE,

Quoted 5 lines

But it looks like these aren't always allocated due to faults? So perhaps:

MTHP_STAT_SHMEM_ALLOC,
MTHP_STAT_SHMEM_FALLBACK,
MTHP_STAT_SHMEM_FALLBACK_CHARGE,

This looks good to me.

Quoted 2 lines

If I've understood the code correctly (I know nothing about shmem), the allocation can be for both mmap(SHARED|ANON) and for tmpfs? So "SHMEM_ANON"

This is allowed, but the 'fd' for tmpfs will be ignored (see ksys_mmap_pgoff()), which is same with anonymous shmem.

Quoted 31 lines

probably isn't quite right?

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
include/linux/huge_mm.h +0 -0
          __MTHP_STAT_COUNT
   };
 
mm/huge_memory.c +2 -0
index 9e52c0db7580..dc15240c1ab3 100644
@@ -557,6 +557,8 @@ DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback,
 MTHP_STAT_ANON_ALLOC_FALLBACK);
   DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT);
   DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK);
   DEFINE_MTHP_STAT_ATTR(anon_swpin_refault, MTHP_STAT_ANON_SWPIN_REFAULT);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc, MTHP_STAT_SHMEM_ANON_ALLOC);
+DEFINE_MTHP_STAT_ATTR(shmem_anon_alloc_fallback,
 MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK);
 

  static struct attribute *stats_attrs[] = {
         &anon_alloc_attr.attr,

mm/huge_memory.c +2 -0
@@ -564,6 +566,8 @@ static struct attribute *stats_attrs[] = {
          &anon_swpout_attr.attr,
          &anon_swpout_fallback_attr.attr,
          &anon_swpin_refault_attr.attr,
+       &shmem_anon_alloc_attr.attr,
+       &shmem_anon_alloc_fallback_attr.attr,
          NULL,
   };
 
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
mm/shmem.c +0 -0
index 8b009e7040b2..4a0aa75ab29c 100644
@@ -1706,11 +1706,14 @@ static struct folio *shmem_alloc_and_add_folio(struct
Quoted 8 lines
Quoted 8 lines

mm/shmem.c static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,

Quoted 8 lines
mm/shmem.c +2 -1
 vm_fault *vmf,
                          pages = 1 << order;
                          index = round_down(index, pages);
                          folio = shmem_alloc_hugefolio(gfp, info, index, order);
-                       if (folio)
+                       if (folio) {
+                               count_mthp_stat(order,
 MTHP_STAT_SHMEM_ANON_ALLOC);

is there any reason why this can't go next to the existing PMD-size stat?

No, will move to the existing PMD-size stat.

#20 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

In reply to Barry Song

On 2024/4/23 09:17, Barry Song wrote:
[...]

Quoted 7 lines
Quoted 7 lines
include/linux/huge_mm.h +0 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
         MTHP_STAT_ANON_SWPOUT,
         MTHP_STAT_ANON_SWPOUT_FALLBACK,
Quoted 8 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 3 lines
include/linux/huge_mm.h +2 -0
         MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

+1

IMO, shmem isn't actually file-backed, but it has file-backed-like characteristics :)

FWIW, perhaps MTHP_STAT_FILE_ALLOC and MTHP_STAT_FILE_ALLOC_FALLBACK would better align with PMD/PTE-mapped THP.

Thanks,
Lance

#21 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 13 lines · Lance Yang

On 2024/4/23 09:17, Barry Song wrote:
[...]

Quoted 10 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
         MTHP_STAT_ANON_SWPOUT,
         MTHP_STAT_ANON_SWPOUT_FALLBACK,
         MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

Seems like you forgot to add the FILE_FALLBACK_CHARGE counter in this patch :)

IIUC, you've excluded the THP_FILE_FALLBACK_CHARGE counter for PTE-mapped mTHP that size < PMD in patch3.

Thanks,
Lance

author reviewer
#22 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 18 lines · Lance Yang
Quoted 13 lines · Lance Yang

On 2024/4/23 09:17, Barry Song wrote:
[...]

Quoted 10 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
          MTHP_STAT_ANON_SWPOUT,
          MTHP_STAT_ANON_SWPOUT_FALLBACK,
          MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

Seems like you forgot to add the FILE_FALLBACK_CHARGE counter in this patch :)

Quoted 2 lines

IIUC, you've excluded the THP_FILE_FALLBACK_CHARGE counter for PTE-mapped mTHP that size < PMD in patch3.

Yes, will add in next version.

#23 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

In reply to Barry Song
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang
---
  include/linux/huge_mm.h | 2 ++
  mm/huge_memory.c        | 4 ++++
  mm/shmem.c              | 5 ++++-
  3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
         MTHP_STAT_ANON_SWPOUT,
         MTHP_STAT_ANON_SWPOUT_FALLBACK,
         MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

The term "anonymous share" in the patch subject is weird to begin with ;) Easy to confuse with anonymous cow-shared memory. Let's just call it "anonymous shmem", which it is under the hood.

... regarding the question: if we add FILE_ALLOC and friends, at least initially, we wouldn't account other large pagecache folios.

... likely we should add that then as well so the counter matches the actual name?

If we later realize that we need separate FILE vs. SHMEM vs. WHATEVER counters, we can always add more fine-grained counters later. Doing it consistently w.r.t. traditional THPs first sounds reasonable.

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#24 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 27 lines · David Hildenbrand
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang

---
  include/linux/huge_mm.h | 2 ++
  mm/huge_memory.c        | 4 ++++
  mm/shmem.c              | 5 ++++-

  3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
          MTHP_STAT_ANON_SWPOUT,
          MTHP_STAT_ANON_SWPOUT_FALLBACK,
          MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Quoted 1 line

The term "anonymous share" in the patch subject is weird to begin with ;) Easy to confuse with anonymous cow-shared memory. Let's just call it "anonymous shmem", which it is under the hood.

Sure.

Quoted 5 lines

... regarding the question: if we add FILE_ALLOC and friends, at least initially, we wouldn't account other large pagecache folios.

... likely we should add that then as well so the counter matches the actual name?

If we later realize that we need separate FILE vs. SHMEM vs. WHATEVER counters, we can always add more fine-grained counters later. Doing it consistently w.r.t. traditional THPs first sounds reasonable.

Um, once we expose it to userspace through the sysfs interface, the sysfs interface should be explicit as much as possible and avoid confusing users, otherwise it will be difficult to change this kind of interface in the future. Personally, I prefer to Ryan's suggestion.

#25 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 42 lines · Baolin Wang
Quoted 27 lines · David Hildenbrand
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang
---
   include/linux/huge_mm.h | 2 ++
   mm/huge_memory.c        | 4 ++++
   mm/shmem.c              | 5 ++++-
   3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
          MTHP_STAT_ANON_SWPOUT,
          MTHP_STAT_ANON_SWPOUT_FALLBACK,
          MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Quoted 3 lines

The term "anonymous share" in the patch subject is weird to begin with ;) Easy to confuse with anonymous cow-shared memory. Let's just call it "anonymous shmem", which it is under the hood.

Sure.

Quoted 6 lines

... regarding the question: if we add FILE_ALLOC and friends, at least initially, we wouldn't account other large pagecache folios.

... likely we should add that then as well so the counter matches the actual name?

Quoted 8 lines
Quoted 3 lines

If we later realize that we need separate FILE vs. SHMEM vs. WHATEVER counters, we can always add more fine-grained counters later. Doing it consistently w.r.t. traditional THPs first sounds reasonable.

Um, once we expose it to userspace through the sysfs interface, the sysfs interface should be explicit as much as possible and avoid confusing users, otherwise it will be difficult to change this kind of interface in the future. Personally, I prefer to Ryan's suggestion.

Inconsistency is confusing. As long as you avoid that, I don't particularly care.

Signature · 3 lines
Cheers,

David / dhildenb
reviewer
#26 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 47 lines · David Hildenbrand
Quoted 46 lines · Baolin Wang
Quoted 27 lines · David Hildenbrand
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang

---
   include/linux/huge_mm.h | 2 ++
   mm/huge_memory.c        | 4 ++++
   mm/shmem.c              | 5 ++++-

   3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
           MTHP_STAT_ANON_SWPOUT,
           MTHP_STAT_ANON_SWPOUT_FALLBACK,
           MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Quoted 3 lines

The term "anonymous share" in the patch subject is weird to begin with ;) Easy to confuse with anonymous cow-shared memory. Let's just call it "anonymous shmem", which it is under the hood.

Sure.

Quoted 9 lines

... regarding the question: if we add FILE_ALLOC and friends, at least initially, we wouldn't account other large pagecache folios.

... likely we should add that then as well so the counter matches the actual name?

If we later realize that we need separate FILE vs. SHMEM vs. WHATEVER counters, we can always add more fine-grained counters later. Doing it consistently w.r.t. traditional THPs first sounds reasonable.

Quoted 6 lines
Quoted 4 lines

Um, once we expose it to userspace through the sysfs interface, the sysfs interface should be explicit as much as possible and avoid confusing users, otherwise it will be difficult to change this kind of interface in the future. Personally, I prefer to Ryan's suggestion.

Inconsistency is confusing. As long as you avoid that, I don't particularly care.

This is a good point. We have been careful to make sure the 2M ANON mTHP stats match the existing PMD-size stats. So we should definitely make sure that any future 2M FILE mTHP stats match too, which I guess means counting both SHMEM and FILE events.

So perhaps it makes more sense to add FILE counters to start with. If we need the SHMEM-specific counters, we could add them later?

I'm happy to go with the crowd on this...

author reviewer
#27 lore raw

Re: [RFC PATCH 5/5] mm: shmem: add anonymous share mTHP counters

Quoted 60 lines · Ryan Roberts
Quoted 47 lines · David Hildenbrand
Quoted 46 lines · Baolin Wang
Quoted 27 lines · David Hildenbrand
Quoted 10 lines · Barry Song
Quoted 8 lines · Baolin Wang
---
    include/linux/huge_mm.h | 2 ++
    mm/huge_memory.c        | 4 ++++
    mm/shmem.c              | 5 ++++-
    3 files changed, 10 insertions(+), 1 deletion(-)
Quoted 15 lines

include/linux/huge_mm.h enum mthp_stat_item { · line 274

Quoted 10 lines
include/linux/huge_mm.h +2 -0
index 26b6fa98d8ac..67b9c1acad31 100644
@@ -270,6 +270,8 @@ enum mthp_stat_item {
           MTHP_STAT_ANON_SWPOUT,
           MTHP_STAT_ANON_SWPOUT_FALLBACK,
           MTHP_STAT_ANON_SWPIN_REFAULT,
+       MTHP_STAT_SHMEM_ANON_ALLOC,
+       MTHP_STAT_SHMEM_ANON_ALLOC_FALLBACK,

not quite sure about this. for 2MB pmd-mapped THP shmem, we count them as FILE_THP.
here we are counting as SHMEM_ANON. To me, SHMEM_ANON is more correct but it doesn't align with pmd-mapped THP. David, Ryan, what do you think?

Quoted 3 lines

The term "anonymous share" in the patch subject is weird to begin with ;) Easy to confuse with anonymous cow-shared memory. Let's just call it "anonymous shmem", which it is under the hood.

Sure.

Quoted 9 lines

... regarding the question: if we add FILE_ALLOC and friends, at least initially, we wouldn't account other large pagecache folios.

... likely we should add that then as well so the counter matches the actual name?

If we later realize that we need separate FILE vs. SHMEM vs. WHATEVER counters, we can always add more fine-grained counters later. Doing it consistently w.r.t. traditional THPs first sounds reasonable.

Quoted 6 lines
Quoted 4 lines

Um, once we expose it to userspace through the sysfs interface, the sysfs interface should be explicit as much as possible and avoid confusing users, otherwise it will be difficult to change this kind of interface in the future. Personally, I prefer to Ryan's suggestion.

Inconsistency is confusing. As long as you avoid that, I don't particularly care.

This is a good point. We have been careful to make sure the 2M ANON mTHP stats match the existing PMD-size stats. So we should definitely make sure that any future 2M FILE mTHP stats match too, which I guess means counting both SHMEM and FILE events.

Quoted 4 lines

So perhaps it makes more sense to add FILE counters to start with. If we need the SHMEM-specific counters, we could add them later?

I'm happy to go with the crowd on this...

(Seems I'm the only one who prefers the term 'SHMEM_' now.) Fine, I have no strong preference, and let's keep consistency first. Thanks guys.

reviewer
#28 lore raw
In reply to Baolin Wang
Quoted 4 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 8 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled. Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

Quoted 22 lines

TODO:
- More testing and provide some performance data.
- Need more discussion about the large folio allocation strategy for a 'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify the 'file' size, which need to follow the anonymous mTHP rule too?
- Do not split the large folio when share memory swap out.
- Can swap in a large folio for share memory.

Baolin Wang (5):
mm: memory: extend finish_fault() to support large folio
mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
mm: shmem: add THP validation for PMD-mapped THP related statistics
mm: shmem: add mTHP support for anonymous share pages
mm: shmem: add anonymous share mTHP counters

 include/linux/huge_mm.h |   4 +-
 mm/huge_memory.c        |   8 ++-
 mm/memory.c             |  25 +++++++---
 mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------
 4 files changed, 108 insertions(+), 36 deletions(-)
#29 lore raw
Quoted 36 lines · Ryan Roberts
Quoted 4 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 8 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled. Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

Quoted 8 lines

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Matches what I had in mind.

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#30 lore raw
In reply to Ryan Roberts
Quoted 14 lines · Ryan Roberts
Quoted 4 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 8 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 4 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks for those workloads) that predominantly use shared anon memory?

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Quoted 11 lines
Quoted 7 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled. Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 5 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

Quoted 8 lines

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

Quoted 2 lines

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

IMO, I do not want to change the global 'huge=' mount option, which can control both anon shmem and tmpfs, but mTHP now is only applied for anon shmem. So let's keep it be same with the global sysfs interface: /sys/kernel/mm/transparent_hugepage/shmem_enabled.

For tmpfs large folio strategy, I plan to address it later, and we may need more discussion to determine if it should follow the file large folio strategy or not (no investigation now).

Thanks for reviewing.

Quoted 21 lines
Quoted 21 lines

TODO:
- More testing and provide some performance data.
- Need more discussion about the large folio allocation strategy for a 'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify the 'file' size, which need to follow the anonymous mTHP rule too?
- Do not split the large folio when share memory swap out.
- Can swap in a large folio for share memory.

Baolin Wang (5):
mm: memory: extend finish_fault() to support large folio
mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
mm: shmem: add THP validation for PMD-mapped THP related statistics
mm: shmem: add mTHP support for anonymous share pages
mm: shmem: add anonymous share mTHP counters

  include/linux/huge_mm.h |   4 +-
  mm/huge_memory.c        |   8 ++-
  mm/memory.c             |  25 +++++++---
  mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------
  4 files changed, 108 insertions(+), 36 deletions(-)
reviewer
#31 lore raw
Quoted 23 lines · Baolin Wang
Quoted 15 lines · Ryan Roberts
Quoted 5 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 8 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 4 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks for those workloads) that predominantly use shared anon memory?

Quoted 6 lines

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 37 lines
Quoted 13 lines
Quoted 12 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'. The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 3 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 5 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

Quoted 8 lines

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough understanding is:

- /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
- huge=... controls tmpfs allocations
- deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as if they were mounted with huge=never or huge=always

Is that correct? If so, then I think it still makes sense to support per-size deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

Quoted 6 lines
Quoted 3 lines

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

IMO, I do not want to change the global 'huge=' mount option, which can control both anon shmem and tmpfs, but mTHP now is only applied for anon shmem. So let's

How does huge= control anon shmem? I thought it was only for mounted filesystems; so tmpfs? Perhaps my mental model for how this works is broken...

Quoted 6 lines

keep it be same with the global sysfs interface:
/sys/kernel/mm/transparent_hugepage/shmem_enabled.

For tmpfs large folio strategy, I plan to address it later, and we may need more discussion to determine if it should follow the file large folio strategy or not (no investigation now).

OK. But until you get to tmpfs, you'll need an interim definition for what it means if a per-size control is set to "inherit" and the top-level control is set to deny/force.

Quoted 2 lines

Thanks for reviewing.

No problem! Thanks for doing the work!

Quoted 24 lines
Quoted 24 lines
Quoted 24 lines

TODO:
  - More testing and provide some performance data.
  - Need more discussion about the large folio allocation strategy for a 'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify
the 'file' size, which need to follow the anonymous mTHP rule too?
  - Do not split the large folio when share memory swap out.
  - Can swap in a large folio for share memory.

Baolin Wang (5):
   mm: memory: extend finish_fault() to support large folio
   mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
   mm: shmem: add THP validation for PMD-mapped THP related statistics
   mm: shmem: add mTHP support for anonymous share pages
   mm: shmem: add anonymous share mTHP counters

  include/linux/huge_mm.h |   4 +-
  mm/huge_memory.c        |   8 ++-
  mm/memory.c             |  25 +++++++---
  mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------

  4 files changed, 108 insertions(+), 36 deletions(-)
author reviewer
#32 lore raw
Quoted 79 lines · Ryan Roberts
Quoted 23 lines · Baolin Wang
Quoted 15 lines · Ryan Roberts
Quoted 5 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 8 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 4 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks for those workloads) that predominantly use shared anon memory?

Quoted 6 lines

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 37 lines
Quoted 13 lines
Quoted 12 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'. The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 3 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 5 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

Quoted 8 lines

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough understanding is:

Quoted 8 lines

- /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
- huge=... controls tmpfs allocations
- deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as if they were mounted with huge=never or huge=always

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 2 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation, so we should not use anonymous control to override tmpfs control, which seems a little mess?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

IMO, I do not want to change the global 'huge=' mount option, which can control both anon shmem and tmpfs, but mTHP now is only applied for anon shmem. So let's

How does huge= control anon shmem? I thought it was only for mounted filesystems; so tmpfs? Perhaps my mental model for how this works is broken...

Sorry for noise, you are right. So this is still the reason I don't want to change the semantics of 'huge=', which is used to control tmpfs.

Quoted 40 lines
Quoted 6 lines

keep it be same with the global sysfs interface:
/sys/kernel/mm/transparent_hugepage/shmem_enabled.

For tmpfs large folio strategy, I plan to address it later, and we may need more discussion to determine if it should follow the file large folio strategy or not (no investigation now).

OK. But until you get to tmpfs, you'll need an interim definition for what it means if a per-size control is set to "inherit" and the top-level control is set to deny/force.

Quoted 2 lines

Thanks for reviewing.

No problem! Thanks for doing the work!

Quoted 24 lines
Quoted 24 lines
Quoted 24 lines

TODO:
  - More testing and provide some performance data.
  - Need more discussion about the large folio allocation strategy for a
'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify
the 'file' size, which need to follow the anonymous mTHP rule too?
  - Do not split the large folio when share memory swap out.
  - Can swap in a large folio for share memory.

Baolin Wang (5):
   mm: memory: extend finish_fault() to support large folio
   mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
   mm: shmem: add THP validation for PMD-mapped THP related statistics
   mm: shmem: add mTHP support for anonymous share pages
   mm: shmem: add anonymous share mTHP counters

   include/linux/huge_mm.h |   4 +-
   mm/huge_memory.c        |   8 ++-
   mm/memory.c             |  25 +++++++---
   mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------
   4 files changed, 108 insertions(+), 36 deletions(-)
reviewer
#33 lore raw
Quoted 95 lines · Baolin Wang
Quoted 81 lines · Ryan Roberts
Quoted 25 lines · Baolin Wang
Quoted 16 lines · Ryan Roberts
Quoted 5 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 9 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 6 lines

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 37 lines
Quoted 13 lines
Quoted 12 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'. The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 3 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 5 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

Quoted 8 lines

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough understanding is:

Quoted 8 lines

  - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
  - huge=... controls tmpfs allocations
  - deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as if they were mounted with huge=never or huge=always

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 7 lines
Quoted 2 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation, so we should not use anonymous control to override tmpfs control, which seems a little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

echo deny > /mm/transparent_hugepage/shmem_enabled
echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 53 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

IMO, I do not want to change the global 'huge=' mount option, which can control both anon shmem and tmpfs, but mTHP now is only applied for anon shmem. So let's

How does huge= control anon shmem? I thought it was only for mounted filesystems; so tmpfs? Perhaps my mental model for how this works is broken...

Sorry for noise, you are right. So this is still the reason I don't want to change the semantics of 'huge=', which is used to control tmpfs.

Quoted 40 lines
Quoted 6 lines

keep it be same with the global sysfs interface:
/sys/kernel/mm/transparent_hugepage/shmem_enabled.

For tmpfs large folio strategy, I plan to address it later, and we may need more discussion to determine if it should follow the file large folio strategy or not (no investigation now).

OK. But until you get to tmpfs, you'll need an interim definition for what it means if a per-size control is set to "inherit" and the top-level control is set to deny/force.

Quoted 2 lines

Thanks for reviewing.

No problem! Thanks for doing the work!

Quoted 24 lines
Quoted 24 lines
Quoted 24 lines

TODO:
   - More testing and provide some performance data.
   - Need more discussion about the large folio allocation strategy for a 'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify
the 'file' size, which need to follow the anonymous mTHP rule too?
   - Do not split the large folio when share memory swap out.
   - Can swap in a large folio for share memory.

Baolin Wang (5):
    mm: memory: extend finish_fault() to support large folio
    mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
    mm: shmem: add THP validation for PMD-mapped THP related statistics
    mm: shmem: add mTHP support for anonymous share pages
    mm: shmem: add anonymous share mTHP counters

   include/linux/huge_mm.h |   4 +-
   mm/huge_memory.c        |   8 ++-
   mm/memory.c             |  25 +++++++---
   mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------

   4 files changed, 108 insertions(+), 36 deletions(-)
author reviewer
#34 lore raw
Quoted 104 lines · Ryan Roberts
Quoted 95 lines · Baolin Wang
Quoted 81 lines · Ryan Roberts
Quoted 25 lines · Baolin Wang
Quoted 16 lines · Ryan Roberts
Quoted 5 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 9 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 6 lines

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 37 lines
Quoted 13 lines
Quoted 12 lines

The primary strategy is that, the use of huge pages for anonymous shared pages still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'. The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 3 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 5 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

Quoted 8 lines

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled, plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough understanding is:

Quoted 8 lines

  - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
  - huge=... controls tmpfs allocations
  - deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as if they were mounted with huge=never or huge=always

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 7 lines
Quoted 2 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation, so we should not use anonymous control to override tmpfs control, which seems a little mess?

Quoted 6 lines

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

Quoted 57 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 53 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

IMO, I do not want to change the global 'huge=' mount option, which can control both anon shmem and tmpfs, but mTHP now is only applied for anon shmem. So let's

How does huge= control anon shmem? I thought it was only for mounted filesystems; so tmpfs? Perhaps my mental model for how this works is broken...

Sorry for noise, you are right. So this is still the reason I don't want to change the semantics of 'huge=', which is used to control tmpfs.

Quoted 40 lines
Quoted 6 lines

keep it be same with the global sysfs interface:
/sys/kernel/mm/transparent_hugepage/shmem_enabled.

For tmpfs large folio strategy, I plan to address it later, and we may need more discussion to determine if it should follow the file large folio strategy or not (no investigation now).

OK. But until you get to tmpfs, you'll need an interim definition for what it means if a per-size control is set to "inherit" and the top-level control is set to deny/force.

Quoted 2 lines

Thanks for reviewing.

No problem! Thanks for doing the work!

Quoted 24 lines
Quoted 24 lines
Quoted 24 lines

TODO:
   - More testing and provide some performance data.
   - Need more discussion about the large folio allocation strategy for a
'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to specify
the 'file' size, which need to follow the anonymous mTHP rule too?
   - Do not split the large folio when share memory swap out.
   - Can swap in a large folio for share memory.

Baolin Wang (5):
    mm: memory: extend finish_fault() to support large folio
    mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
    mm: shmem: add THP validation for PMD-mapped THP related statistics
    mm: shmem: add mTHP support for anonymous share pages
    mm: shmem: add anonymous share mTHP counters

    include/linux/huge_mm.h |   4 +-
    mm/huge_memory.c        |   8 ++-
    mm/memory.c             |  25 +++++++---
    mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------
    4 files changed, 108 insertions(+), 36 deletions(-)
reviewer
#35 lore raw
Quoted 121 lines · Baolin Wang
Quoted 118 lines · Ryan Roberts
Quoted 103 lines · Baolin Wang
Quoted 96 lines · Ryan Roberts
Quoted 26 lines · Baolin Wang
Quoted 17 lines · Ryan Roberts
Quoted 5 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 10 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages,
also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 6 lines

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 42 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared pages
still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 3 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 9 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 7 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce
/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

   - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
   - huge=... controls tmpfs allocations
   - deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as
     if they were mounted with huge=never or huge=always

Quoted 1 line

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 8 lines
Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation, so we should not use anonymous control to override tmpfs control, which seems a little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 5 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to modify the value in /mm/transparent_hugepage/enabled.

Quoted 4 lines
Quoted 4 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

Quoted 60 lines
Quoted 60 lines
Quoted 60 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

Of course the huge= mount option would also need to take a per-size option in this case. e.g. huge=2048kB:advise,64kB:always

Quoted 7 lines
Quoted 4 lines

IMO, I do not want to change the global 'huge=' mount option, which can control
both anon shmem and tmpfs, but mTHP now is only applied for anon shmem. So let's

How does huge= control anon shmem? I thought it was only for mounted filesystems; so tmpfs? Perhaps my mental model for how this works is broken...

Sorry for noise, you are right. So this is still the reason I don't want to change the semantics of 'huge=', which is used to control tmpfs.

Quoted 44 lines
Quoted 8 lines

keep it be same with the global sysfs interface:
/sys/kernel/mm/transparent_hugepage/shmem_enabled.

For tmpfs large folio strategy, I plan to address it later, and we may need more
discussion to determine if it should follow the file large folio strategy or not
(no investigation now).

OK. But until you get to tmpfs, you'll need an interim definition for what it means if a per-size control is set to "inherit" and the top-level control is set
to deny/force.

Quoted 2 lines

Thanks for reviewing.

No problem! Thanks for doing the work!

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

TODO:
    - More testing and provide some performance data.
    - Need more discussion about the large folio allocation strategy for a 'regular
file' operation created by memfd_create(), for example using ftruncate(fd) to
specify
the 'file' size, which need to follow the anonymous mTHP rule too?
    - Do not split the large folio when share memory swap out.
    - Can swap in a large folio for share memory.

Baolin Wang (5):
     mm: memory: extend finish_fault() to support large folio
     mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
     mm: shmem: add THP validation for PMD-mapped THP related statistics
     mm: shmem: add mTHP support for anonymous share pages
     mm: shmem: add anonymous share mTHP counters

    include/linux/huge_mm.h |   4 +-
    mm/huge_memory.c        |   8 ++-
    mm/memory.c             |  25 +++++++---
    mm/shmem.c              | 107 ++++++++++++++++++++++++++++++----------

    4 files changed, 108 insertions(+), 36 deletions(-)
author reviewer
#36 lore raw
Quoted 140 lines · Ryan Roberts
Quoted 121 lines · Baolin Wang
Quoted 118 lines · Ryan Roberts
Quoted 103 lines · Baolin Wang
Quoted 96 lines · Ryan Roberts
Quoted 26 lines · Baolin Wang
Quoted 17 lines · Ryan Roberts
Quoted 5 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 10 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages,
also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 6 lines

As far as I know, some database related workloads make extensive use of shared anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 42 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared pages
still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 3 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 9 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 7 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce
/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

   - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
   - huge=... controls tmpfs allocations
   - deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they
act as
     if they were mounted with huge=never or huge=always

Quoted 1 line

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 8 lines
Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation, so we should not use anonymous control to override tmpfs control, which seems a little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 5 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

Quoted 8 lines

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but I think it should follow the same strategy as anon mTHP as you said.

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

OK. And I also prefer that "force" should be a noop too, since anon shmem control should not configure tmpfs huge page allocation.

reviewer
#37 lore raw
Quoted 171 lines · Baolin Wang
Quoted 151 lines · Ryan Roberts
Quoted 132 lines · Baolin Wang
Quoted 129 lines · Ryan Roberts
Quoted 115 lines · Baolin Wang
Quoted 104 lines · Ryan Roberts
Quoted 30 lines · Baolin Wang
Quoted 21 lines · Ryan Roberts
Quoted 7 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation
through commit 19eaf44954df, that can allow THP to be configured through the
sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 12 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages,
also including the anonymous shared pages, in order to enjoy the benefits of
mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss
etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

As far as I know, some database related workloads make extensive use of shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also
uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 44 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared pages
still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 4 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to
depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 9 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 8 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the
same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

    - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
    - huge=... controls tmpfs allocations
    - deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as
      if they were mounted with huge=never or huge=always

Quoted 1 line

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

Quoted 6 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation,
so we should not use anonymous control to override tmpfs control, which seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 5 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

Quoted 8 lines

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 7 lines
Quoted 4 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

OK. And I also prefer that "force" should be a noop too, since anon shmem control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force" will be an alias for "always" and "deny" will be an alias for "never"?

I was just a bit concerned about later changing that behavior to also impact tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it, I don't see that as a problem.

#38 lore raw
Quoted 180 lines · Ryan Roberts
Quoted 171 lines · Baolin Wang
Quoted 151 lines · Ryan Roberts
Quoted 132 lines · Baolin Wang
Quoted 129 lines · Ryan Roberts
Quoted 115 lines · Baolin Wang
Quoted 104 lines · Ryan Roberts
Quoted 30 lines · Baolin Wang
Quoted 21 lines · Ryan Roberts
Quoted 7 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation
through commit 19eaf44954df, that can allow THP to be configured through the
sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 12 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages,
also including the anonymous shared pages, in order to enjoy the benefits of
mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss
etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

As far as I know, some database related workloads make extensive use of shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also
uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 44 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared pages
still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 4 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to
depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 9 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 8 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the
same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

    - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
    - huge=... controls tmpfs allocations
    - deny and force in shmem_enabled are equivalent to never and always for mmap(SHARED|ANON) but additionally override all tmpfs mounts so they
act as
      if they were mounted with huge=never or huge=always

Quoted 1 line

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

Quoted 6 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation,
so we should not use anonymous control to override tmpfs control, which seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 5 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

Quoted 8 lines

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 7 lines
Quoted 4 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

OK. And I also prefer that "force" should be a noop too, since anon shmem control should not configure tmpfs huge page allocation.

Quoted 6 lines

I guess technically they won't be noops, but (for the non-PMD-sizes) "force" will be an alias for "always" and "deny" will be an alias for "never"?

I was just a bit concerned about later changing that behavior to also impact tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it, I don't see that as a problem.

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Signature · 3 lines
Cheers,

David / dhildenb
reviewer
#39 lore raw
Quoted 194 lines · David Hildenbrand
Quoted 193 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 165 lines · Ryan Roberts
Quoted 140 lines · Baolin Wang
Quoted 137 lines · Ryan Roberts
Quoted 123 lines · Baolin Wang
Quoted 111 lines · Ryan Roberts
Quoted 33 lines · Baolin Wang
Quoted 24 lines · Ryan Roberts
Quoted 7 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation
through commit 19eaf44954df, that can allow THP to be configured through the
sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 15 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios,
therefore, users expect to apply an unified mTHP strategy for anonymous pages,
also including the anonymous shared pages, in order to enjoy the benefits of
mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss
etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

As far as I know, some database related workloads make extensive use of shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also
uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 47 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared pages
still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 5 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to
depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 10 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by
default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 8 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the
same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

     - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
     - huge=... controls tmpfs allocations
     - deny and force in shmem_enabled are equivalent to never and always for
       mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as
       if they were mounted with huge=never or huge=always

Quoted 2 lines

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

Quoted 6 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation,
so we should not use anonymous control to override tmpfs control, which seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 6 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 4 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 7 lines
Quoted 4 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

OK. And I also prefer that "force" should be a noop too, since anon shmem control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force" will be an alias for "always" and "deny" will be an alias for "never"?

Quoted 6 lines
Quoted 3 lines

I was just a bit concerned about later changing that behavior to also impact tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it, I don't see that as a problem.

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Well I think it amounts to the same thing; there isn't much point in forbidding "force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

So I think the question is just 'what should happen when "force" is configured for a non-PMD-sized mTHP'?

I think the answer is 'for now, it behaves like "always", but in future it will also force any tmpfs mounts to consider that mTHP size even if that size wasn't specified by mount='

#40 lore raw
Quoted 199 lines · Ryan Roberts
Quoted 198 lines · David Hildenbrand
Quoted 196 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 165 lines · Ryan Roberts
Quoted 140 lines · Baolin Wang
Quoted 137 lines · Ryan Roberts
Quoted 123 lines · Baolin Wang
Quoted 111 lines · Ryan Roberts
Quoted 33 lines · Baolin Wang
Quoted 24 lines · Ryan Roberts
Quoted 7 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation
through commit 19eaf44954df, that can allow THP to be configured through the
sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

Quoted 15 lines

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios,
therefore, users expect to apply an unified mTHP strategy for anonymous pages,
also including the anonymous shared pages, in order to enjoy the benefits of
mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss
etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

As far as I know, some database related workloads make extensive use of shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also
uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 47 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared pages
still follows the global control determined by the mount option "huge=" parameter
or the sysfs interface at '/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is enabled.
Subsequently, the mTHP sysfs interface (/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled) is checked to determine the mTHP size that can be used for large folio allocation
for these anonymous shared pages.

Quoted 5 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to
depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 10 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by
default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 8 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the
same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

     - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
     - huge=... controls tmpfs allocations
     - deny and force in shmem_enabled are equivalent to never and
always for
       mmap(SHARED|ANON) but additionally override all tmpfs mounts so they
act as
       if they were mounted with huge=never or huge=always

Quoted 2 lines

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

Quoted 6 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation,
so we should not use anonymous control to override tmpfs control, which seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 6 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 4 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 7 lines
Quoted 4 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

OK. And I also prefer that "force" should be a noop too, since anon shmem control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force" will be an alias for "always" and "deny" will be an alias for "never"?

I was just a bit concerned about later changing that behavior to also impact tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it, I don't see that as a problem.

Quoted 8 lines
Quoted 2 lines

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Well I think it amounts to the same thing; there isn't much point in forbidding "force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

The default for PMD should be "inherit", for the other mTHP sizes it should be "never".

So we should fail if:
* Setting top-level to "force" when any non-PMD size is "inherit"
* Setting "inherit" of a non-PMD size when the top-level is force

Both will only happen if someone messes with the mTHP configuration manually.

And we should only offer "force" as an option for PMD-sized mTHP as long as the others are not supported. See below.

Quoted 3 lines

So I think the question is just 'what should happen when "force" is configured for a non-PMD-sized mTHP'?

We should hide it and not offer a configuration toggle that is inactive.

If someone wants to sense support for other mTHP "force" settings in the future, they can just parse if the "shmem_enabled" toggle offers "force" as an option. Then they know that it can actually be enabled and will also do what is promised.

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#41 lore raw
Quoted 209 lines · David Hildenbrand
Quoted 199 lines · Ryan Roberts
Quoted 198 lines · David Hildenbrand
Quoted 196 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 165 lines · Ryan Roberts
Quoted 140 lines · Baolin Wang
Quoted 137 lines · Ryan Roberts
Quoted 123 lines · Baolin Wang
Quoted 111 lines · Ryan Roberts
Quoted 33 lines · Baolin Wang
Quoted 24 lines · Ryan Roberts
Quoted 22 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP)
allocation
through commit 19eaf44954df, that can allow THP to be configured
through the
sysfs interface located at
'/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule
configured through the sysfs interface, and can only use the PMD-mapped
THP, that is not reasonable. Many implement anonymous page sharing
through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage
scenarios,
therefore, users expect to apply an unified mTHP strategy for anonymous
pages,
also including the anonymous shared pages, in order to enjoy the
benefits of
mTHP. For example, lower latency than PMD-mapped THP, smaller memory
bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB
miss
etc.

Quoted 5 lines

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and off-the-shelf benchmarks
for those workloads) that predominantly use shared anon memory?

Quoted 8 lines

As far as I know, some database related workloads make extensive use of shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely also
uses shared anonymous memory. And I still need to do some investigation to measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 47 lines
Quoted 15 lines
Quoted 14 lines

The primary strategy is that, the use of huge pages for anonymous shared
pages
still follows the global control determined by the mount option "huge="
parameter
or the sysfs interface at
'/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global 'huge' switch is
enabled.
Subsequently, the mTHP sysfs interface
(/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for large folio
allocation
for these anonymous shared pages.

Quoted 5 lines

I'm not sure about this proposed control mechanism; won't it break compatibility? I could be wrong, but I don't think shmem's use of THP used to
depend upon the value of /sys/kernel/mm/transparent_hugepage/enabled? So it

Yes, I realized this after more testing.

Quoted 10 lines

doesn't make sense to me that we now depend upon the /sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled values (which by
default disables all sizes except 2M, which is set to "inherit" from /sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set of options (always/never/within_size/advise/deny/force) to enabled (always/madvise/never)

Quoted 8 lines

Perhaps it would be cleaner to do the same trick we did for enabled; Introduce /mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which can have all the
same values as the top-level /sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes will be set to "never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 5 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to 'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my rough
understanding is:

      - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls mmap(SHARED|ANON) allocations (mostly; see rule 3)
      - huge=... controls tmpfs allocations
      - deny and force in shmem_enabled are equivalent to never and always for
        mmap(SHARED|ANON) but additionally override all tmpfs mounts so they act as
        if they were mounted with huge=never or huge=always

Quoted 2 lines

Is that correct? If so, then I think it still makes sense to support per-size

Correct.

Quoted 3 lines

deny/force. Certainly if a per-size control is set to "inherit" and the top-level control is set to deny or force, you would need that to mean something.

Quoted 6 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs allocation,
so we should not use anonymous control to override tmpfs control, which seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the value is deny or force. So if you have:

Quoted 6 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is "inherit", in which case the value in /mm/transparent_hugepage/enabled is used for that size.

That approach enables us to 1) maintain back-compat and 2) control each size independently

1) is met because the default is that all sizes are initially set to "never", except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled) which is initially set to inherit. So any mTHP unaware SW can still modify /mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 4 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size (echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 7 lines
Quoted 4 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to support the deny and force values. When applied to non-PMD sizes, "deny" can just be a noop for now, because there was no way to configure a tmpfs mount for non-PMD size THP in the first place. But I'm not sure what to do with "force"?

OK. And I also prefer that "force" should be a noop too, since anon shmem control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force" will be an alias for "always" and "deny" will be an alias for "never"?

I was just a bit concerned about later changing that behavior to also impact tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it, I don't see that as a problem.

Quoted 8 lines
Quoted 2 lines

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Well I think it amounts to the same thing; there isn't much point in forbidding "force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

Quoted 5 lines

The default for PMD should be "inherit", for the other mTHP sizes it should be "never".

So we should fail if:
* Setting top-level to "force" when any non-PMD size is "inherit"
* Setting "inherit" of a non-PMD size when the top-level is force

IMO, for tmpfs this is true, but for anon shmem, this 2 cases should not fail.

So I think we should allow this configuration, but for tmpfs huge page allocation, we will not check the mTHP.

Quoted 11 lines

Both will only happen if someone messes with the mTHP configuration manually.

And we should only offer "force" as an option for PMD-sized mTHP as long as the others are not supported. See below.

Quoted 3 lines

So I think the question is just 'what should happen when "force" is configured for a non-PMD-sized mTHP'?

We should hide it and not offer a configuration toggle that is inactive.

If someone wants to sense support for other mTHP "force" settings in the future, they can just parse if the "shmem_enabled" toggle offers "force" as an option. Then they know that it can actually be enabled and will also do what is promised.

Sounds good to me.

#42 lore raw
Quoted 280 lines · Baolin Wang
Quoted 277 lines · David Hildenbrand
Quoted 265 lines · Ryan Roberts
Quoted 255 lines · David Hildenbrand
Quoted 254 lines · Ryan Roberts
Quoted 245 lines · Baolin Wang
Quoted 217 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 178 lines · Ryan Roberts
Quoted 159 lines · Baolin Wang
Quoted 144 lines · Ryan Roberts
Quoted 53 lines · Baolin Wang
Quoted 39 lines · Ryan Roberts

On 22/04/2024 08:02, Baolin Wang wrote:
Anonymous pages have already been supported for multi-size
(mTHP)
allocation
through commit 19eaf44954df, that can allow THP to be
configured
through the
sysfs interface located at
'/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the
anonymous mTHP rule
configured through the sysfs interface, and can only use
the PMD-mapped
THP, that is not reasonable. Many implement anonymous page
sharing
through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage
scenarios,
therefore, users expect to apply an unified mTHP strategy
for anonymous
pages,
also including the anonymous shared pages, in order to
enjoy the
benefits of
mTHP. For example, lower latency than PMD-mapped THP,
smaller memory
bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to
reduce TLB
miss
etc.

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and
off-the-shelf
benchmarks
for those workloads) that predominantly use shared anon memory?

As far as I know, some database related workloads make extensive use of
shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or MySQL likely
also
uses shared anonymous memory. And I still need to do some investigation to
measure the performance.

Quoted 1 line

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 61 lines
Quoted 27 lines

The primary strategy is that, the use of huge pages for
anonymous shared
pages
still follows the global control determined by the mount
option "huge="
parameter
or the sysfs interface at
'/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global
'huge' switch is
enabled.
Subsequently, the mTHP sysfs interface
(/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for
large folio
allocation
for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it
break
compatibility? I could be wrong, but I don't think shmem's
use of THP
used to
depend upon the value of
/sys/kernel/mm/transparent_hugepage/enabled?
So it

Yes, I realized this after more testing.

Quoted 24 lines

doesn't make sense to me that we now depend upon the
/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled
values
(which by
default disables all sizes except 2M, which is set to
"inherit" from
/sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set
of options
(always/never/within_size/advise/deny/force) to enabled
(always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for
enabled;
Introduce
/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which
can have all
the
same values as the top-level
/sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes
will be set to
"never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values from
top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

Quoted 7 lines

For mTHP's shmem_enabled interface, we can just keep below values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem, 'deny' is equal to
'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my
rough
understanding is:

      - /sys/kernel/mm/transparent_hugepage/shmem_enabled
controls
        mmap(SHARED|ANON) allocations (mostly; see rule 3)
      - huge=... controls tmpfs allocations
      - deny and force in shmem_enabled are equivalent to
never and
always for
        mmap(SHARED|ANON) but additionally override all tmpfs
mounts so they
act as
        if they were mounted with huge=never or huge=always

Quoted 3 lines

Is that correct? If so, then I think it still makes sense to support
per-size

Correct.

Quoted 5 lines

deny/force. Certainly if a per-size control is set to "inherit" and the
top-level control is set to deny or force, you would need that to mean
something.

Quoted 9 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs
allocation,
so we should not use anonymous control to override tmpfs control, which
seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave
the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the
value is deny or force. So if you have:

Quoted 8 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return
false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow
the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is
"inherit", in which case the value in /mm/transparent_hugepage/enabled is used
for that size.

That approach enables us to 1) maintain back-compat and 2) control each size
independently

1) is met because the default is that all sizes are initially set to "never",
except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled)
which is initially set to inherit. So any mTHP unaware SW can still modify
/mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 6 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size
(echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without
having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but
I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 8 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to
support the deny and force values. When applied to non-PMD sizes, "deny" can
just be a noop for now, because there was no way to configure a tmpfs mount for
non-PMD size THP in the first place. But I'm not sure what to do with "force"?

Quoted 3 lines

OK. And I also prefer that "force" should be a noop too, since anon shmem
control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force"
will be an alias for "always" and "deny" will be an alias for "never"?

Quoted 8 lines
Quoted 5 lines

I was just a bit concerned about later changing that behavior to also impact
tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it,
I don't see that as a problem.

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Quoted 7 lines

Well I think it amounts to the same thing; there isn't much point in forbidding
"force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set
first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

The default for PMD should be "inherit", for the other mTHP sizes it should be "never".

Quoted 6 lines
Quoted 3 lines

So we should fail if:
* Setting top-level to "force" when any non-PMD size is "inherit"
* Setting "inherit" of a non-PMD size when the top-level is force

IMO, for tmpfs this is true, but for anon shmem, this 2 cases should not fail.

If force does not apply to an mTHP size, it should fail if it would get inherited. Until it applies and we enable it.

I'm still confused about all the toggles here, so could be I am missing something.

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#43 lore raw
Quoted 285 lines · David Hildenbrand
Quoted 284 lines · Baolin Wang
Quoted 280 lines · David Hildenbrand
Quoted 265 lines · Ryan Roberts
Quoted 255 lines · David Hildenbrand
Quoted 254 lines · Ryan Roberts
Quoted 245 lines · Baolin Wang
Quoted 217 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 178 lines · Ryan Roberts
Quoted 159 lines · Baolin Wang
Quoted 144 lines · Ryan Roberts
Quoted 54 lines · Baolin Wang

On 2024/4/23 18:41, Ryan Roberts wrote:
On 22/04/2024 08:02, Baolin Wang wrote:
Anonymous pages have already been supported for multi-size
(mTHP)
allocation
through commit 19eaf44954df, that can allow THP to be
configured
through the
sysfs interface located at
'/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the
anonymous mTHP rule
configured through the sysfs interface, and can only use
the PMD-mapped
THP, that is not reasonable. Many implement anonymous page
sharing
through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage
scenarios,
therefore, users expect to apply an unified mTHP strategy
for anonymous
pages,
also including the anonymous shared pages, in order to
enjoy the
benefits of
mTHP. For example, lower latency than PMD-mapped THP,
smaller memory
bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to
reduce TLB
miss
etc.

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and
off-the-shelf
benchmarks
for those workloads) that predominantly use shared anon memory?

As far as I know, some database related workloads make
extensive use of
shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or
MySQL likely
also
uses shared anonymous memory. And I still need to do some
investigation to
measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

Quoted 68 lines

The primary strategy is that, the use of huge pages for
anonymous shared
pages
still follows the global control determined by the mount
option "huge="
parameter
or the sysfs interface at
'/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global
'huge' switch is
enabled.
Subsequently, the mTHP sysfs interface
(/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for
large folio
allocation
for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it
break
compatibility? I could be wrong, but I don't think shmem's
use of THP
used to
depend upon the value of
/sys/kernel/mm/transparent_hugepage/enabled?
So it

Yes, I realized this after more testing.

doesn't make sense to me that we now depend upon the
/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled
values
(which by
default disables all sizes except 2M, which is set to
"inherit" from
/sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set
of options
(always/never/within_size/advise/deny/force) to enabled
(always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for
enabled;
Introduce
/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which
can have all
the
same values as the top-level
/sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes
will be set to
"never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values
from
top-level
'/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

For mTHP's shmem_enabled interface, we can just keep below
values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem,
'deny' is equal to
'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the docs, but my
rough
understanding is:

       - /sys/kernel/mm/transparent_hugepage/shmem_enabled controls
         mmap(SHARED|ANON) allocations (mostly; see rule 3)
       - huge=... controls tmpfs allocations
       - deny and force in shmem_enabled are equivalent to never and
always for
         mmap(SHARED|ANON) but additionally override all tmpfs mounts so they
act as
         if they were mounted with huge=never or huge=always

Quoted 3 lines

Is that correct? If so, then I think it still makes sense to support
per-size

Correct.

Quoted 5 lines

deny/force. Certainly if a per-size control is set to "inherit" and the
top-level control is set to deny or force, you would need that to mean
something.

Quoted 9 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs
allocation,
so we should not use anonymous control to override tmpfs control, which
seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave
the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the
value is deny or force. So if you have:

Quoted 8 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return
false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow
the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is
"inherit", in which case the value in /mm/transparent_hugepage/enabled is used
for that size.

That approach enables us to 1) maintain back-compat and 2) control each size
independently

1) is met because the default is that all sizes are initially set to "never",
except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled)
which is initially set to inherit. So any mTHP unaware SW can still modify
/mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 6 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size
(echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without
having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but
I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 8 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to
support the deny and force values. When applied to non-PMD sizes, "deny" can
just be a noop for now, because there was no way to configure a tmpfs mount for
non-PMD size THP in the first place. But I'm not sure what to do with "force"?

Quoted 3 lines

OK. And I also prefer that "force" should be a noop too, since anon shmem
control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force"
will be an alias for "always" and "deny" will be an alias for "never"?

Quoted 8 lines
Quoted 5 lines

I was just a bit concerned about later changing that behavior to also impact
tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it,
I don't see that as a problem.

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Quoted 7 lines

Well I think it amounts to the same thing; there isn't much point in forbidding
"force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set
first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

The default for PMD should be "inherit", for the other mTHP sizes it should be "never".

So we should fail if:
* Setting top-level to "force" when any non-PMD size is "inherit"
* Setting "inherit" of a non-PMD size when the top-level is force

Quoted 6 lines
Quoted 2 lines

IMO, for tmpfs this is true, but for anon shmem, this 2 cases should not fail.

If force does not apply to an mTHP size, it should fail if it would get inherited. Until it applies and we enable it.

I'm still confused about all the toggles here, so could be I am missing something.

Yes, this is a little messy:(

After thinking more, considering that 'force' is used to override the tmpfs mount option, and 'inherit' will inherit the global setting. Your suggestion will make the logic eary to understand (though it is valid for anon shmem mTHP allocations, which are not part of this scenario), Ryan, what do you think?

reviewer
#44 lore raw
Quoted 297 lines · Baolin Wang
Quoted 285 lines · David Hildenbrand
Quoted 284 lines · Baolin Wang
Quoted 280 lines · David Hildenbrand
Quoted 265 lines · Ryan Roberts
Quoted 255 lines · David Hildenbrand
Quoted 254 lines · Ryan Roberts
Quoted 245 lines · Baolin Wang
Quoted 217 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 178 lines · Ryan Roberts
Quoted 159 lines · Baolin Wang
Quoted 147 lines · Ryan Roberts

On 24/04/2024 07:55, Baolin Wang wrote:

On 2024/4/23 18:41, Ryan Roberts wrote:
On 22/04/2024 08:02, Baolin Wang wrote:
Anonymous pages have already been supported for multi-size
(mTHP)
allocation
through commit 19eaf44954df, that can allow THP to be
configured
through the
sysfs interface located at
'/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the
anonymous mTHP rule
configured through the sysfs interface, and can only use
the PMD-mapped
THP, that is not reasonable. Many implement anonymous page
sharing
through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage
scenarios,
therefore, users expect to apply an unified mTHP strategy
for anonymous
pages,
also including the anonymous shared pages, in order to
enjoy the
benefits of
mTHP. For example, lower latency than PMD-mapped THP,
smaller memory
bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to
reduce TLB
miss
etc.

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and
off-the-shelf
benchmarks
for those workloads) that predominantly use shared anon memory?

As far as I know, some database related workloads make
extensive use of
shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or
MySQL likely
also
uses shared anonymous memory. And I still need to do some
investigation to
measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

The primary strategy is that, the use of huge pages for
anonymous shared
pages
still follows the global control determined by the mount
option "huge="
parameter
or the sysfs interface at
'/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global
'huge' switch is
enabled.
Subsequently, the mTHP sysfs interface
(/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for
large folio
allocation
for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it
break
compatibility? I could be wrong, but I don't think shmem's
use of THP
used to
depend upon the value of
/sys/kernel/mm/transparent_hugepage/enabled?
So it

Yes, I realized this after more testing.

doesn't make sense to me that we now depend upon the
/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled
values
(which by
default disables all sizes except 2M, which is set to
"inherit" from
/sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set
of options
(always/never/within_size/advise/deny/force) to enabled
(always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for
enabled;
Introduce
/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which
can have all
the
same values as the top-level
/sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes
will be set to
"never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values
from
top-level
'/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

For mTHP's shmem_enabled interface, we can just keep below
values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem,
'deny' is equal to
'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the
docs, but my
rough
understanding is:

       - /sys/kernel/mm/transparent_hugepage/shmem_enabled
controls
         mmap(SHARED|ANON) allocations (mostly; see rule 3)
       - huge=... controls tmpfs allocations
       - deny and force in shmem_enabled are equivalent to
never and
always for
         mmap(SHARED|ANON) but additionally override all tmpfs
mounts so they
act as
         if they were mounted with huge=never or huge=always

Is that correct? If so, then I think it still makes sense to
support
per-size

Correct.

Quoted 5 lines

deny/force. Certainly if a per-size control is set to
"inherit" and the
top-level control is set to deny or force, you would need that
to mean
something.

Quoted 9 lines

IMHO, the '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface should only control the anonymous shmem. And 'huge=' controls tmpfs
allocation,
so we should not use anonymous control to override tmpfs control, which
seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave
the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the
value is deny or force. So if you have:

Quoted 8 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return
false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow
the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is
"inherit", in which case the value in /mm/transparent_hugepage/enabled is used
for that size.

That approach enables us to 1) maintain back-compat and 2) control each size
independently

1) is met because the default is that all sizes are initially set to "never",
except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled)
which is initially set to inherit. So any mTHP unaware SW can still modify
/mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 6 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size
(echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without
having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but
I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 8 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to
support the deny and force values. When applied to non-PMD sizes, "deny" can
just be a noop for now, because there was no way to configure a tmpfs mount for
non-PMD size THP in the first place. But I'm not sure what to do with "force"?

Quoted 3 lines

OK. And I also prefer that "force" should be a noop too, since anon shmem
control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force"
will be an alias for "always" and "deny" will be an alias for "never"?

Quoted 8 lines
Quoted 5 lines

I was just a bit concerned about later changing that behavior to also impact
tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it,
I don't see that as a problem.

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Quoted 7 lines

Well I think it amounts to the same thing; there isn't much point in forbidding
"force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set
first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

The default for PMD should be "inherit", for the other mTHP sizes it should be "never".

So we should fail if:
* Setting top-level to "force" when any non-PMD size is "inherit"
* Setting "inherit" of a non-PMD size when the top-level is force

Quoted 8 lines
Quoted 2 lines

IMO, for tmpfs this is true, but for anon shmem, this 2 cases should not fail.

If force does not apply to an mTHP size, it should fail if it would get inherited. Until it applies and we enable it.

I'm still confused about all the toggles here, so could be I am missing something.

Quoted 6 lines

Yes, this is a little messy:(

After thinking more, considering that 'force' is used to override the tmpfs mount option, and 'inherit' will inherit the global setting. Your suggestion will make the logic eary to understand (though it is valid for anon shmem mTHP allocations, which are not part of this scenario), Ryan, what do you think?

I was thinking that ever returning an error code when trying to set "force" for /mm/transparent_hugepage/shmem_enabled would be weird behavior and to be avoided if possible.

But as always, David makes a clear and compelling case. I'm happy to go with his suggestion.

I wonder if we should do the same for "deny"? The argument I previously made was that non-PMD-size mTHP is always effectively denied while tmpfs doesn't support mTHP, so we can allow "deny" and for now it would just be an alias for "never". The semantics don't change when you introduce tmpfs support.

Personally I lean towards allowing "deny" from day 1. David?

#45 lore raw
Quoted 305 lines · Ryan Roberts
Quoted 297 lines · Baolin Wang
Quoted 285 lines · David Hildenbrand
Quoted 284 lines · Baolin Wang
Quoted 280 lines · David Hildenbrand
Quoted 265 lines · Ryan Roberts
Quoted 255 lines · David Hildenbrand
Quoted 254 lines · Ryan Roberts
Quoted 245 lines · Baolin Wang
Quoted 217 lines · Ryan Roberts
Quoted 181 lines · Baolin Wang
Quoted 178 lines · Ryan Roberts
Quoted 168 lines · Baolin Wang

On 2024/4/24 16:26, Ryan Roberts wrote:
On 24/04/2024 07:55, Baolin Wang wrote:

On 2024/4/23 18:41, Ryan Roberts wrote:
On 22/04/2024 08:02, Baolin Wang wrote:
Anonymous pages have already been supported for multi-size
(mTHP)
allocation
through commit 19eaf44954df, that can allow THP to be
configured
through the
sysfs interface located at
'/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the
anonymous mTHP rule
configured through the sysfs interface, and can only use
the PMD-mapped
THP, that is not reasonable. Many implement anonymous page
sharing
through
mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage
scenarios,
therefore, users expect to apply an unified mTHP strategy
for anonymous
pages,
also including the anonymous shared pages, in order to
enjoy the
benefits of
mTHP. For example, lower latency than PMD-mapped THP,
smaller memory
bloat
than PMD-mapped THP, contiguous PTEs on ARM architecture to
reduce TLB
miss
etc.

This sounds like a very useful addition!

Out of interest, can you point me at any workloads (and
off-the-shelf
benchmarks
for those workloads) that predominantly use shared anon memory?

As far as I know, some database related workloads make
extensive use of
shared
anonymous page, such as PolarDB[1] in our Alibaba fleet, or
MySQL likely
also
uses shared anonymous memory. And I still need to do some
investigation to
measure the performance.

[1] https://github.com/ApsaraDB/PolarDB-for-PostgreSQL

Thanks for the pointer!

The primary strategy is that, the use of huge pages for
anonymous shared
pages
still follows the global control determined by the mount
option "huge="
parameter
or the sysfs interface at
'/sys/kernel/mm/transparent_hugepage/shmem_enabled'.
The utilization of mTHP is allowed only when the global
'huge' switch is
enabled.
Subsequently, the mTHP sysfs interface
(/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled)
is checked to determine the mTHP size that can be used for
large folio
allocation
for these anonymous shared pages.

I'm not sure about this proposed control mechanism; won't it
break
compatibility? I could be wrong, but I don't think shmem's
use of THP
used to
depend upon the value of
/sys/kernel/mm/transparent_hugepage/enabled?
So it

Yes, I realized this after more testing.

doesn't make sense to me that we now depend upon the
/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled
values
(which by
default disables all sizes except 2M, which is set to
"inherit" from
/sys/kernel/mm/transparent_hugepage/enabled).

The other problem is that shmem_enabled has a different set
of options
(always/never/within_size/advise/deny/force) to enabled
(always/madvise/never)

Perhaps it would be cleaner to do the same trick we did for
enabled;
Introduce
/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled, which
can have all
the
same values as the top-level
/sys/kernel/mm/transparent_hugepage/shmem_enabled,
plus the additional "inherit" option. By default all sizes
will be set to
"never" except 2M, which is set to "inherit".

Sounds good to me. But I do not want to copy all same values
from
top-level
'/sys/kernel/mm/transparent_hugepage/shmem_enabled':
always within_size advise never deny force

For mTHP's shmem_enabled interface, we can just keep below
values:
always within_size advise never

Cause when checking if mTHP can be used for anon shmem,
'deny' is equal to
'never', and 'force' is equal to 'always'.

I'll admit it wasn't completely clear to me after reading the
docs, but my
rough
understanding is:

       - /sys/kernel/mm/transparent_hugepage/shmem_enabled
controls
         mmap(SHARED|ANON) allocations (mostly; see rule 3)
       - huge=... controls tmpfs allocations
       - deny and force in shmem_enabled are equivalent to
never and
always for
         mmap(SHARED|ANON) but additionally override all tmpfs
mounts so they
act as
         if they were mounted with huge=never or huge=always

Is that correct? If so, then I think it still makes sense to
support
per-size

Correct.

deny/force. Certainly if a per-size control is set to
"inherit" and the
top-level control is set to deny or force, you would need that
to mean
something.

IMHO, the
'/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled' interface
should only control the anonymous shmem. And 'huge=' controls
tmpfs
allocation,
so we should not use anonymous control to override tmpfs
control, which
seems a
little mess?

I agree it would be cleaner to only handle mmap(SHARED|ANON) here, and leave
the
tmpfs stuff for another time. But my point is that /mm/transparent_hugepage/shmem_enabled already interferes with tmpfs if the
value is deny or force. So if you have:

Quoted 8 lines
Quoted 1 line

echo deny > /mm/transparent_hugepage/shmem_enabled

IIUC, this global control will cause shmem_is_huge() to always return
false, so
no matter how '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' is set, anonymous shmem will not use mTHP. No?

No, that's not how '/mm/transparent_hugepage/hugepage-xxxkB/enabled' works, and I think '/mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled' should follow
the established pattern.

For anon-private, each size is controlled by its /mm/transparent_hugepage/hugepage-xxxkB/enabled value. Unless that value is
"inherit", in which case the value in /mm/transparent_hugepage/enabled is used
for that size.

That approach enables us to 1) maintain back-compat and 2) control each size
independently

1) is met because the default is that all sizes are initially set to "never",
except the PMD-size (e.g. /mm/transparent_hugepage/hugepage-2048kB/enabled)
which is initially set to inherit. So any mTHP unaware SW can still modify
/mm/transparent_hugepage/enabled and it will still only apply to PMD size.

Quoted 6 lines

2) is met because mTHP aware SW can come along and e.g. enable the 64K size
(echo always > /mm/transparent_hugepage/hugepage-64kB/enabled) without
having to
modify the value in /mm/transparent_hugepage/enabled.

Thanks for explanation. Initially, I want to make ‘/mm/transparent_hugepage/shmem_enabled’ be a global control for huge page, but
I think it should follow the same strategy as anon mTHP as you said.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

echo inherit > /mm/transparent_hugepage/hugepage-64kB/shmem_enabled

What does that mean?

Quoted 8 lines

So I think /mm/transparent_hugepage/hugepage-xxxkB/shmem_enabled will need to
support the deny and force values. When applied to non-PMD sizes, "deny" can
just be a noop for now, because there was no way to configure a tmpfs mount for
non-PMD size THP in the first place. But I'm not sure what to do with "force"?

Quoted 3 lines

OK. And I also prefer that "force" should be a noop too, since anon shmem
control should not configure tmpfs huge page allocation.

I guess technically they won't be noops, but (for the non-PMD-sizes) "force"
will be an alias for "always" and "deny" will be an alias for "never"?

Quoted 8 lines
Quoted 5 lines

I was just a bit concerned about later changing that behavior to also impact
tmpfs once tmpfs supports mTHP; could that cause breaks? But thinking about it,
I don't see that as a problem.

Is the question what should happen if we "inherit" "force" or if someone specifies "force" for a mTP size explicitly?

Quoted 7 lines

Well I think it amounts to the same thing; there isn't much point in forbidding
"force" to be set directly because it can still be set indirectly through "inherit". We can't forbid indirectly setting it, because "inherit" could be set
first, then the top-level shmem_enabled changed to "force" after - and we wouldn't want to fail that.

The default for PMD should be "inherit", for the other mTHP sizes it should be "never".

So we should fail if:
* Setting top-level to "force" when any non-PMD size is "inherit"
* Setting "inherit" of a non-PMD size when the top-level is force

Quoted 8 lines
Quoted 2 lines

IMO, for tmpfs this is true, but for anon shmem, this 2 cases should not fail.

If force does not apply to an mTHP size, it should fail if it would get inherited. Until it applies and we enable it.

I'm still confused about all the toggles here, so could be I am missing something.

Quoted 6 lines

Yes, this is a little messy:(

After thinking more, considering that 'force' is used to override the tmpfs mount option, and 'inherit' will inherit the global setting. Your suggestion will make the logic eary to understand (though it is valid for anon shmem mTHP allocations, which are not part of this scenario), Ryan, what do you think?

Quoted 6 lines

I was thinking that ever returning an error code when trying to set "force" for /mm/transparent_hugepage/shmem_enabled would be weird behavior and to be avoided if possible.

But as always, David makes a clear and compelling case. I'm happy to go with his suggestion.

I'm afraid "as always" is far from true :D

Quoted 7 lines

I wonder if we should do the same for "deny"? The argument I previously made was that non-PMD-size mTHP is always effectively denied while tmpfs doesn't support mTHP, so we can allow "deny" and for now it would just be an alias for "never". The semantics don't change when you introduce tmpfs support.

Personally I lean towards allowing "deny" from day 1. David?

If it has clear semantics that can be fulfilled as of today, sure, we can have it.

Signature · 3 lines
Cheers,

David / dhildenb