add mTHP support for anonymous shmem

Move highest_order() and next_order() out of the CONFIG_TRANSPARENT_HUGEPAGE macro, which can be common functions to be used.

Signed-off-by Baolin Wang <[email protected]>

---
 include/linux/huge_mm.h | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

include/linux/huge_mm.h +11 -11

index 017cee864080..e49b56c40a11 100644
@@ -106,6 +106,17 @@ extern struct kobj_attribute shmem_enabled_attr;
 #define HPAGE_PUD_MASK	(~(HPAGE_PUD_SIZE - 1))
 #define HPAGE_PUD_SIZE	((1UL) << HPAGE_PUD_SHIFT)
 
+static inline int highest_order(unsigned long orders)
+{
+	return fls_long(orders) - 1;
+}
+
+static inline int next_order(unsigned long *orders, int prev)
+{
+	*orders &= ~BIT(prev);
+	return highest_order(*orders);
+}
+
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
 
 extern unsigned long transparent_hugepage_flags;

@@ -138,17 +149,6 @@ static inline bool hugepage_flags_enabled(void)
 	       huge_anon_orders_madvise;
 }
 
-static inline int highest_order(unsigned long orders)
-{
-	return fls_long(orders) - 1;
-}
-
-static inline int next_order(unsigned long *orders, int prev)
-{
-	*orders &= ~BIT(prev);
-	return highest_order(*orders);
-}
-
 /*
  * Do the below checks:
  *   - For file vma, check if the linear page offset of vma is

Quoted 2 lines · Baolin Wang

Move highest_order() and next_order() out of the CONFIG_TRANSPARENT_HUGEPAGE macro, which can be common functions to be used.

Sorry if I haven't kept up with the discussion, but why is this needed? I wouldn't expect a need to iterate over orders if THP is compile-time disabled because we will never try to allocate THP?

Quoted 4 lines · Ryan Roberts

Quoted 2 lines · Baolin Wang

Move highest_order() and next_order() out of the CONFIG_TRANSPARENT_HUGEPAGE macro, which can be common functions to be used.

Quoted 3 lines

Sorry if I haven't kept up with the discussion, but why is this needed? I wouldn't expect a need to iterate over orders if THP is compile-time disabled because we will never try to allocate THP?

Cause I don't want to add some dummy functions to avoid building errors if CONFIG_TRANSPARENT_HUGEPAGE is not enabled in patch 6. Another thought is that the pagecache can also allocate a large folio even when THP is not enabled, so these helpers may be used in the future (not sure though).

Anyway, I also have no strong perference for this patch, below dummy functions can also work for me:

include/linux/huge_mm.h +10 -0

index c15bebb2cf53..7aa802ee2ce5 100644
@@ -586,6 +586,16 @@ static inline bool thp_migration_supported(void)
  {
         return false;
  }
+
+static inline int highest_order(unsigned long orders)
+{
+        return 0;
+}
+
+static inline int next_order(unsigned long *orders, int prev)
+{
+        return 0;
+}
  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */

Quoted 7 lines · Baolin Wang

Quoted 4 lines · Ryan Roberts

Quoted 2 lines · Baolin Wang

Move highest_order() and next_order() out of the CONFIG_TRANSPARENT_HUGEPAGE macro, which can be common functions to be used.

Quoted 8 lines

Quoted 3 lines

Sorry if I haven't kept up with the discussion, but why is this needed? I wouldn't expect a need to iterate over orders if THP is compile-time disabled because we will never try to allocate THP?

Cause I don't want to add some dummy functions to avoid building errors if CONFIG_TRANSPARENT_HUGEPAGE is not enabled in patch 6. Another thought is that the pagecache can also allocate a large folio even when THP is not enabled, so these helpers may be used in the future (not sure though).

OK, I'll admit I haven't looked at the latter patches yet - I'd like to conclude on the interface and mapping/alignment strategy first.

But it wasn't necessary to access these functions for the anon/private case without CONFIG_TRANSPARENT_HUGEPAGE, so I'm wondering why it's needed for shmem case. I would expect that they don't need to be defined at all.

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
 mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------
 1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +7 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct page *page;
+	struct folio *folio;
 	vm_fault_t ret;
 	bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) &&
 		      !(vma->vm_flags & VM_SHARED);
+	int type, nr_pages, i;
+	unsigned long addr = vmf->address;
 
 	/* Did we COW the page? */
 	if (is_cow)

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
 			return VM_FAULT_OOM;
 	}
 
+	folio = page_folio(page);
+	nr_pages = folio_nr_pages(folio);
+
+	if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c +16 -10

+	}
 	vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-				      vmf->address, &vmf->ptl);
+				       addr, &vmf->ptl);
 	if (!vmf->pte)
 		return VM_FAULT_NOPAGE;
 
 	/* Re-check under ptl */
-	if (likely(!vmf_pte_changed(vmf))) {
-		struct folio *folio = page_folio(page);
-		int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
-
-		set_pte_range(vmf, folio, page, 1, vmf->address);
-		add_mm_counter(vma->vm_mm, type, 1);
-		ret = 0;
-	} else {
-		update_mmu_tlb(vma, vmf->address, vmf->pte);
+	if (nr_pages == 1 && unlikely(vmf_pte_changed(vmf))) {
+		update_mmu_tlb(vma, addr, vmf->pte);
+		ret = VM_FAULT_NOPAGE;
+		goto unlock;
+	} else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
+		for (i = 0; i < nr_pages; i++)
+			update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
 		ret = VM_FAULT_NOPAGE;
+		goto unlock;
 	}
 
+	set_pte_range(vmf, folio, page, nr_pages, addr);
+	type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
+	add_mm_counter(vma->vm_mm, type, nr_pages);
+	ret = 0;
+
+unlock:
 	pte_unmap_unlock(vmf->pte, vmf->ptl);
 	return ret;
 }

Quoted 34 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
  mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------
  1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +7 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
  {
  	struct vm_area_struct *vma = vmf->vma;
  	struct page *page;
+	struct folio *folio;
  	vm_fault_t ret;
  	bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) &&
  		      !(vma->vm_flags & VM_SHARED);
+	int type, nr_pages, i;
+	unsigned long addr = vmf->address;
 
  	/* Did we COW the page? */
  	if (is_cow)

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
  			return VM_FAULT_OOM;
  	}
 
+	folio = page_folio(page);
+	nr_pages = folio_nr_pages(folio);
+
+	if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4799

Quoted 10 lines

mm/memory.c +10 -0

+		nr_pages = 1;
+	} else if (nr_pages > 1) {
+		unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
+		unsigned long end = start + nr_pages * PAGE_SIZE;
+
+		/* In case the folio size in page cache beyond the VMA limits. */
+		addr = max(start, vma->vm_start);
+		nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT;
+
+		page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with vmf->address?

For example, let's say we have a file who's size is 64K and which is cached in a single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

start=0 and end=64K I think?

Additionally, I think this path will end up mapping the entire folio (as long as it fits in the VMA). But this bypasses the fault-around configuration. As I think I mentioned against the RFC, this will inflate the RSS of the process and can cause behavioural changes as a result. I believe the current advice is to disable fault-around to prevent this kind of bloat when needed.

It might be that you need a special variant of finish_fault() for shmem?

Quoted 37 lines

mm/memory.c +16 -10

+	}
  	vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-				      vmf->address, &vmf->ptl);
+				       addr, &vmf->ptl);
  	if (!vmf->pte)
  		return VM_FAULT_NOPAGE;
 
  	/* Re-check under ptl */
-	if (likely(!vmf_pte_changed(vmf))) {
-		struct folio *folio = page_folio(page);
-		int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
-
-		set_pte_range(vmf, folio, page, 1, vmf->address);
-		add_mm_counter(vma->vm_mm, type, 1);
-		ret = 0;
-	} else {
-		update_mmu_tlb(vma, vmf->address, vmf->pte);
+	if (nr_pages == 1 && unlikely(vmf_pte_changed(vmf))) {
+		update_mmu_tlb(vma, addr, vmf->pte);
+		ret = VM_FAULT_NOPAGE;
+		goto unlock;
+	} else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
+		for (i = 0; i < nr_pages; i++)
+			update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
  		ret = VM_FAULT_NOPAGE;
+		goto unlock;
  	}
 
+	set_pte_range(vmf, folio, page, nr_pages, addr);
+	type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
+	add_mm_counter(vma->vm_mm, type, nr_pages);
+	ret = 0;
+
+unlock:
  	pte_unmap_unlock(vmf->pte, vmf->ptl);
  	return ret;
  }

Quoted 50 lines · Ryan Roberts

Quoted 34 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
   mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------
   1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +7 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
   {
   	struct vm_area_struct *vma = vmf->vma;
   	struct page *page;
+	struct folio *folio;
   	vm_fault_t ret;
   	bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) &&
   		      !(vma->vm_flags & VM_SHARED);
+	int type, nr_pages, i;
+	unsigned long addr = vmf->address;
 
   	/* Did we COW the page? */
   	if (is_cow)

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
   			return VM_FAULT_OOM;
   	}
 
+	folio = page_folio(page);
+	nr_pages = folio_nr_pages(folio);
+
+	if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4799

Quoted 10 lines

mm/memory.c +10 -0

+		nr_pages = 1;
+	} else if (nr_pages > 1) {
+		unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
+		unsigned long end = start + nr_pages * PAGE_SIZE;
+
+		/* In case the folio size in page cache beyond the VMA limits. */
+		addr = max(start, vma->vm_start);
+		nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT;
+
+		page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with vmf->address?

Quoted 3 lines

For example, let's say we have a file who's size is 64K and which is cached in a single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

For shmem, this doesn't happen because the VA is aligned with the hugepage size in the shmem_get_unmapped_area() function. See patch 7.

Quoted 53 lines · Baolin Wang

Quoted 50 lines · Ryan Roberts

Quoted 39 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation, to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
   mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------

   1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +1 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
    {
        struct vm_area_struct *vma = vmf->vma;
        struct page *page;
+    struct folio *folio;
        vm_fault_t ret;

       bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED); +    int type, nr_pages, i;
+    unsigned long addr = vmf->address;
       /* Did we COW the page? */
       if (is_cow)

mm/memory.c +10 -0

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
                return VM_FAULT_OOM;
        }
+    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
+
+    if (unlikely(userfaultfd_armed(vma))) {
+        nr_pages = 1;
+    } else if (nr_pages > 1) {
+        unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * 
 PAGE_SIZE);
+        unsigned long end = start + nr_pages * PAGE_SIZE;
+
+        /* In case the folio size in page cache beyond the VMA 

Quoted 5 lines

limits. */
+        addr = max(start, vma->vm_start);
+        nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT; +
+        page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with vmf->address?

Quoted 6 lines

Quoted 3 lines

For example, let's say we have a file who's size is 64K and which is cached in a single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

For shmem, this doesn't happen because the VA is aligned with the hugepage size in the shmem_get_unmapped_area() function. See patch 7.

Does that cover mremap() and MAP_FIXED as well.

We should try doing this as cleanly as possible, to prepare for the future / corner cases.

Quoted 49 lines · Ryan Roberts

Quoted 33 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
  mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------

  1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +1 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
   {
       struct vm_area_struct *vma = vmf->vma;
       struct page *page;
+    struct folio *folio;
       vm_fault_t ret;

      bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED); +    int type, nr_pages, i;
+    unsigned long addr = vmf->address;
        /* Did we COW the page? */
      if (is_cow)

mm/memory.c +3 -0

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
               return VM_FAULT_OOM;
       }
   +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
+
+    if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4798

Quoted 10 lines

mm/memory.c +10 -0

+        nr_pages = 1;
+    } else if (nr_pages > 1) {
+        unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
+        unsigned long end = start + nr_pages * PAGE_SIZE;
+
+        /* In case the folio size in page cache beyond the VMA limits. */
+        addr = max(start, vma->vm_start);
+        nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT;
+
+        page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with vmf->address?

Quoted 3 lines

For example, let's say we have a file who's size is 64K and which is cached in a single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

For shmem, this doesn't happen because the VA is aligned with the hugepage size in the shmem_get_unmapped_area() function. See patch 7.

Quoted 2 lines

start=0 and end=64K I think?

Yes. Unfortunately, some file systems that support large mappings do not perform alignment for multi-size THP (non-PMD sized, for example: 64K). I think this requires modification to __get_unmapped_area--->thp_get_unmapped_area_vmflags() or file->f_op->get_unmapped_area() to align VA for multi-size THP in future.

So before adding that VA alignment changes, only allow building the large folio mapping for anonymous shmem:

mm/memory.c +0 -0

index 936377220b77..9e4d51826d23 100644

mm/memory.c +0 -0

                 nr_pages = 1;
         } else if (nr_pages > 1) {

                unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);

Quoted 5 lines

Additionally, I think this path will end up mapping the entire folio (as long as it fits in the VMA). But this bypasses the fault-around configuration. As I think I mentioned against the RFC, this will inflate the RSS of the process and can cause behavioural changes as a result. I believe the current advice is to disable fault-around to prevent this kind of bloat when needed.

With above change, I do not think this is a problem? since users already want to use mTHP for anonymous shmem.

Quoted 38 lines

It might be that you need a special variant of finish_fault() for shmem?

Quoted 35 lines

mm/memory.c +15 -10

+    }
       vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-                      vmf->address, &vmf->ptl);
+                       addr, &vmf->ptl);
       if (!vmf->pte)
           return VM_FAULT_NOPAGE;
         /* Re-check under ptl */
-    if (likely(!vmf_pte_changed(vmf))) {
-        struct folio *folio = page_folio(page);
-        int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
-
-        set_pte_range(vmf, folio, page, 1, vmf->address);
-        add_mm_counter(vma->vm_mm, type, 1);
-        ret = 0;
-    } else {
-        update_mmu_tlb(vma, vmf->address, vmf->pte);
+    if (nr_pages == 1 && unlikely(vmf_pte_changed(vmf))) {
+        update_mmu_tlb(vma, addr, vmf->pte);
+        ret = VM_FAULT_NOPAGE;
+        goto unlock;
+    } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
+        for (i = 0; i < nr_pages; i++)
+            update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
           ret = VM_FAULT_NOPAGE;
+        goto unlock;
       }
   +    set_pte_range(vmf, folio, page, nr_pages, addr);
+    type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
+    add_mm_counter(vma->vm_mm, type, nr_pages);
+    ret = 0;
+
+unlock:
       pte_unmap_unlock(vmf->pte, vmf->ptl);
       return ret;
   }

Quoted 56 lines · Baolin Wang

Quoted 52 lines · Ryan Roberts

Quoted 33 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
   mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------
   1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +6 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
   {
       struct vm_area_struct *vma = vmf->vma;
       struct page *page;
+    struct folio *folio;
       vm_fault_t ret;
       bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) &&
                 !(vma->vm_flags & VM_SHARED);
+    int type, nr_pages, i;
+    unsigned long addr = vmf->address;
         /* Did we COW the page? */
       if (is_cow)

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
               return VM_FAULT_OOM;
       }
   +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
+
+    if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4798

Quoted 10 lines

mm/memory.c +10 -0

+        nr_pages = 1;
+    } else if (nr_pages > 1) {
+        unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
+        unsigned long end = start + nr_pages * PAGE_SIZE;
+
+        /* In case the folio size in page cache beyond the VMA limits. */
+        addr = max(start, vma->vm_start);
+        nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT;
+
+        page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with vmf->address?

For example, let's say we have a file who's size is 64K and which is cached in a single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

Quoted 2 lines

For shmem, this doesn't happen because the VA is aligned with the hugepage size in the shmem_get_unmapped_area() function. See patch 7.

Certainly agree that shmem can always make sure that it packs a vma in a way such that its folios are naturally aligned in VA when faulting in memory. If you mremap it, that alignment will be lost; I don't think that would be a problem

for a single process; mremap will take care of moving the ptes correctly and this path is not involved.

But what about the case when a process mmaps a shmem region, then forks, then the child mremaps the shmem region. Then the parent faults in a THP into the region (nicely aligned). Then the child faults in the same offset in the region and gets the THP that the parent allocated; that THP will be aligned in the parent's VM space but not in the child's.

Quoted 6 lines

Quoted 1 line

start=0 and end=64K I think?

Yes. Unfortunately, some file systems that support large mappings do not perform alignment for multi-size THP (non-PMD sized, for example: 64K). I think this requires modification to __get_unmapped_area--->thp_get_unmapped_area_vmflags() or file->f_op->get_unmapped_area() to align VA for multi-size THP in future.

By nature of the fact that a file mapping is shared between multiple processes and each process can map it where ever it wants down to 1 page granularity, its impossible for any THP containing a part of that file to be VA-aligned in every process it is mapped in.

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4789

Quoted 13 lines

So before adding that VA alignment changes, only allow building the large folio mapping for anonymous shmem:

mm/memory.c +1 -1

index 936377220b77..9e4d51826d23 100644
@@ -4786,7 +4786,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
         folio = page_folio(page);
         nr_pages = folio_nr_pages(folio);
 
-       if (unlikely(userfaultfd_armed(vma))) {
+       if (unlikely(userfaultfd_armed(vma)) || !vma_is_anon_shmem(vma)) {

If the above theoretical flow for fork & mremap is valid, then I don't think this is sufficient.

Quoted 52 lines

mm/memory.c +0 -0

                 nr_pages = 1;
         } else if (nr_pages > 1) {
                 unsigned long start = ALIGN_DOWN(vmf->address, nr_pages *

PAGE_SIZE);

Quoted 5 lines

Additionally, I think this path will end up mapping the entire folio (as long as it fits in the VMA). But this bypasses the fault-around configuration. As I think I mentioned against the RFC, this will inflate the RSS of the process and can cause behavioural changes as a result. I believe the current advice is to disable fault-around to prevent this kind of bloat when needed.

With above change, I do not think this is a problem? since users already want to use mTHP for anonymous shmem.

Quoted 38 lines

It might be that you need a special variant of finish_fault() for shmem?

Quoted 35 lines

mm/memory.c +15 -10

+    }
       vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-                      vmf->address, &vmf->ptl);
+                       addr, &vmf->ptl);
       if (!vmf->pte)
           return VM_FAULT_NOPAGE;
         /* Re-check under ptl */
-    if (likely(!vmf_pte_changed(vmf))) {
-        struct folio *folio = page_folio(page);
-        int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
-
-        set_pte_range(vmf, folio, page, 1, vmf->address);
-        add_mm_counter(vma->vm_mm, type, 1);
-        ret = 0;
-    } else {
-        update_mmu_tlb(vma, vmf->address, vmf->pte);
+    if (nr_pages == 1 && unlikely(vmf_pte_changed(vmf))) {
+        update_mmu_tlb(vma, addr, vmf->pte);
+        ret = VM_FAULT_NOPAGE;
+        goto unlock;
+    } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
+        for (i = 0; i < nr_pages; i++)
+            update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
           ret = VM_FAULT_NOPAGE;
+        goto unlock;
       }
   +    set_pte_range(vmf, folio, page, nr_pages, addr);
+    type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
+    add_mm_counter(vma->vm_mm, type, nr_pages);
+    ret = 0;
+
+unlock:
       pte_unmap_unlock(vmf->pte, vmf->ptl);
       return ret;
   }

Quoted 63 lines · Ryan Roberts

Quoted 61 lines · Baolin Wang

Quoted 50 lines · Ryan Roberts

Quoted 33 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
   mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------

   1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +1 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
    {
        struct vm_area_struct *vma = vmf->vma;
        struct page *page;
+    struct folio *folio;
        vm_fault_t ret;

       bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED); +    int type, nr_pages, i;
+    unsigned long addr = vmf->address;
         /* Did we COW the page? */
       if (is_cow)

mm/memory.c +3 -0

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
                return VM_FAULT_OOM;
        }
    +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
+
+    if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4798

Quoted 10 lines

mm/memory.c +10 -0

+        nr_pages = 1;
+    } else if (nr_pages > 1) {
+        unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
+        unsigned long end = start + nr_pages * PAGE_SIZE;
+
+        /* In case the folio size in page cache beyond the VMA limits. */
+        addr = max(start, vma->vm_start);
+        nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT;
+
+        page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with
vmf->address?

Quoted 5 lines

For example, let's say we have a file who's size is 64K and which is cached in a
single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

For shmem, this doesn't happen because the VA is aligned with the hugepage size in the shmem_get_unmapped_area() function. See patch 7.

Quoted 3 lines

Certainly agree that shmem can always make sure that it packs a vma in a way such that its folios are naturally aligned in VA when faulting in memory. If you mremap it, that alignment will be lost; I don't think that would be a problem

When mremap it, it will also call shmem_get_unmapped_area() to align the VA, but for mremap() with MAP_FIXED flag as David pointed out, yes, this patch may be not work perfectly.

Quoted 9 lines

for a single process; mremap will take care of moving the ptes correctly and this path is not involved.

But what about the case when a process mmaps a shmem region, then forks, then the child mremaps the shmem region. Then the parent faults in a THP into the region (nicely aligned). Then the child faults in the same offset in the region and gets the THP that the parent allocated; that THP will be aligned in the parent's VM space but not in the child's.

Sorry, I did not get your point here. IIUC, the child's VA will also be aligned if the child mremap is not set MAP_FIXED, since the child's mremap will still call shmem_get_unmapped_area() to find an aligned new VA.

Please correct me if I missed your point.

Quoted 8 lines

Quoted 7 lines

Quoted 2 lines

start=0 and end=64K I think?

Yes. Unfortunately, some file systems that support large mappings do not perform alignment for multi-size THP (non-PMD sized, for example: 64K). I think this requires modification to __get_unmapped_area--->thp_get_unmapped_area_vmflags() or file->f_op->get_unmapped_area() to align VA for multi-size THP in future.

Quoted 4 lines

By nature of the fact that a file mapping is shared between multiple processes and each process can map it where ever it wants down to 1 page granularity, its impossible for any THP containing a part of that file to be VA-aligned in every process it is mapped in.

Yes, so let me re-polish this patch. Thanks.

Quoted 70 lines

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4789

Quoted 13 lines

So before adding that VA alignment changes, only allow building the large folio mapping for anonymous shmem:

mm/memory.c +1 -1

index 936377220b77..9e4d51826d23 100644
@@ -4786,7 +4786,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
          folio = page_folio(page);
          nr_pages = folio_nr_pages(folio);
 
-       if (unlikely(userfaultfd_armed(vma))) {
+       if (unlikely(userfaultfd_armed(vma)) || !vma_is_anon_shmem(vma)) {

If the above theoretical flow for fork & mremap is valid, then I don't think this is sufficient.

Quoted 53 lines

mm/memory.c +0 -0

                  nr_pages = 1;
          } else if (nr_pages > 1) {

                 unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);

Quoted 6 lines

Additionally, I think this path will end up mapping the entire folio (as long as
it fits in the VMA). But this bypasses the fault-around configuration. As I think I mentioned against the RFC, this will inflate the RSS of the process and can cause behavioural changes as a result. I believe the current advice is to disable fault-around to prevent this kind of bloat when needed.

With above change, I do not think this is a problem? since users already want to use mTHP for anonymous shmem.

Quoted 38 lines

It might be that you need a special variant of finish_fault() for shmem?

Quoted 35 lines

mm/memory.c +15 -10

+    }
        vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
-                      vmf->address, &vmf->ptl);
+                       addr, &vmf->ptl);
        if (!vmf->pte)
            return VM_FAULT_NOPAGE;
          /* Re-check under ptl */
-    if (likely(!vmf_pte_changed(vmf))) {
-        struct folio *folio = page_folio(page);
-        int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
-
-        set_pte_range(vmf, folio, page, 1, vmf->address);
-        add_mm_counter(vma->vm_mm, type, 1);
-        ret = 0;
-    } else {
-        update_mmu_tlb(vma, vmf->address, vmf->pte);
+    if (nr_pages == 1 && unlikely(vmf_pte_changed(vmf))) {
+        update_mmu_tlb(vma, addr, vmf->pte);
+        ret = VM_FAULT_NOPAGE;
+        goto unlock;
+    } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) {
+        for (i = 0; i < nr_pages; i++)
+            update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pte + i);
            ret = VM_FAULT_NOPAGE;
+        goto unlock;
        }
    +    set_pte_range(vmf, folio, page, nr_pages, addr);
+    type = is_cow ? MM_ANONPAGES : mm_counter_file(folio);
+    add_mm_counter(vma->vm_mm, type, nr_pages);
+    ret = 0;
+
+unlock:
        pte_unmap_unlock(vmf->pte, vmf->ptl);
        return ret;
    }

Quoted 66 lines · Baolin Wang

Quoted 63 lines · Ryan Roberts

Quoted 61 lines · Baolin Wang

Quoted 50 lines · Ryan Roberts

Quoted 33 lines · Baolin Wang

Add large folio mapping establishment support for finish_fault() as a preparation,
to support multi-size THP allocation of anonymous shmem pages in the following patches.

Signed-off-by Baolin Wang <[email protected]>

---
    mm/memory.c | 43 +++++++++++++++++++++++++++++++++----------
    1 file changed, 33 insertions(+), 10 deletions(-)

mm/memory.c +6 -0

index eea6e4984eae..936377220b77 100644
@@ -4747,9 +4747,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
    {
        struct vm_area_struct *vma = vmf->vma;
        struct page *page;
+    struct folio *folio;
        vm_fault_t ret;
        bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) &&
                  !(vma->vm_flags & VM_SHARED);
+    int type, nr_pages, i;
+    unsigned long addr = vmf->address;
          /* Did we COW the page? */
        if (is_cow)

@@ -4780,24 +4783,44 @@ vm_fault_t finish_fault(struct vm_fault *vmf)
                return VM_FAULT_OOM;
        }
    +    folio = page_folio(page);
+    nr_pages = folio_nr_pages(folio);
+
+    if (unlikely(userfaultfd_armed(vma))) {

mm/memory.c vm_fault_t finish_fault(struct vm_fault *vmf) · line 4798

Quoted 10 lines

mm/memory.c +10 -0

+        nr_pages = 1;
+    } else if (nr_pages > 1) {
+        unsigned long start = ALIGN_DOWN(vmf->address, nr_pages * PAGE_SIZE);
+        unsigned long end = start + nr_pages * PAGE_SIZE;
+
+        /* In case the folio size in page cache beyond the VMA limits. */
+        addr = max(start, vma->vm_start);
+        nr_pages = (min(end, vma->vm_end) - addr) >> PAGE_SHIFT;
+
+        page = folio_page(folio, (addr - start) >> PAGE_SHIFT);

I still don't really follow the logic in this else if block. Isn't it possible that finish_fault() gets called with a page from a folio that isn't aligned with
vmf->address?

Quoted 5 lines

For example, let's say we have a file who's size is 64K and which is cached in a
single large folio in the page cache. But the file is mapped into a process at VA 16K to 80K. Let's say we fault on the first page (VA=16K). You will calculate

For shmem, this doesn't happen because the VA is aligned with the hugepage size in the shmem_get_unmapped_area() function. See patch 7.

Quoted 7 lines

Quoted 3 lines

Certainly agree that shmem can always make sure that it packs a vma in a way such that its folios are naturally aligned in VA when faulting in memory. If you mremap it, that alignment will be lost; I don't think that would be a problem

When mremap it, it will also call shmem_get_unmapped_area() to align the VA, but for mremap() with MAP_FIXED flag as David pointed out, yes, this patch may be not work perfectly.

Assuming this works similarly to anon mTHP, remapping to an arbitrary address shouldn't be a problem within a single process; the previously allocated folios will now be unaligned, but they will be correctly mapped so it doesn't break anything. And new faults will allocate folios so that they are as large as allowed by the sysfs interface AND which do not overlap with any non-none pte AND which are naturally aligned. It's when you start sharing with other processes that the fun and games start...

Quoted 10 lines

Quoted 9 lines

for a single process; mremap will take care of moving the ptes correctly and this path is not involved.

But what about the case when a process mmaps a shmem region, then forks, then the child mremaps the shmem region. Then the parent faults in a THP into the region (nicely aligned). Then the child faults in the same offset in the region and gets the THP that the parent allocated; that THP will be aligned in the parent's VM space but not in the child's.

Quoted 3 lines

Sorry, I did not get your point here. IIUC, the child's VA will also be aligned if the child mremap is not set MAP_FIXED, since the child's mremap will still call shmem_get_unmapped_area() to find an aligned new VA.

In general, you shouldn't be relying on the vma bounds being aligned to a THP boundary.

Quoted 1 line

Please correct me if I missed your point.

(I'm not 100% sure this is definitely how it works, but seems the only sane way to me):

Let's imagine we have a process that maps 4 pages of shared anon memory at VA=64K:

   mmap(64K, 16K, PROT_X, MAP_SHARED | MAP_ANONYMOUS | MAP_FIXED, ...)

Then it forks a child, and the child moves the mapping to VA=68K:

   mremap(64K, 16K, 16K, MREMAP_FIXED | MREMAP_MAYMOVE, 68K)

Then the parent writes to address 64K (offset 0 in the shared region); this will fault and cause a 16K mTHP to be allocated and mapped, covering the whole region at 64K-80K in the parent.

Then the child reads address 68K (offset 0 in the shared region); this will fault and cause the previously allocated 16K folio to be looked up and it must be mapped in the child between 68K-84K. This is not naturally aligned in the child.

For the child, your code will incorrectly calculate start/end as 64K-80K.

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise", "never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

By default, PMD-sized hugepages have enabled="inherit" and all other hugepage sizes have enabled="never" for '/sys/kernel/mm/transparent_hugepage/hugepages-xxkB/shmem_enabled'.

In addition, if top level value is 'force', then only PMD-sized hugepages have enabled="inherit", otherwise configuration will be failed and vice versa. That means now we will avoid using non-PMD sized THP to override the global huge allocation.

Signed-off-by Baolin Wang <[email protected]>

---
 Documentation/admin-guide/mm/transhuge.rst | 29 +++++++
 include/linux/huge_mm.h                    | 10 +++
 mm/huge_memory.c                           | 11 +--
 mm/shmem.c                                 | 96 ++++++++++++++++++++++
 4 files changed, 138 insertions(+), 8 deletions(-)

Documentation/admin-guide/mm/transhuge.rst +29 -0

index 076443cc10a6..a28496e15bdb 100644
@@ -332,6 +332,35 @@ deny
 force
     Force the huge option on for all - very useful for testing;
 
+Anonymous shmem can also use "multi-size THP" (mTHP) by adding a new sysfs knob
+to control mTHP allocation: /sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/shmem_enabled.
+Its value for each mTHP is essentially consistent with the global setting, except
+for the addition of 'inherit' to ensure compatibility with the global settings.
+always
+    Attempt to allocate <size> huge pages every time we need a new page;
+
+inherit
+    Inherit the top-level "shmem_enabled" value. By default, PMD-sized hugepages
+    have enabled="inherit" and all other hugepage sizes have enabled="never";
+
+never
+    Do not allocate <size> huge pages;
+
+within_size
+    Only allocate <size> huge page if it will be fully within i_size.
+    Also respect fadvise()/madvise() hints;
+
+advise
+    Only allocate <size> huge pages if requested with fadvise()/madvise();
+
+deny
+    Has the same semantics as 'never', now mTHP allocation policy is only
+    used for anonymous shmem and no not override tmpfs.
+
+force
+    Has the same semantics as 'always', now mTHP allocation policy is only
+    used for anonymous shmem and no not override tmpfs.
+
 Need of application restart
 ===========================
 

include/linux/huge_mm.h +10 -0

index e49b56c40a11..dbd6b3f56210 100644
@@ -6,6 +6,7 @@
 #include <linux/mm_types.h>
 
 #include <linux/fs.h> /* only for vma_is_dax() */
+#include <linux/kobject.h>
 
 vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf);
 int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,

@@ -63,6 +64,7 @@ ssize_t single_hugepage_flag_show(struct kobject *kobj,
 				  struct kobj_attribute *attr, char *buf,
 				  enum transparent_hugepage_flag flag);
 extern struct kobj_attribute shmem_enabled_attr;
+extern struct kobj_attribute thpsize_shmem_enabled_attr;
 
 /*
  * Mask of all large folio orders supported for anonymous THP; all orders up to

@@ -265,6 +267,14 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma,
 	return __thp_vma_allowable_orders(vma, vm_flags, tva_flags, orders);
 }
 
+struct thpsize {
+	struct kobject kobj;
+	struct list_head node;
+	int order;
+};
+
+#define to_thpsize(kobj) container_of(kobj, struct thpsize, kobj)
+
 enum mthp_stat_item {
 	MTHP_STAT_ANON_FAULT_ALLOC,
 	MTHP_STAT_ANON_FAULT_FALLBACK,

mm/huge_memory.c +3 -8

index 9efb6fefc391..d3080a8843f2 100644
@@ -449,14 +449,6 @@ static void thpsize_release(struct kobject *kobj);
 static DEFINE_SPINLOCK(huge_anon_orders_lock);
 static LIST_HEAD(thpsize_list);
 
-struct thpsize {
-	struct kobject kobj;
-	struct list_head node;
-	int order;
-};
-
-#define to_thpsize(kobj) container_of(kobj, struct thpsize, kobj)
-
 static ssize_t thpsize_enabled_show(struct kobject *kobj,
 				    struct kobj_attribute *attr, char *buf)
 {

@@ -517,6 +509,9 @@ static struct kobj_attribute thpsize_enabled_attr =
 
 static struct attribute *thpsize_attrs[] = {
 	&thpsize_enabled_attr.attr,
+#ifdef CONFIG_SHMEM
+	&thpsize_shmem_enabled_attr.attr,
+#endif
 	NULL,
 };
 

mm/shmem.c +96 -0

index a383ea9a89a5..59cc26d44344 100644
@@ -131,6 +131,14 @@ struct shmem_options {
 #define SHMEM_SEEN_QUOTA 32
 };
 
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+static unsigned long huge_anon_shmem_orders_always __read_mostly;
+static unsigned long huge_anon_shmem_orders_madvise __read_mostly;
+static unsigned long huge_anon_shmem_orders_inherit __read_mostly;
+static unsigned long huge_anon_shmem_orders_within_size __read_mostly;
+static DEFINE_SPINLOCK(huge_anon_shmem_orders_lock);
+#endif
+
 #ifdef CONFIG_TMPFS
 static unsigned long shmem_default_max_blocks(void)
 {

@@ -4687,6 +4695,12 @@ void __init shmem_init(void)
 		SHMEM_SB(shm_mnt->mnt_sb)->huge = shmem_huge;
 	else
 		shmem_huge = SHMEM_HUGE_NEVER; /* just in case it was patched */
+
+	/*
+	 * Default to setting PMD-sized THP to inherit the global setting and
+	 * disable all other multi-size THPs, when anonymous shmem uses mTHP.
+	 */
+	huge_anon_shmem_orders_inherit = BIT(HPAGE_PMD_ORDER);
 #endif
 	return;
 

@@ -4746,6 +4760,11 @@ static ssize_t shmem_enabled_store(struct kobject *kobj,
 			huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)
 		return -EINVAL;
 
+	/* Do not override huge allocation policy with non-PMD sized mTHP */
+	if (huge == SHMEM_HUGE_FORCE &&
+	    huge_anon_shmem_orders_inherit != BIT(HPAGE_PMD_ORDER))
+		return -EINVAL;
+
 	shmem_huge = huge;
 	if (shmem_huge > SHMEM_HUGE_DENY)
 		SHMEM_SB(shm_mnt->mnt_sb)->huge = shmem_huge;

@@ -4753,6 +4772,83 @@ static ssize_t shmem_enabled_store(struct kobject *kobj,
 }
 
 struct kobj_attribute shmem_enabled_attr = __ATTR_RW(shmem_enabled);
+
+static ssize_t thpsize_shmem_enabled_show(struct kobject *kobj,
+					  struct kobj_attribute *attr, char *buf)
+{
+	int order = to_thpsize(kobj)->order;
+	const char *output;
+
+	if (test_bit(order, &huge_anon_shmem_orders_always))
+		output = "[always] inherit within_size advise never deny [force]";
+	else if (test_bit(order, &huge_anon_shmem_orders_inherit))
+		output = "always [inherit] within_size advise never deny force";
+	else if (test_bit(order, &huge_anon_shmem_orders_within_size))
+		output = "always inherit [within_size] advise never deny force";
+	else if (test_bit(order, &huge_anon_shmem_orders_madvise))
+		output = "always inherit within_size [advise] never deny force";
+	else
+		output = "always inherit within_size advise [never] [deny] force";
+
+	return sysfs_emit(buf, "%s\n", output);
+}
+
+static ssize_t thpsize_shmem_enabled_store(struct kobject *kobj,
+					   struct kobj_attribute *attr,
+					   const char *buf, size_t count)
+{
+	int order = to_thpsize(kobj)->order;
+	ssize_t ret = count;
+
+	if (sysfs_streq(buf, "always") || sysfs_streq(buf, "force")) {
+		spin_lock(&huge_anon_shmem_orders_lock);
+		clear_bit(order, &huge_anon_shmem_orders_inherit);
+		clear_bit(order, &huge_anon_shmem_orders_madvise);
+		clear_bit(order, &huge_anon_shmem_orders_within_size);
+		set_bit(order, &huge_anon_shmem_orders_always);
+		spin_unlock(&huge_anon_shmem_orders_lock);
+	} else if (sysfs_streq(buf, "inherit")) {
+		/* Do not override huge allocation policy with non-PMD sized mTHP */
+		if (shmem_huge == SHMEM_HUGE_FORCE &&
+		    order != HPAGE_PMD_ORDER)
+			return -EINVAL;
+
+		spin_lock(&huge_anon_shmem_orders_lock);
+		clear_bit(order, &huge_anon_shmem_orders_always);
+		clear_bit(order, &huge_anon_shmem_orders_madvise);
+		clear_bit(order, &huge_anon_shmem_orders_within_size);
+		set_bit(order, &huge_anon_shmem_orders_inherit);
+		spin_unlock(&huge_anon_shmem_orders_lock);
+	} else if (sysfs_streq(buf, "within_size")) {
+		spin_lock(&huge_anon_shmem_orders_lock);
+		clear_bit(order, &huge_anon_shmem_orders_always);
+		clear_bit(order, &huge_anon_shmem_orders_inherit);
+		clear_bit(order, &huge_anon_shmem_orders_madvise);
+		set_bit(order, &huge_anon_shmem_orders_within_size);
+		spin_unlock(&huge_anon_shmem_orders_lock);
+	} else if (sysfs_streq(buf, "madvise")) {
+		spin_lock(&huge_anon_shmem_orders_lock);
+		clear_bit(order, &huge_anon_shmem_orders_always);
+		clear_bit(order, &huge_anon_shmem_orders_inherit);
+		clear_bit(order, &huge_anon_shmem_orders_within_size);
+		set_bit(order, &huge_anon_shmem_orders_madvise);
+		spin_unlock(&huge_anon_shmem_orders_lock);
+	} else if (sysfs_streq(buf, "never") || sysfs_streq(buf, "deny")) {
+		spin_lock(&huge_anon_shmem_orders_lock);
+		clear_bit(order, &huge_anon_shmem_orders_always);
+		clear_bit(order, &huge_anon_shmem_orders_inherit);
+		clear_bit(order, &huge_anon_shmem_orders_within_size);
+		clear_bit(order, &huge_anon_shmem_orders_madvise);
+		spin_unlock(&huge_anon_shmem_orders_lock);
+	} else {
+		ret = -EINVAL;
+	}
+
+	return ret;
+}
+
+struct kobj_attribute thpsize_shmem_enabled_attr =
+	__ATTR(shmem_enabled, 0644, thpsize_shmem_enabled_show, thpsize_shmem_enabled_store);
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE && CONFIG_SYSFS */
 
 #else /* !CONFIG_SHMEM */

Quoted 8 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise", "never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

that pattern here, it means the top level can always take "force" without any weird error checking. And we would allow "force" on the PMD-sized control but not on the others - again this is easy to error check.

Does this pattern make more sense? If so, is it too late to change hugepages-xxkB/enabled interface?

Quoted 12 lines · Ryan Roberts

Quoted 8 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise", "never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

Quoted 8 lines

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Agree. This is also our usage of 'inherit'.

Quoted 6 lines

that pattern here, it means the top level can always take "force" without any weird error checking. And we would allow "force" on the PMD-sized control but not on the others - again this is easy to error check.

Does this pattern make more sense? If so, is it too late to change hugepages-xxkB/enabled interface?

IMO, this sounds reasonable to me. Let's see what others think, David?

Quoted 18 lines · Baolin Wang

Quoted 15 lines · Ryan Roberts

Quoted 8 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise", "never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 7 lines

Quoted 5 lines

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Agree. This is also our usage of 'inherit'.

Quoted 20 lines · David Hildenbrand

Quoted 19 lines · Baolin Wang

Quoted 16 lines · Ryan Roberts

Quoted 9 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP by disabling the global knob. (I think that's the future we're heading to, where we'd have an "auto" mode that can be set on the global toggle).

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

Quoted 29 lines · David Hildenbrand

Quoted 20 lines · David Hildenbrand

Quoted 19 lines · Baolin Wang

Quoted 16 lines · Ryan Roberts

Quoted 9 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Agree. This is also our usage of 'inherit'.

Quoted 8 lines

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP by disabling the global knob. (I think that's the future we're heading to, where we'd have an "auto" mode that can be set on the global toggle).

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

- It is an error to set "force" for any size except PMD-size

- It is an error to set "force" for the global control if any size except PMD- size is set to "inherit"

- It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

Quoted 58 lines · Ryan Roberts

Quoted 31 lines · David Hildenbrand

Quoted 28 lines · David Hildenbrand

Quoted 27 lines · Baolin Wang

Quoted 23 lines · Ryan Roberts

Quoted 9 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is just
for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Quoted 8 lines

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP by disabling the global knob. (I think that's the future we're heading to, where we'd have an "auto" mode that can be set on the global toggle).

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

  - It is an error to set "force" for any size except PMD-size

  - It is an error to set "force" for the global control if any size except PMD- size is set to "inherit"

  - It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

Quoted 6 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

The strategy for tmpfs supporting mTHP will require more discussions and evaluations in the future. However, regardless of the strategy (explicit mTHP control or page cache control), I think it would be possible to use 'force' to override previous strategies for some testing purposes. This appears to be permissible according to the explanation in the current documentation: "force the huge option on for all - very useful for testing". So it seems not permanent?

Quoted 29 lines · David Hildenbrand

Quoted 20 lines · David Hildenbrand

Quoted 19 lines · Baolin Wang

Quoted 16 lines · Ryan Roberts

Quoted 9 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Agree. This is also our usage of 'inherit'.

Quoted 8 lines

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP by disabling the global knob. (I think that's the future we're heading to, where we'd have an "auto" mode that can be set on the global toggle).

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

- It is an error to set "force" for any size except PMD-size

- It is an error to set "force" for the global control if any size except PMD- size is set to "inherit"

- It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

Quoted 56 lines · Ryan Roberts

Quoted 29 lines · David Hildenbrand

Quoted 20 lines · David Hildenbrand

Quoted 19 lines · Baolin Wang

Quoted 16 lines · Ryan Roberts

Quoted 9 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one. Perhaps we should have only allowed the PMD-sized enable=inherit (this is just for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Agree. This is also our usage of 'inherit'.

Quoted 8 lines

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP by disabling the global knob. (I think that's the future we're heading to, where we'd have an "auto" mode that can be set on the global toggle).

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

- It is an error to set "force" for any size except PMD-size

- It is an error to set "force" for the global control if any size except PMD- size is set to "inherit"

- It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

Quoted 6 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Quoted 71 lines · David Hildenbrand

Quoted 63 lines · Ryan Roberts

Quoted 41 lines · David Hildenbrand

Quoted 30 lines · David Hildenbrand

Quoted 29 lines · Baolin Wang

Quoted 25 lines · Ryan Roberts

Quoted 10 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is just
for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP
by disabling the global knob. (I think that's the future we're heading to, where
we'd have an "auto" mode that can be set on the global toggle).

Quoted 2 lines

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

   - It is an error to set "force" for any size except PMD-size

   - It is an error to set "force" for the global control if any size except PMD-
     size is set to "inherit"

   - It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

Quoted 6 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

Quoted 3 lines

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Also noting that memory waste is not really a problem when a write to a shmem file allocates a large folio that stays within boundaries of that write; issues only pop up if you end up over-allocating, especially, during page faults where you have not that much clue about what to do (single address, no real range provided).

There is the other issue that wasting large chunks of contiguous memory on stuff that barely benefits from it. With memory that maybe never gets evicted, there is no automatic "handing back" of that memory to the system to be used by something else. With ordinary files, that's a bit different. But I did not look closer into that issue yet, it's one of the reasons MADV_HUGEPAGE was added IIRC.

Quoted 82 lines · David Hildenbrand

Quoted 71 lines · David Hildenbrand

Quoted 63 lines · Ryan Roberts

Quoted 41 lines · David Hildenbrand

Quoted 30 lines · David Hildenbrand

Quoted 29 lines · Baolin Wang

Quoted 25 lines · Ryan Roberts

Quoted 10 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is just
for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP
by disabling the global knob. (I think that's the future we're heading to, where
we'd have an "auto" mode that can be set on the global toggle).

Quoted 2 lines

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

   - It is an error to set "force" for any size except PMD-size

   - It is an error to set "force" for the global control if any size except PMD-
     size is set to "inherit"

   - It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

Quoted 6 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

Quoted 3 lines

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Also noting that memory waste is not really a problem when a write to a shmem file allocates a large folio that stays within boundaries of that write; issues only pop up if you end up over-allocating, especially, during page faults where you have not that much clue about what to do (single address, no real range provided).

Quoted 5 lines

There is the other issue that wasting large chunks of contiguous memory on stuff that barely benefits from it. With memory that maybe never gets evicted, there is no automatic "handing back" of that memory to the system to be used by something else. With ordinary files, that's a bit different. But I did not look closer into that issue yet, it's one of the reasons MADV_HUGEPAGE was added IIRC.

OK understood. Although, with tmpfs you're not going to mmap it then randomly extend the file through page faults - mmap doesn't permit that, I don't think? So presumably the user must explicitly set the size of the file first? Are you suggesting there are a lot of use cases where a large tmpfs file is created, mmaped then only accessed sparsely?

Quoted 82 lines · David Hildenbrand

Quoted 71 lines · David Hildenbrand

Quoted 63 lines · Ryan Roberts

Quoted 41 lines · David Hildenbrand

Quoted 30 lines · David Hildenbrand

Quoted 29 lines · Baolin Wang

Quoted 25 lines · Ryan Roberts

Quoted 10 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface 'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP, with a value similar to the top level 'shmem_enabled', which can be set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is just
for legacy back compat after all, I don't think there is any use case where changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP
by disabling the global knob. (I think that's the future we're heading to, where
we'd have an "auto" mode that can be set on the global toggle).

Quoted 2 lines

But I am just making up use cases ;) I think it will be valuable and just doing it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

   - It is an error to set "force" for any size except PMD-size

   - It is an error to set "force" for the global control if any size except
PMD-
     size is set to "inherit"

   - It is an error to set "inherit" for any size except PMD-size if the global control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest UX from the user's point of view when they start seeing errors.

Quoted 6 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error checking becomes permanent).

Quoted 3 lines

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Also noting that memory waste is not really a problem when a write to a shmem file allocates a large folio that stays within boundaries of that write; issues only pop up if you end up over-allocating, especially, during page faults where you have not that much clue about what to do (single address, no real range provided).

Quoted 5 lines

There is the other issue that wasting large chunks of contiguous memory on stuff that barely benefits from it. With memory that maybe never gets evicted, there is no automatic "handing back" of that memory to the system to be used by something else. With ordinary files, that's a bit different. But I did not look closer into that issue yet, it's one of the reasons MADV_HUGEPAGE was added IIRC.

OK understood. Although, with tmpfs you're not going to mmap it then randomly extend the file through page faults - mmap doesn't permit that, I don't think? So presumably the user must explicitly set the size of the file first? Are you suggesting there are a lot of use cases where a large tmpfs file is created, mmaped then only accessed sparsely?

Quoted 102 lines · Ryan Roberts

Quoted 100 lines · David Hildenbrand

Quoted 83 lines · David Hildenbrand

Quoted 74 lines · Ryan Roberts

Quoted 48 lines · David Hildenbrand

Quoted 37 lines · David Hildenbrand

Quoted 36 lines · Baolin Wang

Quoted 32 lines · Ryan Roberts

Quoted 7 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface
'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP,
with a value similar to the top level 'shmem_enabled', which can be

Quoted 8 lines

set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top
level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to
allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is just
for legacy back compat after all, I don't think there is any use case where
changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP
by disabling the global knob. (I think that's the future we're heading to, where
we'd have an "auto" mode that can be set on the global toggle).

Quoted 3 lines

But I am just making up use cases ;) I think it will be valuable and just doing
it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And
yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

    - It is an error to set "force" for any size except PMD-size

    - It is an error to set "force" for the global control if any size except PMD-
      size is set to "inherit"

    - It is an error to set "inherit" for any size except PMD-size if the global
      control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest
UX from the user's point of view when they start seeing errors.

Quoted 7 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error
checking becomes permanent).

Quoted 3 lines

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Also noting that memory waste is not really a problem when a write to a shmem file allocates a large folio that stays within boundaries of that write; issues only pop up if you end up over-allocating, especially, during page faults where you have not that much clue about what to do (single address, no real range provided).

There is the other issue that wasting large chunks of contiguous memory on stuff that barely benefits from it. With memory that maybe never gets evicted, there is no automatic "handing back" of that memory to the system to be used by something else. With ordinary files, that's a bit different. But I did not look closer into that issue yet, it's one of the reasons MADV_HUGEPAGE was added IIRC.

Quoted 5 lines

OK understood. Although, with tmpfs you're not going to mmap it then randomly extend the file through page faults - mmap doesn't permit that, I don't think? So presumably the user must explicitly set the size of the file first? Are you suggesting there are a lot of use cases where a large tmpfs file is created, mmaped then only accessed sparsely?

I don't know about "a lot of use cases", but for VMs that's certainly how it's used.

Quoted 103 lines · David Hildenbrand

Quoted 102 lines · Ryan Roberts

Quoted 100 lines · David Hildenbrand

Quoted 83 lines · David Hildenbrand

Quoted 74 lines · Ryan Roberts

Quoted 48 lines · David Hildenbrand

Quoted 37 lines · David Hildenbrand

Quoted 36 lines · Baolin Wang

Quoted 32 lines · Ryan Roberts

Quoted 7 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface
'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP,
with a value similar to the top level 'shmem_enabled', which can be

Quoted 8 lines

set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top
level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to
allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is just
for legacy back compat after all, I don't think there is any use case where
changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any THP
by disabling the global knob. (I think that's the future we're heading to, where
we'd have an "auto" mode that can be set on the global toggle).

Quoted 3 lines

But I am just making up use cases ;) I think it will be valuable and just doing
it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And
yes, I had forgotten about the glorious "auto" future. So probably continuing all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

    - It is an error to set "force" for any size except PMD-size

    - It is an error to set "force" for the global control if any size except
PMD-
      size is set to "inherit"

    - It is an error to set "inherit" for any size except PMD-size if the
global
      control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest
UX from the user's point of view when they start seeing errors.

Quoted 7 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force". But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be more suited to the approach the page cache takes to transparently ramp up the folio size as it faults more in. (Just saying there is a chance that this error
checking becomes permanent).

Quoted 3 lines

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Also noting that memory waste is not really a problem when a write to a shmem file allocates a large folio that stays within boundaries of that write; issues only pop up if you end up over-allocating, especially, during page faults where you have not that much clue about what to do (single address, no real range provided).

There is the other issue that wasting large chunks of contiguous memory on stuff that barely benefits from it. With memory that maybe never gets evicted, there is no automatic "handing back" of that memory to the system to be used by something else. With ordinary files, that's a bit different. But I did not look closer into that issue yet, it's one of the reasons MADV_HUGEPAGE was added IIRC.

Quoted 8 lines

Quoted 5 lines

OK understood. Although, with tmpfs you're not going to mmap it then randomly extend the file through page faults - mmap doesn't permit that, I don't think? So presumably the user must explicitly set the size of the file first? Are you suggesting there are a lot of use cases where a large tmpfs file is created, mmaped then only accessed sparsely?

I don't know about "a lot of use cases", but for VMs that's certainly how it's used.

Gottya, thanks. And out of curiosity, what's the benefit of using tmpfs rather than private (or shared) anonymous memory for VMs?

Quoted 116 lines · Ryan Roberts

Quoted 115 lines · David Hildenbrand

Quoted 114 lines · Ryan Roberts

Quoted 112 lines · David Hildenbrand

Quoted 91 lines · David Hildenbrand

Quoted 79 lines · Ryan Roberts

Quoted 51 lines · David Hildenbrand

Quoted 39 lines · David Hildenbrand

Quoted 38 lines · Baolin Wang

Quoted 34 lines · Ryan Roberts

Quoted 7 lines · Baolin Wang

To support the use of mTHP with anonymous shmem, add a new sysfs interface
'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/' directory for each mTHP to control whether shmem is enabled for that mTHP,
with a value similar to the top level 'shmem_enabled', which can be

Quoted 8 lines

set to:
"always", "inherit (to inherit the top level setting)", "within_size", "advise",
"never", "deny", "force". These values follow the same semantics as the top
level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
to 'always' to keep compatibility.

We decided at [1] to not allow 'force' for non-PMD-sizes.

[1]
https://lore.kernel.org/linux-mm/[email protected]/

However, thinking about this a bit more, I wonder if the decision we made to
allow all hugepages-xxkB/enabled controls to take "inherit" was the wrong
one.
Perhaps we should have only allowed the PMD-sized enable=inherit (this is
just
for legacy back compat after all, I don't think there is any use case where
changing multiple mTHP size controls atomically is actually useful). Applying

Quoted 1 line

Quoted 1 line

Agree. This is also our usage of 'inherit'.

Missed that one: there might be use cases in the future once we would start defaulting to "inherit" for all knobs (a distro might default to that) and default-enable THP in the global knob. Then, it would be easy to disable any
THP
by disabling the global knob. (I think that's the future we're heading to, where
we'd have an "auto" mode that can be set on the global toggle).

Quoted 3 lines

But I am just making up use cases ;) I think it will be valuable and just doing
it consistently now might be cleaner.

I agree that consistency between enabled and shmem_enabled is top priority. And
yes, I had forgotten about the glorious "auto" future. So probably continuing
all sizes to select "inherit" is best.

But for shmem_enabled, that means we need the following error checking:

     - It is an error to set "force" for any size except PMD-size

     - It is an error to set "force" for the global control if any size except
PMD-
       size is set to "inherit"

     - It is an error to set "inherit" for any size except PMD-size if the global
       control is set to "force".

Certainly not too difficult to code and prove to be correct, but not the nicest
UX from the user's point of view when they start seeing errors.

Quoted 10 lines

I think we previously said this would likely be temporary, and if/when tmpfs gets mTHP support, we could simplify and allow all sizes to be set to "force".
But I wonder if tmpfs would ever need explicit mTHP control? Maybe it would be
more suited to the approach the page cache takes to transparently ramp up the
folio size as it faults more in. (Just saying there is a chance that this error
checking becomes permanent).

Quoted 3 lines

Note that with shmem you're inherently facing the same memory waste issues etc as you would with anonymous memory. (sometimes even worse, if you're running shmem that's configured to be unswappable!).

Also noting that memory waste is not really a problem when a write to a shmem file allocates a large folio that stays within boundaries of that write; issues
only pop up if you end up over-allocating, especially, during page faults where
you have not that much clue about what to do (single address, no real range provided).

There is the other issue that wasting large chunks of contiguous memory on stuff
that barely benefits from it. With memory that maybe never gets evicted, there is no automatic "handing back" of that memory to the system to be used by something else. With ordinary files, that's a bit different. But I did not look
closer into that issue yet, it's one of the reasons MADV_HUGEPAGE was added IIRC.

Quoted 9 lines

Quoted 8 lines

Quoted 5 lines

OK understood. Although, with tmpfs you're not going to mmap it then randomly extend the file through page faults - mmap doesn't permit that, I don't think? So presumably the user must explicitly set the size of the file first? Are you suggesting there are a lot of use cases where a large tmpfs file is created, mmaped then only accessed sparsely?

I don't know about "a lot of use cases", but for VMs that's certainly how it's used.

There are more details around that and the sparsity (memory ballooning, virtio-mem, free page reporting), but it might distract here :) I'll note that shmem+THP is known to be problematic with memory ballooning.

Quoted 2 lines

Gottya, thanks. And out of curiosity, what's the benefit of using tmpfs rather than private (or shared) anonymous memory for VMs?

The primary use case I know of is sharing VM memory with other processes (usually not child processes): DPDK/SPDK and other vhost-user variants (such as virtiofs) mmap() all guest memory to access it directly (some sort of multi-process hypervisors). They either use real-file-based shmem or memfd (essentially the same without a named file) for that.

Then, there is live-hypervisor upgrade, whereby you start a second hypervisor process that will take over. People use shmem for that, so you can minimize downtime by migrating guest memory simply by mmap'ing the shmem file into the new hypervisor.

Shared anonymous memory is basically never used (I only know of one corner case in QEMU).

I would assume that there are also DBs making use of rather sparse shmem? But no expert on that.

Quoted 1 line

Quoted 1 line

mm/shmem.c:1780:10: warning: variable 'folio' is used uninitialized whenever 'while' loop exits because its condition is false [-Wsometimes-uninitialized]

     1780 |                 while (suitable_orders) {
          |                        ^~~~~~~~~~~~~~~
    mm/shmem.c:1795:7: note: uninitialized use occurs here
     1795 |         if (!folio)
          |              ^~~~~
    mm/shmem.c:1780:10: note: remove the condition if it is always true
     1780 |                 while (suitable_orders) {
          |                        ^~~~~~~~~~~~~~~
          |                        1
    mm/shmem.c:1750:21: note: initialize the variable 'folio' to silence this warning
     1750 |         struct folio *folio;
          |                            ^
          |                             = NULL
    mm/shmem.c:1564:20: warning: unused function 'shmem_show_mpol' [-Wunused-function]
     1564 | static inline void shmem_show_mpol(struct seq_file *seq, struct mempolicy *mpol)

Hey Baolin,

I found a compilation issue that failed one[1] of my configurations after applying this series. The error message is as follows:

mm/shmem.c: In function ‘shmem_get_unmapped_area’:
././include/linux/compiler_types.h:460:45: error: call to ‘__compiletime_assert_481’ declared with attribute error: BUILD_BUG failed
_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
^
././include/linux/compiler_types.h:441:25: note: in definition of macro ‘__compiletime_assert’

                          prefix ## suffix();                             \
                          ^~~~~~

././include/linux/compiler_types.h:460:9: note: in expansion of macro ‘_compiletime_assert’
_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) ^~~~~~~~~~~~~~~~~~~
./include/linux/build_bug.h:39:37: note: in expansion of macro ‘compiletime_assert’

  #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
                                      ^~~~~~~~~~~~~~~~~~
./include/linux/build_bug.h:59:21: note: in expansion of macro ‘BUILD_BUG_ON_MSG’
  #define BUILD_BUG() BUILD_BUG_ON_MSG(1, "BUILD_BUG failed")
                      ^~~~~~~~~~~~~~~~
./include/linux/huge_mm.h:97:28: note: in expansion of macro ‘BUILD_BUG’
  #define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; })
                             ^~~~~~~~~
./include/linux/huge_mm.h:104:35: note: in expansion of macro ‘HPAGE_PMD_SHIFT’
  #define HPAGE_PMD_SIZE  ((1UL) << HPAGE_PMD_SHIFT)
                                    ^~~~~~~~~~~~~~~
mm/shmem.c:2419:36: note: in expansion of macro ‘HPAGE_PMD_SIZE’
         unsigned long hpage_size = HPAGE_PMD_SIZE;
                                    ^~~~~~~~~~~~~~~

It seems like we need to handle the case where CONFIG_PGTABLE_HAS_HUGE_LEAVES is undefined.

[1] export ARCH=arm64 && make allnoconfig && make olddefconfig && make -j$(nproc)

Hi Lance,

Quoted 31 lines · Lance Yang

Hey Baolin,

I found a compilation issue that failed one[1] of my configurations after applying this series. The error message is as follows:

mm/shmem.c: In function ‘shmem_get_unmapped_area’:
././include/linux/compiler_types.h:460:45: error: call to ‘__compiletime_assert_481’ declared with attribute error: BUILD_BUG failed
_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
^
././include/linux/compiler_types.h:441:25: note: in definition of macro ‘__compiletime_assert’

                          prefix ## suffix();                             \
                          ^~~~~~

././include/linux/compiler_types.h:460:9: note: in expansion of macro ‘_compiletime_assert’
_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) ^~~~~~~~~~~~~~~~~~~
./include/linux/build_bug.h:39:37: note: in expansion of macro ‘compiletime_assert’

  #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
                                      ^~~~~~~~~~~~~~~~~~
./include/linux/build_bug.h:59:21: note: in expansion of macro ‘BUILD_BUG_ON_MSG’
  #define BUILD_BUG() BUILD_BUG_ON_MSG(1, "BUILD_BUG failed")
                      ^~~~~~~~~~~~~~~~
./include/linux/huge_mm.h:97:28: note: in expansion of macro ‘BUILD_BUG’
  #define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; })
                             ^~~~~~~~~
./include/linux/huge_mm.h:104:35: note: in expansion of macro ‘HPAGE_PMD_SHIFT’
  #define HPAGE_PMD_SIZE  ((1UL) << HPAGE_PMD_SHIFT)
                                    ^~~~~~~~~~~~~~~
mm/shmem.c:2419:36: note: in expansion of macro ‘HPAGE_PMD_SIZE’
         unsigned long hpage_size = HPAGE_PMD_SIZE;
                                    ^~~~~~~~~~~~~~~

Quoted 4 lines

It seems like we need to handle the case where CONFIG_PGTABLE_HAS_HUGE_LEAVES is undefined.

[1] export ARCH=arm64 && make allnoconfig && make olddefconfig && make -j$(nproc)

Thanks for reporting. I can move the use of HPAGE_PMD_SIZE to after the check for CONFIG_TRANSPARENT_HUGEPAGE, which can avoid the building error:

mm/shmem.c +1 -1

index 1af2f0aa384d..d603e36e0f4f 100644
@@ -2416,7 +2416,7 @@ unsigned long shmem_get_unmapped_area(struct file
 *file,
         unsigned long inflated_len;
         unsigned long inflated_addr;
         unsigned long inflated_offset;
-       unsigned long hpage_size = HPAGE_PMD_SIZE;
+       unsigned long hpage_size;
 
         if (len > TASK_SIZE)

                 return -ENOMEM;

mm/shmem.c +1 -0

@@ -2446,6 +2446,7 @@ unsigned long shmem_get_unmapped_area(struct file
 *file,
         if (uaddr == addr)
                 return addr;
 
+       hpage_size = HPAGE_PMD_SIZE;
         if (shmem_huge != SHMEM_HUGE_FORCE) {
                 struct super_block *sb;

                 unsigned long __maybe_unused hpage_orders;

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level

'/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

Use the page fault latency tool to measure the performance of 1G anonymous shmem with 32 threads on my machine environment with: ARM64 Architecture, 32 cores, 125G memory:
base: mm-unstable
user-time sys_time faults_per_sec_per_cpu faults_per_sec
0.04s 3.10s 83516.416 2669684.890

mm-unstable + patchset, anon shmem mTHP disabled
user-time sys_time faults_per_sec_per_cpu faults_per_sec
0.02s 3.14s 82936.359 2630746.027

mm-unstable + patchset, anon shmem 64K mTHP enabled
user-time sys_time faults_per_sec_per_cpu faults_per_sec
0.08s 0.31s 678630.231 17082522.495

From the data above, it is observed that the patchset has a minimal impact when mTHP is not enabled (some fluctuations observed during testing). When enabling 64K mTHP, there is a significant improvement of the page fault latency.

TODO:
- Support mTHP for tmpfs.
- Do not split the large folio when share memory swap out.
- Can swap in a large folio for share memory.

Changes from RFC:
- Rebase the patch set against the new mm-unstable branch, per Lance.
- Add a new patch to export highest_order() and next_order().
- Add a new patch to align mTHP size in shmem_get_unmapped_area().
- Handle the uffd case and the VMA limits case when building mapping for large folio in the finish_fault() function, per Ryan.
- Remove unnecessary 'order' variable in patch 3, per Kefeng.
- Keep the anon shmem counters' name consistency.
- Modify the strategy to support mTHP for anonymous shmem, discussed with Ryan and David.
- Add reviewed tag from Barry.
- Update the commit message.

Baolin Wang (8):
mm: move highest_order() and next_order() out of the THP config
mm: memory: extend finish_fault() to support large folio
mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
mm: shmem: add THP validation for PMD-mapped THP related statistics
mm: shmem: add multi-size THP sysfs interface for anonymous shmem
mm: shmem: add mTHP support for anonymous shmem
mm: shmem: add mTHP size alignment in shmem_get_unmapped_area
mm: shmem: add mTHP counters for anonymous shmem

 Documentation/admin-guide/mm/transhuge.rst |  29 ++
 include/linux/huge_mm.h                    |  35 ++-
 mm/huge_memory.c                           |  17 +-
 mm/memory.c                                |  43 ++-
 mm/shmem.c                                 | 335 ++++++++++++++++++---
 5 files changed, 387 insertions(+), 72 deletions(-)

Quoted 13 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 3 lines

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level

Didn't we agree that "force" would not be supported for now, and would return an error when attempting to set for a non-PMD-size hugepage-XXkb/shmem_enabled (or indirectly through inheritance)?

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

Use the page fault latency tool to measure the performance of 1G anonymous shmem with 32 threads on my machine environment with: ARM64 Architecture, 32 cores, 125G memory:
base: mm-unstable
user-time sys_time faults_per_sec_per_cpu faults_per_sec
0.04s 3.10s 83516.416 2669684.890

mm-unstable + patchset, anon shmem mTHP disabled
user-time sys_time faults_per_sec_per_cpu faults_per_sec
0.02s 3.14s 82936.359 2630746.027

mm-unstable + patchset, anon shmem 64K mTHP enabled
user-time sys_time faults_per_sec_per_cpu faults_per_sec
0.08s 0.31s 678630.231 17082522.495

From the data above, it is observed that the patchset has a minimal impact when mTHP is not enabled (some fluctuations observed during testing). When enabling 64K mTHP, there is a significant improvement of the page fault latency.

TODO:
- Support mTHP for tmpfs.
- Do not split the large folio when share memory swap out.
- Can swap in a large folio for share memory.

Changes from RFC:
- Rebase the patch set against the new mm-unstable branch, per Lance.
- Add a new patch to export highest_order() and next_order().
- Add a new patch to align mTHP size in shmem_get_unmapped_area().
- Handle the uffd case and the VMA limits case when building mapping for large folio in the finish_fault() function, per Ryan.
- Remove unnecessary 'order' variable in patch 3, per Kefeng.
- Keep the anon shmem counters' name consistency.
- Modify the strategy to support mTHP for anonymous shmem, discussed with Ryan and David.
- Add reviewed tag from Barry.
- Update the commit message.

Baolin Wang (8):
mm: move highest_order() and next_order() out of the THP config
mm: memory: extend finish_fault() to support large folio
mm: shmem: add an 'order' parameter for shmem_alloc_hugefolio()
mm: shmem: add THP validation for PMD-mapped THP related statistics
mm: shmem: add multi-size THP sysfs interface for anonymous shmem
mm: shmem: add mTHP support for anonymous shmem
mm: shmem: add mTHP size alignment in shmem_get_unmapped_area
mm: shmem: add mTHP counters for anonymous shmem

 Documentation/admin-guide/mm/transhuge.rst |  29 ++
 include/linux/huge_mm.h                    |  35 ++-
 mm/huge_memory.c                           |  17 +-
 mm/memory.c                                |  43 ++-
 mm/shmem.c                                 | 335 ++++++++++++++++++---
 5 files changed, 387 insertions(+), 72 deletions(-)

Signature · 1 line

2.39.3

Quoted 13 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 8 lines

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

Quoted 23 lines · Daniel Gomez

Quoted 13 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 8 lines

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

Quoted 5 lines

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

mTHP are implemented using large folios, just like traditional PMD-sized THP are. (you really should explore the history of mTHP and how it all works internally)

The biggest challenge with memory that cannot be evicted on memory pressure to be reclaimed (in contrast to your ordinary files in the pagecache) is memory waste, well, and placement of large chunks of memory in general, during page faults.

In the worst case (no swap), you allocate a large chunk of memory once and it will stick around until freed: no reclaim of that memory.

That's the reason why THP for anonymous memory and SHMEM have toggles to manually enable and configure them, in contrast to the pagecache. The same was done for mTHP for anonymous memory, and now (anon) shmem follows.

There are plans to have, at some point, have it all working automatically, but a lot for that for anonymous memory (and shmem similarly) is still missing and unclear.

Signature · 4 lines

-- 
Cheers,

David / dhildenb

Quoted 30 lines · David Hildenbrand

Quoted 28 lines · Daniel Gomez

Quoted 13 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 8 lines

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

Quoted 3 lines

mTHP are implemented using large folios, just like traditional PMD-sized THP are. (you really should explore the history of mTHP and how it all works internally)

I'll check more in deep the code. By any chance are any of you going to be at LSFMM this year? I have this session [1] scheduled for Wednesday and it would be nice to get your feedback on it and if you see this working together with mTHP/THP.

[1] https://lore.kernel.org/all/4ktpayu66noklllpdpspa3vm5gbmb5boxskcj2q6qn7md3pwwt@kvlu64pqwjzl/

Quoted 6 lines

The biggest challenge with memory that cannot be evicted on memory pressure to be reclaimed (in contrast to your ordinary files in the pagecache) is memory waste, well, and placement of large chunks of memory in general, during page faults.

Quoted 2 lines

In the worst case (no swap), you allocate a large chunk of memory once and it will stick around until freed: no reclaim of that memory.

I can see that path being triggered by some fstests but only for THP (where we can actually reclaim memory).

Quoted 31 lines · Daniel Gomez

Quoted 30 lines · David Hildenbrand

Quoted 28 lines · Daniel Gomez

Quoted 13 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 8 lines

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

Quoted 9 lines

Quoted 3 lines

mTHP are implemented using large folios, just like traditional PMD-sized THP are. (you really should explore the history of mTHP and how it all works internally)

I'll check more in deep the code. By any chance are any of you going to be at LSFMM this year? I have this session [1] scheduled for Wednesday and it would be nice to get your feedback on it and if you see this working together with mTHP/THP.

I'll be around and will attend that session! But note that I am still scratching my head what to do with "ordinary" shmem, especially because of the weird way shmem behaves in contrast to real files (below). Some input from Hugh might be very helpful.

Example: you write() to a shmem file and populate a 2M THP. Then, nobody touches that file for a long time. There are certainly other mmap() users that could better benefit from that THP ... and without swap that THP will be trapped there possibly a long time (unless I am missing an important piece of shmem THP design :) )? Sure, if we only have THP's it's nice, that's just not the reality unfortunately. IIRC, that's one of the reasons why THP for shmem can be enabled/disabled. But again, still scratching my head ...

Note that this patch set only tackles anonymous shmem (MAP_SHARED|MAP_ANON), which is in 99.999% of all cases only accessed via page tables (memory allocated during page faults). I think there are ways to grab the fd (/proc/self/fd), but IIRC only corner cases read/write that.

So in that sense, anonymous shmem (this patch set) behaves mostly like ordinary anonymous memory, and likely there is not much overlap with other "allocate large folios during read/write/fallocate" as in [1]. swap might have an overlap.

The real confusion begins when we have ordinary shmem: some users never mmap it and only read/write, some users never read/write it and only mmap it and some (less common?) users do both.

And shmem really is special: it looks like "just another file", but memory-consumption and reclaim wise it behaves just like anonymous memory. It might be swappable ("usually very limited backing disk space available") or it might not.

In a subthread here we are discussing what to do with that special "shmem_enabled = force" mode ... and it's all complicated I think.

Quoted 8 lines

[1] https://lore.kernel.org/all/4ktpayu66noklllpdpspa3vm5gbmb5boxskcj2q6qn7md3pwwt@kvlu64pqwjzl/

Quoted 6 lines

The biggest challenge with memory that cannot be evicted on memory pressure to be reclaimed (in contrast to your ordinary files in the pagecache) is memory waste, well, and placement of large chunks of memory in general, during page faults.

Quoted 5 lines

Quoted 2 lines

In the worst case (no swap), you allocate a large chunk of memory once and it will stick around until freed: no reclaim of that memory.

I can see that path being triggered by some fstests but only for THP (where we can actually reclaim memory).

Is that when we punch-hole a partial THP and split it? I'd be interested in what that test does.

Signature · 4 lines

-- 
Cheers,

David / dhildenb

Quoted 30 lines · David Hildenbrand

Quoted 23 lines · Daniel Gomez

Quoted 13 lines · Baolin Wang

Anonymous pages have already been supported for multi-size (mTHP) allocation through commit 19eaf44954df, that can allow THP to be configured through the sysfs interface located at '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'.

However, the anonymous shared pages will ignore the anonymous mTHP rule configured through the sysfs interface, and can only use the PMD-mapped THP, that is not reasonable. Many implement anonymous page sharing through mmap(MAP_SHARED | MAP_ANONYMOUS), especially in database usage scenarios, therefore, users expect to apply an unified mTHP strategy for anonymous pages, also including the anonymous shared pages, in order to enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture to reduce TLB miss etc.

Quoted 8 lines

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

Quoted 5 lines

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

Quoted 3 lines

mTHP are implemented using large folios, just like traditional PMD-sized THP are. (you really should explore the history of mTHP and how it all works internally)

I'll check more in deep the code. By any chance are any of you going to be at LSFMM this year? I have this session [1] scheduled for Wednesday and it would be nice to get your feedback on it and if you see this working together with mTHP/THP.

[1] https://lore.kernel.org/all/4ktpayu66noklllpdpspa3vm5gbmb5boxskcj2q6qn7md3pwwt@kvlu64pqwjzl/

Quoted 10 lines · Daniel Gomez

Quoted 8 lines · Baolin Wang

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

Quoted 5 lines

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

mTHP are implemented using large folios, just like traditional PMD-sized THP are.

The biggest challenge with memory that cannot be evicted on memory pressure to be reclaimed (in contrast to your ordinary files in the pagecache) is memory waste, well, and placement of large chunks of memory in general, during page faults.

In the worst case (no swap), you allocate a large chunk of memory once and it will stick around until freed: no reclaim of that memory.

That's the reason why THP for anonymous memory and SHMEM have toggles to manually enable and configure them, in contrast to the pagecache. The same was done for mTHP for anonymous memory, and now (anon) shmem follows.

There are plans to have, at some point, have it all working automatically, but a lot for that for anonymous memory (and shmem similarly) is still missing and unclear.

Quoted 28 lines · David Hildenbrand

Quoted 10 lines · Daniel Gomez

Quoted 8 lines · Baolin Wang

The primary strategy is similar to supporting anonymous mTHP. Introduce a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', which can have all the same values as the top-level '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new additional "inherit" option. By default all sizes will be set to "never" except PMD size, which is set to "inherit". This ensures backward compatibility with the shmem enabled of the top level, meanwhile also allows independent control of shmem enabled for each mTHP.

Quoted 5 lines

I'm trying to understand the adoption of mTHP and how it fits into the adoption of (large) folios that the kernel is moving towards. Can you, or anyone involved here, explain this? How much do they overlap, and can we benefit from having both? Is there any argument against the adoption of large folios here that I might have missed?

mTHP are implemented using large folios, just like traditional PMD-sized THP are.

The biggest challenge with memory that cannot be evicted on memory pressure to be reclaimed (in contrast to your ordinary files in the pagecache) is memory waste, well, and placement of large chunks of memory in general, during page faults.

In the worst case (no swap), you allocate a large chunk of memory once and it will stick around until freed: no reclaim of that memory.

Quoted 7 lines

That's the reason why THP for anonymous memory and SHMEM have toggles to manually enable and configure them, in contrast to the pagecache. The same was done for mTHP for anonymous memory, and now (anon) shmem follows.

There are plans to have, at some point, have it all working automatically, but a lot for that for anonymous memory (and shmem similarly) is still missing and unclear.

Whereas the use for large folios for filesystems is already automatic, so long as the filesystem supports it. We do this in readahead and write path already for iomap, we opportunistically use large folios if we can, otherwise we use smaller folios.

So a recommended approach by Matthew was to use the readahead and write path, just as in iomap to determine the size of the folio to use [0]. The use of large folios would also be automatic and not require any knobs at all.

The mTHP approach would be growing the "THP" use in filesystems by the only single filesystem to use THP. Meanwhile use of large folios is already automatic with the approach taken by iomap.

We're at a crux where it does beg the question if we should continue to chug on with tmpfs being special and doing things differently extending the old THP interface with mTHP, or if it should just use large folios using the same approach as iomap did.

From my perspective the more shared code the better, and the more shared
paths the better. There is a chance to help test swap with large folios instead of splitting the folios for swap, and that would could be done first with tmpfs. I have not evaluated the difference in testing or how we could get the most of shared code if we take a mTHP approach or the iomap approach for tmpfs, that should be considered.

Are there other things to consider? Does this require some dialog at LSFMM?

Quoted 6 lines · Luis Chamberlain

From my perspective the more shared code the better, and the more shared
paths the better. There is a chance to help test swap with large folios instead of splitting the folios for swap, and that would could be done first with tmpfs. I have not evaluated the difference in testing or how we could get the most of shared code if we take a mTHP approach or the iomap approach for tmpfs, that should be considered.

I don't have a clear picture yet of what might be best for ordinary shmem (IOW, not MAP_SHARED|MAP_PRIVATE), and I'm afraid there is no easy answer.

As long as we don't end up wasting memory, it's not obviously bad.

But some
things might be tricky (see my example about large folios stranding in shmem and never being able to be really reclaimed+reused for better purposes)

I'll note that mTHP really is just (supposed to be) a user interface to enable the various folio sizes (well, and to expose better per-size stats), not more.

From that point of view, it's just a filter. Enable all, and you get the same behavior as you likely would in the pagecache mode.

Quoted 2 lines

Are there other things to consider? Does this require some dialog at LSFMM?

As raised in my reply to Daniel, I'll be at LSF/MM and happy to discuss. I'm also not a SHMEM expert, so I'm hoping at some point we'd get feedback from Hugh.