Skip to contentSkip to conversation

fix MADV_COLLAPSE issue if THP settings are disabled

PATCH Superseded Baolin Wang 6 participants 19 messages Jun 23, 2025 – Jun 24, 2025

Subsystems (5)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Baolin Wang

author reviewer
#0 lore raw

v3 cover fix MADV_COLLAPSE issue if THP settings are disabled

When invoking thp_vma_allowable_orders(), if the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system. And the MADV_COLLAPSE is an example of such a case, that means it will not set TVA_ENFORCE_SYSFS when calling thp_vma_allowable_orders().

As we discussed in the previous thread [1], the MADV_COLLAPSE will ignore the system-wide anon/shmem THP sysfs settings, which means that even though we have disabled the anon/shmem THP configuration, MADV_COLLAPSE will still attempt to collapse into a anon/shmem THP. This violates the rule we have agreed upon: never means never.

For example, system administrators who disabled THP everywhere must indeed very much not want THP to be used for whatever reason - having individual programs being able to quietly override this is very surprising and likely to cause headaches for those who desire this not to happen on their systems.

This patch set will address the MADV_COLLAPSE issue.

Test
====
1. Tested the mm selftests and found no regressions.
2. With toggling different Anon mTHP settings, the allocation and madvise collapse for anonymous pages work well.
3. With toggling different shmem mTHP settings, the allocation and madvise collapse for shmem work well.
4. Tested the large order allocation for tmpfs, and works as expected.

Hi Dev, Nico and Zi,
I dropped your reviewed or tested tags for patch 1, since patch 1 was refactored according to Lorenzo's suggestions, please help review it again. Thanks.

[1] https://lore.kernel.org/all/[email protected]/

Changes from v2:
- Update the commit message and cover letter, per Lorenzo. Thanks.
- Simplify the logic in thp_vma_allowable_orders(), per Lorenzo and David. Thanks.

Changes from v1:
- Update the commit message, per Zi.
- Add Zi's reviewed tag. Thanks.
- Update the shmem logic.

Baolin Wang (2):
mm: huge_memory: disallow hugepages if the system-wide THP sysfs
settings are disabled
mm: shmem: disallow hugepages if the system-wide shmem THP sysfs
settings are disabled

 include/linux/huge_mm.h                 | 51 ++++++++++++++++++-------
 mm/shmem.c                              |  6 +--
 tools/testing/selftests/mm/khugepaged.c |  8 +---
 3 files changed, 43 insertions(+), 22 deletions(-)
Signature · 1 line
2.43.5
author reviewer
#1 lore raw

PATCH v3 1/2 mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER)) "

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

---
 include/linux/huge_mm.h                 | 51 ++++++++++++++++++-------
 tools/testing/selftests/mm/khugepaged.c |  6 +--
 2 files changed, 39 insertions(+), 18 deletions(-)
include/linux/huge_mm.h +38 -13
index 4d5bb67dc4ec..ab70ca4e704b 100644
@@ -267,6 +267,42 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
 					 unsigned long tva_flags,
 					 unsigned long orders);
 
+/* Strictly mask requested anonymous orders according to sysfs settings. */
+static inline unsigned long __thp_mask_anon_orders(unsigned long vm_flags,
+		unsigned long tva_flags, unsigned long orders)
+{
+	const unsigned long always = READ_ONCE(huge_anon_orders_always);
+	const unsigned long madvise = READ_ONCE(huge_anon_orders_madvise);
+	const unsigned long inherit = READ_ONCE(huge_anon_orders_inherit);
+	const unsigned long never = ~(always | madvise | inherit);
+	const bool inherit_never = !hugepage_global_enabled();
+
+	/* Disallow orders that are set to NEVER directly ... */
+	orders &= ~never;
+
+	/* ... or through inheritance (global == NEVER). */
+	if (inherit_never)
+		orders &= ~inherit;
+
+	/*
+	 * Otherwise, we only enforce sysfs settings if asked. In addition,
+	 * if the user sets a sysfs mode of madvise and if TVA_ENFORCE_SYSFS
+	 * is not set, we don't bother checking whether the VMA has VM_HUGEPAGE
+	 * set.
+	 */
+	if (!(tva_flags & TVA_ENFORCE_SYSFS))
+		return orders;
+
+	/* We already excluded never inherit above. */
+	if (vm_flags & VM_HUGEPAGE)
+		return orders & (always | madvise | inherit);
+
+	if (hugepage_global_always())
+		return orders & (always | inherit);
+
+	return orders & always;
+}
+
 /**
  * thp_vma_allowable_orders - determine hugepage orders that are allowed for vma
  * @vma:  the vm area to check
@@ -289,19 +325,8 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma,
 				       unsigned long orders)
 {
 	/* Optimization to check if required orders are enabled early. */
-	if ((tva_flags & TVA_ENFORCE_SYSFS) && vma_is_anonymous(vma)) {
-		unsigned long mask = READ_ONCE(huge_anon_orders_always);
-
-		if (vm_flags & VM_HUGEPAGE)
-			mask |= READ_ONCE(huge_anon_orders_madvise);
-		if (hugepage_global_always() ||
-		    ((vm_flags & VM_HUGEPAGE) && hugepage_global_enabled()))
-			mask |= READ_ONCE(huge_anon_orders_inherit);
-
-		orders &= mask;
-		if (!orders)
-			return 0;
-	}
+	if (vma_is_anonymous(vma))
+		orders = __thp_mask_anon_orders(vm_flags, tva_flags, orders);
 
 	return __thp_vma_allowable_orders(vma, vm_flags, tva_flags, orders);
 }
tools/testing/selftests/mm/khugepaged.c +1 -5
index 4341ce6b3b38..85bfff53dba6 100644
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
 
 	printf("%s...", msg);
 
-	/*
-	 * Prevent khugepaged interference and tests that MADV_COLLAPSE
-	 * ignores /sys/kernel/mm/transparent_hugepage/enabled
-	 */
-	settings.thp_enabled = THP_NEVER;
+	settings.thp_enabled = THP_ALWAYS;
 	settings.shmem_enabled = SHMEM_NEVER;
 	thp_push_settings(&settings);
 
Signature · 1 line
2.43.5
Reviewed-by on v3 1/2
#2 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 30 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER)) "

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

Quoted 8 lines

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

Thanks! Sounds good.

Quoted 2 lines

Appreciate it though I'm not so bothered about attribution :) but just to say, of course the 'never' stuff is David's idea (and a good one!) :)

Quoted 1 line

LGTM so:

Quoted 79 lines
---
 include/linux/huge_mm.h                 | 51 ++++++++++++++++++-------
 tools/testing/selftests/mm/khugepaged.c |  6 +--
 2 files changed, 39 insertions(+), 18 deletions(-)
include/linux/huge_mm.h +38 -13
index 4d5bb67dc4ec..ab70ca4e704b 100644
@@ -267,6 +267,42 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
 					 unsigned long tva_flags,
 					 unsigned long orders);
 
+/* Strictly mask requested anonymous orders according to sysfs settings. */
+static inline unsigned long __thp_mask_anon_orders(unsigned long vm_flags,
+		unsigned long tva_flags, unsigned long orders)
+{
+	const unsigned long always = READ_ONCE(huge_anon_orders_always);
+	const unsigned long madvise = READ_ONCE(huge_anon_orders_madvise);
+	const unsigned long inherit = READ_ONCE(huge_anon_orders_inherit);
+	const unsigned long never = ~(always | madvise | inherit);
+	const bool inherit_never = !hugepage_global_enabled();
+
+	/* Disallow orders that are set to NEVER directly ... */
+	orders &= ~never;
+
+	/* ... or through inheritance (global == NEVER). */
+	if (inherit_never)
+		orders &= ~inherit;
+
+	/*
+	 * Otherwise, we only enforce sysfs settings if asked. In addition,
+	 * if the user sets a sysfs mode of madvise and if TVA_ENFORCE_SYSFS
+	 * is not set, we don't bother checking whether the VMA has VM_HUGEPAGE
+	 * set.
+	 */
+	if (!(tva_flags & TVA_ENFORCE_SYSFS))
+		return orders;
+
+	/* We already excluded never inherit above. */
+	if (vm_flags & VM_HUGEPAGE)
+		return orders & (always | madvise | inherit);
+
+	if (hugepage_global_always())
+		return orders & (always | inherit);
+
+	return orders & always;
+}
+
 /**
  * thp_vma_allowable_orders - determine hugepage orders that are allowed for vma
  * @vma:  the vm area to check
@@ -289,19 +325,8 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma,
 				       unsigned long orders)
 {
 	/* Optimization to check if required orders are enabled early. */
-	if ((tva_flags & TVA_ENFORCE_SYSFS) && vma_is_anonymous(vma)) {
-		unsigned long mask = READ_ONCE(huge_anon_orders_always);
-
-		if (vm_flags & VM_HUGEPAGE)
-			mask |= READ_ONCE(huge_anon_orders_madvise);
-		if (hugepage_global_always() ||
-		    ((vm_flags & VM_HUGEPAGE) && hugepage_global_enabled()))
-			mask |= READ_ONCE(huge_anon_orders_inherit);
-
-		orders &= mask;
-		if (!orders)
-			return 0;
-	}
+	if (vma_is_anonymous(vma))
+		orders = __thp_mask_anon_orders(vm_flags, tva_flags, orders);
 
 	return __thp_vma_allowable_orders(vma, vm_flags, tva_flags, orders);
 }
tools/testing/selftests/mm/khugepaged.c +0 -0
index 4341ce6b3b38..85bfff53dba6 100644

tools/testing/selftests/mm/khugepaged.c static void __madvise_collapse(const char *msg, char *p, int nr_hpages, · line 504

Quoted 9 lines
tools/testing/selftests/mm/khugepaged.c +1 -5
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
 
 	printf("%s...", msg);
 
-	/*
-	 * Prevent khugepaged interference and tests that MADV_COLLAPSE
-	 * ignores /sys/kernel/mm/transparent_hugepage/enabled
-	 */
-	settings.thp_enabled = THP_NEVER;
+	settings.thp_enabled = THP_ALWAYS;

Good spot!

Quoted 6 lines
tools/testing/selftests/mm/khugepaged.c +0 -1
 	settings.shmem_enabled = SHMEM_NEVER;
 	thp_push_settings(&settings);
 
--

2.43.5

author reviewer
#3 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 40 lines · Lorenzo Stoakes
Quoted 30 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER)) "

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

Quoted 8 lines

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

Quoted 6 lines

Thanks! Sounds good.

Quoted 2 lines

Appreciate it though I'm not so bothered about attribution :) but just to say, of course the 'never' stuff is David's idea (and a good one!) :)

Yes, I should also add:

Quoted 5 lines
Quoted 1 line

LGTM so:

Thanks.

#4 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

In reply to Baolin Wang
Quoted 7 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

Quoted 5 lines

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Should we update the man page for madv_collapse ?
https://man7.org/linux/man-pages/man2/madvise.2.html

MADV_COLLAPSE is independent of any sysfs (see sysfs(5)) setting under /sys/kernel/mm/transparent_hugepage, both in terms of determining THP eligibility, and allocation semantics. See Linux kernel source file Documentation/admin-guide/mm/transhuge.rst for more information. MADV_COLLAPSE also ignores huge= tmpfs mount when operating on tmpfs files. Allocation for the new hugepage may enter direct reclaim and/or compaction, regardless of VMA flags (though VM_NOHUGEPAGE is still respected).

So this effectively changes the uABI, right?

Quoted 19 lines

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER)) "

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

Quoted 8 lines

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

It’s a bit odd that the old test case expects collapsing to succeed even when we’ve set it
to ‘never’.
Setting it to ‘always’ doesn’t seem to test anything as a counterpart.

I assume the goal is to test that setting it to ‘never’ prevents collapsing?

Quoted 4 lines

---

Thanks
Barry

author reviewer
#5 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 29 lines · Barry Song
Quoted 7 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

Quoted 5 lines

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Should we update the man page for madv_collapse ?
https://man7.org/linux/man-pages/man2/madvise.2.html

MADV_COLLAPSE is independent of any sysfs (see sysfs(5)) setting under /sys/kernel/mm/transparent_hugepage, both in terms of determining THP eligibility, and allocation semantics. See Linux kernel source file Documentation/admin-guide/mm/transhuge.rst for more information. MADV_COLLAPSE also ignores huge= tmpfs mount when operating on tmpfs files. Allocation for the new hugepage may enter direct reclaim and/or compaction, regardless of VMA flags (though VM_NOHUGEPAGE is still respected).

Quoted 1 line

So this effectively changes the uABI, right?

Good point. Will update the man page.

Quoted 28 lines
Quoted 19 lines

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER)) "

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

Quoted 8 lines

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

Quoted 6 lines

It’s a bit odd that the old test case expects collapsing to succeed even when we’ve set it
to ‘never’.
Setting it to ‘always’ doesn’t seem to test anything as a counterpart.

I assume the goal is to test that setting it to ‘never’ prevents collapsing?

The original logic will prevent khugepaged by setting THP_NEVER, allowing only madvise_collapse() to perform THP collapse. And this is the logic this patchset tries to fix, which is to also prevent madvise_collapse() from performing THP collapse when system-wide THP sysfs settings are disabled.

Therefore, it should be changed to THP_ALWAYS here to allow madvise_collapse() to perform THP collapse.

Of course, the current logic cannot completely disable khugepaged, but I haven't found a better way to modify it. As David suggested, changing to MADVISE mode would cause some test cases to fail because some tests previously set MADV_NOHUGEPAGE, and now there is no other way to clear the MADV_NOHUGEPAGE flag except for setting MADV_HUGEPAGE. As a result, khugepaged cannot be completely disabled either.

So I think we should introduce a new method to clear MADV_NOHUGEPAGE flag without setting MADV_HUGEPAGE in the future.

#6 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

In reply to Baolin Wang
Quoted 6 lines
tools/testing/selftests/mm/khugepaged.c +0 -0
index 4341ce6b3b38..85bfff53dba6 100644

tools/testing/selftests/mm/khugepaged.c static void __madvise_collapse(const char *msg, char *p, int nr_hpages, · line 504

Quoted 9 lines
tools/testing/selftests/mm/khugepaged.c +1 -5
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
 
  	printf("%s...", msg);
 
-	/*
-	 * Prevent khugepaged interference and tests that MADV_COLLAPSE
-	 * ignores /sys/kernel/mm/transparent_hugepage/enabled
-	 */
-	settings.thp_enabled = THP_NEVER;
+	settings.thp_enabled = THP_ALWAYS;

Would MADVISE mode also work here? If we don't set MADV_HUGEPAGE, then khugepaged should be excluded, correct?

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#7 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 17 lines · David Hildenbrand
Quoted 6 lines
tools/testing/selftests/mm/khugepaged.c +0 -0
index 4341ce6b3b38..85bfff53dba6 100644

tools/testing/selftests/mm/khugepaged.c static void __madvise_collapse(const char *msg, · line 503

Quoted 8 lines
tools/testing/selftests/mm/khugepaged.c +1 -5
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg, 
 char *p, int nr_hpages,
       printf("%s...", msg);
-    /*
-     * Prevent khugepaged interference and tests that MADV_COLLAPSE
-     * ignores /sys/kernel/mm/transparent_hugepage/enabled
-     */
-    settings.thp_enabled = THP_NEVER;
+    settings.thp_enabled = THP_ALWAYS;

Would MADVISE mode also work here? If we don't set MADV_HUGEPAGE, then khugepaged should be excluded, correct?

I tried this, but some test cases failed. As I replied to Barry, it's because some tests previously set MADV_NOHUGEPAGE, and now there is no way to clear the MADV_NOHUGEPAGE flag except by setting MADV_HUGEPAGE.

#8 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 23 lines · Baolin Wang
Quoted 11 lines · David Hildenbrand
Quoted 11 lines
tools/testing/ renamed from tools/testing/selftests/mm/khugepaged.c +0 -0

selftests/mm/khugepaged.c
index 4341ce6b3b38..85bfff53dba6 100644

tools/testing/selftests/mm/khugepaged.c +0 -2
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg,
 char *p, int nr_hpages,
       printf("%s...", msg);
-    /*
-     * Prevent khugepaged interference and tests that MADV_COLLAPSE
Quoted 8 lines

tools/testing/selftests/mm/khugepaged.c static void __madvise_collapse(const char *msg, char *p, int nr_hpages, · line 504

Quoted 4 lines
tools/testing/selftests/mm/khugepaged.c +1 -3
-     * ignores /sys/kernel/mm/transparent_hugepage/enabled
-     */
-    settings.thp_enabled = THP_NEVER;
+    settings.thp_enabled = THP_ALWAYS;

Would MADVISE mode also work here? If we don't set MADV_HUGEPAGE, then khugepaged should be excluded, correct?

Quoted 3 lines

I tried this, but some test cases failed. As I replied to Barry, it's because some tests previously set MADV_NOHUGEPAGE, and now there is no way to clear the MADV_NOHUGEPAGE flag except by setting MADV_HUGEPAGE.

Okay, can you add that detail to the patch description. I suspect we really want a way to undo what MADV_NOHUGEPAGE/MADV_NOHUGEPAGE did (if only naming wouldn't be complicated: MADV_DEFAULT_HUGEPAGE, hmmmm).

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#9 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 24 lines · David Hildenbrand
Quoted 23 lines · Baolin Wang
Quoted 19 lines · David Hildenbrand
Quoted 7 lines
tools/testing/ renamed from tools/testing/selftests/mm/khugepaged.c +0 -0

selftests/mm/khugepaged.c
index 4341ce6b3b38..85bfff53dba6 100644

tools/testing/selftests/mm/khugepaged.c static void __madvise_collapse(const char *msg, · line 503

Quoted 8 lines
tools/testing/selftests/mm/khugepaged.c +1 -5
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg,
 char *p, int nr_hpages,
        printf("%s...", msg);
-    /*
-     * Prevent khugepaged interference and tests that MADV_COLLAPSE
-     * ignores /sys/kernel/mm/transparent_hugepage/enabled
-     */
-    settings.thp_enabled = THP_NEVER;
+    settings.thp_enabled = THP_ALWAYS;

Would MADVISE mode also work here? If we don't set MADV_HUGEPAGE, then khugepaged should be excluded, correct?

Quoted 5 lines
Quoted 3 lines

I tried this, but some test cases failed. As I replied to Barry, it's because some tests previously set MADV_NOHUGEPAGE, and now there is no way to clear the MADV_NOHUGEPAGE flag except by setting MADV_HUGEPAGE.

Okay, can you add that detail to the patch description.

Sure. Will do.

Quoted 1 line

I suspect we really want a way to undo what MADV_NOHUGEPAGE/MADV_NOHUGEPAGE did (if only naming wouldn't be complicated: MADV_DEFAULT_HUGEPAGE, hmmmm).

reviewer Reviewed-by on v3 1/2
#10 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

In reply to Baolin Wang
Quoted 39 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER)) "

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

Quoted 7 lines
---
 include/linux/huge_mm.h                 | 51 ++++++++++++++++++-------
 tools/testing/selftests/mm/khugepaged.c |  6 +--
 2 files changed, 39 insertions(+), 18 deletions(-)

The code looks much cleaner. Thanks.

--
Best Regards,
Yan, Zi

reviewer
#11 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

In reply to Baolin Wang
Quoted 16 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

Quoted 9 lines

"
/*
* If we are here, we've succeeded in replacing all the native pages
* in the page cache with a single hugepage. If a mm were to fault-in
* this memory (mapped by a suitably aligned VMA), we'd get the hugepage
* and map it by a PMD, regardless of sysfs THP settings. As such, let's
* analogously elide sysfs THP settings here.
*/
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER))

So the behaviour now is: First check whether THP settings converge to never. Then, if enforce_sysfs is not set, return immediately. So in this khugepaged code will it be better to call __thp_vma_allowable_orders()? If the sysfs settings are changed to never before hitting collapse_pte_mapped_thp(), then right now we will return SCAN_VMA_CHECK from here, whereas, the comment says "regardless of sysfs THP settings", which should include "regardless of whether the sysfs settings say never".

Quoted 107 lines

"

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

To address this issue, the current strategy should be:

If no hugepage modes are enabled for the desired orders, nor can we enable them by inheriting from a 'global' enabled setting - then it must be the case that all desired orders either specify or inherit 'NEVER' - and we must abort.

Meanwhile, we should fix the khugepaged selftest for MADV_COLLAPSE by enabling THP.

---
  include/linux/huge_mm.h                 | 51 ++++++++++++++++++-------
  tools/testing/selftests/mm/khugepaged.c |  6 +--
  2 files changed, 39 insertions(+), 18 deletions(-)
include/linux/huge_mm.h +38 -13
index 4d5bb67dc4ec..ab70ca4e704b 100644
@@ -267,6 +267,42 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
  					 unsigned long tva_flags,
  					 unsigned long orders);
 
+/* Strictly mask requested anonymous orders according to sysfs settings. */
+static inline unsigned long __thp_mask_anon_orders(unsigned long vm_flags,
+		unsigned long tva_flags, unsigned long orders)
+{
+	const unsigned long always = READ_ONCE(huge_anon_orders_always);
+	const unsigned long madvise = READ_ONCE(huge_anon_orders_madvise);
+	const unsigned long inherit = READ_ONCE(huge_anon_orders_inherit);
+	const unsigned long never = ~(always | madvise | inherit);
+	const bool inherit_never = !hugepage_global_enabled();
+
+	/* Disallow orders that are set to NEVER directly ... */
+	orders &= ~never;
+
+	/* ... or through inheritance (global == NEVER). */
+	if (inherit_never)
+		orders &= ~inherit;
+
+	/*
+	 * Otherwise, we only enforce sysfs settings if asked. In addition,
+	 * if the user sets a sysfs mode of madvise and if TVA_ENFORCE_SYSFS
+	 * is not set, we don't bother checking whether the VMA has VM_HUGEPAGE
+	 * set.
+	 */
+	if (!(tva_flags & TVA_ENFORCE_SYSFS))
+		return orders;
+
+	/* We already excluded never inherit above. */
+	if (vm_flags & VM_HUGEPAGE)
+		return orders & (always | madvise | inherit);
+
+	if (hugepage_global_always())
+		return orders & (always | inherit);
+
+	return orders & always;
+}
+
  /**
   * thp_vma_allowable_orders - determine hugepage orders that are allowed for vma
   * @vma:  the vm area to check
@@ -289,19 +325,8 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma,
  				       unsigned long orders)
  {
  	/* Optimization to check if required orders are enabled early. */
-	if ((tva_flags & TVA_ENFORCE_SYSFS) && vma_is_anonymous(vma)) {
-		unsigned long mask = READ_ONCE(huge_anon_orders_always);
-
-		if (vm_flags & VM_HUGEPAGE)
-			mask |= READ_ONCE(huge_anon_orders_madvise);
-		if (hugepage_global_always() ||
-		    ((vm_flags & VM_HUGEPAGE) && hugepage_global_enabled()))
-			mask |= READ_ONCE(huge_anon_orders_inherit);
-
-		orders &= mask;
-		if (!orders)
-			return 0;
-	}
+	if (vma_is_anonymous(vma))
+		orders = __thp_mask_anon_orders(vm_flags, tva_flags, orders);
 
  	return __thp_vma_allowable_orders(vma, vm_flags, tva_flags, orders);
  }
tools/testing/selftests/mm/khugepaged.c +1 -5
index 4341ce6b3b38..85bfff53dba6 100644
@@ -501,11 +501,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
 
  	printf("%s...", msg);
 
-	/*
-	 * Prevent khugepaged interference and tests that MADV_COLLAPSE
-	 * ignores /sys/kernel/mm/transparent_hugepage/enabled
-	 */
-	settings.thp_enabled = THP_NEVER;
+	settings.thp_enabled = THP_ALWAYS;
  	settings.shmem_enabled = SHMEM_NEVER;
  	thp_push_settings(&settings);
 
author reviewer
#12 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 28 lines · Dev Jain
Quoted 16 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

Quoted 9 lines

"
/*
  * If we are here, we've succeeded in replacing all the native pages
  * in the page cache with a single hugepage. If a mm were to fault-in
  * this memory (mapped by a suitably aligned VMA), we'd get the hugepage
  * and map it by a PMD, regardless of sysfs THP settings. As such, let's
  * analogously elide sysfs THP settings here.
  */
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER))

Quoted 7 lines

So the behaviour now is: First check whether THP settings converge to never. Then, if enforce_sysfs is not set, return immediately. So in this khugepaged code will it be better to call __thp_vma_allowable_orders()? If the sysfs settings are changed to never before hitting collapse_pte_mapped_thp(), then right now we will return SCAN_VMA_CHECK from here, whereas, the comment says "regardless of sysfs THP settings", which should include "regardless of whether the sysfs settings say never".

Sounds reasonable to me. Thanks.

I will change thp_vma_allowable_order() to __thp_vma_allowable_orders() in the collapse_pte_mapped_thp() function to maintain consistency with the original logic.

Lorenzo and David, how do you think? Thanks.

author reviewer
#13 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 39 lines · Baolin Wang
Quoted 28 lines · Dev Jain
Quoted 16 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

Quoted 9 lines

"
/*
  * If we are here, we've succeeded in replacing all the native pages
  * in the page cache with a single hugepage. If a mm were to fault-in
  * this memory (mapped by a suitably aligned VMA), we'd get the hugepage
  * and map it by a PMD, regardless of sysfs THP settings. As such, let's
  * analogously elide sysfs THP settings here.
  */
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER))

Quoted 7 lines

So the behaviour now is: First check whether THP settings converge to never. Then, if enforce_sysfs is not set, return immediately. So in this khugepaged code will it be better to call __thp_vma_allowable_orders()? If the sysfs settings are changed to never before hitting collapse_pte_mapped_thp(), then right now we will return SCAN_VMA_CHECK from here, whereas, the comment says "regardless of sysfs THP settings", which should include "regardless of whether the sysfs settings say never".

Quoted 5 lines

Sounds reasonable to me. Thanks.

I will change thp_vma_allowable_order() to __thp_vma_allowable_orders() in the collapse_pte_mapped_thp() function to maintain consistency with the original logic.

Lorenzo and David, how do you think? Thanks.

After thinking more, since collapse_pte_mapped_thp() is only used for file/shmem collapse, changing to __thp_vma_allowable_orders() has no effect. So I prefer to leave it as is.

reviewer
#14 lore raw

Re: [PATCH v3 1/2] mm: huge_memory: disallow hugepages if the system-wide THP sysfs settings are disabled

Quoted 42 lines · Baolin Wang
Quoted 39 lines · Baolin Wang
Quoted 35 lines · Dev Jain
Quoted 16 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. Whilst it makes sense for the callers who do not specify this flag, it creates a odd and surprising situation where a sysadmin specifying 'never' for all THP sizes still observing THP pages being allocated and used on the system.

The motivating case for this is MADV_COLLAPSE. The MADV_COLLAPSE will ignore the system-wide Anon THP sysfs settings, which means that even though we have disabled the Anon THP configuration, MADV_COLLAPSE will still attempt to collapse into a Anon THP. This violates the rule we have agreed upon: never means never.

Currently, besides MADV_COLLAPSE not setting TVA_ENFORCE_SYSFS, there is only one other instance where TVA_ENFORCE_SYSFS is not set, which is in the collapse_pte_mapped_thp() function, but I believe this is reasonable from its comments:

Quoted 9 lines

"
/*
  * If we are here, we've succeeded in replacing all the native pages
  * in the page cache with a single hugepage. If a mm were to fault-in
  * this memory (mapped by a suitably aligned VMA), we'd get the hugepage
  * and map it by a PMD, regardless of sysfs THP settings. As such, let's
  * analogously elide sysfs THP settings here.
  */
if (!thp_vma_allowable_order(vma, vma->vm_flags, 0, PMD_ORDER))

So the behaviour now is: First check whether THP settings converge to never. Then, if enforce_sysfs is not set, return immediately. So in this khugepaged code will it be better to call __thp_vma_allowable_orders()? If the sysfs settings are changed to never before hitting collapse_pte_mapped_thp(), then right now we will return SCAN_VMA_CHECK from here, whereas, the comment says "regardless of sysfs THP settings", which should include "regardless of whether the sysfs settings say never".

Quoted 7 lines
Quoted 5 lines

Sounds reasonable to me. Thanks.

I will change thp_vma_allowable_order() to __thp_vma_allowable_orders() in the collapse_pte_mapped_thp() function to maintain consistency with the original logic.

Lorenzo and David, how do you think? Thanks.

After thinking more, since collapse_pte_mapped_thp() is only used for file/shmem collapse, changing to __thp_vma_allowable_orders() has no effect. So I prefer to leave it as is.

Oops my bad, thanks.

author reviewer
#15 lore raw

PATCH v3 2/2 mm: shmem: disallow hugepages if the system-wide shmem THP sysfs settings are disabled

In reply to Baolin Wang

When invoking thp_vma_allowable_orders(), if the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. And the MADV_COLLAPSE is an example of such a case.

The MADV_COLLAPSE will ignore the system-wide shmem THP sysfs settings, which means that even though we have disabled the shmem THP configuration, MADV_COLLAPSE will still attempt to collapse into a shmem THP. This violates the rule we have agreed upon: never means never.

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

To fix the MADV_COLLAPSE issue for shmem, then the current strategy should be:

For shmem, if none of always, madvise, within_size, and inherit have enabled PMD-sized THP, then MADV_COLLAPSE will be prohibited from collapsing PMD-sized THP.

For tmpfs, if the mount option is set with the 'huge=never' parameter, then MADV_COLLAPSE will be prohibited from collapsing PMD-sized THP.

Meanwhile, we should fix the khugepaged selftest for shmem MADV_COLLAPSE by enabling shmem THP.

---
 mm/shmem.c                              | 6 +++---
 tools/testing/selftests/mm/khugepaged.c | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)
mm/shmem.c +3 -3
index 2b19965d27df..e3f51fab2b7d 100644
@@ -637,7 +637,7 @@ static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index
 		return 0;
 	if (shmem_huge == SHMEM_HUGE_DENY)
 		return 0;
-	if (shmem_huge_force || shmem_huge == SHMEM_HUGE_FORCE)
+	if (shmem_huge == SHMEM_HUGE_FORCE)
 		return maybe_pmd_order;
 
 	/*
@@ -672,7 +672,7 @@ static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index
 
 		fallthrough;
 	case SHMEM_HUGE_ADVISE:
-		if (vm_flags & VM_HUGEPAGE)
+		if (shmem_huge_force || (vm_flags & VM_HUGEPAGE))
 			return maybe_pmd_order;
 		fallthrough;
 	default:
@@ -1806,7 +1806,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	/* Allow mTHP that will be fully within i_size. */
 	mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);
 
-	if (vm_flags & VM_HUGEPAGE)
+	if (shmem_huge_force || (vm_flags & VM_HUGEPAGE))
 		mask |= READ_ONCE(huge_shmem_orders_madvise);
 
 	if (global_orders > 0)
tools/testing/selftests/mm/khugepaged.c +1 -1
index 85bfff53dba6..9517ed99c382 100644
@@ -502,7 +502,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
 	printf("%s...", msg);
 
 	settings.thp_enabled = THP_ALWAYS;
-	settings.shmem_enabled = SHMEM_NEVER;
+	settings.shmem_enabled = SHMEM_ALWAYS;
 	thp_push_settings(&settings);
 
 	/* Clear VM_NOHUGEPAGE */
Signature · 1 line
2.43.5
Reviewed-by on v3 2/2
#16 lore raw

Re: [PATCH v3 2/2] mm: shmem: disallow hugepages if the system-wide shmem THP sysfs settings are disabled

Quoted 17 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), if the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. And the MADV_COLLAPSE is an example of such a case.

The MADV_COLLAPSE will ignore the system-wide shmem THP sysfs settings, which means that even though we have disabled the shmem THP configuration, MADV_COLLAPSE will still attempt to collapse into a shmem THP. This violates the rule we have agreed upon: never means never.

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

To fix the MADV_COLLAPSE issue for shmem, then the current strategy should be:

For shmem, if none of always, madvise, within_size, and inherit have enabled PMD-sized THP, then MADV_COLLAPSE will be prohibited from collapsing PMD-sized THP.

Quoted 8 lines

For tmpfs, if the mount option is set with the 'huge=never' parameter, then MADV_COLLAPSE will be prohibited from collapsing PMD-sized THP.

Meanwhile, we should fix the khugepaged selftest for shmem MADV_COLLAPSE by enabling shmem THP.

LGTM, so:

Quoted 52 lines
---
 mm/shmem.c                              | 6 +++---
 tools/testing/selftests/mm/khugepaged.c | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)
mm/shmem.c +3 -3
index 2b19965d27df..e3f51fab2b7d 100644
@@ -637,7 +637,7 @@ static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index
 		return 0;
 	if (shmem_huge == SHMEM_HUGE_DENY)
 		return 0;
-	if (shmem_huge_force || shmem_huge == SHMEM_HUGE_FORCE)
+	if (shmem_huge == SHMEM_HUGE_FORCE)
 		return maybe_pmd_order;
 
 	/*
@@ -672,7 +672,7 @@ static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index
 
 		fallthrough;
 	case SHMEM_HUGE_ADVISE:
-		if (vm_flags & VM_HUGEPAGE)
+		if (shmem_huge_force || (vm_flags & VM_HUGEPAGE))
 			return maybe_pmd_order;
 		fallthrough;
 	default:
@@ -1806,7 +1806,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	/* Allow mTHP that will be fully within i_size. */
 	mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);
 
-	if (vm_flags & VM_HUGEPAGE)
+	if (shmem_huge_force || (vm_flags & VM_HUGEPAGE))
 		mask |= READ_ONCE(huge_shmem_orders_madvise);
 
 	if (global_orders > 0)
tools/testing/selftests/mm/khugepaged.c +1 -2
index 85bfff53dba6..9517ed99c382 100644
@@ -502,7 +502,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
 	printf("%s...", msg);
 
 	settings.thp_enabled = THP_ALWAYS;
-	settings.shmem_enabled = SHMEM_NEVER;
+	settings.shmem_enabled = SHMEM_ALWAYS;
 	thp_push_settings(&settings);
 
 	/* Clear VM_NOHUGEPAGE */
--

2.43.5

#17 lore raw

Re: [PATCH v3 2/2] mm: shmem: disallow hugepages if the system-wide shmem THP sysfs settings are disabled

In reply to Baolin Wang
Quoted 9 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), if the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. And the MADV_COLLAPSE is an example of such a case.

The MADV_COLLAPSE will ignore the system-wide shmem THP sysfs settings, which means that even though we have disabled the shmem THP configuration, MADV_COLLAPSE will still attempt to collapse into a shmem THP. This violates the rule we have agreed upon: never means never.

Quoted 7 lines

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

To fix the MADV_COLLAPSE issue for shmem, then the current strategy should be:

For shmem, if none of always, madvise, within_size, and inherit have enabled PMD-sized THP, then MADV_COLLAPSE will be prohibited from collapsing PMD-sized THP.

I assume we could rephrase that to "For shmem, if "shmem_enabled" is set to either "none" or "deny", then MADV_COLLAPSE will be prohibited from collapsing."

Or am I missing a case?

[...]

Quoted 23 lines
mm/shmem.c +2 -2
@@ -672,7 +672,7 @@ static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index
 
  		fallthrough;
  	case SHMEM_HUGE_ADVISE:
-		if (vm_flags & VM_HUGEPAGE)
+		if (shmem_huge_force || (vm_flags & VM_HUGEPAGE))
  			return maybe_pmd_order;
  		fallthrough;
  	default:
@@ -1806,7 +1806,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
  	/* Allow mTHP that will be fully within i_size. */
  	mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);
 
-	if (vm_flags & VM_HUGEPAGE)
+	if (shmem_huge_force || (vm_flags & VM_HUGEPAGE))
  		mask |= READ_ONCE(huge_shmem_orders_madvise);
 
  	if (global_orders > 0)
tools/testing/selftests/mm/khugepaged.c +0 -0
index 85bfff53dba6..9517ed99c382 100644

tools/testing/selftests/mm/khugepaged.c static void __madvise_collapse(const char *msg, char *p, int nr_hpages, · line 506

Quoted 6 lines
tools/testing/selftests/mm/khugepaged.c +1 -1
@@ -502,7 +502,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,
  	printf("%s...", msg);
 
  	settings.thp_enabled = THP_ALWAYS;
-	settings.shmem_enabled = SHMEM_NEVER;
+	settings.shmem_enabled = SHMEM_ALWAYS;
  	thp_push_settings(&settings);

Same question as for the other case.

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#18 lore raw

Re: [PATCH v3 2/2] mm: shmem: disallow hugepages if the system-wide shmem THP sysfs settings are disabled

Quoted 13 lines · David Hildenbrand
Quoted 12 lines · Baolin Wang

When invoking thp_vma_allowable_orders(), if the TVA_ENFORCE_SYSFS flag is not specified, we will ignore the THP sysfs settings. And the MADV_COLLAPSE is an example of such a case.

The MADV_COLLAPSE will ignore the system-wide shmem THP sysfs settings, which means that even though we have disabled the shmem THP configuration, MADV_COLLAPSE will still attempt to collapse into a shmem THP. This violates the rule we have agreed upon: never means never.

Another rule for madvise, referring to David's suggestion: “allowing for collapsing in a VM without VM_HUGEPAGE in the "madvise" mode would be fine".

Quoted 6 lines
Quoted 4 lines

To fix the MADV_COLLAPSE issue for shmem, then the current strategy should be:

For shmem, if none of always, madvise, within_size, and inherit have enabled PMD-sized THP, then MADV_COLLAPSE will be prohibited from collapsing PMD-sized THP.

I assume we could rephrase that to "For shmem, if "shmem_enabled" is set to either "none" or "deny", then MADV_COLLAPSE will be prohibited from collapsing."

Yes. Setting 'deny' will also prevent MADV_COLLAPSE(), and there is no 'none' option for 'shmem_enabled'. Will update the commit mesasge to make it clear.