Skip to contentSkip to conversation

mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"

PATCH Applied Zach O'Keefe 5 participants 10 messages Sep 25, 2023 – Oct 9, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

Fixes 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")
Cc Yang Shi <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc David Hildenbrand <[email protected]>

---
I've updated the changelog to reflect discussions in [1] -- leaving ack to David / Matthew on whether to take the patch.

Changed from v3[1]:
- [akpm / David H. / M. Wilcox] Updated log to capture email discussion
Changed from v2[2]:
- Fixed false negative in smaps check when !dax && ->huge_fault
Changed from v1[3]:
- [Saurabhi] Allow ->huge_fault handler to handle fault, if it exists

[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/CAAa6QmQw+F=o6htOn=6ADD6mwvMO=Ow_67f3ifBv3GpXx9Xg_g@mail.gmail.com/

---
 mm/huge_memory.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
mm/huge_memory.c +13 -7
index 0f93a73115f73..797fe617e51ab 100644
@@ -100,11 +100,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags,
 		return in_pf;
 
 	/*
-	 * Special VMA and hugetlb VMA.
+	 * khugepaged special VMA and hugetlb VMA.
 	 * Must be checked after dax since some dax mappings may have
 	 * VM_MIXEDMAP set.
 	 */
-	if (vm_flags & VM_NO_KHUGEPAGED)
+	if (!in_pf && !smaps && (vm_flags & VM_NO_KHUGEPAGED))
 		return false;
 
 	/*
@@ -132,12 +132,18 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags,
 					   !hugepage_flags_always())))
 		return false;
 
-	/* Only regular file is valid */
-	if (!in_pf && file_thp_enabled(vma))
-		return true;
-
-	if (!vma_is_anonymous(vma))
+	if (!vma_is_anonymous(vma)) {
+		/*
+		 * Trust that ->huge_fault() handlers know what they are doing
+		 * in fault path.
+		 */
+		if (((in_pf || smaps)) && vma->vm_ops->huge_fault)
+			return true;
+		/* Only regular file is valid in collapse path */
+		if (((!in_pf || smaps)) && file_thp_enabled(vma))
+			return true;
 		return false;
+	}
 
 	if (vma_is_temporary_stack(vma))
 		return false;
Signature · 1 line
2.42.0.515.g380fc7ccd1-goog
#1 lore raw
Quoted 25 lines · Zach O'Keefe

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Quoted 4 lines

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

Looks good to me. Reviewed-by: Yang Shi <[email protected]>

Quoted 71 lines
Fixes 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")
Cc Yang Shi <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc David Hildenbrand <[email protected]>

---
I've updated the changelog to reflect discussions in [1] -- leaving ack to David / Matthew on whether to take the patch.

Changed from v3[1]:
- [akpm / David H. / M. Wilcox] Updated log to capture email discussion
Changed from v2[2]:
- Fixed false negative in smaps check when !dax && ->huge_fault
Changed from v1[3]:
- [Saurabhi] Allow ->huge_fault handler to handle fault, if it exists

[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/CAAa6QmQw+F=o6htOn=6ADD6mwvMO=Ow_67f3ifBv3GpXx9Xg_g@mail.gmail.com/

---
 mm/huge_memory.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
mm/huge_memory.c +13 -8
index 0f93a73115f73..797fe617e51ab 100644
@@ -100,11 +100,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags,
                return in_pf;
 
        /*
-        * Special VMA and hugetlb VMA.
+        * khugepaged special VMA and hugetlb VMA.
         * Must be checked after dax since some dax mappings may have
         * VM_MIXEDMAP set.
         */
-       if (vm_flags & VM_NO_KHUGEPAGED)
+       if (!in_pf && !smaps && (vm_flags & VM_NO_KHUGEPAGED))
                return false;
 
        /*
@@ -132,12 +132,18 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags,
                                           !hugepage_flags_always())))
                return false;
 
-       /* Only regular file is valid */
-       if (!in_pf && file_thp_enabled(vma))
-               return true;
-
-       if (!vma_is_anonymous(vma))
+       if (!vma_is_anonymous(vma)) {
+               /*
+                * Trust that ->huge_fault() handlers know what they are doing
+                * in fault path.
+                */
+               if (((in_pf || smaps)) && vma->vm_ops->huge_fault)
+                       return true;
+               /* Only regular file is valid in collapse path */
+               if (((!in_pf || smaps)) && file_thp_enabled(vma))
+                       return true;
                return false;
+       }
 
        if (vma_is_temporary_stack(vma))
                return false;
--

2.42.0.515.g380fc7ccd1-goog

maintainer
#2 lore raw
In reply to Zach O'Keefe
Quoted 24 lines · On Mon, 25 Sep 2023 13:01:10 -0700 "Zach O'Keefe" <[email protected]> wrote:

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Quoted 4 lines

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

It's unclear what are the userspace visible impacts of this change. Which makes it hard for others to determine whether -stable kernels should be patched.

Quoted 2 lines
Fixes 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

It's nice to include a Closes: link after a Reported-by:. Then readers are better able to answer the above question.

Quoted 4 lines
Cc Yang Shi <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc David Hildenbrand <[email protected]>
#3 lore raw
Quoted 27 lines · Andrew Morton
Quoted 24 lines · On Mon, 25 Sep 2023 13:01:10 -0700 "Zach O'Keefe" <[email protected]> wrote:

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Quoted 8 lines
Quoted 4 lines

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

It's unclear what are the userspace visible impacts of this change. Which makes it hard for others to determine whether -stable kernels should be patched.

IMO, I don't think this change is suitable for -stable; the only users that would have been affected are those that maintain out-of-tree drivers / code that hooked into ->huge_fault() or used VM_MIXEDMAP + THP. No users of the in-tree kernel would have been affected. It's still a good "fix" to make going forward (and certainly happy to be able to help Saurabh out).

+ greg k-h for vis / to confirm.

Quoted 5 lines
Quoted 2 lines
Fixes 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

It's nice to include a Closes: link after a Reported-by:. Then readers are better able to answer the above question.

Ah, apologies, Andrew; I didn't know such a tag existed -- I'll be sure to include it in the future.

Quoted 5 lines
Quoted 4 lines
Cc Yang Shi <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc David Hildenbrand <[email protected]>
maintainer
#4 lore raw
In reply to Zach O'Keefe
Quoted 32 lines · On Mon, 25 Sep 2023 13:01:10 -0700 "Zach O'Keefe" <[email protected]> wrote:

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

...

mm/huge_memory.c bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, · line 101

Quoted 2 lines
mm/huge_memory.c +0 -0
@@ -100,11 +100,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags,
 		return in_pf;
 

Ryan's "mm: thp: introduce anon_orders and anon_always_mask sysfs files" changes hugepage_vma_check() to return an unsigned int, so this patch will need some rework to fit in after that.

However Ryan's overall series "variable-order, large folios for anonymous memory" is in early days and might not make it.

And as I don't know what is the urgency of this patch ("mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"), I'm unable to decide which patch needs to come first (thus requiring rework of the other patch).

Please discuss!

#5 lore raw
Quoted 41 lines · Andrew Morton
Quoted 32 lines · On Mon, 25 Sep 2023 13:01:10 -0700 "Zach O'Keefe" <[email protected]> wrote:

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

...

mm/huge_memory.c bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, · line 101

Quoted 2 lines
mm/huge_memory.c +0 -0
@@ -100,11 +100,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags,
  		return in_pf;
 

Ryan's "mm: thp: introduce anon_orders and anon_always_mask sysfs files" changes hugepage_vma_check() to return an unsigned int, so this patch will need some rework to fit in after that.

Quoted 8 lines

However Ryan's overall series "variable-order, large folios for anonymous memory" is in early days and might not make it.

And as I don't know what is the urgency of this patch ("mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"), I'm unable to decide which patch needs to come first (thus requiring rework of the other patch).

Please discuss!

IMHO clearly this one.

Signature · 3 lines
Cheers,

David / dhildenb
maintainer
#6 lore raw
Quoted 7 lines · On Fri, 6 Oct 2023 20:52:30 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 5 lines

And as I don't know what is the urgency of this patch ("mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"), I'm unable to decide which patch needs to come first (thus requiring rework of the other patch).

Please discuss!

IMHO clearly this one.

OK. I'll drop the "variable-order, large folios for anonymous memory" v6 series for now.

reviewer
#7 lore raw
Quoted 6 lines · Andrew Morton
Quoted 4 lines · On Fri, 6 Oct 2023 20:52:30 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 4 lines

And as I don't know what is the urgency of this patch ("mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"), I'm unable to decide which patch needs to come first (thus requiring rework of the other patch).

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Please discuss!

IMHO clearly this one.

OK. I'll drop the "variable-order, large folios for anonymous memory" v6 series for now.

Yep, agreed!

#8 lore raw
Quoted 8 lines · Ryan Roberts
Quoted 6 lines · Andrew Morton
Quoted 4 lines · On Fri, 6 Oct 2023 20:52:30 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 4 lines

And as I don't know what is the urgency of this patch ("mm/thp: fix "mm: thp: kill __transhuge_page_enabled()"), I'm unable to decide which patch needs to come first (thus requiring rework of the other patch).

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Please discuss!

IMHO clearly this one.

OK. I'll drop the "variable-order, large folios for anonymous memory" v6 series for now.

Yep, agreed!

Thank all and sorry for the late response ; have been buried as of late.

Best,
Zach

Acked-by on v4 1/1
#9 lore raw
In reply to Zach O'Keefe
Quoted 35 lines · Zach O'Keefe

The 6.0 commits:

commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")

merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks.

During the process, the semantics of the fault path check changed in two ways:

1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path).
2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines.

During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general).

Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits.

Fixes 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()")
Cc Yang Shi <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc David Hildenbrand <[email protected]>
Quoted 3 lines

---
I've updated the changelog to reflect discussions in [1] -- leaving ack to David / Matthew on whether to take the patch.

Works for me.

Signature · 3 lines
Cheers,

David / dhildenb