Skip to contentSkip to conversation

mm: make page_mapped_in_vma() hugetlb walk aware

PATCH Superseded Jane Chu 2 participants 4 messages Jan 21, 2025 – Feb 24, 2025

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jane Chu

author
#0 lore raw

PATCH 1/1 mm: make page_mapped_in_vma() hugetlb walk aware

When a process consumes a UE in a page, the memory failure handler attempts to collect information for a potential SIGBUS. If the page is an anonymous page, page_mapped_in_vma(page, vma) is invoked in order to
1. retrieve the vaddr from the process' address space,
2. verify that the vaddr is indeed mapped to the poisoned page,
where 'page' is the precise small page with UE.

It's been observed that when injecting poison to a non-head subpage of an anonymous hugetlb page, no SIGBUS show up; while injecting to the head page produces a SIGBUS. The casue is that, though hugetlb_walk() returns a valid pmd entry (on x86), but check_pte() detects mismatch between the head page per the pmd and the input subpage. Thus the vaddr is considered not mapped to the subpage and the process is not collected for SIGBUS purpose. This is the calling stack
collect_procs_anon
page_mapped_in_vma
page_vma_mapped_walk
hugetlb_walk
huge_pte_lock
check_pte

It seems that the most obvious place to fix the issue is by making page_mapped_in_vma() hugetlb walk aware. The precise subpage in the input is useful in providing PAGE_SIZE granularity vaddr.

---
 mm/page_vma_mapped.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
mm/page_vma_mapped.c +12 -1
index 81839a9e74f1..bc036060cc68 100644
@@ -342,15 +342,26 @@ unsigned long page_mapped_in_vma(const struct page *page,
 {
 	const struct folio *folio = page_folio(page);
 	struct page_vma_mapped_walk pvmw = {
-		.pfn = page_to_pfn(page),
 		.nr_pages = 1,
 		.vma = vma,
 		.flags = PVMW_SYNC,
 	};
 
+	/* fine granularity address is always preferred */
 	pvmw.address = vma_address(vma, page_pgoff(folio, page), 1);
 	if (pvmw.address == -EFAULT)
 		goto out;
+
+	/*
+	 * Hugetlb doesn't support partial page-mapping, hugetlb_walk()
+	 * simply assumes hugetlb pte, hence feed the headpage pfn for
+	 * the walk and pte check.
+	 */
+	if (folio_test_hugetlb(folio))
+		pvmw.pfn = folio_pfn(folio);
+	else
+		pvmw.pfn = page_to_pfn(page);
+
 	if (!page_vma_mapped_walk(&pvmw))
 		return -EFAULT;
 	page_vma_mapped_walk_done(&pvmw);
Signature · 1 line
2.39.3
#1 lore raw
Quoted 22 lines · Jane Chu

When a process consumes a UE in a page, the memory failure handler attempts to collect information for a potential SIGBUS. If the page is an anonymous page, page_mapped_in_vma(page, vma) is invoked in order to
1. retrieve the vaddr from the process' address space,
2. verify that the vaddr is indeed mapped to the poisoned page,
where 'page' is the precise small page with UE.

It's been observed that when injecting poison to a non-head subpage of an anonymous hugetlb page, no SIGBUS show up; while injecting to the head page produces a SIGBUS. The casue is that, though hugetlb_walk() returns a valid pmd entry (on x86), but check_pte() detects mismatch between the head page per the pmd and the input subpage. Thus the vaddr is considered not mapped to the subpage and the process is not collected for SIGBUS purpose. This is the calling stack
collect_procs_anon
page_mapped_in_vma
page_vma_mapped_walk
hugetlb_walk
huge_pte_lock
check_pte

Quoted 3 lines

It seems that the most obvious place to fix the issue is by making page_mapped_in_vma() hugetlb walk aware. The precise subpage in the input is useful in providing PAGE_SIZE granularity vaddr.

I don't like this solution because it adds yet another special case for hugetlb. If we don't split a PMD-mapped THP, we'd have the same problem, right?

check_pte() would succeed if we set pvmw->pfn to folio_pfn() and pvmw->nr_pages to folio_nr_pages(), right? I just don't know what else might be affected by that.

I like one of these two options:

@@ -206,6 +206,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)

                pvmw->pte = hugetlb_walk(vma, pvmw->address, size);
                if (!pvmw->pte)
                        return false;

+ pvmw->pte += pvmw->address & (size - PAGE_SIZE);

                pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte);
                if (!check_pte(pvmw))

(that needs a bit of tidying up; you can't just do that, but I think you get the basic idea -- correct the pte to point to the precise page instead of the hugetlb pfn)

The option I really prefer is much more work but matches our preferred direction of getting rid of hugetlb specific code. Something like this:

@@ -192,27 +192,6 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)

        if (pvmw->pmd && !pvmw->pte)
                return not_found(pvmw);

- if (unlikely(is_vm_hugetlb_page(vma))) {

-               struct hstate *hstate = hstate_vma(vma);
-               unsigned long size = huge_page_size(hstate);
-               /* The only possible mapping was handled on last iteration */

[...]
- pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte);
- if (!check_pte(pvmw))
- return not_found(pvmw);
- return true;
- }
-

        end = vma_address_end(pvmw);
        if (pvmw->pte)
                goto next_pte;
@@ -229,7 +208,19 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw                        continue;
                }
                pud = pud_offset(p4d, pvmw->address);
-               if (!pud_present(*pud)) {
+               pude = *pud;
+               if (pud_trans_huge(pude) ||
+                   (pud_present(pude) && pud_devmap(pude))) {
+                       pvmw->ptl = pud_lock(mm, pvmw->pud);
+                       ...
+                       if (likely(pud_trans_huge(pude) || pud_devmap(pude))) {

+ if (pvmw->flags & PVMW_MIGRATION)
+ return not_found(pvmw);
+ if (!check_pud(pud_pfn(pude), pvmw))
+ return not_found(pvmw);
+ return true;
+ }

+               } else if (!pud_present(pude)) {
                        step_forward(pvmw, PUD_SIZE);
                        continue;
                }

ie get rid of all the hugetlb-specific code, and add support for the PUD level to the common code. You'd also need to write check_pud().

I'll understand if you don't want to do all the extra work. And thanks for tracking down this bug.

#2 lore raw

Thanks for the quick comment!

Quoted 45 lines · Matthew Wilcox
Quoted 22 lines · Jane Chu

When a process consumes a UE in a page, the memory failure handler attempts to collect information for a potential SIGBUS. If the page is an anonymous page, page_mapped_in_vma(page, vma) is invoked in order to
1. retrieve the vaddr from the process' address space,
2. verify that the vaddr is indeed mapped to the poisoned page,
where 'page' is the precise small page with UE.

It's been observed that when injecting poison to a non-head subpage of an anonymous hugetlb page, no SIGBUS show up; while injecting to the head page produces a SIGBUS. The casue is that, though hugetlb_walk() returns a valid pmd entry (on x86), but check_pte() detects mismatch between the head page per the pmd and the input subpage. Thus the vaddr is considered not mapped to the subpage and the process is not collected for SIGBUS purpose. This is the calling stack
collect_procs_anon
page_mapped_in_vma
page_vma_mapped_walk
hugetlb_walk
huge_pte_lock
check_pte

Quoted 3 lines

It seems that the most obvious place to fix the issue is by making page_mapped_in_vma() hugetlb walk aware. The precise subpage in the input is useful in providing PAGE_SIZE granularity vaddr.

I don't like this solution because it adds yet another special case for hugetlb. If we don't split a PMD-mapped THP, we'd have the same problem, right?

check_pte() would succeed if we set pvmw->pfn to folio_pfn() and pvmw->nr_pages to folio_nr_pages(), right? I just don't know what else might be affected by that.

I like one of these two options:

(quoted diff) +1 -0
@@ -206,6 +206,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
                 pvmw->pte = hugetlb_walk(vma, pvmw->address, size);
                 if (!pvmw->pte)
                         return false;
+               pvmw->pte += pvmw->address & (size - PAGE_SIZE);
 
                 pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte);
                 if (!check_pte(pvmw))
Quoted 3 lines

(that needs a bit of tidying up; you can't just do that, but I think you get the basic idea -- correct the pte to point to the precise page instead of the hugetlb pfn)

That'll work, let me think about how to tidy it up.  More below.

Quoted 44 lines

The option I really prefer is much more work but matches our preferred direction of getting rid of hugetlb specific code. Something like this:

(quoted diff) +13 -11
@@ -192,27 +192,6 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
         if (pvmw->pmd && !pvmw->pte)
                 return not_found(pvmw);
 
-       if (unlikely(is_vm_hugetlb_page(vma))) {
-               struct hstate *hstate = hstate_vma(vma);
-               unsigned long size = huge_page_size(hstate);
-               /* The only possible mapping was handled on last iteration */
[...]
-               pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte);
-               if (!check_pte(pvmw))
-                       return not_found(pvmw);
-               return true;
-       }
-
         end = vma_address_end(pvmw);
         if (pvmw->pte)
                 goto next_pte;
@@ -229,7 +208,19 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw                        continue;
                 }
                 pud = pud_offset(p4d, pvmw->address);
-               if (!pud_present(*pud)) {
+               pude = *pud;
+               if (pud_trans_huge(pude) ||
+                   (pud_present(pude) && pud_devmap(pude))) {
+                       pvmw->ptl = pud_lock(mm, pvmw->pud);
+                       ...
+                       if (likely(pud_trans_huge(pude) || pud_devmap(pude))) {
+                               if (pvmw->flags & PVMW_MIGRATION)
+                                       return not_found(pvmw);
+                               if (!check_pud(pud_pfn(pude), pvmw))
+                                       return not_found(pvmw);
+                               return true;
+                       }
+               } else if (!pud_present(pude)) {
                         step_forward(pvmw, PUD_SIZE);
                         continue;
                 }
 
Quoted 2 lines

ie get rid of all the hugetlb-specific code, and add support for the PUD level to the common code. You'd also need to write check_pud().

Good idea!  I'd like to give this more generic approach a try as well.

Quoted 3 lines

I'll understand if you don't want to do all the extra work. And thanks for tracking down this bug.

Thanks a lot!

-jane

#3 lore raw

An update below. Also, add Peter.

Quoted 46 lines · On 1/20/2025 9:20 PM, [email protected] wrote:

Thanks for the quick comment!

Quoted 43 lines · Matthew Wilcox
Quoted 22 lines · Jane Chu

When a process consumes a UE in a page, the memory failure handler attempts to collect information for a potential SIGBUS. If the page is an anonymous page, page_mapped_in_vma(page, vma) is invoked in order to
   1. retrieve the vaddr from the process' address space,
   2. verify that the vaddr is indeed mapped to the poisoned page, where 'page' is the precise small page with UE.

It's been observed that when injecting poison to a non-head subpage of an anonymous hugetlb page, no SIGBUS show up; while injecting to the head page produces a SIGBUS. The casue is that, though hugetlb_walk() returns a valid pmd entry (on x86), but check_pte() detects mismatch between the head page per the pmd and the input subpage. Thus the vaddr is considered not mapped to the subpage and the process is not collected for SIGBUS purpose.  This is the calling stack collect_procs_anon
         page_mapped_in_vma
           page_vma_mapped_walk
             hugetlb_walk
               huge_pte_lock
                 check_pte

Quoted 3 lines

It seems that the most obvious place to fix the issue is by making page_mapped_in_vma() hugetlb walk aware. The precise subpage in the input is useful in providing PAGE_SIZE granularity vaddr.

I don't like this solution because it adds yet another special case for hugetlb.  If we don't split a PMD-mapped THP, we'd have the same problem, right?

check_pte() would succeed if we set pvmw->pfn to folio_pfn() and pvmw->nr_pages to folio_nr_pages(), right?  I just don't know what else might be affected by that.

I like one of these two options:

(quoted diff) +1 -0
@@ -206,6 +206,7 @@ bool page_vma_mapped_walk(struct 
 page_vma_mapped_walk *pvmw)
                  pvmw->pte = hugetlb_walk(vma, pvmw->address, size);
                  if (!pvmw->pte)
                          return false;
+               pvmw->pte += pvmw->address & (size - PAGE_SIZE);
 
Quoted 7 lines
Quoted 6 lines
(quoted diff) +0 -0
                  pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte);

                 if (!check_pte(pvmw))

(that needs a bit of tidying up; you can't just do that, but I think you get the basic idea -- correct the pte to point to the precise page instead of the hugetlb pfn)

That'll work, let me think about how to tidy it up.  More below.

It appears that check_pte() is supposed to be able to check range overlap for leaf pte among other things.

But the currently implementation doesn't do that.  Fixing this takes care of the subject issue.

More below.

Quoted 48 lines
Quoted 48 lines

The option I really prefer is much more work but matches our preferred direction of getting rid of hugetlb specific code.  Something like this:

(quoted diff) +0 -4
@@ -192,27 +192,6 @@ bool page_vma_mapped_walk(struct 
 page_vma_mapped_walk *pvmw)
          if (pvmw->pmd && !pvmw->pte)
                  return not_found(pvmw);
 
-       if (unlikely(is_vm_hugetlb_page(vma))) {
-               struct hstate *hstate = hstate_vma(vma);
-               unsigned long size = huge_page_size(hstate);
-               /* The only possible mapping was handled on last 

iteration */
[...]
-               pvmw->ptl = huge_pte_lock(hstate, mm, pvmw->pte); -               if (!check_pte(pvmw))
-                       return not_found(pvmw);
-               return true;
-       }
-
         end = vma_address_end(pvmw);
         if (pvmw->pte)
                 goto next_pte;

(quoted diff) +13 -1
@@ -229,7 +208,19 @@ bool page_vma_mapped_walk(struct 
 page_vma_mapped_walk *pvmw                        continue;
                  }
                  pud = pud_offset(p4d, pvmw->address);
-               if (!pud_present(*pud)) {
+               pude = *pud;
+               if (pud_trans_huge(pude) ||
+                   (pud_present(pude) && pud_devmap(pude))) {
+                       pvmw->ptl = pud_lock(mm, pvmw->pud);
+                       ...
+                       if (likely(pud_trans_huge(pude) || 
 pud_devmap(pude))) {
+                               if (pvmw->flags & PVMW_MIGRATION)
+                                       return not_found(pvmw);
+                               if (!check_pud(pud_pfn(pude), pvmw))
+                                       return not_found(pvmw);
+                               return true;
+                       }
+               } else if (!pud_present(pude)) {
                          step_forward(pvmw, PUD_SIZE);
                          continue;

                 }

Quoted 3 lines
Quoted 2 lines

ie get rid of all the hugetlb-specific code, and add support for the PUD level to the common code.  You'd also need to write check_pud().

Good idea!  I'd like to give this more generic approach a try as well.

I ran a simple experiment on page_vma_mapped_walk() checking on a PMD level hugetlb page but with the hugetlb{} logic commented out.

The experiment includes both test/move_pages and the memory poison tests, and the latter hit a NULL pointer deref. in try_to_unmap_one()  that I am looking into.

Besides, there are broader things to consider, such as pxd_trans_huge() are defined under CONFIG_TRANSPARENT_HUGEPAGE that doesn't apply to hugetlb.  Mabybe try pxd_leaf()? but they're not the same thing, does the  difference matter?

Hugetlb doesn't use pxd_lock(), it has its own locking though after peeling off the wrappers appear to be  the similar thing, but not quite the same thing.

There are also the hugetlb unique PMD sharing and implied locking requirement.

Maybe there are more as I dig further, it's becoming more like a project on its own which I will update on a separate thread in future.

Thanks,

-jane

Quoted 9 lines
Quoted 3 lines

I'll understand if you don't want to do all the extra work.  And thanks for tracking down this bug.

Thanks a lot!

-jane