mm/hugetlb: fix copy_hugetlb_page_range() to use ->pt_share_count
Subsystems (2)
PATCH v2 1/1 mm/hugetlb: fix copy_hugetlb_page_range() to use ->pt_share_count
commit 59d9094df3d79 ("mm: hugetlb: independent PMD page table shared count") introduced ->pt_share_count dedicated to hugetlb PMD share count tracking, but omitted fixing copy_hugetlb_page_range(), leaving the function relying on page_count() for tracking that no longer works.
When lazy page table copy for hugetlb is disabled, that is, revert commit bcd51a3c679d ("hugetlb: lazy page table copies in fork()") fork()'ing with hugetlb PMD sharing quickly lockup -
[ 239.446559] watchdog: BUG: soft lockup - CPU#75 stuck for 27s! [ 239.446611] RIP: 0010:native_queued_spin_lock_slowpath+0x7e/0x2e0 [ 239.446631] Call Trace: [ 239.446633] <TASK> [ 239.446636] _raw_spin_lock+0x3f/0x60 [ 239.446639] copy_hugetlb_page_range+0x258/0xb50 [ 239.446645] copy_page_range+0x22b/0x2c0 [ 239.446651] dup_mmap+0x3e2/0x770 [ 239.446654] dup_mm.constprop.0+0x5e/0x230 [ 239.446657] copy_process+0xd17/0x1760 [ 239.446660] kernel_clone+0xc0/0x3e0 [ 239.446661] __do_sys_clone+0x65/0xa0 [ 239.446664] do_syscall_64+0x82/0x930 [ 239.446668] ? count_memcg_events+0xd2/0x190 [ 239.446671] ? syscall_trace_enter+0x14e/0x1f0 [ 239.446676] ? syscall_exit_work+0x118/0x150 [ 239.446677] ? arch_exit_to_user_mode_prepare.constprop.0+0x9/0xb0 [ 239.446681] ? clear_bhb_loop+0x30/0x80 [ 239.446684] ? clear_bhb_loop+0x30/0x80 [ 239.446686] entry_SYSCALL_64_after_hwframe+0x76/0x7e
There are two options to resolve the potential latent issue:
1. warn against PMD sharing in copy_hugetlb_page_range(),
2. fix it.
This patch opts for the second option.
--- mm/hugetlb.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-)
mm/hugetlb.c +4 -9
@@ -5594,18 +5594,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,break ; } -/* -* If the pagetables are shared don't copy or take references. -* -* dst_pte == src_pte is the common case of src/dest sharing. -* However, src could have 'unshared' and dst shares with -* another vma. So page_count of ptep page is checked instead -* to reliably determine whether pte is shared. -*/ -if (page_count (virt_to_page (dst_pte)) >1 ) { +#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING +/* If the pagetables are shared don't copy or take references. */ +if (ptdesc_pmd_pts_count (virt_to_ptdesc (dst_pte)) >0 ) { addr |= last_addr_mask;continue ; } +#endif dst_ptl =huge_pte_lock (h, dst, dst_pte); src_ptl =huge_pte_lockptr (h, src, src_pte);
Signature · 1 line
2.43.5
Quoted 48 lines · Jane Chu
commit
59d9094df3d79("mm: hugetlb: independent PMD page table shared count") introduced ->pt_share_count dedicated to hugetlb PMD share count tracking, but omitted fixingcopy_hugetlb_page_range(), leaving the function relying onpage_count()for tracking that no longer works.When lazy page table copy for hugetlb is disabled, that is, revert commit
bcd51a3c679d("hugetlb: lazy page table copies infork()") fork()'ing with hugetlb PMD sharing quickly lockup -[ 239.446559] watchdog: BUG: soft lockup - CPU#75 stuck for 27s! [ 239.446611] RIP: 0010:native_queued_spin_lock_slowpath+0x7e/0x2e0 [ 239.446631] Call Trace: [ 239.446633] <TASK> [ 239.446636] _raw_spin_lock+0x3f/0x60 [ 239.446639] copy_hugetlb_page_range+0x258/0xb50 [ 239.446645] copy_page_range+0x22b/0x2c0 [ 239.446651] dup_mmap+0x3e2/0x770 [ 239.446654] dup_mm.constprop.0+0x5e/0x230 [ 239.446657] copy_process+0xd17/0x1760 [ 239.446660] kernel_clone+0xc0/0x3e0 [ 239.446661] __do_sys_clone+0x65/0xa0 [ 239.446664] do_syscall_64+0x82/0x930 [ 239.446668] ? count_memcg_events+0xd2/0x190 [ 239.446671] ? syscall_trace_enter+0x14e/0x1f0 [ 239.446676] ? syscall_exit_work+0x118/0x150 [ 239.446677] ? arch_exit_to_user_mode_prepare.constprop.0+0x9/0xb0 [ 239.446681] ? clear_bhb_loop+0x30/0x80 [ 239.446684] ? clear_bhb_loop+0x30/0x80 [ 239.446686] entry_SYSCALL_64_after_hwframe+0x76/0x7eThere are two options to resolve the potential latent issue:
1. warn against PMD sharing incopy_hugetlb_page_range(),
2. fix it.
This patch opts for the second option.Fixes 59d9094df3d79 ("mm: hugetlb: independent PMD page table shared count")Signed-off-by Jane Chu <[email protected]>Reviewed-by Harry Yoo <[email protected]>--- mm/hugetlb.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-)mm/hugetlb.c +0
-0@@ -5594,18 +5594,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,break ;
mm/hugetlb.c int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · line 5604
Quoted 10 lines
mm/hugetlb.c +0
-8} -/* -* If the pagetables are shared don't copy or take references. -* -* dst_pte == src_pte is the common case of src/dest sharing. -* However, src could have 'unshared' and dst shares with -* another vma. So page_count of ptep page is checked instead -* to reliably determine whether pte is shared. -*/
I think you ignored my question to v1 regarding the change of comment.
Signature · 3 lines
Cheers David / dhildenb
On 9/11/2025 1:17 AM, David Hildenbrand wrote:
[..]
Quoted 11 lines
Quoted 11 lines
mm/hugetlb.c +0
-2@@ -5594,18 +5594,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, break; } - /* -* If the pagetables are shared don't copy or take references.
Quoted 9 lines
mm/hugetlb.c int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · line 5604
Quoted 6 lines
mm/hugetlb.c +0
-6-* -* dst_pte == src_pte is the common case of src/dest sharing. -* However, src could have 'unshared' and dst shares with -* another vma. So page_count of ptep page is checked instead -* to reliably determine whether pte is shared. -*/ I think you ignored my question to v1 regarding the change of comment.
Sorry David, didn't mean disrespect, I missed your earlier comments. I just replied.
Thanks,
-jane
Quoted 25 lines · On 11.09.25 21:58, [email protected] wrote:
On 9/11/2025 1:17 AM, David Hildenbrand wrote:
[..]Quoted 11 lines
Quoted 11 lines
mm/hugetlb.c +0
-2@@ -5594,18 +5594,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, break; } - /* -* If the pagetables are shared don't copy or take references. Quoted 9 lines
mm/hugetlb.c int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · line 5604
Quoted 6 lines
mm/hugetlb.c +0
-6-* -* dst_pte == src_pte is the common case of src/dest sharing. -* However, src could have 'unshared' and dst shares with -* another vma. So page_count of ptep page is checked instead -* to reliably determine whether pte is shared. -*/ I think you ignored my question to v1 regarding the change of comment.
Quoted 1 line
Sorry David, didn't mean disrespect, I missed your earlier comments.
No worries. Replied there about possibly simplifying the comment further.
BTW, I think the code would be even clearer if we would have
ptdesc_pmd_is_shared()
that is simply a wrapper around the ptdesc_pmd_pts_count() check.
Signature · 3 lines
Cheers David / dhildenb
Quoted 28 lines · David Hildenbrand
Quoted 25 lines · On 11.09.25 21:58, [email protected] wrote:
On 9/11/2025 1:17 AM, David Hildenbrand wrote:
[..]Quoted 11 lines
Quoted 11 lines
mm/hugetlb.c +0
-2@@ -5594,18 +5594,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, break; } - /* -* If the pagetables are shared don't copy or take references. Quoted 9 lines
mm/hugetlb.c int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, · line 5604
Quoted 6 lines
mm/hugetlb.c +0
-6-* -* dst_pte == src_pte is the common case of src/dest sharing. -* However, src could have 'unshared' and dst shares with -* another vma. So page_count of ptep page is checked instead -* to reliably determine whether pte is shared. -*/ I think you ignored my question to v1 regarding the change of comment.
Quoted 1 line
Sorry David, didn't mean disrespect, I missed your earlier comments.
Quoted 7 lines
No worries. Replied there about possibly simplifying the comment further.
BTW, I think the code would be even clearer if we would have
ptdesc_pmd_is_shared()that is simply a wrapper around the
ptdesc_pmd_pts_count()check.
Agreed, will add ptdesc_pmd_is_shared().
thanks a lot!
-jane>