Skip to contentSkip to conversation

mm/hugetlb: Fix missing hugetlb_lock for memcg resv uncharge

PATCH In review Peter Xu 3 participants 9 messages Apr 17, 2024 – Apr 19, 2024

Subsystems (4)
lore
Share & tools
lore

Peter Xu

author
#0 lore raw

cover mm/hugetlb: Fix missing hugetlb_lock for memcg resv uncharge

Should fix the recent syzbot report for:

https://lore.kernel.org/all/[email protected]/

Patch 1 is a small fixup where I notice mm-unstable crashes with the new hugetlb memcg accounting when testing the changes.

Patch 2 should be the fix to the reported issue.

Patch 3 is an oneliner to add an assertion similar to what found the issue in patch 2.

Only smoke tested over a bunch of hugetlb unit tests. Reviews welcomed.

Thanks,

Peter Xu (3):
fixup! mm: always initialise folio->_deferred_list mm/hugetlb: Fix missing hugetlb_lock for resv uncharge mm/hugetlb: Assert hugetlb_lock in __hugetlb_cgroup_commit_charge

 mm/hugetlb.c        | 5 ++++-
 mm/hugetlb_cgroup.c | 2 +-
 mm/memcontrol.c     | 1 +
 3 files changed, 6 insertions(+), 2 deletions(-)
Signature · 1 line
2.44.0
author
#1 lore raw

PATCH 1/3 fixup! mm: always initialise folio->_deferred_list

Current mm-unstable will hit this when running test_hugetlb_memcg. This fixes the crash for me.

---
 mm/memcontrol.c | 1 +
 1 file changed, 1 insertion(+)
mm/memcontrol.c +1 -0
index 1840ba4c355d..7703ced535a3 100644
@@ -7529,6 +7529,7 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 
 	VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
 	VM_BUG_ON_FOLIO(folio_order(folio) > 1 &&
+			!folio_test_hugetlb(folio) &&
 			!list_empty(&folio->_deferred_list), folio);
 
 	/*
Signature · 1 line
2.44.0
#2 lore raw

Re: [PATCH 1/3] fixup! mm: always initialise folio->_deferred_list

Quoted 13 lines · Peter Xu

Current mm-unstable will hit this when running test_hugetlb_memcg. This fixes the crash for me.

---
 mm/memcontrol.c | 1 +
 1 file changed, 1 insertion(+)
mm/memcontrol.c +0 -0
index 1840ba4c355d..7703ced535a3 100644

mm/memcontrol.c static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug) · line 7533

Quoted 5 lines
mm/memcontrol.c +1 -0
@@ -7529,6 +7529,7 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 
 	VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
 	VM_BUG_ON_FOLIO(folio_order(folio) > 1 &&
+			!folio_test_hugetlb(folio) &&
 			!list_empty(&folio->_deferred_list), folio);

Hum. I thought we didn't get here for hugetlb folios. What stacktrace did you get?

I'm basing it on comments like this:

                /* hugetlb has its own memcg */
                if (folio_test_hugetlb(folio)) {
                        if (lruvec) {
                                unlock_page_lruvec_irqrestore(lruvec, flags);
                                lruvec = NULL;
                        }
                        free_huge_folio(folio);
                        continue;
                }
author
#3 lore raw

Re: [PATCH 1/3] fixup! mm: always initialise folio->_deferred_list

Quoted 3 lines · Matthew Wilcox
Quoted 2 lines · Peter Xu

Current mm-unstable will hit this when running test_hugetlb_memcg. This fixes the crash for me.

[1]

Quoted 12 lines
Quoted 12 lines
---
 mm/memcontrol.c | 1 +
 1 file changed, 1 insertion(+)
mm/memcontrol.c +0 -0
index 1840ba4c355d..7703ced535a3 100644
@@ -7529,6 +7529,7 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 
Quoted 7 lines

mm/memcontrol.c static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug) · line 7533

Quoted 4 lines
mm/memcontrol.c +1 -0
 	VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
 	VM_BUG_ON_FOLIO(folio_order(folio) > 1 &&
+			!folio_test_hugetlb(folio) &&
 			!list_empty(&folio->_deferred_list), folio);

Hum. I thought we didn't get here for hugetlb folios. What stacktrace did you get?

A normal hugetlb free path iirc. You can try the test case, I mentioned the reproducer [1] above. It crashes constantly.

Quoted 12 lines

I'm basing it on comments like this:

                /* hugetlb has its own memcg */
                if (folio_test_hugetlb(folio)) {
                        if (lruvec) {
                                unlock_page_lruvec_irqrestore(lruvec, flags);
                                lruvec = NULL;
                        }
                        free_huge_folio(folio);
                        continue;
                }

Hugetlb does have its own memcg but I guess now it's even more than that, see the patch merged months ago:

https://lore.kernel.org/all/[email protected]/

Especially:

@@ -1902,6 +1902,7 @@ void free_huge_folio(struct folio *folio)
pages_per_huge_page(h), folio);
hugetlb_cgroup_uncharge_folio_rsvd(hstate_index(h),
pages_per_huge_page(h), folio);
+ mem_cgroup_uncharge(folio);
if (restore_reserve)
h->resv_huge_pages++;

The comment above looks a bit confusing to me, as memcg is not the only thing special for hugetlb pages. Explicitly mentioning it there made me feel like hugetlb can be freed like a normal compound page if memcg is not a problem, but looks like not yet?

Signature · 1 line
Peter Xu
author
#4 lore raw

PATCH 2/3 mm/hugetlb: Fix missing hugetlb_lock for resv uncharge

In reply to Peter Xu

There is a recent report on UFFDIO_COPY over hugetlb:

https://lore.kernel.org/all/[email protected]/

350: lockdep_assert_held(&hugetlb_lock);

Should be an issue in hugetlb but triggered in an userfault context, where it goes into the unlikely path where two threads modifying the resv map together. Mike has a fix in that path for resv uncharge but it looks like the locking criteria was overlooked: hugetlb_cgroup_uncharge_folio_rsvd() will update the cgroup pointer, so it requires to be called with the lock held.

Looks like a stable material, so have it copied.

Reported-by [email protected]
Cc Mina Almasry <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc linux-stable <[email protected]>
Fixes 79aa925bf239 ("hugetlb_cgroup: fix reservation accounting")
---
 mm/hugetlb.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
mm/hugetlb.c +4 -1
index 26ab9dfc7d63..3158a55ce567 100644
@@ -3247,9 +3247,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 
 		rsv_adjust = hugepage_subpool_put_pages(spool, 1);
 		hugetlb_acct_memory(h, -rsv_adjust);
-		if (deferred_reserve)
+		if (deferred_reserve) {
+			spin_lock_irq(&hugetlb_lock);
 			hugetlb_cgroup_uncharge_folio_rsvd(hstate_index(h),
 					pages_per_huge_page(h), folio);
+			spin_unlock_irq(&hugetlb_lock);
+		}
 	}
 
 	if (!memcg_charge_ret)
Signature · 1 line
2.44.0
Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] mm/hugetlb: Fix missing hugetlb_lock for resv uncharge

Quoted 14 lines · Peter Xu

There is a recent report on UFFDIO_COPY over hugetlb:

https://lore.kernel.org/all/[email protected]/

350: lockdep_assert_held(&hugetlb_lock);

Should be an issue in hugetlb but triggered in an userfault context, where it goes into the unlikely path where two threads modifying the resv map together. Mike has a fix in that path for resv uncharge but it looks like the locking criteria was overlooked: hugetlb_cgroup_uncharge_folio_rsvd() will update the cgroup pointer, so it requires to be called with the lock held.

Quoted 8 lines

Looks like a stable material, so have it copied.

Reported-by [email protected]
Cc Mina Almasry <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc linux-stable <[email protected]>
Fixes 79aa925bf239 ("hugetlb_cgroup: fix reservation accounting")
Quoted 25 lines
---
 mm/hugetlb.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
mm/hugetlb.c +4 -2
index 26ab9dfc7d63..3158a55ce567 100644
@@ -3247,9 +3247,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 
                rsv_adjust = hugepage_subpool_put_pages(spool, 1);
                hugetlb_acct_memory(h, -rsv_adjust);
-               if (deferred_reserve)
+               if (deferred_reserve) {
+                       spin_lock_irq(&hugetlb_lock);
                        hugetlb_cgroup_uncharge_folio_rsvd(hstate_index(h),
                                        pages_per_huge_page(h), folio);
+                       spin_unlock_irq(&hugetlb_lock);
+               }
        }
 
        if (!memcg_charge_ret)
--

2.44.0

Signature · 2 lines
Thanks,
Mina
author
#6 lore raw

PATCH 3/3 mm/hugetlb: Assert hugetlb_lock in __hugetlb_cgroup_commit_charge

In reply to Peter Xu

This is similar to __hugetlb_cgroup_uncharge_folio() where it relies on holding hugetlb_lock. Add the similar assertion like the other one, since it looks like such things may help some day.

---
 mm/hugetlb_cgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb_cgroup.c +1 -1
index aa4486bd3904..e20339a346b9 100644
@@ -308,7 +308,7 @@ static void __hugetlb_cgroup_commit_charge(int idx, unsigned long nr_pages,
 {
 	if (hugetlb_cgroup_disabled() || !h_cg)
 		return;
-
+	lockdep_assert_held(&hugetlb_lock);
 	__set_hugetlb_cgroup(folio, h_cg, rsvd);
 	if (!rsvd) {
 		unsigned long usage =
Signature · 1 line
2.44.0
Reviewed-by on v1 3/3
#7 lore raw

Re: [PATCH 3/3] mm/hugetlb: Assert hugetlb_lock in __hugetlb_cgroup_commit_charge

Quoted 6 lines · Peter Xu

This is similar to __hugetlb_cgroup_uncharge_folio() where it relies on holding hugetlb_lock. Add the similar assertion like the other one, since it looks like such things may help some day.

Quoted 9 lines
---
 mm/hugetlb_cgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb_cgroup.c +0 -0
index aa4486bd3904..e20339a346b9 100644

mm/hugetlb_cgroup.c static void __hugetlb_cgroup_commit_charge(int idx, unsigned long nr_pages, · line 311

Quoted 5 lines
mm/hugetlb_cgroup.c +1 -1
@@ -308,7 +308,7 @@ static void __hugetlb_cgroup_commit_charge(int idx, unsigned long nr_pages,
 {
        if (hugetlb_cgroup_disabled() || !h_cg)
                return;
-
+       lockdep_assert_held(&hugetlb_lock);

Maybe also remove the comment on the top of the function:

/* Should be called with hugetlb_lock held */

Now that the function asserts, the comment seems redundant, but up to you.

Quoted 6 lines
mm/hugetlb_cgroup.c +0 -1
        __set_hugetlb_cgroup(folio, h_cg, rsvd);
        if (!rsvd) {
                unsigned long usage =
--

2.44.0

Signature · 2 lines
Thanks,
Mina
author
#8 lore raw

Re: [PATCH 3/3] mm/hugetlb: Assert hugetlb_lock in __hugetlb_cgroup_commit_charge

Quoted 9 lines · Mina Almasry
Quoted 6 lines · Peter Xu

This is similar to __hugetlb_cgroup_uncharge_folio() where it relies on holding hugetlb_lock. Add the similar assertion like the other one, since it looks like such things may help some day.

Thanks.

Quoted 18 lines
Quoted 10 lines
---
 mm/hugetlb_cgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb_cgroup.c +0 -0
index aa4486bd3904..e20339a346b9 100644

mm/hugetlb_cgroup.c static void __hugetlb_cgroup_commit_charge(int idx, unsigned long nr_pages, · line 311

Quoted 5 lines
mm/hugetlb_cgroup.c +1 -1
@@ -308,7 +308,7 @@ static void __hugetlb_cgroup_commit_charge(int idx, unsigned long nr_pages,
 {
        if (hugetlb_cgroup_disabled() || !h_cg)
                return;
-
+       lockdep_assert_held(&hugetlb_lock);

Maybe also remove the comment on the top of the function:

Quoted 3 lines

/* Should be called with hugetlb_lock held */

Now that the function asserts, the comment seems redundant, but up to you.

IMHO there's no harm to be verbose in this case, so I'll just keep it simple to be as-is.

Thanks,

Signature · 1 line
Peter Xu