Skip to contentSkip to conversation

mm/khugepaged: clear MMF_VM_HUGEPAGE on mm_slot_alloc() failure

PATCH In review Ye Liu 5 participants 10 messages May 6 – May 11

Subsystems (2)
lore
Share & tools
lore

Ye Liu

author
#0 lore raw

PATCH v2 1/1 mm/khugepaged: clear MMF_VM_HUGEPAGE on mm_slot_alloc() failure

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---
Changes since v1:
- Add Fixes tag as suggested by Dev Jain and Lance Yang

 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +3 -1
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}
 
 	spin_lock(&khugepaged_mm_lock);
 	mm_slot_insert(mm_slots_hash, mm, slot);
Signature · 1 line
2.43.0
reviewer Reviewed-by on v2 1/1
#1 lore raw
Quoted 12 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Quoted 4 lines

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

Nit: the title does not exactly match the original commit subject.

It should be

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for "struct mm_slot"")

which includes quotes around "struct mm_slot".

No need to resend; I think Andrew can fix it up when applying :)

Quoted 2 lines

---

Thanks, LGTM.

reviewer Reviewed-by on v2 1/1
#2 lore raw
In reply to Ye Liu
Quoted 12 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Quoted 6 lines

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---

Good catch. Feel free to add:

reviewer Reviewed-by on v2 1/1
#3 lore raw
In reply to Ye Liu
Quoted 29 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---
Changes since v1:
- Add Fixes tag as suggested by Dev Jain and Lance Yang

 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 565

Quoted 8 lines
mm/khugepaged.c +3 -1
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

Note that, a racing khugepaged_enter_vma() may back off when it sees that MMF_VM_HUGEPAGE is set, but then the above clears the flag after slot alloc failure. So we end up not registering the mm with khugepaged. But I am sure no one cares, we are in much big trouble if slot alloc is failing.

Although one could argue the same about this patch, I will still say it is important to fix "flag is set but not registered with khugepaged" because that just feels wrong.

Quoted 3 lines
mm/khugepaged.c +0 -0
 	spin_lock(&khugepaged_mm_lock);
 	mm_slot_insert(mm_slots_hash, mm, slot);
reviewer
#4 lore raw
Quoted 34 lines · Dev Jain
Quoted 31 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---
Changes since v1:
- Add Fixes tag as suggested by Dev Jain and Lance Yang

 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
Quoted 12 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 565

Quoted 6 lines
mm/khugepaged.c +3 -1
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

Note that, a racing khugepaged_enter_vma() may back off when it sees that MMF_VM_HUGEPAGE is set, but then the above clears the flag after slot alloc failure. So we end up not registering the mm with khugepaged. But I am sure no one cares, we are in much big trouble if slot alloc is failing.

Right. A racing khugepaged_enter_vma() can see MMF_VM_HUGEPAGE is set and return, then !slot clears it again. If there is no later khugepaged_enter_vma(), the mm still wouldn't get registered :)

Quoted 3 lines

Although one could argue the same about this patch, I will still say it is important to fix "flag is set but not registered with khugepaged" because that just feels wrong.

+1 that is still better then keeping MMF_VM_HUGEPAGE set forever: any later attempt can retry and install the mm_slot :)

Cheers, Lance

#5 lore raw
Quoted 49 lines · Lance Yang
Quoted 34 lines · Dev Jain
Quoted 31 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---
Changes since v1:
- Add Fixes tag as suggested by Dev Jain and Lance Yang

 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
Quoted 12 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 565

Quoted 6 lines
mm/khugepaged.c +3 -1
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

Note that, a racing khugepaged_enter_vma() may back off when it sees that MMF_VM_HUGEPAGE is set, but then the above clears the flag after slot alloc failure. So we end up not registering the mm with khugepaged. But I am sure no one cares, we are in much big trouble if slot alloc is failing.

Quoted 3 lines

Right. A racing khugepaged_enter_vma() can see MMF_VM_HUGEPAGE is set and return, then !slot clears it again. If there is no later khugepaged_enter_vma(), the mm still wouldn't get registered :)

So why not

mm/khugepaged.c +5 -2
index 5f4e009593e0..78735f34250a 100644
@@ -437,13 +437,16 @@ void __khugepaged_enter(struct mm_struct *mm)
 
        /* __khugepaged_exit() must not run from under us */
        VM_BUG_ON_MM(collapse_test_exit(mm), mm);
-       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm)))
-               return;
 
        slot = mm_slot_alloc(mm_slot_cache);
        if (!slot)
                return;
 
+       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
+               mm_slot_free(mm_slot_cache, slot);
+               return;
+       }
+
        spin_lock(&khugepaged_mm_lock);
        mm_slot_insert(mm_slots_hash, mm, slot);
        /*

Arguably, on the race described above, likely the thread seeing the MMF_VM_HUGEPAGE would likely similarly have failed the allocation.

I'm fine with either, just wanted to raise the (cleaner looking?) alternative where we just properly back off?

Signature · 3 lines
Cheers,

David
reviewer
#6 lore raw
Quoted 81 lines · David Hildenbrand (Arm)
Quoted 49 lines · Lance Yang
Quoted 34 lines · Dev Jain
Quoted 31 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---
Changes since v1:
- Add Fixes tag as suggested by Dev Jain and Lance Yang

 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
Quoted 12 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 448

Quoted 6 lines
mm/khugepaged.c +3 -1
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

Note that, a racing khugepaged_enter_vma() may back off when it sees that MMF_VM_HUGEPAGE is set, but then the above clears the flag after slot alloc failure. So we end up not registering the mm with khugepaged. But I am sure no one cares, we are in much big trouble if slot alloc is failing.

Quoted 3 lines

Right. A racing khugepaged_enter_vma() can see MMF_VM_HUGEPAGE is set and return, then !slot clears it again. If there is no later khugepaged_enter_vma(), the mm still wouldn't get registered :)

So why not

mm/khugepaged.c +5 -2
index 5f4e009593e0..78735f34250a 100644
@@ -437,13 +437,16 @@ void __khugepaged_enter(struct mm_struct *mm)
 
       /* __khugepaged_exit() must not run from under us */
       VM_BUG_ON_MM(collapse_test_exit(mm), mm);
-       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm)))
-               return;
 
       slot = mm_slot_alloc(mm_slot_cache);
       if (!slot)
               return;
 
+       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
+               mm_slot_free(mm_slot_cache, slot);
+               return;
+       }
+
       spin_lock(&khugepaged_mm_lock);
       mm_slot_insert(mm_slots_hash, mm, slot);
       /*
Quoted 2 lines

Arguably, on the race described above, likely the thread seeing the MMF_VM_HUGEPAGE would likely similarly have failed the allocation.

Right, LGTM!

Quoted 2 lines

I'm fine with either, just wanted to raise the (cleaner looking?) alternative where we just properly back off?

Dev suggested the same thing[1] on v1 as well. We should have gone that way :)

Allocating the slot first and only setting MMF_VM_HUGEPAGE after that makes the race go away. If mm_slot_alloc() fails, there is nothing to undo.

[1] https://lore.kernel.org/linux-mm/[email protected]/

Cheers, Lance

reviewer
#7 lore raw
In reply to David Hildenbrand (Arm)
Quoted 81 lines · David Hildenbrand (Arm)
Quoted 49 lines · Lance Yang
Quoted 34 lines · Dev Jain
Quoted 31 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

Fixes 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")

---
Changes since v1:
- Add Fixes tag as suggested by Dev Jain and Lance Yang

 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
Quoted 12 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 448

Quoted 6 lines
mm/khugepaged.c +3 -1
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

Note that, a racing khugepaged_enter_vma() may back off when it sees that MMF_VM_HUGEPAGE is set, but then the above clears the flag after slot alloc failure. So we end up not registering the mm with khugepaged. But I am sure no one cares, we are in much big trouble if slot alloc is failing.

Quoted 3 lines

Right. A racing khugepaged_enter_vma() can see MMF_VM_HUGEPAGE is set and return, then !slot clears it again. If there is no later khugepaged_enter_vma(), the mm still wouldn't get registered :)

So why not

mm/khugepaged.c +5 -2
index 5f4e009593e0..78735f34250a 100644
@@ -437,13 +437,16 @@ void __khugepaged_enter(struct mm_struct *mm)
 
        /* __khugepaged_exit() must not run from under us */
        VM_BUG_ON_MM(collapse_test_exit(mm), mm);
-       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm)))
-               return;
 
        slot = mm_slot_alloc(mm_slot_cache);
        if (!slot)
                return;
 
+       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
+               mm_slot_free(mm_slot_cache, slot);
+               return;
+       }
+
        spin_lock(&khugepaged_mm_lock);
        mm_slot_insert(mm_slots_hash, mm, slot);
        /*
Quoted 5 lines

Arguably, on the race described above, likely the thread seeing the MMF_VM_HUGEPAGE would likely similarly have failed the allocation.

I'm fine with either, just wanted to raise the (cleaner looking?) alternative where we just properly back off?

Yes this is also fine - I am overthinking but I wasn't going this way because ... A process doing THP allocations will fail on the mm_flags_test_and_set everytime after the first time. So we will have an unconditional overhead of mm_slot_alloc in the fault handler.

Again I am overthinking : )

Quoted 1 line
#8 lore raw
Quoted 40 lines · Dev Jain
Quoted 37 lines · David Hildenbrand (Arm)
Quoted 5 lines · Lance Yang

Right. A racing khugepaged_enter_vma() can see MMF_VM_HUGEPAGE is set and return, then !slot clears it again. If there is no later khugepaged_enter_vma(), the mm still wouldn't get registered :)

So why not

mm/khugepaged.c +5 -2
index 5f4e009593e0..78735f34250a 100644
@@ -437,13 +437,16 @@ void __khugepaged_enter(struct mm_struct *mm)
 
        /* __khugepaged_exit() must not run from under us */
        VM_BUG_ON_MM(collapse_test_exit(mm), mm);
-       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm)))
-               return;
 
        slot = mm_slot_alloc(mm_slot_cache);
        if (!slot)
                return;
 
+       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
+               mm_slot_free(mm_slot_cache, slot);
+               return;
+       }
+
        spin_lock(&khugepaged_mm_lock);
        mm_slot_insert(mm_slots_hash, mm, slot);
        /*

Arguably, on the race described above, likely the thread seeing the MMF_VM_HUGEPAGE would likely similarly have failed the allocation.

Quoted 6 lines
Quoted 2 lines

I'm fine with either, just wanted to raise the (cleaner looking?) alternative where we just properly back off?

Yes this is also fine - I am overthinking but I wasn't going this way because ... A process doing THP allocations will fail on the mm_flags_test_and_set everytime after the first time.

We should perform a mm_flags_test(MMF_VM_HUGEPAGE, vma->vm_mm) test before calling the function when the flag might not be set yet: in khugepaged_enter_vma()

khugepaged_fork() should only get called once per process.

Which makes sense, because mm_flags_test_and_set() is expensive.

Signature · 3 lines
Cheers,

David
reviewer
#9 lore raw
Quoted 41 lines · David Hildenbrand (Arm)
Quoted 40 lines · Dev Jain
Quoted 37 lines · David Hildenbrand (Arm)
Quoted 5 lines · Lance Yang

Right. A racing khugepaged_enter_vma() can see MMF_VM_HUGEPAGE is set and return, then !slot clears it again. If there is no later khugepaged_enter_vma(), the mm still wouldn't get registered :)

So why not

mm/khugepaged.c +5 -2
index 5f4e009593e0..78735f34250a 100644
@@ -437,13 +437,16 @@ void __khugepaged_enter(struct mm_struct *mm)
 
        /* __khugepaged_exit() must not run from under us */
        VM_BUG_ON_MM(collapse_test_exit(mm), mm);
-       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm)))
-               return;
 
        slot = mm_slot_alloc(mm_slot_cache);
        if (!slot)
                return;
 
+       if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
+               mm_slot_free(mm_slot_cache, slot);
+               return;
+       }
+
        spin_lock(&khugepaged_mm_lock);
        mm_slot_insert(mm_slots_hash, mm, slot);
        /*

Arguably, on the race described above, likely the thread seeing the MMF_VM_HUGEPAGE would likely similarly have failed the allocation.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I'm fine with either, just wanted to raise the (cleaner looking?) alternative where we just properly back off?

Yes this is also fine - I am overthinking but I wasn't going this way because ... A process doing THP allocations will fail on the mm_flags_test_and_set everytime after the first time.

We should perform a mm_flags_test(MMF_VM_HUGEPAGE, vma->vm_mm) test before calling the function when the flag might not be set yet: in khugepaged_enter_vma()

Ah that slipped my mind, you are right.

Quoted 5 lines

khugepaged_fork() should only get called once per process.

Which makes sense, because mm_flags_test_and_set() is expensive.