Skip to contentSkip to conversation

mm/khugepaged: clear MMF_VM_HUGEPAGE on mm_slot_alloc() failure

PATCH Superseded Ye Liu 3 participants 4 messages May 1 – May 5

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Ye Liu

author
#0 lore raw

PATCH 1/1 mm/khugepaged: clear MMF_VM_HUGEPAGE on mm_slot_alloc() failure

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

---
 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +3 -1
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}
 
 	spin_lock(&khugepaged_mm_lock);
 	mm_slot_insert(mm_slots_hash, mm, slot);
Signature · 1 line
2.43.0
reviewer
#1 lore raw
Quoted 25 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

---
 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 565

Quoted 8 lines
mm/khugepaged.c +3 -1
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

You could just move the test_and_set() after this no? So if slot allocation fails then MMF_VM_HUGEPAGE is never set.

Fixes tag is probably required but not sure about Ccing stable.

Quoted 3 lines
mm/khugepaged.c +0 -0
 	spin_lock(&khugepaged_mm_lock);
 	mm_slot_insert(mm_slots_hash, mm, slot);
reviewer
#2 lore raw
Quoted 30 lines · Dev Jain
Quoted 27 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

---
 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
Quoted 8 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 565

Quoted 6 lines
mm/khugepaged.c +3 -1
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

You could just move the test_and_set() after this no?

Yep, that sounds better :)

Just one small thing: if we move the test_and_set(), after mm_slot_alloc(), we need to free the slot when test_and_set() says that the flag was already set. Otherwise the racing caller that loses would leak the slot :)

Something like:

slot = mm_slot_alloc(mm_slot_cache);
if (!slot)

	return;
if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
	mm_slot_free(mm_slot_cache, slot);
	return;
}
Quoted 3 lines

So if slot allocation fails then MMF_VM_HUGEPAGE is never set.

Fixes tag is probably required but not sure about Ccing stable.

Yes, Fixes sounds good. khugepaged collapse is best-effort though, anyway :)

So I'm also not sure about Ccing stable.

Quoted 3 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 568

Quoted 3 lines
mm/khugepaged.c +0 -0
 	spin_lock(&khugepaged_mm_lock);
 	mm_slot_insert(mm_slots_hash, mm, slot);

Cheers, Lance

reviewer
#3 lore raw
Quoted 50 lines · Lance Yang
Quoted 30 lines · Dev Jain
Quoted 27 lines · Ye Liu

From: Ye Liu <[email protected]>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the corresponding mm_slot. If mm_slot_alloc() fails, the function returns with the flag set but without inserting the mm into the khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as registered (MMF_VM_HUGEPAGE set), but will never be scanned by khugepaged. Future attempts to register the mm are skipped since khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails, restoring the ability to retry registration later.

---
 mm/khugepaged.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/khugepaged.c +0 -0
index 7d48d4fbd5f3..60ab7c1b61dd 100644
@@ -559,8 +559,10 @@ void __khugepaged_enter(struct mm_struct *mm)
 		return;
 
Quoted 8 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 565

Quoted 6 lines
mm/khugepaged.c +3 -1
 	slot = mm_slot_alloc(mm_slot_cache);
-	if (!slot)
+	if (!slot) {
+		mm_flags_clear(MMF_VM_HUGEPAGE, mm);
 		return;
+	}

You could just move the test_and_set() after this no?

Yep, that sounds better :)

Just one small thing: if we move the test_and_set(), after mm_slot_alloc(), we need to free the slot when test_and_set() says that the flag was already set. Otherwise the racing caller that loses would leak the slot :)

Something like:

Quoted 8 lines

slot = mm_slot_alloc(mm_slot_cache);
if (!slot)

	return;
if (unlikely(mm_flags_test_and_set(MMF_VM_HUGEPAGE, mm))) {
	mm_slot_free(mm_slot_cache, slot);
	return;
}

My bad, I missed the fact that test-and-set may also fail. So I think Ye's patch is also fine.

Quoted 15 lines
Quoted 4 lines

So if slot allocation fails then MMF_VM_HUGEPAGE is never set.

Fixes tag is probably required but not sure about Ccing stable.

Yes, Fixes sounds good. khugepaged collapse is best-effort though, anyway :)

So I'm also not sure about Ccing stable.

Quoted 3 lines

mm/khugepaged.c void __khugepaged_enter(struct mm_struct *mm) · line 568

Quoted 3 lines
mm/khugepaged.c +0 -0
 	spin_lock(&khugepaged_mm_lock);
 	mm_slot_insert(mm_slots_hash, mm, slot);

Cheers, Lance