Skip to contentSkip to conversation

mm, thp: display [never] for defrag when THP is set to never

PATCH In review Yuanhe Shu 3 participants 7 messages Feb 22, 2024 – Feb 26, 2024

Subsystems (2)
lore
Share & tools
lore

Yuanhe Shu

author
#0 lore raw

PATCH 1/1 mm, thp: display [never] for defrag when THP is set to never

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

---
 mm/huge_memory.c | 38 ++++++++++++++++++++++++--------------
 1 file changed, 24 insertions(+), 14 deletions(-)
mm/huge_memory.c +24 -14
index 94c958f7ebb5..03cd22a87b93 100644
@@ -349,21 +349,25 @@ static ssize_t defrag_show(struct kobject *kobj,
 {
 	const char *output;
 
-	if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG,
-		     &transparent_hugepage_flags))
-		output = "[always] defer defer+madvise madvise never";
-	else if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_FLAG,
-			  &transparent_hugepage_flags))
-		output = "always [defer] defer+madvise madvise never";
-	else if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_OR_MADV_FLAG,
-			  &transparent_hugepage_flags))
-		output = "always defer [defer+madvise] madvise never";
-	else if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_REQ_MADV_FLAG,
-			  &transparent_hugepage_flags))
-		output = "always defer defer+madvise [madvise] never";
-	else
+	if (!test_bit(TRANSPARENT_HUGEPAGE_FLAG, &transparent_hugepage_flags) &&
+	    !test_bit(TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG, &transparent_hugepage_flags))
 		output = "always defer defer+madvise madvise [never]";
-
+	else {
+		if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG,
+			&transparent_hugepage_flags))
+			output = "[always] defer defer+madvise madvise never";
+		else if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_FLAG,
+				&transparent_hugepage_flags))
+			output = "always [defer] defer+madvise madvise never";
+		else if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_OR_MADV_FLAG,
+				&transparent_hugepage_flags))
+			output = "always defer [defer+madvise] madvise never";
+		else if (test_bit(TRANSPARENT_HUGEPAGE_DEFRAG_REQ_MADV_FLAG,
+				&transparent_hugepage_flags))
+			output = "always defer defer+madvise [madvise] never";
+		else
+			output = "always defer defer+madvise madvise [never]";
+	}
 	return sysfs_emit(buf, "%s\n", output);
 }
 
@@ -371,6 +375,12 @@ static ssize_t defrag_store(struct kobject *kobj,
 			    struct kobj_attribute *attr,
 			    const char *buf, size_t count)
 {
+	if (!test_bit(TRANSPARENT_HUGEPAGE_FLAG, &transparent_hugepage_flags) &&
+	    !test_bit(TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG, &transparent_hugepage_flags)) {
+		pr_warn("transparent_hugepage not enabled, defrag takes no effect\n");
+		return -EINVAL;
+	}
+
 	if (sysfs_streq(buf, "always")) {
 		clear_bit(TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_FLAG, &transparent_hugepage_flags);
 		clear_bit(TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_OR_MADV_FLAG, &transparent_hugepage_flags);
Signature · 1 line
2.39.3
#1 lore raw
Quoted 11 lines · Yuanhe Shu

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

---

No, I don't think we want such a dependency between both options.

You might just end up breaking existing scripts (enable defrag before enabling THP) for no good reason.

Signature · 3 lines
Cheers,

David / dhildenb
#2 lore raw
Quoted 9 lines · David Hildenbrand
Quoted 8 lines · Yuanhe Shu

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

Quoted 8 lines
Quoted 3 lines

---

No, I don't think we want such a dependency between both options.

You might just end up breaking existing scripts (enable defrag before enabling THP) for no good reason.

In certain situations where khugepaged_thread is NULL, it would be more reasonable for the value of /sys/kernel/mm/transparent_hugepage/khugepaged/defrag to be 0. The patch should include a fix for this case.

Thanks

#3 lore raw
Quoted 17 lines · Liu Song
Quoted 15 lines · David Hildenbrand
Quoted 11 lines · Yuanhe Shu

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

---

No, I don't think we want such a dependency between both options.

Quoted 7 lines
Quoted 3 lines

You might just end up breaking existing scripts (enable defrag before enabling THP) for no good reason.

In certain situations where khugepaged_thread is NULL, it would be more reasonable for the value of /sys/kernel/mm/transparent_hugepage/khugepaged/defrag to be 0. The patch should include a fix for this case.

Why?

We have a bunch of THP toggles. They reside in "/sys/kernel/mm/transparent_hugepage/", indicating that they are THP specific.

Some of them are only in effect if some other toggles are set.

That is very common practice.

If you think something could be confusing, maybe clarify the doc? I don't immediately see why any code changes are required, really.

Signature · 3 lines
Cheers,

David / dhildenb
#4 lore raw
Quoted 28 lines · David Hildenbrand
Quoted 17 lines · Liu Song
Quoted 15 lines · David Hildenbrand
Quoted 11 lines · Yuanhe Shu

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

---

No, I don't think we want such a dependency between both options.

Quoted 7 lines
Quoted 3 lines

You might just end up breaking existing scripts (enable defrag before enabling THP) for no good reason.

In certain situations where khugepaged_thread is NULL, it would be more reasonable for the value of /sys/kernel/mm/transparent_hugepage/khugepaged/defrag to be 0. The patch should include a fix for this case.

Why?

Quoted 7 lines

We have a bunch of THP toggles. They reside in "/sys/kernel/mm/transparent_hugepage/", indicating that they are THP specific.

Some of them are only in effect if some other toggles are set.

That is very common practice.

If you think something could be confusing, maybe clarify the doc? I don't immediately see why any code changes are required, really.

We should explain this in the documentation, but to be honest, many people don't read the documentation, and even after we explicitly disable THP with transparent_hugepage=never, khugepaged/defrag is still set to 1, and users come asking why it's still defragging. We can't expect all users to have technical expertise, or to diligently read through the documentation; it would obviously be best if we could avoid user confusion altogether.

Thanks

#5 lore raw
Quoted 41 lines · Liu Song
Quoted 32 lines · David Hildenbrand
Quoted 17 lines · Liu Song
Quoted 15 lines · David Hildenbrand
Quoted 11 lines · Yuanhe Shu

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

---

No, I don't think we want such a dependency between both options.

Quoted 7 lines
Quoted 3 lines

You might just end up breaking existing scripts (enable defrag before enabling THP) for no good reason.

In certain situations where khugepaged_thread is NULL, it would be more reasonable for the value of /sys/kernel/mm/transparent_hugepage/khugepaged/defrag to be 0. The patch should include a fix for this case.

Why?

We have a bunch of THP toggles. They reside in "/sys/kernel/mm/transparent_hugepage/", indicating that they are THP specific.

Quoted 6 lines

Some of them are only in effect if some other toggles are set.

That is very common practice.

If you think something could be confusing, maybe clarify the doc? I don't immediately see why any code changes are required, really.

Quoted 7 lines

We should explain this in the documentation, but to be honest, many people don't read the documentation, and even after we explicitly disable THP with transparent_hugepage=never, khugepaged/defrag is still set to 1, and users come asking why it's still defragging. We can't expect all users to have technical expertise, or to diligently read through the documentation; it would obviously be best if we could avoid user confusion altogether.

Then tell your users if a feature is enabled, other toggles that affect that feature are ignored. Basic 1on1.

If my care engine is off but it's not in neutral, it's not suddenly driving. And most users understand that reasoning.

Signature · 3 lines
Cheers,

David / dhildenb
#6 lore raw
Quoted 50 lines · David Hildenbrand
Quoted 41 lines · Liu Song
Quoted 32 lines · David Hildenbrand
Quoted 17 lines · Liu Song
Quoted 15 lines · David Hildenbrand
Quoted 11 lines · Yuanhe Shu

When transparent_hugepage is set to never by cmdline or echo, defrag still show what it used to be and can be modified which makes user confusing whether defrag would take effect.

Actually if transparent_hugepage is set to never, defrag will not take effect. Just Display never and remain unchangeable to for defrag when transparent_hugepage is set to never.

---

No, I don't think we want such a dependency between both options.

Quoted 7 lines
Quoted 3 lines

You might just end up breaking existing scripts (enable defrag before enabling THP) for no good reason.

In certain situations where khugepaged_thread is NULL, it would be more reasonable for the value of /sys/kernel/mm/transparent_hugepage/khugepaged/defrag to be 0. The patch should include a fix for this case.

Why?

We have a bunch of THP toggles. They reside in "/sys/kernel/mm/transparent_hugepage/", indicating that they are THP specific.

Quoted 6 lines

Some of them are only in effect if some other toggles are set.

That is very common practice.

If you think something could be confusing, maybe clarify the doc? I don't immediately see why any code changes are required, really.

Quoted 7 lines

We should explain this in the documentation, but to be honest, many people don't read the documentation, and even after we explicitly disable THP with transparent_hugepage=never, khugepaged/defrag is still set to 1, and users come asking why it's still defragging. We can't expect all users to have technical expertise, or to diligently read through the documentation; it would obviously be best if we could avoid user confusion altogether.

Quoted 1 line

Then tell your users if a feature is enabled

s/enabled/disabled/
s/1on1/101/

Time for a break :D

Anyhow, there has to be a pretty good reason to change the semantics of these toggles that have been around forever. "users could be confused" is a not sufficient.

Signature · 3 lines
Cheers,

David / dhildenb