mm: huge_memory: don't start_stop_khugepaged for non-PMD THP
Subsystems (2)
PATCH 1/1 mm: huge_memory: don't start_stop_khugepaged for non-PMD THP
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call start_stop_khugepaged() for non-PMD THP, as it would only waste CPU time.
--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -1
@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err; err =start_stop_khugepaged ();
Signature · 1 line
2.34.1
Ryan Roberts
reviewerQuoted 27 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +0
-0
mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 8 lines
mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err; err =start_stop_khugepaged ();
Personally I see this as a bit of a layering violation; its start_stop_khugepaged() that should decide the policy for when to start and stop the daemon. thpsize_enabled_store() should just be calling start_stop_khugepaged() to notify that something potentially pertinent to the a policy decision has changed.
And I don't think this should be a hot path; I'd expect the global thp size configuration to be set at boot and remain pretty static. Do you have evidence to the contrary?
Quoted 35 lines · Ryan Roberts
Quoted 34 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err;
Quoted 7 lines
mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 1 line
mm/huge_memory.c +0
-0err =start_stop_khugepaged ();Personally I see this as a bit of a layering violation; its
start_stop_khugepaged()that should decide the policy for when to start and stop the daemon.thpsize_enabled_store()should just be callingstart_stop_khugepaged()to notify that something potentially pertinent to the a policy decision has changed.
Agreed, skimming the subject I was under the impression that we would be fixing something here.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 37 lines · David Hildenbrand
Quoted 35 lines · Ryan Roberts
Quoted 34 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err;
Quoted 8 lines
Quoted 7 lines
mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 1 line
mm/huge_memory.c +0
-0err =start_stop_khugepaged ();Personally I see this as a bit of a layering violation; its
start_stop_khugepaged()that should decide the policy for when to start and stop the daemon.thpsize_enabled_store()should just be callingstart_stop_khugepaged()to notify that something potentially pertinent to the a policy decision has changed.
My impression is that it slightly deviates from the huge page documentation in Documentation/admin-guide/mm/transhuge.rst.
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
non-PMD size is not involved in khugepaged, but I agree the policy might change in the future.
Quoted 2 lines
Agreed, skimming the subject I was under the impression that we would be fixing something here.
working on another swapin_enabled and reviewing the enabled source code. I don't need this startstop for all sizes in that case, so I made a quick adjustment
to this part as well. If neither of you likes it, that's fine with me :-)
Quoted 6 lines
--
Cheers,David / dhildenb
Thanks
Barry
Ryan Roberts
reviewerQuoted 47 lines · Barry Song
Quoted 37 lines · David Hildenbrand
Quoted 35 lines · Ryan Roberts
Quoted 34 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err;Quoted 8 lines
Quoted 7 lines
mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 1 line
mm/huge_memory.c +0
-0err =start_stop_khugepaged ();Personally I see this as a bit of a layering violation; its
start_stop_khugepaged()that should decide the policy for when to start and stop the daemon.thpsize_enabled_store()should just be callingstart_stop_khugepaged()to notify that something potentially pertinent to the a policy decision has changed.
Quoted 8 lines
My impression is that it slightly deviates from the huge page documentation in
Documentation/admin-guide/mm/transhuge.rst.khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
But start_stop_khugepaged() doesn't unconditionally start khugepaged, it takes action based on hugepage_pmd_enabled() which only returns true if there are any pmd sized THP enabled (currently looking at anon and file, but should also look at shmem in future; that's a known bug that's been there forever). So I don't think it is inconsistent with the documentation?
Quoted 21 lines
non-PMD size is not involved in khugepaged, but I agree the policy might change in the future.
Quoted 2 lines
Agreed, skimming the subject I was under the impression that we would be fixing something here.
working on another
swapin_enabledand reviewing the enabled source code. I don't need this startstop for all sizes in that case, so I made a quick adjustment
to this part as well. If neither of you likes it, that's fine with me :-)Quoted 6 lines
--
Cheers,David / dhildenb
Thanks
Barry
Quoted 58 lines · Ryan Roberts
Quoted 47 lines · Barry Song
Quoted 37 lines · David Hildenbrand
Quoted 35 lines · Ryan Roberts
Quoted 34 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err;Quoted 8 lines
Quoted 7 lines
mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 1 line
mm/huge_memory.c +0
-0err =start_stop_khugepaged ();Personally I see this as a bit of a layering violation; its
start_stop_khugepaged()that should decide the policy for when to start and stop the daemon.thpsize_enabled_store()should just be callingstart_stop_khugepaged()to notify that something potentially pertinent to the a policy decision has changed.Quoted 8 lines
My impression is that it slightly deviates from the huge page documentation in
Documentation/admin-guide/mm/transhuge.rst.khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
Quoted 5 lines
But
start_stop_khugepaged()doesn't unconditionally start khugepaged, it takes action based onhugepage_pmd_enabled()which only returns true if there are any pmd sized THP enabled (currently looking at anon and file, but should also look at shmem in future; that's a known bug that's been there forever). So I don't think it is inconsistent with the documentation?
My point is that, as a code reader, if non-PMD sizes are never involved with khugepaged,
we can proceed without needing to check the lower-layer code. otherwise it is a bit
confusing, especially since start_stop_khugepaged() unconditionally calls set_recommended_min_free_kbytes() for all sizes, regardless of whether they are PMD.
However, I agree with your point that if non-PMD sizes are eventually managed by khugepaged, then this approach is flawed. This is the downside of this change.
Quoted 14 lines
Quoted 14 lines
non-PMD size is not involved in khugepaged, but I agree the policy might change in the future.
Quoted 2 lines
Agreed, skimming the subject I was under the impression that we would be fixing something here.
working on another
swapin_enabledand reviewing the enabled source code. I don't need this startstop for all sizes in that case, so I made a quick adjustment
to this part as well. If neither of you likes it, that's fine with me :-)Quoted 1 line
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
--
Cheers,David / dhildenb
Thanks
Barry
Ryan Roberts
reviewerQuoted 73 lines · Barry Song
Quoted 58 lines · Ryan Roberts
Quoted 47 lines · Barry Song
Quoted 37 lines · David Hildenbrand
Quoted 35 lines · Ryan Roberts
Quoted 34 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err;Quoted 8 lines
Quoted 7 lines
mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 1 line
mm/huge_memory.c +0
-0err =start_stop_khugepaged ();Personally I see this as a bit of a layering violation; its
start_stop_khugepaged()that should decide the policy for when to start and stop the daemon.thpsize_enabled_store()should just be callingstart_stop_khugepaged()to notify that something potentially pertinent to the a policy decision has changed.Quoted 8 lines
My impression is that it slightly deviates from the huge page documentation in
Documentation/admin-guide/mm/transhuge.rst.khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
Quoted 5 lines
But
start_stop_khugepaged()doesn't unconditionally start khugepaged, it takes action based onhugepage_pmd_enabled()which only returns true if there are any pmd sized THP enabled (currently looking at anon and file, but should also look at shmem in future; that's a known bug that's been there forever). So I don't think it is inconsistent with the documentation?My point is that, as a code reader, if non-PMD sizes are never involved with khugepaged,
we can proceed without needing to check the lower-layer code. otherwise it is a bit
confusing, especially sincestart_stop_khugepaged()unconditionally callsset_recommended_min_free_kbytes()for all sizes, regardless of whether they are PMD.
Quoted 2 lines
However, I agree with your point that if non-PMD sizes are eventually managed by khugepaged, then this approach is flawed. This is the downside of this change.
Yes, I'd hope that eventually khugepaged will support non-PMD sizes. That's something we are hoping to look at later this year. So my personal preference is to leave as is.
Quoted 23 lines
Quoted 15 lines
Quoted 15 lines
non-PMD size is not involved in khugepaged, but I agree the policy might change in the future.
Quoted 2 lines
Agreed, skimming the subject I was under the impression that we would be fixing something here.
working on another
swapin_enabledand reviewing the enabled source code. I don't need this startstop for all sizes in that case, so I made a quick adjustment
to this part as well. If neither of you likes it, that's fine with me :-)Quoted 1 line
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
--
Cheers,David / dhildenb
Thanks
Barry
Quoted 44 lines · Ryan Roberts
Quoted 27 lines · Barry Song
From: Barry Song <[email protected]>
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
It seems unnecessary to call
start_stop_khugepaged()for non-PMD THP, as it would only waste CPU time.Cc Lance Yang <[email protected]>Cc Ryan Roberts <[email protected]>Cc Baolin Wang <[email protected]>Cc David Hildenbrand <[email protected]>Cc Yang Shi <[email protected]>Cc Zi Yan <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/huge_memory.c +0
-0mm/huge_memory.c static ssize_t thpsize_enabled_store(struct kobject *kobj, · line 520
Quoted 8 lines
mm/huge_memory.c +1
-1@@ -514,7 +514,7 @@ static ssize_t thpsize_enabled_store(struct kobject *kobj, }else ret = -EINVAL ; -if (ret >0 ) { +if (ret >0 && order ==HPAGE_PMD_ORDER ) {int err; err =start_stop_khugepaged ();Personally I see this as a bit of a layering violation; its
start_stop_khugepaged()that should decide the policy for when to start and stop the daemon.thpsize_enabled_store()should just be callingstart_stop_khugepaged()to notify that something potentially pertinent to the a policy decision has changed.
Quoted 3 lines
And I don't think this should be a hot path; I'd expect the global thp size configuration to be set at boot and remain pretty static. Do you have evidence to the contrary?
Absolutely no, I was reading and patching the doc "Documentation/admin-guide/mm/transhuge.rst"
and was also copying-paste your "enabled" code to "swapin_enabled" with dropping start_stop_khugepaged() in that case. I believe I don't need it for enabling/disabling
swapin as swapin is a subset of mTHP not a main entry.
just feel a bit inconsistency between the doc and the code at least for this moment:
khugepaged will be automatically started when PMD-sized THP is enabled (either of the per-size anon control or the top-level control are set to "always" or "madvise"), and it'll be automatically shutdown when PMD-sized THP is disabled (when both the per-size anon control and the top-level control are "never").
which triggered me to make this change.
Thanks
Barry