mm: introduce per-order mTHP split counters
Subsystems (4)
v3 cover mm: introduce per-order mTHP split counters
Hi all,
At present, the split counters in THP statistics no longer include PTE-mapped mTHP. Therefore, we want to introduce per-order mTHP split counters to monitor the frequency of mTHP splits. This will assist developers in better analyzing and optimizing system performance.
/sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
split
split_failed
split_deferred
---
Changes since v2 [2]
====================
- mm: add per-order mTHP split counters
- Pick AB from Barry - thanks!
- Pick RB from Baolin - thanks!
- Pick RB from Ryan - thanks!
- Make things more readable (per Barry and Baolin)
- mm: add docs for per-order mTHP split counters
- Pick RB from Barry - thanks!
- Improve the doc as suggested by Ryan
- Remove the outdated note (per Ryan)
Changes since v1 [1]
====================
- mm: add per-order mTHP split counters
- Update the changelog
- Drop '_page' from mTHP split counter names (per David and Ryan)
- Store the order of the folio in a variable and reuse it later (per Bang)
- mm: add docs for per-order mTHP split counters
- Improve the doc as suggested by Ryan
[1] https://lore.kernel.org/linux-mm/[email protected]
[2] https://lore.kernel.org/linux-mm/[email protected]
Lance Yang (2):
mm: add per-order mTHP split counters
mm: add docs for per-order mTHP split counters
Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- include/linux/huge_mm.h | 3 +++ mm/huge_memory.c | 12 ++++++++++-- 3 files changed, 29 insertions(+), 6 deletions(-)
base-commit: ec34ecf3924f09a694e5c7d8d7e785b82f67f9f0
Signature · 1 line
2.45.2
PATCH v3 1/2 mm: add per-order mTHP split counters
Currently, the split counters in THP statistics no longer include PTE-mapped mTHP. Therefore, we propose introducing per-order mTHP split counters to monitor the frequency of mTHP splits. This will help developers better analyze and optimize system performance.
/sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
split
split_failed
split_deferred
--- include/linux/huge_mm.h | 3 +++ mm/huge_memory.c | 12 ++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-)
include/linux/huge_mm.h +3 -0
@@ -284,6 +284,9 @@ enum mthp_stat_item {MTHP_STAT_FILE_ALLOC ,MTHP_STAT_FILE_FALLBACK ,MTHP_STAT_FILE_FALLBACK_CHARGE , +MTHP_STAT_SPLIT , +MTHP_STAT_SPLIT_FAILED , +MTHP_STAT_SPLIT_DEFERRED , __MTHP_STAT_COUNT };
mm/huge_memory.c +10 -2
@@ -559,6 +559,9 @@ DEFINE_MTHP_STAT_ATTR(swpout_fallback, MTHP_STAT_SWPOUT_FALLBACK);DEFINE_MTHP_STAT_ATTR (file_alloc,MTHP_STAT_FILE_ALLOC );DEFINE_MTHP_STAT_ATTR (file_fallback,MTHP_STAT_FILE_FALLBACK );DEFINE_MTHP_STAT_ATTR (file_fallback_charge,MTHP_STAT_FILE_FALLBACK_CHARGE ); +DEFINE_MTHP_STAT_ATTR (split,MTHP_STAT_SPLIT ); +DEFINE_MTHP_STAT_ATTR (split_failed,MTHP_STAT_SPLIT_FAILED ); +DEFINE_MTHP_STAT_ATTR (split_deferred,MTHP_STAT_SPLIT_DEFERRED );static struct attribute *stats_attrs[] = { &anon_fault_alloc_attr.attr,
@@ -569,6 +572,9 @@ static struct attribute *stats_attrs[] = { &file_alloc_attr.attr, &file_fallback_attr.attr, &file_fallback_charge_attr.attr, + &split_attr.attr, + &split_failed_attr.attr, + &split_deferred_attr.attr, NULL, };
@@ -3068,7 +3074,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,XA_STATE_ORDER (xas, &folio->mapping->i_pages, folio->index, new_order);struct anon_vma *anon_vma =NULL ;struct address_space *mapping =NULL ; -bool is_thp =folio_test_pmd_mappable (folio); +int order =folio_order (folio);int extra_pins, ret;pgoff_t end;bool is_hzp;
@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )count_vm_event (!ret ?THP_SPLIT_PAGE :THP_SPLIT_PAGE_FAILED ); +count_mthp_stat (order, !ret ?MTHP_STAT_SPLIT :MTHP_STAT_SPLIT_FAILED );return ret; }
@@ -3307,6 +3314,7 @@ void deferred_split_folio(struct folio *folio)if (list_empty (&folio->_deferred_list)) {if (folio_test_pmd_mappable (folio))count_vm_event (THP_DEFERRED_SPLIT_PAGE ); +count_mthp_stat (folio_order (folio),MTHP_STAT_SPLIT_DEFERRED );list_add_tail (&folio->_deferred_list, &ds_queue->split_queue); ds_queue->split_queue_len++;#ifdef CONFIG_MEMCG
Signature · 1 line
2.45.2
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
mm/huge_memory.c int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, · line 3262
Quoted 6 lines
mm/huge_memory.c +1
-1@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )
We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the future.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
Quoted 10 lines · David Hildenbrand
mm/huge_memory.c int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, · line 3262
Quoted 7 lines
mm/huge_memory.c +1
-1@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )We likely should be using "==
HPAGE_PMD_ORDER" here, to be safe for the future.
I feel this might need to be separate since all other places are using folio_test_pmd_mappable() ?
Quoted 8 lines
--
Cheers,David / dhildenb
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
Quoted 12 lines · Barry Song
Quoted 10 lines · David Hildenbrand
mm/huge_memory.c int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, · line 3262
Quoted 7 lines
mm/huge_memory.c +1
-1@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )We likely should be using "==
HPAGE_PMD_ORDER" here, to be safe for the future.
Quoted 2 lines
I feel this might need to be separate since all other places are using
folio_test_pmd_mappable()?
Likely, but as you are moving away from this ... this counter here does and will always only care about HPAGE_PMD_ORDER.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
Hi David and Barry,
Thanks a lot for paying attention!
Quoted 14 lines · David Hildenbrand
Quoted 12 lines · Barry Song
Quoted 10 lines · David Hildenbrand
mm/huge_memory.c int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, · line 3262
Quoted 7 lines
mm/huge_memory.c +1
-1@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )We likely should be using "==
HPAGE_PMD_ORDER" here, to be safe for the future.
Quoted 5 lines
Quoted 2 lines
I feel this might need to be separate since all other places are using
folio_test_pmd_mappable()?Likely, but as you are moving away from this ... this counter here does and will always only care about
HPAGE_PMD_ORDER.
I appreciate the different opinions on whether we should use ">= HPAGE_PMD_ORDER" or "==" for this check.
In this context, let's leave it as is and stay consistent with folio_test_pmd_mappable() by using ">= HPAGE_PMD_ORDER", what do you think?
Thanks,
Lance
Quoted 6 lines
--
Cheers,David / dhildenb
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
Quoted 25 lines · Lance Yang
Hi David and Barry,
Thanks a lot for paying attention!
Quoted 14 lines · David Hildenbrand
Quoted 12 lines · Barry Song
Quoted 10 lines · David Hildenbrand
mm/huge_memory.c int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, · line 3262
Quoted 7 lines
mm/huge_memory.c +1
-1@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )We likely should be using "==
HPAGE_PMD_ORDER" here, to be safe for the future.Quoted 5 lines
Quoted 2 lines
I feel this might need to be separate since all other places are using
folio_test_pmd_mappable()?Likely, but as you are moving away from this ... this counter here does and will always only care about
HPAGE_PMD_ORDER.
Quoted 6 lines
I appreciate the different opinions on whether we should use ">=
HPAGE_PMD_ORDER" or "==" for this check.In this context, let's leave it as is and stay consistent with
folio_test_pmd_mappable()by using ">=HPAGE_PMD_ORDER", what do you think?
I don't think it's a good idea to add more wrong code that is even harder to grep (folio_test_pmd_mappable would give you candidates that might need attention). But I don't care too much. Maybe someone here can volunteer to clean up these instances to make sure we check PMD-size and not PMD-mappable for these counters that are for PMD-sized folios only, even in the future with larger folios?
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
Quoted 34 lines · David Hildenbrand
Quoted 25 lines · Lance Yang
Hi David and Barry,
Thanks a lot for paying attention!
Quoted 14 lines · David Hildenbrand
Quoted 12 lines · Barry Song
Quoted 10 lines · David Hildenbrand
mm/huge_memory.c int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, · line 3262
Quoted 7 lines
mm/huge_memory.c +1
-1@@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (is_thp) +if (order >=HPAGE_PMD_ORDER )We likely should be using "==
HPAGE_PMD_ORDER" here, to be safe for the future.Quoted 5 lines
Quoted 2 lines
I feel this might need to be separate since all other places are using
folio_test_pmd_mappable()?Likely, but as you are moving away from this ... this counter here does and will always only care about
HPAGE_PMD_ORDER.Quoted 6 lines
I appreciate the different opinions on whether we should use ">=
HPAGE_PMD_ORDER" or "==" for this check.In this context, let's leave it as is and stay consistent with
folio_test_pmd_mappable()by using ">=HPAGE_PMD_ORDER", what do you think?
Quoted 6 lines
I don't think it's a good idea to add more wrong code that is even harder to grep (
folio_test_pmd_mappablewould give you candidates that might need attention). But I don't care too much. Maybe someone here can volunteer to clean up these instances to make sure we check PMD-size and not PMD-mappable for these counters that are for PMD-sized folios only, even in the future with larger folios?
Thanks for clarifying! Yes, agreed. We should ensure we check PMD-size, not PMD-mappable here, especially as we consider large folios in the future.
So, let's use "== HPAGE_PMD_ORDER" here ;)
Thanks,
Lance
Quoted 6 lines
--
Cheers,David / dhildenb
Re: [PATCH v3 1/2] mm: add per-order mTHP split counters
Hi Andrew,
Could you please fold the following changes into this patch?
mm/huge_memory.c +1 -1
@@ -3266,7 +3266,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,i_mmap_unlock_read (mapping); out:xas_destroy (&xas); -if (order >=HPAGE_PMD_ORDER ) +if (order ==HPAGE_PMD_ORDER )count_vm_event (!ret ?THP_SPLIT_PAGE :THP_SPLIT_PAGE_FAILED );count_mthp_stat (order, !ret ?MTHP_STAT_SPLIT :MTHP_STAT_SPLIT_FAILED );return ret;
Thanks,
Lance
PATCH v3 2/2 mm: add docs for per-order mTHP split counters
In reply to Lance YangThis commit introduces documentation for mTHP split counters in transhuge.rst.
--- Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
Documentation/admin-guide/mm/transhuge.rst +16 -4
@@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. Monitoring usage ================ -.. note:: - Currently the below counters only record events relating to - PMD-sized THP. Events relating to other THP sizes are not included. - The number of PMD-sized anonymous transparent huge pages currently used by the system is available by reading the AnonHugePages field in ``/proc/meminfo``. To identify what applications are using PMD-sized anonymous transparent huge
@@ -514,6 +510,22 @@ file_fallback_charge falls back to using small pages even though the allocation was successful. +split + is incremented every time a huge page is successfully split into + smaller orders. This can happen for a variety of reasons but a + common reason is that a huge page is old and is being reclaimed. + This action implies splitting any block mappings into PTEs. + +split_failed + is incremented if kernel fails to split huge + page. This can happen if the page was pinned by somebody. + +split_deferred + is incremented when a huge page is put onto split + queue. This happens when a huge page is partially unmapped and + splitting it would free up some memory. Pages on split queue are + going to be split under memory pressure. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help
Signature · 1 line
2.45.2
Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters
Quoted 34 lines · Lance Yang
This commit introduces documentation for mTHP split counters in
transhuge.rst.Reviewed-by Barry Song <[email protected]>Signed-off-by Mingzhe Yang <[email protected]>Signed-off-by Lance Yang <[email protected]>--- Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)Documentation/admin-guide/mm/transhuge.rst +5
-4@@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. Monitoring usage ================ -.. note:: - Currently the below counters only record events relating to - PMD-sized THP. Events relating to other THP sizes are not included. - The number of PMD-sized anonymous transparent huge pages currently used by the system is available by reading the AnonHugePages field in ``/proc/meminfo``. To identify what applications are using PMD-sized anonymous transparent huge@@ -514,6 +510,22 @@ file_fallback_charge falls back to using small pages even though the allocation was successful. +split + is incremented every time a huge page is successfully split into + smaller orders. This can happen for a variety of reasons but a + common reason is that a huge page is old and is being reclaimed. + This action implies splitting any block mappings into PTEs.
Documentation/admin-guide/mm/transhuge.rst file_fallback_charge · line 527
Quoted 10 lines
Documentation/admin-guide/mm/transhuge.rst +10
-0+ +split_failed + is incremented if kernel fails to split huge + page. This can happen if the page was pinned by somebody. + +split_deferred + is incremented when a huge page is put onto split + queue. This happens when a huge page is partially unmapped and + splitting it would free up some memory. Pages on split queue are + going to be split under memory pressure.
".. if splitting is possible." ;)
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters
In reply to Lance YangQuoted 26 lines · Lance Yang
This commit introduces documentation for mTHP split counters in
transhuge.rst.Reviewed-by Barry Song <[email protected]>Signed-off-by Mingzhe Yang <[email protected]>Signed-off-by Lance Yang <[email protected]>--- Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-4@@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. Monitoring usage ================ -.. note:: - Currently the below counters only record events relating to - PMD-sized THP. Events relating to other THP sizes are not included. - The number of PMD-sized anonymous transparent huge pages currently used by the system is available by reading the AnonHugePages field in ``/proc/meminfo``. To identify what applications are using PMD-sized anonymous transparent huge
Documentation/admin-guide/mm/transhuge.rst file_fallback_charge · line 517
Quoted 8 lines
Documentation/admin-guide/mm/transhuge.rst +5
-0@@ -514,6 +510,22 @@ file_fallback_charge falls back to using small pages even though the allocation was successful. +split + is incremented every time a huge page is successfully split into + smaller orders. This can happen for a variety of reasons but a + common reason is that a huge page is old and is being reclaimed. + This action implies splitting any block mappings into PTEs.
nit: the block mappings will already be PTEs if starting with mTHP?
regardless:
Quoted 14 lines
Documentation/admin-guide/mm/transhuge.rst +11
-0+ +split_failed + is incremented if kernel fails to split huge + page. This can happen if the page was pinned by somebody. + +split_deferred + is incremented when a huge page is put onto split + queue. This happens when a huge page is partially unmapped and + splitting it would free up some memory. Pages on split queue are + going to be split under memory pressure. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help
Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters
Quoted 30 lines · Ryan Roberts
Quoted 29 lines · Lance Yang
This commit introduces documentation for mTHP split counters in
transhuge.rst.Reviewed-by Barry Song <[email protected]>Signed-off-by Mingzhe Yang <[email protected]>Signed-off-by Lance Yang <[email protected]>--- Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)Documentation/admin-guide/mm/transhuge.rst +0
-4@@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. Monitoring usage ================ -.. note:: - Currently the below counters only record events relating to - PMD-sized THP. Events relating to other THP sizes are not included. - The number of PMD-sized anonymous transparent huge pages currently used by the system is available by reading the AnonHugePages field in ``/proc/meminfo``. To identify what applications are using PMD-sized anonymous transparent huge@@ -514,6 +510,22 @@ file_fallback_charge falls back to using small pages even though the allocation was successful.
Quoted 7 lines
Documentation/admin-guide/mm/transhuge.rst file_fallback_charge · line 517
Quoted 5 lines
Documentation/admin-guide/mm/transhuge.rst +5
-0+split + is incremented every time a huge page is successfully split into + smaller orders. This can happen for a variety of reasons but a + common reason is that a huge page is old and is being reclaimed. + This action implies splitting any block mappings into PTEs.nit: the block mappings will already be PTEs if starting with mTHP?
Was confused by that as well, so maybe just drop that detail here.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters
Hi David and Ryan,
Thanks for taking time to review!
Updated the doc. How about the following?
split
is incremented every time a huge page is successfully split into smaller orders. This can happen for a variety of reasons but a common reason is that a huge page is old and is being reclaimed.
split_failed
is incremented if kernel fails to split huge page. This can happen if the page was pinned by somebody.
split_deferred
is incremented when a huge page is put onto split queue. This happens when a huge page is partially unmapped and splitting it would free up some memory. Pages on split queue are going to be split under memory pressure, if splitting is possible.
Thanks,
Lance
Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters
In reply to Lance YangHi Andrew,
Could you please fold the following changes into this patch?
Documentation/admin-guide/mm/transhuge.rst +4 -5
@@ -513,17 +513,16 @@ split is incremented every time a huge page is successfully split into smaller orders. This can happen for a variety of reasons but a common reason is that a huge page is old and is being reclaimed. - This action implies splitting any block mappings into PTEs. split_failed is incremented if kernel fails to split huge page. This can happen if the page was pinned by somebody. split_deferred - is incremented when a huge page is put onto split - queue. This happens when a huge page is partially unmapped and - splitting it would free up some memory. Pages on split queue are - going to be split under memory pressure. + is incremented when a huge page is put onto split queue. + This happens when a huge page is partially unmapped and splitting + it would free up some memory. Pages on split queue are going to + be split under memory pressure, if splitting is possible. As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a
Signature · 2 lines
Thanks, Lance