mm: Override mTHP "enabled" defaults at kernel cmdline
Subsystems (4)
PATCH v4 1/1 mm: Override mTHP "enabled" defaults at kernel cmdline
From: Ryan Roberts <[email protected]>
Add thp_anon= cmdline parameter to allow specifying the default enablement of each supported anon THP size. The parameter accepts the following format and can be provided multiple times to configure each size:
thp_anon=<size>,<size>[KMG]:<value>;<size>-<size>[KMG]:<value>
An example:
thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never
See Documentation/admin-guide/mm/transhuge.rst for more details.
Configuring the defaults at boot time is useful to allow early user space to take advantage of mTHP before its been configured through sysfs.
---
-v4:
* use bitmap APIs to set and clear bits. thanks very much for David's comment!
.../admin-guide/kernel-parameters.txt | 9 ++ Documentation/admin-guide/mm/transhuge.rst | 37 +++++-- mm/huge_memory.c | 96 ++++++++++++++++++- 3 files changed, 134 insertions(+), 8 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +9 -0
@@ -6629,6 +6629,15 @@ <deci-seconds>: poll all this frequency 0: no polling (default) + thp_anon= [KNL] + Format: <size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state> + state is one of "always", "madvise", "never" or "inherit". + Can be used to control the default behavior of the + system with respect to anonymous transparent hugepages. + Can be used multiple times for multiple anon THP sizes. + See Documentation/admin-guide/mm/transhuge.rst for more + details. + threadirqs [KNL,EARLY] Force threading of all interrupt handlers except those marked explicitly IRQF_NO_THREAD.
Documentation/admin-guide/mm/transhuge.rst +30 -7
@@ -284,13 +284,36 @@ that THP is shared. Exceeding the number would block the collapse:: A higher value may increase memory footprint for some workloads. -Boot parameter -============== - -You can change the sysfs boot time defaults of Transparent Hugepage -Support by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` -to the kernel command line. +Boot parameters +=============== + +You can change the sysfs boot time default for the top-level "enabled" +control by passing the parameter ``transparent_hugepage=always`` or +``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the +kernel command line. + +Alternatively, each supported anonymous THP size can be controlled by +passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, +where ``<size>`` is the THP size and ``<state>`` is one of ``always``, +``madvise``, ``never`` or ``inherit``. + +For example, the following will set 16K, 32K, 64K THP to ``always``, +set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M +to ``never``:: + + thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never + +``thp_anon=`` may be specified multiple times to configure all THP sizes as +required. If ``thp_anon=`` is specified at least once, any anon THP sizes +not explicitly configured on the command line are implicitly set to +``never``. + +``transparent_hugepage`` setting only affects the global toggle. If +``thp_anon`` is not specified, PMD_ORDER THP will default to ``inherit``. +However, if a valid ``thp_anon`` setting is provided by the user, the +PMD_ORDER THP policy will be overridden. If the policy for PMD_ORDER +is not defined within a valid ``thp_anon``, its policy will default to +``never``. Hugepages in tmpfs/shmem ========================
mm/huge_memory.c +95 -1
114 lines not on this page · raw message
Signature · 1 line
2.34.1
Quoted 15 lines · Barry Song
From: Ryan Roberts <[email protected]>
Add thp_anon= cmdline parameter to allow specifying the default enablement of each supported anon THP size. The parameter accepts the following format and can be provided multiple times to configure each size:
thp_anon=<size>,<size>[KMG]:<value>;<size>-<size>[KMG]:<value>
An example:
thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never
See
Documentation/admin-guide/mm/transhuge.rstfor more details.
Quoted 7 lines
Configuring the defaults at boot time is useful to allow early user space to take advantage of mTHP before its been configured through sysfs.
Signed-off-by Ryan Roberts <[email protected]>Co-developed-by Barry Song <[email protected]>Signed-off-by Barry Song <[email protected]>
LGTM. Feel free to add:
Just a small nit as below.
Quoted 104 lines
---
-v4:
* use bitmap APIs to set and clear bits. thanks very much for David's comment!.../admin-guide/kernel-parameters.txt | 9 ++ Documentation/admin-guide/mm/transhuge.rst | 37 +++++-- mm/huge_memory.c | 96 ++++++++++++++++++- 3 files changed, 134 insertions(+), 8 deletions(-)Documentation/admin-guide/kernel-parameters.txt +9
-0@@ -6629,6 +6629,15 @@ <deci-seconds>: poll all this frequency 0: no polling (default) + thp_anon= [KNL] + Format: <size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state> + state is one of "always", "madvise", "never" or "inherit". + Can be used to control the default behavior of the + system with respect to anonymous transparent hugepages. + Can be used multiple times for multiple anon THP sizes. + See Documentation/admin-guide/mm/transhuge.rst for more + details. + threadirqs [KNL,EARLY] Force threading of all interrupt handlers except those marked explicitly IRQF_NO_THREAD.Documentation/admin-guide/mm/transhuge.rst +30
-7@@ -284,13 +284,36 @@ that THP is shared. Exceeding the number would block the collapse:: A higher value may increase memory footprint for some workloads. -Boot parameter -============== - -You can change the sysfs boot time defaults of Transparent Hugepage -Support by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` -to the kernel command line. +Boot parameters +=============== + +You can change the sysfs boot time default for the top-level "enabled" +control by passing the parameter ``transparent_hugepage=always`` or +``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the +kernel command line. + +Alternatively, each supported anonymous THP size can be controlled by +passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, +where ``<size>`` is the THP size and ``<state>`` is one of ``always``, +``madvise``, ``never`` or ``inherit``. + +For example, the following will set 16K, 32K, 64K THP to ``always``, +set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M +to ``never``:: + + thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never + +``thp_anon=`` may be specified multiple times to configure all THP sizes as +required. If ``thp_anon=`` is specified at least once, any anon THP sizes +not explicitly configured on the command line are implicitly set to +``never``. + +``transparent_hugepage`` setting only affects the global toggle. If +``thp_anon`` is not specified, PMD_ORDER THP will default to ``inherit``. +However, if a valid ``thp_anon`` setting is provided by the user, the +PMD_ORDER THP policy will be overridden. If the policy for PMD_ORDER +is not defined within a valid ``thp_anon``, its policy will default to +``never``. Hugepages in tmpfs/shmem ========================mm/huge_memory.c +5
-1@@ -81,6 +81,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL;unsigned long huge_anon_orders_always__read_mostly ;unsigned long huge_anon_orders_madvise__read_mostly ;unsigned long huge_anon_orders_inherit__read_mostly ; +static bool anon_orders_configured; unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, unsigned long vm_flags,@@ -737,7 +738,10 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)* disable all other sizes. powerpc's PMD_ORDER isn't a compile-time * constant so we have to do this here. */ - huge_anon_orders_inherit =BIT (PMD_ORDER ); +if (!anon_orders_configured) { + huge_anon_orders_inherit =BIT (PMD_ORDER ); + anon_orders_configured = true; + } *hugepage_kobj =kobject_create_and_add ("transparent_hugepage" , mm_kobj);if (unlikely (!*hugepage_kobj)) {@@ -922,6 +926,96 @@ static int __init setup_transparent_hugepage(char *str) }
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 936
Quoted 10 lines
mm/huge_memory.c +8
-0__setup ("transparent_hugepage=" , setup_transparent_hugepage); +static inline int get_order_from_str (const char *size_str) +{ +unsigned long size; +char *endptr; +int order; + + size =memparse (size_str, &endptr); + order =fls (size >>PAGE_SHIFT ) -1 ;
Nit: using get_order() seems more robust?
Quoted 8 lines
mm/huge_memory.c +8
-0+if ((1 << order) & ~THP_ORDERS_ALL_ANON ) { +pr_err ("invalid size %s(order %d) in thp_anon boot parameter\n" , + size_str, order); +return -EINVAL ; + } + +return order; +}
[snip]
Quoted 23 lines · Baolin Wang
Quoted 19 lines · Barry Song
From: Ryan Roberts <[email protected]>
Add thp_anon= cmdline parameter to allow specifying the default enablement of each supported anon THP size. The parameter accepts the following format and can be provided multiple times to configure each size:
thp_anon=<size>,<size>[KMG]:<value>;<size>-<size>[KMG]:<value>
An example:
thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never
See
Documentation/admin-guide/mm/transhuge.rstfor more details.Configuring the defaults at boot time is useful to allow early user space to take advantage of mTHP before its been configured through sysfs.
Quoted 8 lines
Quoted 3 lines
Signed-off-by Ryan Roberts <[email protected]>Co-developed-by Barry Song <[email protected]>Signed-off-by Barry Song <[email protected]>LGTM. Feel free to add:
Reviewed-by Baolin Wang <[email protected]>Tested-by Baolin Wang <[email protected]>
Thanks, Baolin!
Quoted 110 lines
Just a small nit as below.
Quoted 108 lines
---
-v4:
* use bitmap APIs to set and clear bits. thanks very much for David's comment!
.../admin-guide/kernel-parameters.txt| 9 ++Documentation/admin-guide/mm/transhuge.rst| 37 +++++--mm/huge_memory.c| 96 ++++++++++++++++++-3 files changed, 134 insertions(+), 8 deletions(-)Documentation/admin-guide/kernel-parameters.txt +0
-0@@ -6629,6 +6629,15 @@<deci-seconds>: poll all this frequency
0: no polling (default)
+ thp_anon= [KNL]
+ Format: <size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>
+ state is one of "always", "madvise", "never" or "inherit".
+ Can be used to control the default behavior of the
+ system with respect to anonymous transparent hugepages.
+ Can be used multiple times for multiple anon THP sizes.
+ SeeDocumentation/admin-guide/mm/transhuge.rstfor more
+ details.
+
threadirqs [KNL,EARLY]
Force threading of all interrupt handlers except those marked explicitlyIRQF_NO_THREAD.Documentation/admin-guide/mm/transhuge.rst +0
-0@@ -284,13 +284,36 @@ that THP is shared. Exceeding the number would block the collapse::
A higher value may increase memory footprint for some workloads.Documentation/admin-guide/mm/transhuge.rst +30
-7-Boot parameter -============== - -You can change the sysfs boot time defaults of Transparent Hugepage -Support by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` -to the kernel command line. +Boot parameters +=============== + +You can change the sysfs boot time default for the top-level "enabled" +control by passing the parameter ``transparent_hugepage=always`` or +``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the +kernel command line. + +Alternatively, each supported anonymous THP size can be controlled by +passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, +where ``<size>`` is the THP size and ``<state>`` is one of ``always``, +``madvise``, ``never`` or ``inherit``. + +For example, the following will set 16K, 32K, 64K THP to ``always``, +set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M +to ``never``:: + + thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never + +``thp_anon=`` may be specified multiple times to configure all THP sizes as +required. If ``thp_anon=`` is specified at least once, any anon THP sizes +not explicitly configured on the command line are implicitly set to +``never``. + +``transparent_hugepage`` setting only affects the global toggle. If +``thp_anon`` is not specified, PMD_ORDER THP will default to ``inherit``. +However, if a valid ``thp_anon`` setting is provided by the user, the +PMD_ORDER THP policy will be overridden. If the policy for PMD_ORDER +is not defined within a valid ``thp_anon``, its policy will default to +``never``.
Hugepages in tmpfs/shmem
========================mm/huge_memory.c +1
-0@@ -81,6 +81,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; unsigned long huge_anon_orders_always __read_mostly; unsigned long huge_anon_orders_madvise __read_mostly; unsigned long huge_anon_orders_inherit __read_mostly; +static bool anon_orders_configured;
unsigned long __thp_vma_allowable_orders(structvm_area_struct*vma, unsigned longvm_flags,mm/huge_memory.c +4
-1@@ -737,7 +738,10 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)* disable all other sizes. powerpc's PMD_ORDER isn't a compile-time * constant so we have to do this here. */ - huge_anon_orders_inherit = BIT(PMD_ORDER); + if (!anon_orders_configured) { + huge_anon_orders_inherit = BIT(PMD_ORDER); + anon_orders_configured = true; + }
*hugepage_kobj=kobject_create_and_add("transparent_hugepage", mm_kobj); if (unlikely(!*hugepage_kobj)) {mm/huge_memory.c +0
-0@@ -922,6 +926,96 @@ static int __init setup_transparent_hugepage(char *str) } __setup("transparent_hugepage=", setup_transparent_hugepage);mm/huge_memory.c +2
-0+static inline int get_order_from_str (const char *size_str) +{
Quoted 8 lines
Quoted 6 lines
+ unsigned long size;
+ char *endptr;
+ int order;
+
+ size =memparse(size_str, &endptr);
+ order =fls(size >> PAGE_SHIFT)- 1;Nit: using
get_order()seems more robust?
Yes. I agree get_order() is better:
mm/huge_memory.c +1 -1
@@ -933,7 +933,7 @@ static inline int get_order_from_str(const char *size_str)int order; size =memparse (size_str, &endptr); - order =fls (size >>PAGE_SHIFT ) -1 ; + order =get_order (size);if ((1 << order) & ~THP_ORDERS_ALL_ANON ) {pr_err ("invalid size %s(order %d) in thp_anon boot parameter\n" , size_str, order);
Quoted 10 lines
Quoted 9 lines
+ if ((1 << order) & ~THP_ORDERS_ALL_ANON) {
+ pr_err("invalid size %s(order %d) inthp_anonboot parameter\n",
+size_str, order);
+ return-EINVAL;
+ }mm/huge_memory.c +3
-0+ + return order; +}[snip]
Quoted 125 lines · Barry Song
From: Ryan Roberts <[email protected]>
Add thp_anon= cmdline parameter to allow specifying the default enablement of each supported anon THP size. The parameter accepts the following format and can be provided multiple times to configure each size:
thp_anon=<size>,<size>[KMG]:<value>;<size>-<size>[KMG]:<value>
An example:
thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never
See
Documentation/admin-guide/mm/transhuge.rstfor more details.Configuring the defaults at boot time is useful to allow early user space to take advantage of mTHP before its been configured through sysfs.
Signed-off-by Ryan Roberts <[email protected]>Co-developed-by Barry Song <[email protected]>Signed-off-by Barry Song <[email protected]>---
-v4:
* use bitmap APIs to set and clear bits. thanks very much for David's comment!.../admin-guide/kernel-parameters.txt | 9 ++ Documentation/admin-guide/mm/transhuge.rst | 37 +++++-- mm/huge_memory.c | 96 ++++++++++++++++++- 3 files changed, 134 insertions(+), 8 deletions(-)Documentation/admin-guide/kernel-parameters.txt +9
-0@@ -6629,6 +6629,15 @@ <deci-seconds>: poll all this frequency 0: no polling (default) + thp_anon= [KNL] + Format: <size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state> + state is one of "always", "madvise", "never" or "inherit". + Can be used to control the default behavior of the + system with respect to anonymous transparent hugepages. + Can be used multiple times for multiple anon THP sizes. + See Documentation/admin-guide/mm/transhuge.rst for more + details. + threadirqs [KNL,EARLY] Force threading of all interrupt handlers except those marked explicitly IRQF_NO_THREAD.Documentation/admin-guide/mm/transhuge.rst +30
-7@@ -284,13 +284,36 @@ that THP is shared. Exceeding the number would block the collapse:: A higher value may increase memory footprint for some workloads. -Boot parameter -============== - -You can change the sysfs boot time defaults of Transparent Hugepage -Support by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` -to the kernel command line. +Boot parameters +=============== + +You can change the sysfs boot time default for the top-level "enabled" +control by passing the parameter ``transparent_hugepage=always`` or +``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the +kernel command line. + +Alternatively, each supported anonymous THP size can be controlled by +passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, +where ``<size>`` is the THP size and ``<state>`` is one of ``always``, +``madvise``, ``never`` or ``inherit``. + +For example, the following will set 16K, 32K, 64K THP to ``always``, +set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M +to ``never``:: + + thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never + +``thp_anon=`` may be specified multiple times to configure all THP sizes as +required. If ``thp_anon=`` is specified at least once, any anon THP sizes +not explicitly configured on the command line are implicitly set to +``never``. + +``transparent_hugepage`` setting only affects the global toggle. If +``thp_anon`` is not specified, PMD_ORDER THP will default to ``inherit``. +However, if a valid ``thp_anon`` setting is provided by the user, the +PMD_ORDER THP policy will be overridden. If the policy for PMD_ORDER +is not defined within a valid ``thp_anon``, its policy will default to +``never``. Hugepages in tmpfs/shmem ========================mm/huge_memory.c +5
-1@@ -81,6 +81,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL;unsigned long huge_anon_orders_always__read_mostly ;unsigned long huge_anon_orders_madvise__read_mostly ;unsigned long huge_anon_orders_inherit__read_mostly ; +static bool anon_orders_configured; unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, unsigned long vm_flags,@@ -737,7 +738,10 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)* disable all other sizes. powerpc's PMD_ORDER isn't a compile-time * constant so we have to do this here. */ - huge_anon_orders_inherit =BIT (PMD_ORDER ); +if (!anon_orders_configured) { + huge_anon_orders_inherit =BIT (PMD_ORDER ); + anon_orders_configured = true; + } *hugepage_kobj =kobject_create_and_add ("transparent_hugepage" , mm_kobj);if (unlikely (!*hugepage_kobj)) {
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 935
Quoted 10 lines
mm/huge_memory.c +7
-0@@ -922,6 +926,96 @@ static int __init setup_transparent_hugepage(char *str) }__setup ("transparent_hugepage=" , setup_transparent_hugepage); +static inline int get_order_from_str (const char *size_str) +{ +unsigned long size; +char *endptr; +int order; + + size =memparse (size_str, &endptr);
Do we have to also test if is_power_of_2(), and refuse if not? For example, what if someone would pass 3K, would the existing check catch it?
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 936
Quoted 1 line
mm/huge_memory.c +1
-0+ order =fls (size >>PAGE_SHIFT ) -1 ;
Is this a fancy way of writing
order = log2(size >> PAGE_SHIFT);
? :)
Anyhow, if get_order() wraps that, all good.
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 944
Quoted 8 lines
mm/huge_memory.c +8
-0+if ((1 << order) & ~THP_ORDERS_ALL_ANON ) { +pr_err ("invalid size %s(order %d) in thp_anon boot parameter\n" , + size_str, order); +return -EINVAL ; + } + +return order; +}
Apart from that, nothing jumped at me.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 132 lines · David Hildenbrand
Quoted 130 lines · Barry Song
From: Ryan Roberts <[email protected]>
Add thp_anon= cmdline parameter to allow specifying the default enablement of each supported anon THP size. The parameter accepts the following format and can be provided multiple times to configure each size:
thp_anon=<size>,<size>[KMG]:<value>;<size>-<size>[KMG]:<value>
An example:
thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never
See
Documentation/admin-guide/mm/transhuge.rstfor more details.Configuring the defaults at boot time is useful to allow early user space to take advantage of mTHP before its been configured through sysfs.
Signed-off-by Ryan Roberts <[email protected]>Co-developed-by Barry Song <[email protected]>Signed-off-by Barry Song <[email protected]>---
-v4:
* use bitmap APIs to set and clear bits. thanks very much for David's comment!
.../admin-guide/kernel-parameters.txt| 9 ++Documentation/admin-guide/mm/transhuge.rst| 37 +++++--mm/huge_memory.c| 96 ++++++++++++++++++-3 files changed, 134 insertions(+), 8 deletions(-)Documentation/admin-guide/kernel-parameters.txt +0
-0@@ -6629,6 +6629,15 @@<deci-seconds>: poll all this frequency
0: no polling (default)
+ thp_anon= [KNL]
+ Format: <size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>
+ state is one of "always", "madvise", "never" or "inherit".
+ Can be used to control the default behavior of the
+ system with respect to anonymous transparent hugepages.
+ Can be used multiple times for multiple anon THP sizes.
+ SeeDocumentation/admin-guide/mm/transhuge.rstfor more
+ details.
+
threadirqs [KNL,EARLY]
Force threading of all interrupt handlers except those marked explicitlyIRQF_NO_THREAD.Documentation/admin-guide/mm/transhuge.rst +0
-0@@ -284,13 +284,36 @@ that THP is shared. Exceeding the number would block the collapse::
A higher value may increase memory footprint for some workloads.Documentation/admin-guide/mm/transhuge.rst +30
-7-Boot parameter -============== - -You can change the sysfs boot time defaults of Transparent Hugepage -Support by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` -to the kernel command line. +Boot parameters +=============== + +You can change the sysfs boot time default for the top-level "enabled" +control by passing the parameter ``transparent_hugepage=always`` or +``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the +kernel command line. + +Alternatively, each supported anonymous THP size can be controlled by +passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, +where ``<size>`` is the THP size and ``<state>`` is one of ``always``, +``madvise``, ``never`` or ``inherit``. + +For example, the following will set 16K, 32K, 64K THP to ``always``, +set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M +to ``never``:: + + thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never + +``thp_anon=`` may be specified multiple times to configure all THP sizes as +required. If ``thp_anon=`` is specified at least once, any anon THP sizes +not explicitly configured on the command line are implicitly set to +``never``. + +``transparent_hugepage`` setting only affects the global toggle. If +``thp_anon`` is not specified, PMD_ORDER THP will default to ``inherit``. +However, if a valid ``thp_anon`` setting is provided by the user, the +PMD_ORDER THP policy will be overridden. If the policy for PMD_ORDER +is not defined within a valid ``thp_anon``, its policy will default to +``never``.
Hugepages in tmpfs/shmem
========================mm/huge_memory.c +1
-0@@ -81,6 +81,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; unsigned long huge_anon_orders_always __read_mostly; unsigned long huge_anon_orders_madvise __read_mostly; unsigned long huge_anon_orders_inherit __read_mostly; +static bool anon_orders_configured;
unsigned long __thp_vma_allowable_orders(structvm_area_struct*vma, unsigned longvm_flags,mm/huge_memory.c +4
-1@@ -737,7 +738,10 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)* disable all other sizes. powerpc's PMD_ORDER isn't a compile-time * constant so we have to do this here. */ - huge_anon_orders_inherit = BIT(PMD_ORDER); + if (!anon_orders_configured) { + huge_anon_orders_inherit = BIT(PMD_ORDER); + anon_orders_configured = true; + }
*hugepage_kobj=kobject_create_and_add("transparent_hugepage", mm_kobj); if (unlikely(!*hugepage_kobj)) {mm/huge_memory.c +0
-0@@ -922,6 +926,96 @@ static int __init setup_transparent_hugepage(char *str) } __setup("transparent_hugepage=", setup_transparent_hugepage);mm/huge_memory.c +2
-0+static inline int get_order_from_str (const char *size_str) +{
Quoted 8 lines
Quoted 5 lines
+ unsigned long size;
+ char *endptr;
+ int order;
+
+ size =memparse(size_str, &endptr);Do we have to also test if
is_power_of_2(), and refuse if not? For example, what if someone would pass 3K, would the existing check catch it?
no, the existing check can't catch it.
I passed thp_anon=15K-64K:always, then I got 16K enabled:
/ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled [always] inherit madvise never
I can actually check that by:
static inline int get_order_from_str(const char *size_str)
{
unsigned long size;
char *endptr;
int order;
size = memparse(size_str, &endptr);
if (!is_power_of_2(size >> PAGE_SHIFT))
goto err;
order = get_order(size);
if ((1 << order) & ~THP_ORDERS_ALL_ANON)
goto err;
return order;
err:
pr_err("invalid size %s in thp_anon boot parameter\n", size_str);
return -EINVAL;
}Quoted 8 lines
Quoted 2 lines
+ order =
fls(size >> PAGE_SHIFT)- 1;Is this a fancy way of writing
order =
log2(size >> PAGE_SHIFT);? :)
I think ilog2 is implemented by fls ?
Quoted 2 lines
Anyhow, if
get_order()wraps that, all good.
I guess it doesn't check power of 2?
Quoted 11 lines
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 944
Quoted 9 lines
+ if ((1 << order) & ~THP_ORDERS_ALL_ANON) {
+ pr_err("invalid size %s(order %d) inthp_anonboot parameter\n",
+size_str, order);
+ return-EINVAL;
+ }mm/huge_memory.c +3
-0+ + return order; +}Apart from that, nothing jumped at me.
Please take a look at the new get_order_from_str() before I send v5 :-)
Quoted 6 lines
--
Cheers,David / dhildenb
Thanks
Barry
Quoted 11 lines
Quoted 10 lines
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 935
Quoted 7 lines
mm/huge_memory.c +7
-0+static inline int get_order_from_str (const char *size_str) +{ + unsigned long size; + char *endptr; + int order; + + size = memparse(size_str, &endptr);Do we have to also test if
is_power_of_2(), and refuse if not? For example, what if someone would pass 3K, would the existing check catch it?
Quoted 7 lines
no, the existing check can't catch it.
I passed thp_anon=15K-64K:always, then I got 16K enabled:
/ # cat
/sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled[always] inherit madvise never
Okay, so we should document then that start/end of the range must be valid THP sizes.
Quoted 11 lines
I can actually check that by:
static inline int
get_order_from_str(const char *size_str){ unsigned long size; char *endptr; int order; size = memparse(size_str, &endptr); if (!is_power_of_2(size >> PAGE_SHIFT))
No need for the shift.
if (!is_power_of_2(size))
Is likely even more correct if someone would manage to pass something stupid like
16385 (16K + 1)
Quoted 14 lines
goto err; order = get_order(size); if ((1 << order) & ~THP_ORDERS_ALL_ANON) goto err; return order; err: pr_err("invalid size %s in thp_anon boot parameter\n", size_str); return -EINVAL; }Quoted 3 lines
Quoted 2 lines
+ order =
fls(size >> PAGE_SHIFT)- 1;
Quoted 7 lines
Quoted 5 lines
Is this a fancy way of writing
order =
log2(size >> PAGE_SHIFT);? :)
I think ilog2 is implemented by fls ?
Yes, so we should have used that instead. But get_order() is even better.
Quoted 18 lines
Quoted 3 lines
Anyhow, if
get_order()wraps that, all good.I guess it doesn't check power of 2?
Quoted 11 lines
mm/huge_memory.c static int __init setup_transparent_hugepage(char *str) · line 944
Quoted 9 lines
+ if ((1 << order) & ~THP_ORDERS_ALL_ANON) {
+ pr_err("invalid size %s(order %d) inthp_anonboot parameter\n",
+size_str, order);
+ return-EINVAL;
+ }mm/huge_memory.c +3
-0+ + return order; +}Apart from that, nothing jumped at me.
Quoted 2 lines
Please take a look at the new
get_order_from_str()before I send v5 :-)
Besides the shift for is_power_of_2(), LGTM, thanks!
Signature · 3 lines
Cheers, David / dhildenb
Quoted 14 lines · David Hildenbrand
Quoted 14 lines
Quoted 11 lines
Review
Quoted 8 lines
(quoted diff) +7
-0+static inline int get_order_from_str(const char *size_str) +{ + unsigned long size; + char *endptr; + int order; + + size = memparse(size_str, &endptr);Do we have to also test if
is_power_of_2(), and refuse if not? For example, what if someone would pass 3K, would the existing check catch it?no, the existing check can't catch it.
Quoted 8 lines
Quoted 5 lines
I passed thp_anon=15K-64K:always, then I got 16K enabled:
/ # cat
/sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled[always] inherit madvise neverOkay, so we should document then that start/end of the range must be valid THP sizes.
Ack
Quoted 13 lines
Quoted 9 lines
I can actually check that by:
static inline int
get_order_from_str(const char *size_str){
unsigned long size;
char *endptr;
int order;Quoted 3 lines
size =
memparse(size_str, &endptr);if (!is_power_of_2(size >>
PAGE_SHIFT))
Quoted 8 lines
No need for the shift.
if (!is_power_of_2(size))
Is likely even more correct if someone would manage to pass something stupid like
16385 (16K + 1)
Ack
Quoted 44 lines
Quoted 15 lines
goto err;
order =get_order(size);
if ((1 << order) & ~THP_ORDERS_ALL_ANON)
goto err;return order;
err:
pr_err("invalid size %s in thp_anon boot parameter\n", size_str); return-EINVAL;
}Quoted 3 lines
Quoted 2 lines
+ order =
fls(size >> PAGE_SHIFT)- 1;Quoted 7 lines
Quoted 5 lines
Is this a fancy way of writing
order =
log2(size >> PAGE_SHIFT);? :)
I think ilog2 is implemented by fls ?
Yes, so we should have used that instead. But
get_order()is even better.Quoted 18 lines
Quoted 3 lines
Anyhow, if
get_order()wraps that, all good.I guess it doesn't check power of 2?
Quoted 11 lines
Review
Quoted 9 lines
+ if ((1 << order) & ~THP_ORDERS_ALL_ANON) {
+ pr_err("invalid size %s(order %d) inthp_anonboot parameter\n",
+size_str, order);
+ return-EINVAL;
+ }(quoted diff) +3
-0+ + return order; +}Apart from that, nothing jumped at me.
Quoted 4 lines
Quoted 2 lines
Please take a look at the new
get_order_from_str()before I send v5 :-)Besides the shift for
is_power_of_2(), LGTM, thanks!
Thanks, David!
Hi Andrew,
Apologies for sending another squash request. If you'd prefer me to send a new v5 that includes all the changes, please let me know.
Don't shift the size, as it can still detect invalid sizes like 16K+1. Also, document that the size must be a valid THP size.
Documentation/admin-guide/mm/transhuge.rst +3 -2
@@ -294,8 +294,9 @@ kernel command line. Alternatively, each supported anonymous THP size can be controlled by passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, -where ``<size>`` is the THP size and ``<state>`` is one of ``always``, -``madvise``, ``never`` or ``inherit``. +where ``<size>`` is the THP size (must be a power of 2 of PAGE_SIZE and +supported anonymous THP) and ``<state>`` is one of ``always``, ``madvise``, +``never`` or ``inherit``. For example, the following will set 16K, 32K, 64K THP to ``always``, set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M
mm/huge_memory.c +1 -1
@@ -953,7 +953,7 @@ static inline int get_order_from_str(const char *size_str) size =memparse (size_str, &endptr); -if (!is_power_of_2 (size >>PAGE_SHIFT )) +if (!is_power_of_2 (size)) goto err; order =get_order (size);if ((1 << order) & ~THP_ORDERS_ALL_ANON )
Quoted 6 lines
--
Cheers,David / dhildenb
Thanks
Barry
Quoted 133 lines · Barry Song
Quoted 14 lines · David Hildenbrand
Quoted 14 lines
Quoted 11 lines
Review
Quoted 8 lines
(quoted diff) +7
-0+static inline int get_order_from_str(const char *size_str) +{ + unsigned long size; + char *endptr; + int order; + + size = memparse(size_str, &endptr);Do we have to also test if
is_power_of_2(), and refuse if not? For example, what if someone would pass 3K, would the existing check catch it?no, the existing check can't catch it.
Quoted 8 lines
Quoted 5 lines
I passed thp_anon=15K-64K:always, then I got 16K enabled:
/ # cat
/sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled[always] inherit madvise neverOkay, so we should document then that start/end of the range must be valid THP sizes.
Ack
Quoted 13 lines
Quoted 9 lines
I can actually check that by:
static inline int
get_order_from_str(const char *size_str){
unsigned long size;
char *endptr;
int order;Quoted 3 lines
size =
memparse(size_str, &endptr);if (!is_power_of_2(size >>
PAGE_SHIFT))Quoted 8 lines
No need for the shift.
if (!is_power_of_2(size))
Is likely even more correct if someone would manage to pass something stupid like
16385 (16K + 1)
Ack
Quoted 44 lines
Quoted 15 lines
goto err;
order =get_order(size);
if ((1 << order) & ~THP_ORDERS_ALL_ANON)
goto err;return order;
err:
pr_err("invalid size %s in thp_anon boot parameter\n", size_str); return-EINVAL;
}Quoted 3 lines
Quoted 2 lines
+ order =
fls(size >> PAGE_SHIFT)- 1;Quoted 7 lines
Quoted 5 lines
Is this a fancy way of writing
order =
log2(size >> PAGE_SHIFT);? :)
I think ilog2 is implemented by fls ?
Yes, so we should have used that instead. But
get_order()is even better.Quoted 18 lines
Quoted 3 lines
Anyhow, if
get_order()wraps that, all good.I guess it doesn't check power of 2?
Quoted 11 lines
Review
Quoted 9 lines
+ if ((1 << order) & ~THP_ORDERS_ALL_ANON) {
+ pr_err("invalid size %s(order %d) inthp_anonboot parameter\n",
+size_str, order);
+ return-EINVAL;
+ }(quoted diff) +3
-0+ + return order; +}Apart from that, nothing jumped at me.
Quoted 4 lines
Quoted 2 lines
Please take a look at the new
get_order_from_str()before I send v5 :-)Besides the shift for
is_power_of_2(), LGTM, thanks!Thanks, David!
Hi Andrew,
Apologies for sending another squash request. If you'd prefer me to send a new v5 that includes all the changes, please let me know.
Don't shift the size, as it can still detect invalid sizes like 16K+1. Also, document that the size must be a valid THP size.
Documentation/admin-guide/mm/transhuge.rst +3
-2@@ -294,8 +294,9 @@ kernel command line. Alternatively, each supported anonymous THP size can be controlled by passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, -where ``<size>`` is the THP size and ``<state>`` is one of ``always``, -``madvise``, ``never`` or ``inherit``. +where ``<size>`` is the THP size (must be a power of 2 of PAGE_SIZE and +supported anonymous THP) and ``<state>`` is one of ``always``, ``madvise``, +``never`` or ``inherit``. For example, the following will set 16K, 32K, 64K THP to ``always``, set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2Mmm/huge_memory.c +0
-0
mm/huge_memory.c static inline int get_order_from_str(const char *size_str) · line 957
Quoted 6 lines
mm/huge_memory.c +1
-1@@ -953,7 +953,7 @@ static inline int get_order_from_str(const char *size_str) size =memparse (size_str, &endptr); -if (!is_power_of_2 (size >>PAGE_SHIFT )) +if (!is_power_of_2 (size)) goto err;
Reading your documentation above, do we also want to test "if (size < PAGE_SIZE)", or is that implicitly covered? (likely not I assume?)
I assume it's implicitly covered: if we pass "1k" , it would be mapped to "4k" (order-0) and that is not a valid mTHP size, right?
I would appreciate a quick v5, just so can see the final result more easily :)
Signature · 3 lines
Cheers, David / dhildenb
Quoted 138 lines · David Hildenbrand
Quoted 136 lines · Barry Song
Quoted 14 lines · David Hildenbrand
Quoted 14 lines
Quoted 11 lines
Review
Quoted 8 lines
(quoted diff) +7
-0+static inline int get_order_from_str(const char *size_str) +{ + unsigned long size; + char *endptr; + int order; + + size = memparse(size_str, &endptr);Do we have to also test if
is_power_of_2(), and refuse if not? For example, what if someone would pass 3K, would the existing check catch it?no, the existing check can't catch it.
Quoted 8 lines
Quoted 5 lines
I passed thp_anon=15K-64K:always, then I got 16K enabled:
/ # cat
/sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled[always] inherit madvise neverOkay, so we should document then that start/end of the range must be valid THP sizes.
Ack
Quoted 13 lines
Quoted 4 lines
I can actually check that by:
static inline int
get_order_from_str(const char *size_str)Quoted 8 lines
{ unsigned long size; char *endptr; int order; size = memparse(size_str, &endptr); if (!is_power_of_2(size >> PAGE_SHIFT))Quoted 8 lines
No need for the shift.
if (!is_power_of_2(size))
Is likely even more correct if someone would manage to pass something stupid like
16385 (16K + 1)
Ack
Quoted 44 lines
Quoted 15 lines
goto err; order = get_order(size); if ((1 << order) & ~THP_ORDERS_ALL_ANON) goto err; return order; err: pr_err("invalid size %s in thp_anon boot parameter\n", size_str); return -EINVAL; }Quoted 3 lines
Quoted 2 lines
+ order =
fls(size >> PAGE_SHIFT)- 1;Quoted 7 lines
Quoted 5 lines
Is this a fancy way of writing
order =
log2(size >> PAGE_SHIFT);? :)
I think ilog2 is implemented by fls ?
Yes, so we should have used that instead. But
get_order()is even better.Quoted 18 lines
Quoted 3 lines
Anyhow, if
get_order()wraps that, all good.I guess it doesn't check power of 2?
Quoted 11 lines
Review
Quoted 9 lines
(quoted diff) +8
-0+ if ((1 << order) & ~THP_ORDERS_ALL_ANON) { + pr_err("invalid size %s(order %d) in thp_anon boot parameter\n", + size_str, order); + return -EINVAL; + } + + return order; +}Apart from that, nothing jumped at me.
Quoted 4 lines
Quoted 2 lines
Please take a look at the new
get_order_from_str()before I send v5 :-)Besides the shift for
is_power_of_2(), LGTM, thanks!Thanks, David!
Hi Andrew,
Apologies for sending another squash request. If you'd prefer me to send a new v5 that includes all the changes, please let me know.
Don't shift the size, as it can still detect invalid sizes like 16K+1. Also, document that the size must be a valid THP size.
Documentation/admin-guide/mm/transhuge.rst +3
-2@@ -294,8 +294,9 @@ kernel command line. Alternatively, each supported anonymous THP size can be controlled by passing ``thp_anon=<size>,<size>[KMG]:<state>;<size>-<size>[KMG]:<state>``, -where ``<size>`` is the THP size and ``<state>`` is one of ``always``, -``madvise``, ``never`` or ``inherit``. +where ``<size>`` is the THP size (must be a power of 2 of PAGE_SIZE and +supported anonymous THP) and ``<state>`` is one of ``always``, ``madvise``, +``never`` or ``inherit``. For example, the following will set 16K, 32K, 64K THP to ``always``, set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2Mmm/huge_memory.c +0
-0@@ -953,7 +953,7 @@ static inline int get_order_from_str(const char *size_str) size =memparse (size_str, &endptr);
Quoted 7 lines
mm/huge_memory.c static inline int get_order_from_str(const char *size_str) · line 957
Quoted 3 lines
mm/huge_memory.c +1
-1-if (!is_power_of_2 (size >>PAGE_SHIFT )) +if (!is_power_of_2 (size)) goto err;Reading your documentation above, do we also want to test "if (size <
PAGE_SIZE)", or is that implicitly covered? (likely not I assume?)
as we also check the order is valid. so size <PAGE_SIZE will get invalid order.
static inline int get_order_from_str(const char *size_str)
{
unsigned long size;
char *endptr;
int order;
size = memparse(size_str, &endptr);
if (!is_power_of_2(size >> PAGE_SHIFT))
goto err;
order = get_order(size);
if ((1 << order) & ~THP_ORDERS_ALL_ANON)
goto err;
return order;err:
pr_err("invalid size %s in thp_anon boot parameter\n", size_str);
return -EINVAL;
}Quoted 6 lines
I assume it's implicitly covered: if we pass "1k" , it would be mapped to "4k" (order-0) and that is not a valid mTHP size, right?
I would appreciate a quick v5, just so can see the final result more easily :)
sure.
Quoted 6 lines
--
Cheers,David / dhildenb
Quoted 16 lines · Barry Song
From: Ryan Roberts <[email protected]>
Add thp_anon= cmdline parameter to allow specifying the default enablement of each supported anon THP size. The parameter accepts the following format and can be provided multiple times to configure each size:
thp_anon=<size>,<size>[KMG]:<value>;<size>-<size>[KMG]:<value>
An example:
thp_anon=16K-64K:always;128K,512K:inherit;256K:madvise;1M-2M:never
See
Documentation/admin-guide/mm/transhuge.rstfor more details.
Quoted 8 lines
Configuring the defaults at boot time is useful to allow early user space to take advantage of mTHP before its been configured through sysfs.
Signed-off-by Ryan Roberts <[email protected]>Co-developed-by Barry Song <[email protected]>Signed-off-by Barry Song <[email protected]>---
Hi Andrew,
I saw you have pulled v4. Thanks!
Can you please squash the below changes suggested by Baolin and David?
From af42aa80f45d89798027e44a8711f7737e08b115 Mon Sep 17 00:00:00 2001
From: Barry Song <[email protected]>
Date: Thu, 15 Aug 2024 10:34:16 +1200
Subject: [PATCH] mm: use get_oder() and check size is is_power_of_2
Using get_order() is more robust according to Baolin. It is also better to filter illegal size such as 3KB, 16KB according to David.
--- mm/huge_memory.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
mm/huge_memory.c +9 -6
@@ -952,14 +952,17 @@ static inline int get_order_from_str(const char *size_str)int order; size =memparse (size_str, &endptr); - order =fls (size >>PAGE_SHIFT ) -1 ; -if ((1 << order) & ~THP_ORDERS_ALL_ANON ) { -pr_err ("invalid size %s(order %d) in thp_anon boot parameter\n" , - size_str, order); -return -EINVAL ; - } + +if (!is_power_of_2 (size >>PAGE_SHIFT )) + goto err; + order =get_order (size); +if ((1 << order) & ~THP_ORDERS_ALL_ANON ) + goto err;return order; +err: +pr_err ("invalid size %s in thp_anon boot parameter\n" , size_str); +return -EINVAL ; }static char str_dup[PAGE_SIZE ]__meminitdata ;
Signature · 4 lines
2.34.1 Thanks Barry