mm: Add CONFIG_PAGE_BLOCK_ORDER to select page block order
Subsystems (3)
PATCH v3 1/1 mm: Add CONFIG_PAGE_BLOCK_ORDER to select page block order
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less available MIGRATE_UNMOVABLE and MIGRATE_RECLAIMABLE page blocks since MIGRATE_CMA can't fallback to them.
The CMA_MIN_ALIGNMENT_BYTES increases because it depends on MAX_PAGE_ORDER which depends on ARCH_FORCE_MAX_ORDER. The value of ARCH_FORCE_MAX_ORDER increases on 16k and 64k kernels.
For example, in ARM, the CMA alignment requirement when:
- CONFIG_ARCH_FORCE_MAX_ORDER default value is used
- CONFIG_TRANSPARENT_HUGEPAGE is set:
PAGE_SIZE | MAX_PAGE_ORDER | pageblock_order | CMA_MIN_ALIGNMENT_BYTES
-----------------------------------------------------------------------
4KiB | 10 | 10 | 4KiB * (2 ^ 10) = 4MiB 16Kib | 11 | 11 | 16KiB * (2 ^ 11) = 32MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiB
There are some extreme cases for the CMA alignment requirement when:
- CONFIG_ARCH_FORCE_MAX_ORDER maximum value is set
- CONFIG_TRANSPARENT_HUGEPAGE is NOT set:
- CONFIG_HUGETLB_PAGE is NOT set
PAGE_SIZE | MAX_PAGE_ORDER | pageblock_order | CMA_MIN_ALIGNMENT_BYTES
------------------------------------------------------------------------
4KiB | 15 | 15 | 4KiB * (2 ^ 15) = 128MiB 16Kib | 13 | 13 | 16KiB * (2 ^ 13) = 128MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiB
This affects the CMA reservations for the drivers. If a driver in a 4KiB kernel needs 4MiB of CMA memory, in a 16KiB kernel, the minimal reservation has to be 32MiB due to the alignment requirements:
reserved-memory {
...
cma_test_reserve: cma_test_reserve {
compatible = "shared-dma-pool";
size = <0x0 0x400000>; /* 4 MiB */
...
};
};reserved-memory {
...
cma_test_reserve: cma_test_reserve {
compatible = "shared-dma-pool";
size = <0x0 0x2000000>; /* 32 MiB */
...
};
};Solution: Add a new config CONFIG_PAGE_BLOCK_ORDER that allows to set the page block order in all the architectures. The maximum page block order will be given by ARCH_FORCE_MAX_ORDER.
By default, CONFIG_PAGE_BLOCK_ORDER will have the same value that ARCH_FORCE_MAX_ORDER. This will make sure that current kernel configurations won't be affected by this change. It is a opt-in change.
This patch will allow to have the same CMA alignment requirements for large page sizes (16KiB, 64KiB) as that in 4kb kernels by setting a lower pageblock_order.
Tests:
- Verified that HugeTLB pages work when pageblock_order is 1, 7, 10 on 4k and 16k kernels.
- Verified that Transparent Huge Pages work when pageblock_order is 1, 7, 10 on 4k and 16k kernels.
- Verified that dma-buf heaps allocations work when pageblock_order is 1, 7, 10 on 4k and 16k kernels.
Benchmarks:
The benchmarks compare 16kb kernels with pageblock_order 10 and 7. The reason for the pageblock_order 7 is because this value makes the min CMA alignment requirement the same as that in 4kb kernels (2MB).
- Perform 100K dma-buf heaps (/dev/dma_heap/system) allocations of SZ_8M, SZ_4M, SZ_2M, SZ_1M, SZ_64, SZ_8, SZ_4. Use simpleperf
(https://developer.android.com/ndk/guides/simpleperf) to measure the # of instructions and page-faults on 16k kernels. The benchmark was executed 10 times. The averages are below:
# instructions | #page-faults
order 10 | order 7 | order 10 | order 7
--------------------------------------------------------
13,891,765,770 | 11,425,777,314 | 220 | 217
14,456,293,487 | 12,660,819,302 | 224 | 219
13,924,261,018 | 13,243,970,736 | 217 | 221
13,910,886,504 | 13,845,519,630 | 217 | 221
14,388,071,190 | 13,498,583,098 | 223 | 224
13,656,442,167 | 12,915,831,681 | 216 | 218
13,300,268,343 | 12,930,484,776 | 222 | 218
13,625,470,223 | 14,234,092,777 | 219 | 218
13,508,964,965 | 13,432,689,094 | 225 | 219
13,368,950,667 | 13,683,587,37 | 219 | 225
-------------------------------------------------------------------
13,803,137,433 | 13,131,974,268 | 220 | 220 AveragesThere were 4.85% #instructions when order was 7, in comparison with order 10.
13,803,137,433 - 13,131,974,268 = -671,163,166 (-4.86%)
The number of page faults in order 7 and 10 were the same.
These results didn't show any significant regression when the pageblock_order is set to 7 on 16kb kernels.
- Run speedometer 3.1 (https://browserbench.org/Speedometer3.1/) 5 times on the 16k kernels with pageblock_order 7 and 10.
order 10 | order 7 | order 7 - order 10 | (order 7 - order 10) %
-------------------------------------------------------------------
15.8 | 16.4 | 0.6 | 3.80% 16.4 | 16.2 | -0.2 | -1.22% 16.6 | 16.3 | -0.3 | -1.81% 16.8 | 16.3 | -0.5 | -2.98% 16.6 | 16.8 | 0.2 | 1.20%
-------------------------------------------------------------------
16.44 16.4 -0.04 -0.24% Averages
The results didn't show any significant regression when the pageblock_order is set to 7 on 16kb kernels.
---
Changes in v3:
- Rename ARCH_FORCE_PAGE_BLOCK_ORDER to PAGE_BLOCK_ORDER as per Matthew's suggestion.
- Update comments in pageblock-flags.h for pageblock_order value when THP or HugeTLB are not used.
Changes in v2:
- Add Zi's Acked-by tag.
- Move ARCH_FORCE_PAGE_BLOCK_ORDER config to mm/Kconfig as per Zi and Matthew suggestion so it is available to all the architectures.
- Set ARCH_FORCE_PAGE_BLOCK_ORDER to 10 by default when ARCH_FORCE_MAX_ORDER is not available.
include/linux/pageblock-flags.h | 14 ++++++++++---- mm/Kconfig | 31 +++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/pageblock-flags.h +10 -4
@@ -28,6 +28,12 @@ enum pageblock_bits { NR_PAGEBLOCK_BITS }; +#if defined(CONFIG_PAGE_BLOCK_ORDER ) +#define PAGE_BLOCK_ORDER CONFIG_PAGE_BLOCK_ORDER +#else +#define PAGE_BLOCK_ORDER MAX_PAGE_ORDER +#endif /* CONFIG_PAGE_BLOCK_ORDER */ +#if defined(CONFIG_HUGETLB_PAGE )#ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE
@@ -41,18 +47,18 @@ extern unsigned int pageblock_order;* Huge pages are a constant size, but don't exceed the maximum allocation * granularity. */ -#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, PAGE_BLOCK_ORDER)#endif /* CONFIG_HUGETLB_PAGE_SIZE_VARIABLE */ #elif defined(CONFIG_TRANSPARENT_HUGEPAGE ) -#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, PAGE_BLOCK_ORDER)#else /* CONFIG_TRANSPARENT_HUGEPAGE */ -/* If huge pages are not used, group by MAX_ORDER_NR_PAGES */ -#define pageblock_order MAX_PAGE_ORDER +/* If huge pages are not used, group by PAGE_BLOCK_ORDER */ +#define pageblock_order PAGE_BLOCK_ORDER#endif /* CONFIG_HUGETLB_PAGE */
mm/Kconfig +31 -0
@@ -989,6 +989,37 @@ config CMA_AREAS If unsure, leave the default value "8" in UMA and "20" in NUMA. +# +# Select this config option from the architecture Kconfig, if available, to set +# the max page order for physically contiguous allocations. +# +config ARCH_FORCE_MAX_ORDER + int + +# When ARCH_FORCE_MAX_ORDER is not defined, the default page block order is 10, +# as per include/linux/mmzone.h. +config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER + default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + + help + The page block order refers to the power of two number of pages that + are physically contiguous and can have a migrate type associated to + them. The maximum size of the page block order is limited by + ARCH_FORCE_MAX_ORDER. + + This option allows overriding the default setting when the page + block order requires to be smaller than ARCH_FORCE_MAX_ORDER. + + Reducing pageblock order can negatively impact THP generation + successful rate. If your workloads uses THP heavily, please use this + option with caution. + + Don't change if unsure. + config MEM_SOFT_DIRTY bool "Track memory changes" depends on CHECKPOINT_RESTORE && HAVE_ARCH_SOFT_DIRTY && PROC_FS
Signature · 1 line
2.49.0.967.g6a0df3ecc3-goog
Andrew Morton
maintainerQuoted 5 lines · On Mon, 5 May 2025 17:22:58 -0700 Juan Yescas <[email protected]> wrote:
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (
CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less availableMIGRATE_UNMOVABLEandMIGRATE_RECLAIMABLEpage blocks sinceMIGRATE_CMAcan't fallback to them.
Thanks, I'll add this for testing while we consider the proposal.
Quoted 13 lines
(quoted diff) +13
-0+# as per include/linux/mmzone.h. +config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER + default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + + help + The page block order refers to the power of two number of pages that + are physically contiguous and can have a migrate type associated to + them. The maximum size of the page block order is limited by + ARCH_FORCE_MAX_ORDER.
Review
Quoted 10 lines
(quoted diff) +10
-0+ + This option allows overriding the default setting when the page + block order requires to be smaller than ARCH_FORCE_MAX_ORDER. + + Reducing pageblock order can negatively impact THP generation + successful rate. If your workloads uses THP heavily, please use this + option with caution. + + Don't change if unsure. +
I messed with the text a little.
mm/Kconfig renamed from mm/Kconfig~mm-add-config_page_block_order-to-select-page-block-order-fix +2 -2
@@ -1028,10 +1028,10 @@ config PAGE_BLOCK_ORDER ARCH_FORCE_MAX_ORDER. This option allows overriding the default setting when the page - block order requires to be smaller than ARCH_FORCE_MAX_ORDER. + block order is required to be smaller than ARCH_FORCE_MAX_ORDER. Reducing pageblock order can negatively impact THP generation - successful rate. If your workloads uses THP heavily, please use this + success rate. If your workloads uses THP heavily, please use this option with caution. Don't change if unsure.
_
Quoted 46 lines · Juan Yescas
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (
CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less availableMIGRATE_UNMOVABLEandMIGRATE_RECLAIMABLEpage blocks sinceMIGRATE_CMAcan't fallback to them.The
CMA_MIN_ALIGNMENT_BYTESincreases because it depends onMAX_PAGE_ORDERwhich depends onARCH_FORCE_MAX_ORDER. The value ofARCH_FORCE_MAX_ORDERincreases on 16k and 64k kernels.For example, in ARM, the CMA alignment requirement when:
-
CONFIG_ARCH_FORCE_MAX_ORDERdefault value is used
-CONFIG_TRANSPARENT_HUGEPAGEis set:
PAGE_SIZE|MAX_PAGE_ORDER|pageblock_order|CMA_MIN_ALIGNMENT_BYTES
-----------------------------------------------------------------------4KiB | 10 | 10 | 4KiB * (2 ^ 10) = 4MiB 16Kib | 11 | 11 | 16KiB * (2 ^ 11) = 32MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiBThere are some extreme cases for the CMA alignment requirement when:
-
CONFIG_ARCH_FORCE_MAX_ORDERmaximum value is set
-CONFIG_TRANSPARENT_HUGEPAGEis NOT set:
-CONFIG_HUGETLB_PAGEis NOT set
PAGE_SIZE|MAX_PAGE_ORDER|pageblock_order|CMA_MIN_ALIGNMENT_BYTES
------------------------------------------------------------------------4KiB | 15 | 15 | 4KiB * (2 ^ 15) = 128MiB 16Kib | 13 | 13 | 16KiB * (2 ^ 13) = 128MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiBThis affects the CMA reservations for the drivers. If a driver in a 4KiB kernel needs 4MiB of CMA memory, in a 16KiB kernel, the minimal reservation has to be 32MiB due to the alignment requirements:
reserved-memory { ... cma_test_reserve: cma_test_reserve { compatible = "shared-dma-pool"; size = <0x0 0x400000>; /* 4 MiB */ ... }; };
Quoted 8 lines
reserved-memory {
...
cma_test_reserve: cma_test_reserve {
compatible = "shared-dma-pool";
size = <0x0 0x2000000>; /* 32 MiB */
...
};
};This indeed is a valid problem which reduces available memory for non-CMA page blocks on system required for general memory usage.
Quoted 10 lines
Solution: Add a new config
CONFIG_PAGE_BLOCK_ORDERthat allows to set the page block order in all the architectures. The maximum page block order will be given byARCH_FORCE_MAX_ORDER.By default,
CONFIG_PAGE_BLOCK_ORDERwill have the same value thatARCH_FORCE_MAX_ORDER. This will make sure that current kernel configurations won't be affected by this change. It is a opt-in change.
Right.
Quoted 7 lines
This patch will allow to have the same CMA alignment requirements for large page sizes (16KiB, 64KiB) as that in 4kb kernels by setting a lower
pageblock_order.Tests:
Quoted 8 lines
- Verified that HugeTLB pages work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.- Verified that Transparent Huge Pages work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.- Verified that dma-buf heaps allocations work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.
pageblock_order are choosen as 1, 7 and 10 to cover the entire possible range for ARCH_FORCE_MAX_ORDER. Although kernel CI should test this for all values in the range. Because this now opens up different ranges for different platforms which were never tested earlier.
Quoted 118 lines
Benchmarks:
The benchmarks compare 16kb kernels with
pageblock_order10 and 7. The reason for thepageblock_order7 is because this value makes the min CMA alignment requirement the same as that in 4kb kernels (2MB).- Perform 100K dma-buf heaps (
/dev/dma_heap/system) allocations ofSZ_8M,SZ_4M,SZ_2M,SZ_1M,SZ_64,SZ_8,SZ_4. Use simpleperf
(https://developer.android.com/ndk/guides/simpleperf) to measure the # of instructions and page-faults on 16k kernels. The benchmark was executed 10 times. The averages are below:# instructions | #page-faults order 10 | order 7 | order 10 | order 7 -------------------------------------------------------- 13,891,765,770 | 11,425,777,314 | 220 | 217 14,456,293,487 | 12,660,819,302 | 224 | 219 13,924,261,018 | 13,243,970,736 | 217 | 221 13,910,886,504 | 13,845,519,630 | 217 | 221 14,388,071,190 | 13,498,583,098 | 223 | 224 13,656,442,167 | 12,915,831,681 | 216 | 218 13,300,268,343 | 12,930,484,776 | 222 | 218 13,625,470,223 | 14,234,092,777 | 219 | 218 13,508,964,965 | 13,432,689,094 | 225 | 219 13,368,950,667 | 13,683,587,37 | 219 | 225 ------------------------------------------------------------------- 13,803,137,433 | 13,131,974,268 | 220 | 220 AveragesThere were 4.85% #instructions when order was 7, in comparison with order 10.
13,803,137,433 - 13,131,974,268 = -671,163,166 (-4.86%)The number of page faults in order 7 and 10 were the same.
These results didn't show any significant regression when the
pageblock_orderis set to 7 on 16kb kernels.- Run speedometer 3.1 (https://browserbench.org/Speedometer3.1/) 5 times on the 16k kernels with
pageblock_order7 and 10.order 10 | order 7 | order 7 - order 10 | (order 7 - order 10) %
-------------------------------------------------------------------15.8 | 16.4 | 0.6 | 3.80% 16.4 | 16.2 | -0.2 | -1.22% 16.6 | 16.3 | -0.3 | -1.81% 16.8 | 16.3 | -0.5 | -2.98% 16.6 | 16.8 | 0.2 | 1.20%-------------------------------------------------------------------
16.44 16.4 -0.04 -0.24% AveragesThe results didn't show any significant regression when the
pageblock_orderis set to 7 on 16kb kernels.Cc Andrew Morton <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Liam R. Howlett <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>CC Mike Rapoport <[email protected]>Cc Zi Yan <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Minchan Kim <[email protected]>Signed-off-by Juan Yescas <[email protected]>Acked-by Zi Yan <[email protected]>---
Changes in v3:
- RenameARCH_FORCE_PAGE_BLOCK_ORDERtoPAGE_BLOCK_ORDERas per Matthew's suggestion.
- Update comments inpageblock-flags.hforpageblock_ordervalue when THP or HugeTLB are not used.Changes in v2:
- Add Zi's Acked-by tag.
- MoveARCH_FORCE_PAGE_BLOCK_ORDERconfig to mm/Kconfig as per Zi and Matthew suggestion so it is available to all the architectures.
- SetARCH_FORCE_PAGE_BLOCK_ORDERto 10 by default whenARCH_FORCE_MAX_ORDERis not available.include/linux/pageblock-flags.h | 14 ++++++++++---- mm/Kconfig | 31 +++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 4 deletions(-)include/linux/pageblock-flags.h +8
-2@@ -28,6 +28,12 @@ enum pageblock_bits { NR_PAGEBLOCK_BITS }; +#if defined(CONFIG_PAGE_BLOCK_ORDER ) +#define PAGE_BLOCK_ORDER CONFIG_PAGE_BLOCK_ORDER +#else +#define PAGE_BLOCK_ORDER MAX_PAGE_ORDER +#endif /* CONFIG_PAGE_BLOCK_ORDER */ +#if defined(CONFIG_HUGETLB_PAGE )#ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -41,18 +47,18 @@ extern unsigned int pageblock_order;* Huge pages are a constant size, but don't exceed the maximum allocation * granularity. */ -#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, PAGE_BLOCK_ORDER)#endif /* CONFIG_HUGETLB_PAGE_SIZE_VARIABLE */ #elif defined(CONFIG_TRANSPARENT_HUGEPAGE) -#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, PAGE_BLOCK_ORDER)
include/linux/pageblock-flags.h extern unsigned int pageblock_order; · line 64
Quoted 10 lines
include/linux/pageblock-flags.h +2
-2#else /* CONFIG_TRANSPARENT_HUGEPAGE */ -/* If huge pages are not used, group by MAX_ORDER_NR_PAGES */ -#define pageblock_order MAX_PAGE_ORDER +/* If huge pages are not used, group by PAGE_BLOCK_ORDER */ +#define pageblock_order PAGE_BLOCK_ORDER#endif /* CONFIG_HUGETLB_PAGE */
These all look good.
Quoted 5 lines
mm/Kconfig +0
-0
mm/Kconfig config CMA_AREAS · line 997
Quoted 9 lines
mm/Kconfig +6
-0@@ -989,6 +989,37 @@ config CMA_AREAS If unsure, leave the default value "8" in UMA and "20" in NUMA. +# +# Select this config option from the architecture Kconfig, if available, to set +# the max page order for physically contiguous allocations. +# +config ARCH_FORCE_MAX_ORDER + int
ARCH_FORCE_MAX_ORDER needs to be defined here first before PAGE_BLOCK_ORDER could use that subsequently.But ARCH_FORCE_MAX_ORDER is defined in various architectures in 'int' or 'int "<description>"' formats. So could there be a problem for this config to be defined both in generic and platform code ? But clearly ARCH_FORCE_MAX_ORDER still remains a arch specific config.
#git grep "config ARCH_FORCE_MAX_ORDER" arch/arc/Kconfig:config ARCH_FORCE_MAX_ORDER arch/arm/Kconfig:config ARCH_FORCE_MAX_ORDER arch/arm64/Kconfig:config ARCH_FORCE_MAX_ORDER arch/loongarch/Kconfig:config ARCH_FORCE_MAX_ORDER arch/m68k/Kconfig.cpu:config ARCH_FORCE_MAX_ORDER arch/mips/Kconfig:config ARCH_FORCE_MAX_ORDER arch/nios2/Kconfig:config ARCH_FORCE_MAX_ORDER arch/powerpc/Kconfig:config ARCH_FORCE_MAX_ORDER arch/sh/mm/Kconfig:config ARCH_FORCE_MAX_ORDER arch/sparc/Kconfig:config ARCH_FORCE_MAX_ORDER arch/xtensa/Kconfig:config ARCH_FORCE_MAX_ORDER mm/Kconfig:config ARCH_FORCE_MAX_ORDER
arch/arc/
config ARCH_FORCE_MAX_ORDER
int "Maximum zone order"
arch/arm/
config ARCH_FORCE_MAX_ORDER
int "Order of maximal physically contiguous allocations"
arch/arm64/
config ARCH_FORCE_MAX_ORDER
int
...........
arch/sparc/
config ARCH_FORCE_MAX_ORDER
int "Order of maximal physically contiguous allocations"
mm/Kconfig config CMA_AREAS · line 999
Quoted 2 lines
mm/Kconfig +2
-0+ +# When ARCH_FORCE_MAX_ORDER is not defined, the default page block order is 10,
Just wondering - why the default is 10 ?
mm/Kconfig config CMA_AREAS · line 1003
Quoted 4 lines
mm/Kconfig +4
-0+# as per include/linux/mmzone.h. +config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER
Also why the range is restricted to 10 ?
mm/Kconfig config CMA_AREAS · line 1006
Quoted 3 lines
mm/Kconfig +3
-0+ default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER
We still have the MAX_PAGE_ORDER which maps into ARCH_FORCE_MAX_ORDER when available or otherwise just falls back as 10.
/* Free memory management - zoned buddy allocator. */ #ifndef CONFIG_ARCH_FORCE_MAX_ORDER
#define MAX_PAGE_ORDER 10
#else
#define MAX_PAGE_ORDER CONFIG_ARCH_FORCE_MAX_ORDER #endif
Hence could PAGE_BLOCK_ORDER config description block be simplified as
config PAGE_BLOCK_ORDER
int "Page Block Order"
range 1 MAX_PAGE_ORDER
default MAX_PAGE_ORDER
As MAX_PAGE_ORDER could switch between ARCH_FORCE_MAX_ORDER and 10 as and when required.
mm/Kconfig config CMA_AREAS · line 1012
Quoted 6 lines
mm/Kconfig +6
-0+ + help + The page block order refers to the power of two number of pages that + are physically contiguous and can have a migrate type associated to + them. The maximum size of the page block order is limited by + ARCH_FORCE_MAX_ORDER.
s/ARCH_FORCE_MAX_ORDER/ARCH_FORCE_MAX_ORDER when available on the platform/ ?
Also mention about max range when ARCH_FORCE_MAX_ORDER is not available.
mm/Kconfig config CMA_AREAS · line 1019
Quoted 7 lines
mm/Kconfig +7
-0+ + This option allows overriding the default setting when the page + block order requires to be smaller than ARCH_FORCE_MAX_ORDER. + + Reducing pageblock order can negatively impact THP generation + successful rate. If your workloads uses THP heavily, please use this + option with caution.
Just wondering - could there be any other side effects besides THP ? Will it be better to depend on CONFIG_EXPERT while selecting anything other than the default option i.e ARCH_FORCE_MAX_ORDER or 10 from the value range.
Quoted 6 lines
mm/Kconfig +3
-0+ + Don't change if unsure. + config MEM_SOFT_DIRTY bool "Track memory changes" depends on CHECKPOINT_RESTORE && HAVE_ARCH_SOFT_DIRTY && PROC_FS
Zi Yan
reviewerQuoted 277 lines · Anshuman Khandual
Quoted 46 lines · Juan Yescas
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (
CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less availableMIGRATE_UNMOVABLEandMIGRATE_RECLAIMABLEpage blocks sinceMIGRATE_CMAcan't fallback to them.The
CMA_MIN_ALIGNMENT_BYTESincreases because it depends onMAX_PAGE_ORDERwhich depends onARCH_FORCE_MAX_ORDER. The value ofARCH_FORCE_MAX_ORDERincreases on 16k and 64k kernels.For example, in ARM, the CMA alignment requirement when:
-
CONFIG_ARCH_FORCE_MAX_ORDERdefault value is used
-CONFIG_TRANSPARENT_HUGEPAGEis set:
PAGE_SIZE|MAX_PAGE_ORDER|pageblock_order|CMA_MIN_ALIGNMENT_BYTES
-----------------------------------------------------------------------4KiB | 10 | 10 | 4KiB * (2 ^ 10) = 4MiB 16Kib | 11 | 11 | 16KiB * (2 ^ 11) = 32MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiBThere are some extreme cases for the CMA alignment requirement when:
-
CONFIG_ARCH_FORCE_MAX_ORDERmaximum value is set
-CONFIG_TRANSPARENT_HUGEPAGEis NOT set:
-CONFIG_HUGETLB_PAGEis NOT set
PAGE_SIZE|MAX_PAGE_ORDER|pageblock_order|CMA_MIN_ALIGNMENT_BYTES
------------------------------------------------------------------------4KiB | 15 | 15 | 4KiB * (2 ^ 15) = 128MiB 16Kib | 13 | 13 | 16KiB * (2 ^ 13) = 128MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiBThis affects the CMA reservations for the drivers. If a driver in a 4KiB kernel needs 4MiB of CMA memory, in a 16KiB kernel, the minimal reservation has to be 32MiB due to the alignment requirements:
reserved-memory { ... cma_test_reserve: cma_test_reserve { compatible = "shared-dma-pool"; size = <0x0 0x400000>; /* 4 MiB */ ... }; };Quoted 8 lines
reserved-memory { ... cma_test_reserve: cma_test_reserve { compatible = "shared-dma-pool"; size = <0x0 0x2000000>; /* 32 MiB */ ... }; };This indeed is a valid problem which reduces available memory for non-CMA page blocks on system required for general memory usage.
Quoted 10 lines
Solution: Add a new config
CONFIG_PAGE_BLOCK_ORDERthat allows to set the page block order in all the architectures. The maximum page block order will be given byARCH_FORCE_MAX_ORDER.By default,
CONFIG_PAGE_BLOCK_ORDERwill have the same value thatARCH_FORCE_MAX_ORDER. This will make sure that current kernel configurations won't be affected by this change. It is a opt-in change.Right.
Quoted 7 lines
This patch will allow to have the same CMA alignment requirements for large page sizes (16KiB, 64KiB) as that in 4kb kernels by setting a lower
pageblock_order.Tests:
Quoted 8 lines
- Verified that HugeTLB pages work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.- Verified that Transparent Huge Pages work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.- Verified that dma-buf heaps allocations work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.
pageblock_orderare choosen as 1, 7 and 10 to cover the entire possible range forARCH_FORCE_MAX_ORDER. Although kernel CI should test this for all values in the range. Because this now opens up different ranges for different platforms which were never tested earlier.Quoted 118 lines
Benchmarks:
The benchmarks compare 16kb kernels with
pageblock_order10 and 7. The reason for thepageblock_order7 is because this value makes the min CMA alignment requirement the same as that in 4kb kernels (2MB).- Perform 100K dma-buf heaps (
/dev/dma_heap/system) allocations ofSZ_8M,SZ_4M,SZ_2M,SZ_1M,SZ_64,SZ_8,SZ_4. Use simpleperf
(https://developer.android.com/ndk/guides/simpleperf) to measure the # of instructions and page-faults on 16k kernels. The benchmark was executed 10 times. The averages are below:# instructions | #page-faults order 10 | order 7 | order 10 | order 7 -------------------------------------------------------- 13,891,765,770 | 11,425,777,314 | 220 | 217 14,456,293,487 | 12,660,819,302 | 224 | 219 13,924,261,018 | 13,243,970,736 | 217 | 221 13,910,886,504 | 13,845,519,630 | 217 | 221 14,388,071,190 | 13,498,583,098 | 223 | 224 13,656,442,167 | 12,915,831,681 | 216 | 218 13,300,268,343 | 12,930,484,776 | 222 | 218 13,625,470,223 | 14,234,092,777 | 219 | 218 13,508,964,965 | 13,432,689,094 | 225 | 219 13,368,950,667 | 13,683,587,37 | 219 | 225 ------------------------------------------------------------------- 13,803,137,433 | 13,131,974,268 | 220 | 220 AveragesThere were 4.85% #instructions when order was 7, in comparison with order 10.
13,803,137,433 - 13,131,974,268 = -671,163,166 (-4.86%)The number of page faults in order 7 and 10 were the same.
These results didn't show any significant regression when the
pageblock_orderis set to 7 on 16kb kernels.- Run speedometer 3.1 (https://browserbench.org/Speedometer3.1/) 5 times on the 16k kernels with
pageblock_order7 and 10.order 10 | order 7 | order 7 - order 10 | (order 7 - order 10) %
-------------------------------------------------------------------15.8 | 16.4 | 0.6 | 3.80% 16.4 | 16.2 | -0.2 | -1.22% 16.6 | 16.3 | -0.3 | -1.81% 16.8 | 16.3 | -0.5 | -2.98% 16.6 | 16.8 | 0.2 | 1.20%-------------------------------------------------------------------
16.44 16.4 -0.04 -0.24% AveragesThe results didn't show any significant regression when the
pageblock_orderis set to 7 on 16kb kernels.Cc Andrew Morton <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Liam R. Howlett <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>CC Mike Rapoport <[email protected]>Cc Zi Yan <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Minchan Kim <[email protected]>Signed-off-by Juan Yescas <[email protected]>Acked-by Zi Yan <[email protected]>---
Changes in v3:
- RenameARCH_FORCE_PAGE_BLOCK_ORDERtoPAGE_BLOCK_ORDERas per Matthew's suggestion.
- Update comments inpageblock-flags.hforpageblock_ordervalue when THP or HugeTLB are not used.Changes in v2:
- Add Zi's Acked-by tag.
- MoveARCH_FORCE_PAGE_BLOCK_ORDERconfig to mm/Kconfig as per Zi and Matthew suggestion so it is available to all the architectures.
- SetARCH_FORCE_PAGE_BLOCK_ORDERto 10 by default whenARCH_FORCE_MAX_ORDERis not available.include/linux/pageblock-flags.h | 14 ++++++++++---- mm/Kconfig | 31 +++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 4 deletions(-)include/linux/pageblock-flags.h +8
-2@@ -28,6 +28,12 @@ enum pageblock_bits { NR_PAGEBLOCK_BITS }; +#if defined(CONFIG_PAGE_BLOCK_ORDER ) +#define PAGE_BLOCK_ORDER CONFIG_PAGE_BLOCK_ORDER +#else +#define PAGE_BLOCK_ORDER MAX_PAGE_ORDER +#endif /* CONFIG_PAGE_BLOCK_ORDER */ +#if defined(CONFIG_HUGETLB_PAGE )#ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -41,18 +47,18 @@ extern unsigned int pageblock_order;* Huge pages are a constant size, but don't exceed the maximum allocation * granularity. */ -#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, PAGE_BLOCK_ORDER)#endif /* CONFIG_HUGETLB_PAGE_SIZE_VARIABLE */ #elif defined(CONFIG_TRANSPARENT_HUGEPAGE) -#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, PAGE_BLOCK_ORDER)include/linux/pageblock-flags.h extern unsigned int pageblock_order; · line 64
Quoted 10 lines
include/linux/pageblock-flags.h +2
-2#else /* CONFIG_TRANSPARENT_HUGEPAGE */ -/* If huge pages are not used, group by MAX_ORDER_NR_PAGES */ -#define pageblock_order MAX_PAGE_ORDER +/* If huge pages are not used, group by PAGE_BLOCK_ORDER */ +#define pageblock_order PAGE_BLOCK_ORDER#endif /* CONFIG_HUGETLB_PAGE */ These all look good.
Quoted 5 lines
mm/Kconfig +0
-0mm/Kconfig config CMA_AREAS · line 997
Quoted 9 lines
mm/Kconfig +6
-0@@ -989,6 +989,37 @@ config CMA_AREAS If unsure, leave the default value "8" in UMA and "20" in NUMA. +# +# Select this config option from the architecture Kconfig, if available, to set +# the max page order for physically contiguous allocations. +# +config ARCH_FORCE_MAX_ORDER + int
ARCH_FORCE_MAX_ORDERneeds to be defined here first beforePAGE_BLOCK_ORDERcould use that subsequently.ButARCH_FORCE_MAX_ORDERis defined in various architectures in 'int' or 'int "<description>"' formats. So could there be a problem for this config to be defined both in generic and platform code ? But clearlyARCH_FORCE_MAX_ORDERstill remains a arch specific config.#git grep "config
ARCH_FORCE_MAX_ORDER" arch/arc/Kconfig:configARCH_FORCE_MAX_ORDERarch/arm/Kconfig:configARCH_FORCE_MAX_ORDERarch/arm64/Kconfig:configARCH_FORCE_MAX_ORDERarch/loongarch/Kconfig:configARCH_FORCE_MAX_ORDERarch/m68k/Kconfig.cpu:configARCH_FORCE_MAX_ORDERarch/mips/Kconfig:configARCH_FORCE_MAX_ORDERarch/nios2/Kconfig:configARCH_FORCE_MAX_ORDERarch/powerpc/Kconfig:configARCH_FORCE_MAX_ORDERarch/sh/mm/Kconfig:configARCH_FORCE_MAX_ORDERarch/sparc/Kconfig:configARCH_FORCE_MAX_ORDERarch/xtensa/Kconfig:configARCH_FORCE_MAX_ORDERmm/Kconfig:configARCH_FORCE_MAX_ORDER
arch/arc/config
ARCH_FORCE_MAX_ORDER
int "Maximum zone order"
arch/arm/config
ARCH_FORCE_MAX_ORDER
int "Order of maximal physically contiguous allocations"
arch/arm64/config
ARCH_FORCE_MAX_ORDER
int
...........
arch/sparc/
Quoted 7 lines
config
ARCH_FORCE_MAX_ORDER
int "Order of maximal physically contiguous allocations"mm/Kconfig config CMA_AREAS · line 999
Quoted 2 lines
mm/Kconfig +2
-0+ +# When ARCH_FORCE_MAX_ORDER is not defined, the default page block order is 10,Just wondering - why the default is 10 ?
For x86_64, MAX_PAGE_ORDER is 10. I wonder if it is related.
Quoted 22 lines
mm/Kconfig config CMA_AREAS · line 1003
Quoted 5 lines
mm/Kconfig +4
-0+# as per include/linux/mmzone.h. +config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDERAlso why the range is restricted to 10 ?
mm/Kconfig config CMA_AREAS · line 1006
Quoted 3 lines
mm/Kconfig +3
-0+ default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDERWe still have the
MAX_PAGE_ORDERwhich maps intoARCH_FORCE_MAX_ORDERwhen available or otherwise just falls back as 10./* Free memory management - zoned buddy allocator. */ #ifndef
CONFIG_ARCH_FORCE_MAX_ORDER
#defineMAX_PAGE_ORDER10
#else
#defineMAX_PAGE_ORDERCONFIG_ARCH_FORCE_MAX_ORDER#endif
Quoted 6 lines
Hence could
PAGE_BLOCK_ORDERconfig description block be simplified asconfig
PAGE_BLOCK_ORDER
int "Page Block Order"
range 1MAX_PAGE_ORDER
defaultMAX_PAGE_ORDER
Could this work? MAX_PAGE_ORDER is a macro defined in linux/mmzone.h. Can Kconfig access it? I am not an expert on Kconfig.
Quoted 3 lines
As
MAX_PAGE_ORDERcould switch betweenARCH_FORCE_MAX_ORDERand 10 as and when required.
If the above Kconfig code work, that would be great.
Quoted 12 lines
mm/Kconfig config CMA_AREAS · line 1012
Quoted 7 lines
mm/Kconfig +6
-0+ + help + The page block order refers to the power of two number of pages that + are physically contiguous and can have a migrate type associated to + them. The maximum size of the page block order is limited by + ARCH_FORCE_MAX_ORDER.s/ARCH_FORCE_MAX_ORDER/ARCH_FORCE_MAX_ORDER when available on the platform/ ?
Also mention about max range when
ARCH_FORCE_MAX_ORDERis not available.
Quoted 11 lines
mm/Kconfig config CMA_AREAS · line 1019
Quoted 7 lines
mm/Kconfig +7
-0+ + This option allows overriding the default setting when the page + block order requires to be smaller than ARCH_FORCE_MAX_ORDER. + + Reducing pageblock order can negatively impact THP generation + successful rate. If your workloads uses THP heavily, please use this + option with caution.Just wondering - could there be any other side effects besides THP ? Will it be better to depend on
CONFIG_EXPERTwhile selecting anything other than the default option i.eARCH_FORCE_MAX_ORDERor 10 from the value range.
Another side effect (or maybe benefit) is that things like virtio-balloon free page reporting, virtio-mem using pageblock in their work can have smaller granularity with a reduced pageblock size.
--
Best Regards,
Yan, Zi
Quoted 278 lines · Anshuman Khandual
Quoted 46 lines · Juan Yescas
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (
CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less availableMIGRATE_UNMOVABLEandMIGRATE_RECLAIMABLEpage blocks sinceMIGRATE_CMAcan't fallback to them.The
CMA_MIN_ALIGNMENT_BYTESincreases because it depends onMAX_PAGE_ORDERwhich depends onARCH_FORCE_MAX_ORDER. The value ofARCH_FORCE_MAX_ORDERincreases on 16k and 64k kernels.For example, in ARM, the CMA alignment requirement when:
-
CONFIG_ARCH_FORCE_MAX_ORDERdefault value is used
-CONFIG_TRANSPARENT_HUGEPAGEis set:
PAGE_SIZE|MAX_PAGE_ORDER|pageblock_order|CMA_MIN_ALIGNMENT_BYTES
-----------------------------------------------------------------------4KiB | 10 | 10 | 4KiB * (2 ^ 10) = 4MiB 16Kib | 11 | 11 | 16KiB * (2 ^ 11) = 32MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiBThere are some extreme cases for the CMA alignment requirement when:
-
CONFIG_ARCH_FORCE_MAX_ORDERmaximum value is set
-CONFIG_TRANSPARENT_HUGEPAGEis NOT set:
-CONFIG_HUGETLB_PAGEis NOT set
PAGE_SIZE|MAX_PAGE_ORDER|pageblock_order|CMA_MIN_ALIGNMENT_BYTES
------------------------------------------------------------------------4KiB | 15 | 15 | 4KiB * (2 ^ 15) = 128MiB 16Kib | 13 | 13 | 16KiB * (2 ^ 13) = 128MiB 64KiB | 13 | 13 | 64KiB * (2 ^ 13) = 512MiBThis affects the CMA reservations for the drivers. If a driver in a 4KiB kernel needs 4MiB of CMA memory, in a 16KiB kernel, the minimal reservation has to be 32MiB due to the alignment requirements:
reserved-memory { ... cma_test_reserve: cma_test_reserve { compatible = "shared-dma-pool"; size = <0x0 0x400000>; /* 4 MiB */ ... }; };Quoted 8 lines
reserved-memory { ... cma_test_reserve: cma_test_reserve { compatible = "shared-dma-pool"; size = <0x0 0x2000000>; /* 32 MiB */ ... }; };This indeed is a valid problem which reduces available memory for non-CMA page blocks on system required for general memory usage.
Quoted 10 lines
Solution: Add a new config
CONFIG_PAGE_BLOCK_ORDERthat allows to set the page block order in all the architectures. The maximum page block order will be given byARCH_FORCE_MAX_ORDER.By default,
CONFIG_PAGE_BLOCK_ORDERwill have the same value thatARCH_FORCE_MAX_ORDER. This will make sure that current kernel configurations won't be affected by this change. It is a opt-in change.Right.
Quoted 7 lines
This patch will allow to have the same CMA alignment requirements for large page sizes (16KiB, 64KiB) as that in 4kb kernels by setting a lower
pageblock_order.Tests:
Quoted 8 lines
- Verified that HugeTLB pages work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.- Verified that Transparent Huge Pages work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.- Verified that dma-buf heaps allocations work when
pageblock_orderis 1, 7, 10 on 4k and 16k kernels.
pageblock_orderare choosen as 1, 7 and 10 to cover the entire possible range forARCH_FORCE_MAX_ORDER. Although kernel CI should test this for all values in the range. Because this now opens up different ranges for different platforms which were never tested earlier.Quoted 118 lines
Benchmarks:
The benchmarks compare 16kb kernels with
pageblock_order10 and 7. The reason for thepageblock_order7 is because this value makes the min CMA alignment requirement the same as that in 4kb kernels (2MB).- Perform 100K dma-buf heaps (
/dev/dma_heap/system) allocations ofSZ_8M,SZ_4M,SZ_2M,SZ_1M,SZ_64,SZ_8,SZ_4. Use simpleperf
(https://developer.android.com/ndk/guides/simpleperf) to measure the # of instructions and page-faults on 16k kernels. The benchmark was executed 10 times. The averages are below:# instructions | #page-faults order 10 | order 7 | order 10 | order 7 -------------------------------------------------------- 13,891,765,770 | 11,425,777,314 | 220 | 217 14,456,293,487 | 12,660,819,302 | 224 | 219 13,924,261,018 | 13,243,970,736 | 217 | 221 13,910,886,504 | 13,845,519,630 | 217 | 221 14,388,071,190 | 13,498,583,098 | 223 | 224 13,656,442,167 | 12,915,831,681 | 216 | 218 13,300,268,343 | 12,930,484,776 | 222 | 218 13,625,470,223 | 14,234,092,777 | 219 | 218 13,508,964,965 | 13,432,689,094 | 225 | 219 13,368,950,667 | 13,683,587,37 | 219 | 225 ------------------------------------------------------------------- 13,803,137,433 | 13,131,974,268 | 220 | 220 AveragesThere were 4.85% #instructions when order was 7, in comparison with order 10.
13,803,137,433 - 13,131,974,268 = -671,163,166 (-4.86%)The number of page faults in order 7 and 10 were the same.
These results didn't show any significant regression when the
pageblock_orderis set to 7 on 16kb kernels.- Run speedometer 3.1 (https://browserbench.org/Speedometer3.1/) 5 times on the 16k kernels with
pageblock_order7 and 10.order 10 | order 7 | order 7 - order 10 | (order 7 - order 10) %
-------------------------------------------------------------------15.8 | 16.4 | 0.6 | 3.80% 16.4 | 16.2 | -0.2 | -1.22% 16.6 | 16.3 | -0.3 | -1.81% 16.8 | 16.3 | -0.5 | -2.98% 16.6 | 16.8 | 0.2 | 1.20%-------------------------------------------------------------------
16.44 16.4 -0.04 -0.24% AveragesThe results didn't show any significant regression when the
pageblock_orderis set to 7 on 16kb kernels.Cc Andrew Morton <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Liam R. Howlett <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>CC Mike Rapoport <[email protected]>Cc Zi Yan <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Minchan Kim <[email protected]>Signed-off-by Juan Yescas <[email protected]>Acked-by Zi Yan <[email protected]>---
Changes in v3:
- RenameARCH_FORCE_PAGE_BLOCK_ORDERtoPAGE_BLOCK_ORDERas per Matthew's suggestion.
- Update comments inpageblock-flags.hforpageblock_ordervalue when THP or HugeTLB are not used.Changes in v2:
- Add Zi's Acked-by tag.
- MoveARCH_FORCE_PAGE_BLOCK_ORDERconfig to mm/Kconfig as per Zi and Matthew suggestion so it is available to all the architectures.
- SetARCH_FORCE_PAGE_BLOCK_ORDERto 10 by default whenARCH_FORCE_MAX_ORDERis not available.include/linux/pageblock-flags.h | 14 ++++++++++---- mm/Kconfig | 31 +++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 4 deletions(-)include/linux/pageblock-flags.h +8
-2@@ -28,6 +28,12 @@ enum pageblock_bits { NR_PAGEBLOCK_BITS }; +#if defined(CONFIG_PAGE_BLOCK_ORDER ) +#define PAGE_BLOCK_ORDER CONFIG_PAGE_BLOCK_ORDER +#else +#define PAGE_BLOCK_ORDER MAX_PAGE_ORDER +#endif /* CONFIG_PAGE_BLOCK_ORDER */ +#if defined(CONFIG_HUGETLB_PAGE )#ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -41,18 +47,18 @@ extern unsigned int pageblock_order;* Huge pages are a constant size, but don't exceed the maximum allocation * granularity. */ -#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HUGETLB_PAGE_ORDER, PAGE_BLOCK_ORDER)#endif /* CONFIG_HUGETLB_PAGE_SIZE_VARIABLE */ #elif defined(CONFIG_TRANSPARENT_HUGEPAGE) -#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, MAX_PAGE_ORDER) +#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, PAGE_BLOCK_ORDER)include/linux/pageblock-flags.h extern unsigned int pageblock_order; · line 64
Quoted 10 lines
include/linux/pageblock-flags.h +2
-2#else /* CONFIG_TRANSPARENT_HUGEPAGE */ -/* If huge pages are not used, group by MAX_ORDER_NR_PAGES */ -#define pageblock_order MAX_PAGE_ORDER +/* If huge pages are not used, group by PAGE_BLOCK_ORDER */ +#define pageblock_order PAGE_BLOCK_ORDER#endif /* CONFIG_HUGETLB_PAGE */ These all look good.
Quoted 5 lines
mm/Kconfig +0
-0mm/Kconfig config CMA_AREAS · line 997
Quoted 9 lines
mm/Kconfig +6
-0@@ -989,6 +989,37 @@ config CMA_AREAS If unsure, leave the default value "8" in UMA and "20" in NUMA. +# +# Select this config option from the architecture Kconfig, if available, to set +# the max page order for physically contiguous allocations. +# +config ARCH_FORCE_MAX_ORDER + int
ARCH_FORCE_MAX_ORDERneeds to be defined here first beforePAGE_BLOCK_ORDERcould use that subsequently.ButARCH_FORCE_MAX_ORDERis defined in various architectures in 'int' or 'int "<description>"' formats. So could there be a problem for this config to be defined both in generic and platform code ? But clearlyARCH_FORCE_MAX_ORDERstill remains a arch specific config.#git grep "config
ARCH_FORCE_MAX_ORDER" arch/arc/Kconfig:configARCH_FORCE_MAX_ORDERarch/arm/Kconfig:configARCH_FORCE_MAX_ORDERarch/arm64/Kconfig:configARCH_FORCE_MAX_ORDERarch/loongarch/Kconfig:configARCH_FORCE_MAX_ORDERarch/m68k/Kconfig.cpu:configARCH_FORCE_MAX_ORDERarch/mips/Kconfig:configARCH_FORCE_MAX_ORDERarch/nios2/Kconfig:configARCH_FORCE_MAX_ORDERarch/powerpc/Kconfig:configARCH_FORCE_MAX_ORDERarch/sh/mm/Kconfig:configARCH_FORCE_MAX_ORDERarch/sparc/Kconfig:configARCH_FORCE_MAX_ORDERarch/xtensa/Kconfig:configARCH_FORCE_MAX_ORDERmm/Kconfig:configARCH_FORCE_MAX_ORDER
arch/arc/config
ARCH_FORCE_MAX_ORDER
int "Maximum zone order"
arch/arm/config
ARCH_FORCE_MAX_ORDER
int "Order of maximal physically contiguous allocations"
arch/arm64/config
ARCH_FORCE_MAX_ORDER
int
...........
arch/sparc/
Quoted 8 lines
config
ARCH_FORCE_MAX_ORDER
int "Order of maximal physically contiguous allocations"mm/Kconfig config CMA_AREAS · line 999
Quoted 2 lines
mm/Kconfig +2
-0+ +# When ARCH_FORCE_MAX_ORDER is not defined, the default page block order is 10,Just wondering - why the default is 10 ?
When CONFIG_ARCH_FORCE_MAX_ORDER is not defined, the default is 10 for MAX_PAGE_ORDER in include/linux/mmzone.h.
https://elixir.bootlin.com/linux/v6.15-rc5/source/include/linux/mmzone.h#L30
My understanding is that with the default order 10 for MAX_PAGE_ORDER, we make sure that we can allocate huge pages using the buddy allocator when PAGE_SIZE = 4096. For example, we can allocate 2 huge pages of 2MiB using the buddy allocator:
(2 ^ 10) * 4096 = 4194304 = 4 MiB
Could any of the mm experts confirm this?
Quoted 6 lines
mm/Kconfig config CMA_AREAS · line 1003
Quoted 4 lines
mm/Kconfig +4
-0+# as per include/linux/mmzone.h. +config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDERAlso why the range is restricted to 10 ?
The PAGE_BLOCK_ORDER has to be less or equal to the MAX_PAGE_ORDER when ARCH_FORCE_MAX_ORDER is not defined.
Thanks
Juan
Quoted 54 lines
mm/Kconfig config CMA_AREAS · line 1006
Quoted 4 lines
mm/Kconfig +3
-0+ default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDERWe still have the
MAX_PAGE_ORDERwhich maps intoARCH_FORCE_MAX_ORDERwhen available or otherwise just falls back as 10./* Free memory management - zoned buddy allocator. */ #ifndef
CONFIG_ARCH_FORCE_MAX_ORDER
#defineMAX_PAGE_ORDER10
#else
#defineMAX_PAGE_ORDERCONFIG_ARCH_FORCE_MAX_ORDER#endifHence could
PAGE_BLOCK_ORDERconfig description block be simplified asconfig
PAGE_BLOCK_ORDER
int "Page Block Order"
range 1MAX_PAGE_ORDER
defaultMAX_PAGE_ORDERAs
MAX_PAGE_ORDERcould switch betweenARCH_FORCE_MAX_ORDERand 10 as and when required.mm/Kconfig config CMA_AREAS · line 1012
Quoted 6 lines
mm/Kconfig +6
-0+ + help + The page block order refers to the power of two number of pages that + are physically contiguous and can have a migrate type associated to + them. The maximum size of the page block order is limited by + ARCH_FORCE_MAX_ORDER.s/ARCH_FORCE_MAX_ORDER/ARCH_FORCE_MAX_ORDER when available on the platform/ ?
Also mention about max range when
ARCH_FORCE_MAX_ORDERis not available.mm/Kconfig config CMA_AREAS · line 1019
Quoted 7 lines
mm/Kconfig +7
-0+ + This option allows overriding the default setting when the page + block order requires to be smaller than ARCH_FORCE_MAX_ORDER. + + Reducing pageblock order can negatively impact THP generation + successful rate. If your workloads uses THP heavily, please use this + option with caution.Just wondering - could there be any other side effects besides THP ? Will it be better to depend on
CONFIG_EXPERTwhile selecting anything other than the default option i.eARCH_FORCE_MAX_ORDERor 10 from the value range.Quoted 6 lines
mm/Kconfig +3
-0+ + Don't change if unsure. + config MEM_SOFT_DIRTY bool "Track memory changes" depends on CHECKPOINT_RESTORE && HAVE_ARCH_SOFT_DIRTY && PROC_FS
Andrew Morton
maintainerQuoted 12 lines · On Mon, 5 May 2025 17:22:58 -0700 Juan Yescas <[email protected]> wrote:
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (
CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less availableMIGRATE_UNMOVABLEandMIGRATE_RECLAIMABLEpage blocks sinceMIGRATE_CMAcan't fallback to them.The
CMA_MIN_ALIGNMENT_BYTESincreases because it depends onMAX_PAGE_ORDERwhich depends onARCH_FORCE_MAX_ORDER. The value ofARCH_FORCE_MAX_ORDERincreases on 16k and 64k kernels....
mm/Kconfig config CMA_AREAS · line 1006
Quoted 6 lines
mm/Kconfig +6
-0+config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER + default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER
Do we really need to do this arithmetic within Kconfig? Would it be cleaner to do this at runtime, presumably when calculating pageblock_order?
Quoted 14 lines · Andrew Morton
Quoted 12 lines · On Mon, 5 May 2025 17:22:58 -0700 Juan Yescas <[email protected]> wrote:
Problem: On large page size configurations (16KiB, 64KiB), the CMA alignment requirement (
CMA_MIN_ALIGNMENT_BYTES) increases considerably, and this causes the CMA reservations to be larger than necessary. This means that system will have less availableMIGRATE_UNMOVABLEandMIGRATE_RECLAIMABLEpage blocks sinceMIGRATE_CMAcan't fallback to them.The
CMA_MIN_ALIGNMENT_BYTESincreases because it depends onMAX_PAGE_ORDERwhich depends onARCH_FORCE_MAX_ORDER. The value ofARCH_FORCE_MAX_ORDERincreases on 16k and 64k kernels....
Quoted 10 lines
mm/Kconfig config CMA_AREAS · line 1006
Quoted 6 lines
mm/Kconfig +6
-0+config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER + default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDERDo we really need to do this arithmetic within
Kconfig? Would it be cleaner to do this at runtime, presumably when calculatingpageblock_order?
AFAIK pageblock_order is compile-time constant. Making this a boot parameter was proposed in v1 but explained as not useful. That explanation could be added in the changelog?
Andrew Morton
maintainerQuoted 11 lines · On Tue, 6 May 2025 14:48:19 +0200 Vlastimil Babka <[email protected]> wrote:
Quoted 10 lines
mm/Kconfig config CMA_AREAS · line 1006
Quoted 6 lines
mm/Kconfig +6
-0+config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER + default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDERDo we really need to do this arithmetic within
Kconfig? Would it be cleaner to do this at runtime, presumably when calculatingpageblock_order?
Quoted 1 line
AFAIK
pageblock_orderis compile-time constant.
So it is. Why the heck did we make it lower case?
And pageblock_nr_pages.
Zi Yan
reviewerQuoted 13 lines · Andrew Morton
Quoted 11 lines · On Tue, 6 May 2025 14:48:19 +0200 Vlastimil Babka <[email protected]> wrote:
Quoted 10 lines
mm/Kconfig config CMA_AREAS · line 1006
Quoted 6 lines
mm/Kconfig +6
-0+config PAGE_BLOCK_ORDER + int "Page Block Order" + range 1 10 if !ARCH_FORCE_MAX_ORDER + default 10 if !ARCH_FORCE_MAX_ORDER + range 1 ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDER + default ARCH_FORCE_MAX_ORDER if ARCH_FORCE_MAX_ORDERDo we really need to do this arithmetic within
Kconfig? Would it be cleaner to do this at runtime, presumably when calculatingpageblock_order?
Quoted 5 lines
Quoted 1 line
AFAIK
pageblock_orderis compile-time constant.So it is. Why the heck did we make it lower case?
And
pageblock_nr_pages.
Because when CONFIG_HUGETLB_PAGE_SIZE_VARIABLE, pageblock_order is a variable and set at boot (see set_pageblock_order() in mm_init.c ). So its type is Kconfig dependent.
--
Best Regards,
Yan, Zi