arm64: mm: decrease the section size to reduce the memory reserved for the page map
PATCH 1/1 arm64: mm: decrease the section size to reduce the memory reserved for the page map
For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP do not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.
--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/include/asm/sparsemem.h +1 -1
@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27#endif #endif
--
2.15.0
Quoted 14 lines
-----Original Message-----
From: liwei (CM)
Sent: Friday, December 4, 2020 2:45 PM
To: [email protected]; [email protected]; [email protected]; liwei (CM) <[email protected]>
Cc: fengbaopeng <[email protected]>; [email protected]; [email protected]; Song Bao Hua (Barry Song) <[email protected]>; [email protected]; [email protected]; butao <[email protected]>
Subject: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page map
Quoted 8 lines
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>---
When page size is 4K, for each 1GB memory, we need 16MB vmemmap; For each 128MB memory, we need 2MB vmemmap.
So while we have memory hole like 928MB(1GB-64MB),if SECTION_SIZE_BITS is 30, we waste 15MB; if SECTION_SIZE_BITS is 27, we waste 1MB only.
Quoted 18 lines
arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +1
-2@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27#endif #endif --
Quoted 1 line
2.15.0
Thanks
Barry
Will Deacon
maintainerQuoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0
arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27
We chose '30' to avoid running out of bits in the page flags. What changed?
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds SECTION_SIZE #error Allocator MAX_ORDER exceeds SECTION_SIZE
if I bump up NR_CPUS and NODES_SHIFT.
Will
Quoted 18 lines · Will Deacon
Quoted 17 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0@@ -7,7 +7,7 @@
Quoted 6 lines
arch/arm64/include/asm/sparsemem.h line 10
Quoted 4 lines
arch/arm64/include/asm/sparsemem.h +1
-1#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
I think that for 64-bit there are still plenty of free bits. I didn't check now, but when I played with SPARSEMEM on m68k there were 8 bits for section out of 32.
Quoted 6 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.
I don't think it's related to NR_CPUS and NODES_SHIFT. This seems rather 64K pages that cause this.
Not that is shouldn't be addressed.
Quoted 1 line
Will
Signature · 2 lines
Sincerely yours, Mike.
Quoted 49 lines
-----Original Message-----
From: Mike Rapoport [mailto:[email protected]]
Sent: Saturday, December 5, 2020 12:44 AM
To: Will Deacon <[email protected]>
Cc: liwei (CM) <[email protected]>; [email protected]; fengbaopeng <[email protected]>; [email protected]; [email protected]; Song Bao Hua (Barry Song) <[email protected]>; [email protected]; [email protected]; butao <[email protected]>
Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page mapQuoted 5 lines · Will Deacon
Quoted 4 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Quoted 8 lines
Review
Quoted 8 lines
Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)(quoted diff) +0
-0
b/arch/arm64/include/asm/sparsemem.hQuoted 11 lines
arch/arm64/include/asm/sparsemem.h line 10
Quoted 9 lines
index
1f43fcc79738..8963bd3def28100644arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
I think that for 64-bit there are still plenty of free bits. I didn't check now, but when I played with SPARSEMEM on m68k there were 8 bits for section out of 32.
Quoted 3 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceeds
SECTION_SIZE
Quoted 8 lines
Quoted 3 lines
#error Allocator
MAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.I don't think it's related to
NR_CPUSandNODES_SHIFT. This seems rather 64K pages that cause this.Not that is shouldn't be addressed.
Right now, only 4K PAGES will define ARM64_SWAPPER_USES_SECTION_MAPS. Other cases will use vmemmap_populate_basepages().
The original patch should be only addressing the issue in 4K pages:
https://lore.kernel.org/lkml/[email protected]/
would we do something like the below?
#ifdef CONFIG_ARM64_4K_PAGE
#define SECTION_SIZE_BITS 27
#else
#define SECTION_SIZE_BITS 30
#endif
Quoted 6 lines
Quoted 2 lines
Will
--
Sincerely yours,
Mike.
Thanks
Barry
Quoted 63 lines · Song Bao Hua (Barry Song)
Quoted 49 lines
-----Original Message-----
From: Mike Rapoport [mailto:[email protected]]
Sent: Saturday, December 5, 2020 12:44 AM
To: Will Deacon <[email protected]>
Cc: liwei (CM) <[email protected]>; [email protected]; fengbaopeng <[email protected]>; [email protected]; [email protected]; Song Bao Hua (Barry Song) <[email protected]>; [email protected]; [email protected]; butao <[email protected]>
Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page mapQuoted 5 lines · Will Deacon
Quoted 4 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Quoted 8 lines
Review
Quoted 8 lines
Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)(quoted diff) +0
-0
b/arch/arm64/include/asm/sparsemem.hQuoted 11 lines
arch/arm64/include/asm/sparsemem.h line 10
Quoted 9 lines
index
1f43fcc79738..8963bd3def28100644arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
I think that for 64-bit there are still plenty of free bits. I didn't check now, but when I played with SPARSEMEM on m68k there were 8 bits for section out of 32.
Quoted 3 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceeds
SECTION_SIZEQuoted 8 lines
Quoted 3 lines
#error Allocator
MAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.I don't think it's related to
NR_CPUSandNODES_SHIFT. This seems rather 64K pages that cause this.Not that is shouldn't be addressed.
Right now, only 4K PAGES will define
ARM64_SWAPPER_USES_SECTION_MAPS. Other cases will usevmemmap_populate_basepages().
The original patch should be only addressing the issue in 4K pages:
https://lore.kernel.org/lkml/[email protected]/
Quoted 6 lines
would we do something like the below?
#ifdefCONFIG_ARM64_4K_PAGE
#defineSECTION_SIZE_BITS27
#else
#defineSECTION_SIZE_BITS30
#endif
This is bit arbitrary. Probably 27 can be further reduced for 4K page size. Instead, we should make SECTION_SIZE_BITS explicitly depend upon MAX_ORDER. IOW section size should be the same as the highest order page in the buddy. CONFIG_FORCE_MAX_ZONEORDER is always defined on arm64. A quick test shows SECTION_SIZE_BITS would be 22 on 4K pages and 29 for 64K pages. As a fall back SECTION_SIZE_BITS can still be 30 in case CONFIG_FORCE_MAX_ZONEORDER is not defined.
arch/arm64/include/asm/sparsemem.h +1 -1
@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)#endif #endif
A similar approach exists on ia64 platform as well.
Quoted 89 lines
-----Original Message-----
From: Anshuman Khandual [mailto:[email protected]]
Sent: Monday, December 7, 2020 8:31 PM
To: Song Bao Hua (Barry Song) <[email protected]>; Mike Rapoport <[email protected]>; Will Deacon <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; liwei (CM) <[email protected]>; butao <[email protected]>; [email protected]; fengbaopeng <[email protected]>
Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page mapQuoted 8 lines · Song Bao Hua (Barry Song)
Quoted 8 lines
-----Original Message-----
From: Mike Rapoport [mailto:[email protected]]
Sent: Saturday, December 5, 2020 12:44 AM
To: Will Deacon <[email protected]>
Cc: liwei (CM) <[email protected]>; [email protected]; fengbaopeng <[email protected]>; [email protected];Quoted 55 lines
Quoted 41 lines
Song Bao Hua (Barry Song) <[email protected]>; [email protected]; [email protected]; butao <[email protected]>
Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page mapQuoted 5 lines · Will Deacon
Quoted 4 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Quoted 8 lines
Review
Quoted 8 lines
Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)(quoted diff) +0
-0
b/arch/arm64/include/asm/sparsemem.hQuoted 11 lines
arch/arm64/include/asm/sparsemem.h line 10
Quoted 9 lines
index
1f43fcc79738..8963bd3def28100644arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
I think that for 64-bit there are still plenty of free bits. I didn't check now, but when I played with SPARSEMEM on m68k there were 8 bits for section out of 32.
Quoted 3 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceeds
SECTION_SIZEQuoted 8 lines
Quoted 3 lines
#error Allocator
MAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.I don't think it's related to
NR_CPUSandNODES_SHIFT. This seems rather 64K pages that cause this.Not that is shouldn't be addressed.
Right now, only 4K PAGES will define
ARM64_SWAPPER_USES_SECTION_MAPS. Other cases will usevmemmap_populate_basepages().
The original patch should be only addressing the issue in 4K pages:
https://lore.kernel.org/lkml/[email protected]/Quoted 6 lines
would we do something like the below?
#ifdefCONFIG_ARM64_4K_PAGE
#defineSECTION_SIZE_BITS27
#else
#defineSECTION_SIZE_BITS30
#endif
Quoted 7 lines
This is bit arbitrary. Probably 27 can be further reduced for 4K page size. Instead, we should make
SECTION_SIZE_BITSexplicitly depend uponMAX_ORDER. IOW section size should be the same as the highest order page in the buddy.CONFIG_FORCE_MAX_ZONEORDERis always defined on arm64. A quick test showsSECTION_SIZE_BITSwould be 22 on 4K pages and 29 for 64K pages. As a fall backSECTION_SIZE_BITScan still be 30 in caseCONFIG_FORCE_MAX_ZONEORDERis not defined.
This will break the pmd mapping for vmemmap. As for each 128M(2^27), we can have 2MB pmd mapping.
Quoted 13 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)#endif #endif
Quoted 1 line
A similar approach exists on ia64 platform as well.
Thanks
Barry
(+ Marc)
Quoted 26 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
Quoted 7 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.
Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce MAX_ORDER to fit inside SECTION_SIZE_BITS?
Quoted 34 lines · Ard Biesheuvel
(+ Marc)
Quoted 31 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE#error AllocatorMAX_ORDERexceedsSECTION_SIZE
Quoted 6 lines
Quoted 2 lines
if I bump up
NR_CPUSandNODES_SHIFT.Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce
MAX_ORDERto fit insideSECTION_SIZE_BITS?
Most probably. We are already massively constraint on platforms such as TX1, and dividing the max allocatable range by 8 isn't going to make it work any better...
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 40 lines · Marc Zyngier
Quoted 39 lines · Ard Biesheuvel
(+ Marc)
Quoted 35 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE
#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.
Quoted 7 lines
Quoted 3 lines
Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce
MAX_ORDERto fit insideSECTION_SIZE_BITS?Most probably. We are already massively constraint on platforms such as TX1, and dividing the max allocatable range by 8 isn't going to make it work any better...
I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is reduced it should accomodate the existing MAX_ORDER.
My two pennies.
Quoted 3 lines
M.Signature · 1 line
Jazz is not dead. It just smells funny...
Signature · 2 lines
Sincerely yours, Mike.
Quoted 46 lines · Mike Rapoport
Quoted 44 lines · Marc Zyngier
Quoted 42 lines · Ard Biesheuvel
(+ Marc)
Quoted 27 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
Quoted 8 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE
#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce
MAX_ORDERto fit insideSECTION_SIZE_BITS?
Quoted 9 lines
Quoted 3 lines
Most probably. We are already massively constraint on platforms such as TX1, and dividing the max allocatable range by 8 isn't going to make it work any better...
I don't think
MAX_ORDERshould shrink. Even ifSECTION_SIZE_BITSis reduced it should accomodate the existingMAX_ORDER.My two pennies.
But include/linux/mmzone.h:1170 has this:
#if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS #error Allocator MAX_ORDER exceeds SECTION_SIZE #endif
and Will managed to trigger it after applying this patch.
Quoted 57 lines · Ard Biesheuvel
Quoted 46 lines · Mike Rapoport
Quoted 44 lines · Marc Zyngier
Quoted 42 lines · Ard Biesheuvel
(+ Marc)
Quoted 27 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
Quoted 8 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE
#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce
MAX_ORDERto fit insideSECTION_SIZE_BITS?Quoted 9 lines
Quoted 3 lines
Most probably. We are already massively constraint on platforms such as TX1, and dividing the max allocatable range by 8 isn't going to make it work any better...
I don't think
MAX_ORDERshould shrink. Even ifSECTION_SIZE_BITSis reduced it should accomodate the existingMAX_ORDER.My two pennies.
Quoted 7 lines
But
include/linux/mmzone.h:1170has this:#if (
MAX_ORDER- 1 +PAGE_SHIFT) >SECTION_SIZE_BITS#error AllocatorMAX_ORDERexceedsSECTION_SIZE#endifand Will managed to trigger it after applying this patch.
Right, because with 64K pages section size of 27 bits is not enough to accomodate MAX_ORDER (2^13 pages of 64K).
Which means that definition of SECTION_SIZE_BITS should take MAX_ORDER into account either statically with
#ifdef ARM64_4K_PAGES
#define SECTION_SIZE_BITS <a number>
#elif ARM64_16K_PAGES
#define SECTION_SIZE_BITS <a larger number>
#elif ARM64_64K_PAGES
#define SECTION_SIZE_BITS <even larger number>
#else
#error "and what is the page size?"
#endif
or dynamically, like e.g. ia64 does:
#ifdef CONFIG_FORCE_MAX_ZONEORDER
#if ((CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS) #undef SECTION_SIZE_BITS
#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) #endif
Signature · 2 lines
Sincerely yours, Mike.
Quoted 82 lines · Mike Rapoport
Quoted 57 lines · Ard Biesheuvel
Quoted 46 lines · Mike Rapoport
Quoted 44 lines · Marc Zyngier
Quoted 42 lines · Ard Biesheuvel
(+ Marc)
Quoted 27 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
Quoted 8 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE
#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce
MAX_ORDERto fit insideSECTION_SIZE_BITS?Quoted 9 lines
Quoted 3 lines
Most probably. We are already massively constraint on platforms such as TX1, and dividing the max allocatable range by 8 isn't going to make it work any better...
I don't think
MAX_ORDERshould shrink. Even ifSECTION_SIZE_BITSis reduced it should accomodate the existingMAX_ORDER.My two pennies.
Quoted 7 lines
But
include/linux/mmzone.h:1170has this:#if (
MAX_ORDER- 1 +PAGE_SHIFT) >SECTION_SIZE_BITS#error AllocatorMAX_ORDERexceedsSECTION_SIZE#endifand Will managed to trigger it after applying this patch.
Right, because with 64K pages section size of 27 bits is not enough to accomodate
MAX_ORDER(2^13 pages of 64K).Which means that definition of
SECTION_SIZE_BITSshould takeMAX_ORDERinto account either statically with#ifdef
ARM64_4K_PAGES
#defineSECTION_SIZE_BITS<a number>
#elifARM64_16K_PAGES
#defineSECTION_SIZE_BITS<a larger number>
#elifARM64_64K_PAGES
#defineSECTION_SIZE_BITS<even larger number>
#else
#error "and what is the page size?"
#endif
Quoted 7 lines
or dynamically, like e.g. ia64 does:
#ifdef
CONFIG_FORCE_MAX_ZONEORDER
#if ((CONFIG_FORCE_MAX_ZONEORDER- 1 +PAGE_SHIFT) >SECTION_SIZE_BITS) #undefSECTION_SIZE_BITS
#defineSECTION_SIZE_BITS(CONFIG_FORCE_MAX_ZONEORDER- 1 +PAGE_SHIFT) #endif
I had proposed the same on the other thread here. But with this the SECTION_SIZE_BITS becomes 22 in case of 4K page size reducing to an extent where PMD based vmemmap mapping could not be created. Though have not looked into much details yet.
Using CONFIG_FORCE_MAX_ZONEORDER seems to the right thing to do. But if that does not reasonably work for 4K pages, we might have to hard code it as 27 to have huge page vmemmap mappings.
(+ saberlily + jiapeng)
Quoted 103 lines · Anshuman Khandual
Quoted 82 lines · Mike Rapoport
Quoted 57 lines · Ard Biesheuvel
Quoted 46 lines · Mike Rapoport
Quoted 44 lines · Marc Zyngier
Quoted 42 lines · Ard Biesheuvel
(+ Marc)
Quoted 27 lines · Will Deacon
Quoted 16 lines · Wei Li
For the memory hole, sparse memory model that define
SPARSEMEM_VMEMMAPdo not free the reserved memory for the page map, decrease the section size can reduce the waste of reserved memory.Signed-off-by Wei Li <[email protected]>Signed-off-by Baopeng Feng <[email protected]>Signed-off-by Xia Qing <[email protected]>--- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/include/asm/sparsemem.h +0
-0arch/arm64/include/asm/sparsemem.h line 10
Quoted 5 lines
arch/arm64/include/asm/sparsemem.h +1
-1@@ -7,7 +7,7 @@#ifdef CONFIG_SPARSEMEM #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS -#define SECTION_SIZE_BITS 30 +#define SECTION_SIZE_BITS 27We chose '30' to avoid running out of bits in the page flags. What changed?
Quoted 8 lines
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: AllocatorMAX_ORDERexceedsSECTION_SIZE
#error AllocatorMAX_ORDERexceedsSECTION_SIZEif I bump up
NR_CPUSandNODES_SHIFT.Does this mean we will run into problems with the GICv3 ITS LPI tables again if we are forced to reduce
MAX_ORDERto fit insideSECTION_SIZE_BITS?Quoted 9 lines
Quoted 3 lines
Most probably. We are already massively constraint on platforms such as TX1, and dividing the max allocatable range by 8 isn't going to make it work any better...
I don't think
MAX_ORDERshould shrink. Even ifSECTION_SIZE_BITSis reduced it should accomodate the existingMAX_ORDER.My two pennies.
Quoted 7 lines
But
include/linux/mmzone.h:1170has this:#if (
MAX_ORDER- 1 +PAGE_SHIFT) >SECTION_SIZE_BITS#error AllocatorMAX_ORDERexceedsSECTION_SIZE#endifand Will managed to trigger it after applying this patch.
Right, because with 64K pages section size of 27 bits is not enough to accomodate
MAX_ORDER(2^13 pages of 64K).Which means that definition of
SECTION_SIZE_BITSshould takeMAX_ORDERinto account either statically with#ifdef
ARM64_4K_PAGES
#defineSECTION_SIZE_BITS<a number>
#elifARM64_16K_PAGES
#defineSECTION_SIZE_BITS<a larger number>
#elifARM64_64K_PAGES
#defineSECTION_SIZE_BITS<even larger number>
#else
#error "and what is the page size?"
#endifQuoted 7 lines
or dynamically, like e.g. ia64 does:
#ifdef
CONFIG_FORCE_MAX_ZONEORDER
#if ((CONFIG_FORCE_MAX_ZONEORDER- 1 +PAGE_SHIFT) >SECTION_SIZE_BITS) #undefSECTION_SIZE_BITS
#defineSECTION_SIZE_BITS(CONFIG_FORCE_MAX_ZONEORDER- 1 +PAGE_SHIFT) #endifI had proposed the same on the other thread here. But with this the
SECTION_SIZE_BITSbecomes 22 in case of 4K page size reducing to an extent where PMD based vmemmap mapping could not be created. Though have not looked into much details yet.Using
CONFIG_FORCE_MAX_ZONEORDERseems to the right thing to do. But if that does not reasonably work for 4K pages, we might have to hard code it as 27 to have huge page vmemmap mappings.
.