sparsemem support for RISC-V
Subsystems (8)
v2 cover sparsemem support for RISC-V
This patchset implements sparsemem on RISC-V. The first few patches move some code in existing architectures into common helpers so they can be used by the new RISC-V implementation. The final patch actually adds sparsmem support to RISC-V.
This is the first small step in supporting P2P on RISC-V.
--
Changes in v2:
* Rebase on v4.19-rc8
* Move the STRUCT_PAGE_MAX_SHIFT define into a common header (near the definition of struct page). As suggested by Christoph.
* Clean up the unnecessary nid variable in the memblocks_present() function, per Christoph.
* Collected tags from Palmer and Catalin.
--
Logan Gunthorpe (6):
mm: Introduce common STRUCT_PAGE_MAX_SHIFT define mm/sparse: add common helper to mark all memblocks present
ARM: mm: make use of new memblocks_present() helper
arm64: mm: make use of new memblocks_present() helper
sh: mm: make use of new memblocks_present() helper
RISC-V: Implement sparsemem
arch/arm/mm/init.c | 17 +---------------- arch/arm64/include/asm/memory.h | 9 --------- arch/arm64/mm/init.c | 28 +--------------------------- arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ arch/riscv/kernel/setup.c | 4 +++- arch/riscv/mm/init.c | 8 ++++++++ arch/sh/mm/init.c | 7 +------ include/asm-generic/fixmap.h | 1 + include/linux/mm_types.h | 5 +++++ include/linux/mmzone.h | 6 ++++++ mm/sparse.c | 14 ++++++++++++++ 13 files changed, 91 insertions(+), 63 deletions(-) create mode 100644 arch/riscv/include/asm/sparsemem.h
--
2.19.0
PATCH v2 1/6 mm: Introduce common STRUCT_PAGE_MAX_SHIFT define
This define is used by arm64 to calculate the size of the vmemmap region. It is defined as the log2 of the upper bound on the size of a struct page.
We move it into mm_types.h so it can be defined properly instead of set and checked with a build bug. This also allows us to use the same define for riscv.
--- arch/arm64/include/asm/memory.h | 9 --------- arch/arm64/mm/init.c | 8 -------- include/asm-generic/fixmap.h | 1 + include/linux/mm_types.h | 5 +++++ 4 files changed, 6 insertions(+), 17 deletions(-)
arch/arm64/include/asm/memory.h +0 -9
@@ -34,15 +34,6 @@*/ #define PCI_IO_SIZE SZ_16M -/* -* Log2 of the upper bound of the size of a struct page. Used for sizing -* the vmemmap region only, does not affect actual memory footprint. -* We don't use sizeof(struct page) directly since taking its size here -* requires its definition to be available at this point in the inclusion -* chain, and it may not be a power of 2 in the first place. -*/ -#define STRUCT_PAGE_MAX_SHIFT 6 -/* * VMEMMAP_SIZE - allows the whole linear region to be covered by * a struct page array
arch/arm64/mm/init.c +0 -8
@@ -615,14 +615,6 @@ void __init mem_init(void)BUILD_BUG_ON (TASK_SIZE_32 >TASK_SIZE_64 );#endif -#ifdef CONFIG_SPARSEMEM_VMEMMAP -/* -* Make sure we chose the upper bound of sizeof(struct page) -* correctly when sizing the VMEMMAP array. -*/ -BUILD_BUG_ON (sizeof (struct page ) > (1 <<STRUCT_PAGE_MAX_SHIFT )); -#endif -if (PAGE_SIZE >=16384 &&get_num_physpages () <=128 ) {extern int sysctl_overcommit_memory;/*
include/asm-generic/fixmap.h +1 -0
@@ -16,6 +16,7 @@#define __ASM_GENERIC_FIXMAP_H#include <linux/bug.h> +#include <linux/mm_types.h> #define __fix_to_virt (x) (FIXADDR_TOP - ((x) << PAGE_SHIFT))#define __virt_to_fix (x) ((FIXADDR_TOP - ((x)&PAGE_MASK)) >> PAGE_SHIFT)
include/linux/mm_types.h +5 -0
@@ -206,6 +206,11 @@ struct page {#endif } _struct_page_alignment; +/* +* Used for sizing the vmemmap region on some architectures +*/ +#define STRUCT_PAGE_MAX_SHIFT ilog2(roundup_pow_of_two(sizeof(struct page))) +#define PAGE_FRAG_CACHE_MAX_SIZE __ALIGN_MASK(32768, ~PAGE_MASK)#define PAGE_FRAG_CACHE_MAX_ORDER get_order(PAGE_FRAG_CACHE_MAX_SIZE)
Signature · 1 line
2.19.0
Re: [PATCH v2 1/6] mm: Introduce common STRUCT_PAGE_MAX_SHIFT define
Hi Logan,
Quoted 14 lines · Logan Gunthorpe
This define is used by arm64 to calculate the size of the vmemmap region. It is defined as the log2 of the upper bound on the size of a struct page.
We move it into
mm_types.hso it can be defined properly instead of set and checked with a build bug. This also allows us to use the same define for riscv.Signed-off-by Logan Gunthorpe <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Andrew Morton <[email protected]>Cc Christoph Hellwig <[email protected]>
Quoted 6 lines
--- arch/arm64/include/asm/memory.h | 9 --------- arch/arm64/mm/init.c | 8 -------- include/asm-generic/fixmap.h | 1 + include/linux/mm_types.h | 5 +++++ 4 files changed, 6 insertions(+), 17 deletions(-)
This looks like a really good cleanup to me, thanks:
Will
PATCH v2 2/6 mm/sparse: add common helper to mark all memblocks present
In reply to Logan GunthorpePresently the arches arm64, arm and sh have a function which loops through each memblock and calls memory present. riscv will require a similar function.
Introduce a common memblocks_present() function that can be used by all the arches. Subsequent patches will cleanup the arches that make use of this.
--- include/linux/mmzone.h | 6 ++++++ mm/sparse.c | 14 ++++++++++++++ 2 files changed, 20 insertions(+)
include/linux/mmzone.h +6 -0
@@ -784,6 +784,12 @@ void memory_present(int nid, unsigned long start, unsigned long end);static inline void memory_present (int nid,unsigned long start,unsigned long end) {}#endif +#if defined(CONFIG_SPARSEMEM ) && defined(CONFIG_HAVE_MEMBLOCK ) +void memblocks_present (void ); +#else +static inline void memblocks_present (void ) {} +#endif +#ifdef CONFIG_HAVE_MEMORYLESS_NODES int local_memory_node (int node_id);#else
mm/sparse.c +14 -0
@@ -5,6 +5,7 @@#include <linux/mm.h> #include <linux/slab.h> #include <linux/mmzone.h> +#include <linux/memblock.h> #include <linux/bootmem.h> #include <linux/compiler.h> #include <linux/highmem.h>
@@ -238,6 +239,19 @@ void __init memory_present(int nid, unsigned long start, unsigned long end) } } +#ifdef CONFIG_HAVE_MEMBLOCK +void __init memblocks_present (void ) +{ +struct memblock_region *reg; + +for_each_memblock (memory, reg) { +memory_present (memblock_get_region_node (reg), +memblock_region_memory_base_pfn (reg), +memblock_region_memory_end_pfn (reg)); + } +} +#endif +/* * Subtle, we encode the real pfn into the mem_map such that * the identity pfn - section_mem_map will return the actual
Signature · 1 line
2.19.0
PATCH v2 3/6 ARM: mm: make use of new memblocks_present() helper
In reply to Logan GunthorpeCleanup the arm_memory_present() function seeing it's very similar to other arches.
The new memblocks_present() helper checks for node ids which the arm version did not. However, this is equivalent seeing HAVE_MEMBLOCK_NODE_MAP should be false in this arch and therefore memblock_get_region_node() should return 0.
--- arch/arm/mm/init.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)
arch/arm/mm/init.c +1 -16
@@ -201,21 +201,6 @@ int pfn_valid(unsigned long pfn)EXPORT_SYMBOL (pfn_valid);#endif -#ifndef CONFIG_SPARSEMEM -static void __init arm_memory_present (void ) -{ -} -#else -static void __init arm_memory_present (void ) -{ -struct memblock_region *reg; - -for_each_memblock (memory, reg) -memory_present (0 ,memblock_region_memory_base_pfn (reg), -memblock_region_memory_end_pfn (reg)); -} -#endif -static bool arm_memblock_steal_permitted = true;phys_addr_t __init arm_memblock_steal (phys_addr_t size,phys_addr_t align)
@@ -317,7 +302,7 @@ void __init bootmem_init(void)* Sparsemem tries to allocate bootmem in memory_present(), * so must be done after the fixed reservations */ -arm_memory_present (); +memblocks_present ();/* * sparse_init() needs the bootmem allocator up and running.
Signature · 1 line
2.19.0
PATCH v2 4/6 arm64: mm: make use of new memblocks_present() helper
In reply to Logan GunthorpeCleanup the arm64_memory_present() function seeing it's very similar to other arches.
memblocks_present() is a direct replacement of arm64_memory_present()
--- arch/arm64/mm/init.c | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-)
arch/arm64/mm/init.c +1 -19
@@ -296,24 +296,6 @@ int pfn_valid(unsigned long pfn)EXPORT_SYMBOL (pfn_valid);#endif -#ifndef CONFIG_SPARSEMEM -static void __init arm64_memory_present (void ) -{ -} -#else -static void __init arm64_memory_present (void ) -{ -struct memblock_region *reg; - -for_each_memblock (memory, reg) { -int nid =memblock_get_region_node (reg); - -memory_present (nid,memblock_region_memory_base_pfn (reg), -memblock_region_memory_end_pfn (reg)); - } -} -#endif -static phys_addr_t memory_limit =PHYS_ADDR_MAX ;/*
@@ -506,7 +488,7 @@ void __init bootmem_init(void)* Sparsemem tries to allocate bootmem in memory_present(), so must be * done after the fixed reservations. */ -arm64_memory_present (); +memblocks_present ();sparse_init ();zone_sizes_init (min, max);
Signature · 1 line
2.19.0
PATCH v2 5/6 sh: mm: make use of new memblocks_present() helper
In reply to Logan GunthorpeCleanup the open coded for_each_memblock() loop that is equivalent to the new memblocks_present() helper.
--- arch/sh/mm/init.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
arch/sh/mm/init.c +1 -6
@@ -235,12 +235,7 @@ static void __init do_init_bootmem(void)plat_mem_setup (); -for_each_memblock (memory, reg) { -int nid =memblock_get_region_node (reg); - -memory_present (nid,memblock_region_memory_base_pfn (reg), -memblock_region_memory_end_pfn (reg)); - } +memblocks_present ();sparse_init (); }
Signature · 1 line
2.19.0
PATCH v2 6/6 RISC-V: Implement sparsemem
In reply to Logan GunthorpeThis patch implements sparsemem support for risc-v which helps pave the way for memory hotplug and eventually P2P support.
We introduce Kconfig options for virtual and physical address bits which are used to calculate the size of the vmemmap and set the MAX_PHYSMEM_BITS.
The vmemmap is located directly before the VMALLOC region and sized such that we can allocate enough pages to populate all the virtual address space in the system (similar to the way it's done in arm64).
During initialization, call memblocks_present() and sparse_init(), and provide a stub for vmemmap_populate() (all of which is similar to arm64).
--- arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ arch/riscv/kernel/setup.c | 4 +++- arch/riscv/mm/init.c | 8 ++++++++ 5 files changed, 62 insertions(+), 5 deletions(-) create mode 100644 arch/riscv/include/asm/sparsemem.h
arch/riscv/Kconfig +23 -0
@@ -52,12 +52,32 @@ config ZONE_DMA32 bool default y if 64BIT +config VA_BITS + int + default 32 if 32BIT + default 39 if 64BIT + +config PA_BITS + int + default 34 if 32BIT + default 56 if 64BIT + config PAGE_OFFSET hex default 0xC0000000 if 32BIT && MAXPHYSMEM_2GB default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB +config ARCH_FLATMEM_ENABLE + def_bool y + +config ARCH_SPARSEMEM_ENABLE + def_bool y + select SPARSEMEM_VMEMMAP_ENABLE + +config ARCH_SELECT_MEMORY_MODEL + def_bool ARCH_SPARSEMEM_ENABLE + config STACKTRACE_SUPPORT def_bool y
@@ -92,6 +112,9 @@ config PGTABLE_LEVELS config HAVE_KPROBES def_bool n +config HAVE_ARCH_PFN_VALID + def_bool y + menu "Platform type" choice
arch/riscv/include/asm/pgtable.h +17 -4
@@ -89,6 +89,23 @@ extern pgd_t swapper_pg_dir[];#define __S110 PAGE_SHARED_EXEC#define __S111 PAGE_SHARED_EXEC +#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) +#define VMALLOC_END (PAGE_OFFSET - 1) +#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) + +/* +* Roughly size the vmemmap space to be large enough to fit enough +* struct pages to map half the virtual address space. Then +* position vmemmap directly below the VMALLOC region. +*/ +#define VMEMMAP_SHIFT \ + (CONFIG_VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT) +#define VMEMMAP_SIZE (1UL << VMEMMAP_SHIFT) +#define VMEMMAP_END (VMALLOC_START - 1) +#define VMEMMAP_START (VMALLOC_START - VMEMMAP_SIZE) + +#define vmemmap ((struct page *)VMEMMAP_START) +/* * ZERO_PAGE is a global shared page that is always zero, * used for zero-mapped memory areas, etc.
@@ -411,10 +428,6 @@ static inline void pgtable_cache_init(void)/* No page table caches to initialize */ } -#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) -#define VMALLOC_END (PAGE_OFFSET - 1) -#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) -/* * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32. * Note that PGDIR_SIZE must evenly divide TASK_SIZE.
arch/riscv/include/asm/sparsemem.h new file +11 -0
@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_SPARSEMEM_H +#define __ASM_SPARSEMEM_H + +#ifdef CONFIG_SPARSEMEM +#define MAX_PHYSMEM_BITS CONFIG_PA_BITS +#define SECTION_SIZE_BITS 30 +#endif /* CONFIG_SPARSEMEM */ + +#endif /* __ASM_SPARSEMEM_H */
arch/riscv/kernel/setup.c +3 -1
@@ -205,6 +205,9 @@ static void __init setup_bootmem(void)PFN_PHYS (end_pfn - start_pfn), &memblock.memory, 0); } + +memblocks_present (); +sparse_init (); }void __init setup_arch (char **cmdline_p)
@@ -239,4 +242,3 @@ void __init setup_arch(char **cmdline_p)riscv_fill_hwcap (); } -
arch/riscv/mm/init.c +8 -0
@@ -70,3 +70,11 @@ void free_initrd_mem(unsigned long start, unsigned long end) { }#endif /* CONFIG_BLK_DEV_INITRD */ + +#ifdef CONFIG_SPARSEMEM +int __meminit vmemmap_populate (unsigned long start,unsigned long end,int node, +struct vmem_altmap *altmap) +{ +return vmemmap_populate_basepages (start, end, node); +} +#endif
Signature · 1 line
2.19.0
Re: [PATCH v2 6/6] RISC-V: Implement sparsemem
Hello Logan,
Στις 2018-10-15 20:57, Logan Gunthorpe έγραψε:
Quoted 127 lines
This patch implements sparsemem support for risc-v which helps pave the way for memory hotplug and eventually P2P support.
We introduce
Kconfigoptions for virtual and physical address bits which
are used to calculate the size of the vmemmap and set theMAX_PHYSMEM_BITS.The vmemmap is located directly before the VMALLOC region and sized such that we can allocate enough pages to populate all the virtual address space in the system (similar to the way it's done in arm64).
During initialization, call
memblocks_present()andsparse_init(), and provide a stub forvmemmap_populate()(all of which is similar to arm64).Signed-off-by Logan Gunthorpe <[email protected]>Reviewed-by Palmer Dabbelt <[email protected]>Cc Albert Ou <[email protected]>Cc Andrew Waterman <[email protected]>Cc Olof Johansson <[email protected]>Cc Michael Clark <[email protected]>Cc Rob Herring <[email protected]>Cc Zong Li <[email protected]>--- arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ arch/riscv/kernel/setup.c | 4 +++- arch/riscv/mm/init.c | 8 ++++++++ 5 files changed, 62 insertions(+), 5 deletions(-) create mode 100644 arch/riscv/include/asm/sparsemem.harch/riscv/Kconfig +23
-0@@ -52,12 +52,32 @@ config ZONE_DMA32 bool default y if 64BIT +config VA_BITS + int + default 32 if 32BIT + default 39 if 64BIT + +config PA_BITS + int + default 34 if 32BIT + default 56 if 64BIT + config PAGE_OFFSET hex default 0xC0000000 if 32BIT && MAXPHYSMEM_2GB default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB +config ARCH_FLATMEM_ENABLE + def_bool y + +config ARCH_SPARSEMEM_ENABLE + def_bool y + select SPARSEMEM_VMEMMAP_ENABLE + +config ARCH_SELECT_MEMORY_MODEL + def_bool ARCH_SPARSEMEM_ENABLE + config STACKTRACE_SUPPORT def_bool y@@ -92,6 +112,9 @@ config PGTABLE_LEVELS config HAVE_KPROBES def_bool n +config HAVE_ARCH_PFN_VALID + def_bool y + menu "Platform type" choicearch/riscv/include/asm/pgtable.h +17
-4@@ -89,6 +89,23 @@ extern pgd_t swapper_pg_dir[];#define __S110 PAGE_SHARED_EXEC#define __S111 PAGE_SHARED_EXEC +#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) +#define VMALLOC_END (PAGE_OFFSET - 1) +#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) + +/* +* Roughly size the vmemmap space to be large enough to fit enough +* struct pages to map half the virtual address space. Then +* position vmemmap directly below the VMALLOC region. +*/ +#define VMEMMAP_SHIFT \ + (CONFIG_VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT) +#define VMEMMAP_SIZE (1UL << VMEMMAP_SHIFT) +#define VMEMMAP_END (VMALLOC_START - 1) +#define VMEMMAP_START (VMALLOC_START - VMEMMAP_SIZE) + +#define vmemmap ((struct page *)VMEMMAP_START) +/* * ZERO_PAGE is a global shared page that is always zero, * used for zero-mapped memory areas, etc. @@ -411,10 +428,6 @@ static inline void pgtable_cache_init(void)/* No page table caches to initialize */ } -#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) -#define VMALLOC_END (PAGE_OFFSET - 1) -#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) -/* * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32. * Note that PGDIR_SIZE must evenly divide TASK_SIZE. arch/riscv/include/asm/sparsemem.h new file +0
-0
arch/riscv/include/asm/sparsemem.h line 8
Quoted 8 lines
arch/riscv/include/asm/sparsemem.h new file +8
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_SPARSEMEM_H +#define __ASM_SPARSEMEM_H + +#ifdef CONFIG_SPARSEMEM +#define MAX_PHYSMEM_BITS CONFIG_PA_BITS +#define SECTION_SIZE_BITS 30
Having memory blocks of a minimum size of 1GB doesn't make much sense. It makes it harder to implement hotplug on top of this since we'll only able to add/remove 1GB at a time. ARM used to do the same and they switched to 27bits (https://patchwork.kernel.org/patch/9172845/), ARM64 still uses 1GB, x86 also uses 27bits and most archs also use something below 30. I believe we should go for 27bits as well or even better have this as a compile time option.
BTW memblocks_present is on master now (got merged 3 days ago).
Regards,
N.
Re: [PATCH v2 6/6] RISC-V: Implement sparsemem
Quoted 7 lines · Nick Kossifidis
Having memory blocks of a minimum size of 1GB doesn't make much sense. It makes it harder to implement hotplug on top of this since we'll only able to add/remove 1GB at a time. ARM used to do the same and they switched to 27bits (https://patchwork.kernel.org/patch/9172845/), ARM64 still uses 1GB, x86 also uses 27bits and most archs also use something below 30. I believe we should go for 27bits as well or even better have this as a compile time option.
Thanks, that makes sense. I'll make the change for the next time we submit.
Quoted 1 line
BTW
memblocks_presentis on master now (got merged 3 days ago).
Great! We'll send an updated patch set after the merge window.
Logan
Quoted 6 lines · logang at deltatee.com
This patchset implements sparsemem on RISC-V. The first few patches move some code in existing architectures into common helpers so they can be used by the new RISC-V implementation. The final patch actually adds sparsmem support to RISC-V.
This is the first small step in supporting P2P on RISC-V.
Thanks. I see less maintainer tags for the parts that touch other ports than I would feel comfortable merging. I'm going to let this sit in my inbox for a bit and we'll see if anything collects.
For patch sets I submit that clean up other ports I've attempted to split the patch into N patch sets, where:
* One part adds the generic support, which starts out as dead code.
* One part per arch uses the generic support.
This is a bit of a headache, but it at least allows us to get the RISC-V version that uses the generic support in quickly while waiting on acks from the other arch maintainers.
Like I said, I'll wait a bit and hope people ack.
Thanks!