sparsemem support for RISC-V
Subsystems (3)
v2 cover sparsemem support for RISC-V
This patchset implements sparsemem on RISC-V. The first couple patches from the previous version have been merged into v4.20 and the ARM patches have been sent separately to the ARM maintainers. These are the RISC-V patches only.
This is the first small step in supporting P2P on RISC-V.
--
Changes in v4:
* Rebased onto v5.0-rc1
* Changed the SECTION_SIZE_BITS to 27, per Nick Kossifidis
Changes in v3 (only sent the common patches):
* Rebased on v4.20-rc1
* Minor fixups
* Collected Ack from Will Deacon
Changes in v2:
* Rebase on v4.19-rc8
* Move the STRUCT_PAGE_MAX_SHIFT define into a common header (near the definition of struct page). As suggested by Christoph.
* Clean up the unnecessary nid variable in the memblocks_present() function, per Christoph.
* Collected tags from Palmer and Catalin.
--
Logan Gunthorpe (2):
sh: mm: make use of new memblocks_present() helper
RISC-V: Implement sparsemem
arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ arch/riscv/kernel/setup.c | 4 +++- arch/riscv/mm/init.c | 8 ++++++++ arch/sh/mm/init.c | 7 +------ 6 files changed, 63 insertions(+), 11 deletions(-) create mode 100644 arch/riscv/include/asm/sparsemem.h
--
2.19.0
PATCH v4 1/2 sh: mm: make use of new memblocks_present() helper
Cleanup the open coded for_each_memblock() loop that is equivalent to the new memblocks_present() helper.
--- arch/sh/mm/init.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
arch/sh/mm/init.c +1 -6
@@ -234,12 +234,7 @@ static void __init do_init_bootmem(void)plat_mem_setup (); -for_each_memblock (memory, reg) { -int nid =memblock_get_region_node (reg); - -memory_present (nid,memblock_region_memory_base_pfn (reg), -memblock_region_memory_end_pfn (reg)); - } +memblocks_present ();sparse_init (); }
Signature · 1 line
2.19.0
Re: [PATCH v4 1/2] sh: mm: make use of new memblocks_present() helper
Quoted 2 lines · Logan Gunthorpe
Cleanup the open coded
for_each_memblock()loop that is equivalent to the newmemblocks_present()helper.
Looks good:
Probably no need to keep this in a series with the RISC-V code, this can be queued up by the sh folks independently.
Re: [PATCH v4 1/2] sh: mm: make use of new memblocks_present() helper
Quoted 4 lines · Christoph Hellwig
Reviewed-by Christoph Hellwig <[email protected]>Probably no need to keep this in a series with the RISC-V code, this can be queued up by the sh folks independently.
Oh, yeah, sorry, I obviously didn't pay enough attention to this when I resent it. I sent the similar Arm changes to the appropriate list and, yes, the sh ones should go to that maintainer.
@Palmer: if you can just look at taking the second patch, I'll resubmit the first one to the appropriate list.
Thanks,
Logan
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH v4 2/2 RISC-V: Implement sparsemem
In reply to Logan GunthorpeThis patch implements sparsemem support for risc-v which helps pave the way for memory hotplug and eventually P2P support.
We introduce Kconfig options for virtual and physical address bits which are used to calculate the size of the vmemmap and set the MAX_PHYSMEM_BITS.
The vmemmap is located directly before the VMALLOC region and sized such that we can allocate enough pages to populate all the virtual address space in the system (similar to the way it's done in arm64).
During initialization, call memblocks_present() and sparse_init(), and provide a stub for vmemmap_populate() (all of which is similar to arm64).
--- arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ arch/riscv/kernel/setup.c | 4 +++- arch/riscv/mm/init.c | 8 ++++++++ 5 files changed, 62 insertions(+), 5 deletions(-) create mode 100644 arch/riscv/include/asm/sparsemem.h
arch/riscv/Kconfig +23 -0
@@ -54,12 +54,32 @@ config ZONE_DMA32 bool default y if 64BIT +config VA_BITS + int + default 32 if 32BIT + default 39 if 64BIT + +config PA_BITS + int + default 34 if 32BIT + default 56 if 64BIT + config PAGE_OFFSET hex default 0xC0000000 if 32BIT && MAXPHYSMEM_2GB default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB +config ARCH_FLATMEM_ENABLE + def_bool y + +config ARCH_SPARSEMEM_ENABLE + def_bool y + select SPARSEMEM_VMEMMAP_ENABLE + +config ARCH_SELECT_MEMORY_MODEL + def_bool ARCH_SPARSEMEM_ENABLE + config STACKTRACE_SUPPORT def_bool y
@@ -94,6 +114,9 @@ config PGTABLE_LEVELS config HAVE_KPROBES def_bool n +config HAVE_ARCH_PFN_VALID + def_bool y + menu "Platform type" choice
arch/riscv/include/asm/pgtable.h +17 -4
@@ -89,6 +89,23 @@ extern pgd_t swapper_pg_dir[];#define __S110 PAGE_SHARED_EXEC#define __S111 PAGE_SHARED_EXEC +#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) +#define VMALLOC_END (PAGE_OFFSET - 1) +#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) + +/* +* Roughly size the vmemmap space to be large enough to fit enough +* struct pages to map half the virtual address space. Then +* position vmemmap directly below the VMALLOC region. +*/ +#define VMEMMAP_SHIFT \ + (CONFIG_VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT) +#define VMEMMAP_SIZE (1UL << VMEMMAP_SHIFT) +#define VMEMMAP_END (VMALLOC_START - 1) +#define VMEMMAP_START (VMALLOC_START - VMEMMAP_SIZE) + +#define vmemmap ((struct page *)VMEMMAP_START) +/* * ZERO_PAGE is a global shared page that is always zero, * used for zero-mapped memory areas, etc.
@@ -411,10 +428,6 @@ static inline void pgtable_cache_init(void)/* No page table caches to initialize */ } -#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) -#define VMALLOC_END (PAGE_OFFSET - 1) -#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) -/* * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32. * Note that PGDIR_SIZE must evenly divide TASK_SIZE.
arch/riscv/include/asm/sparsemem.h new file +11 -0
@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_SPARSEMEM_H +#define __ASM_SPARSEMEM_H + +#ifdef CONFIG_SPARSEMEM +#define MAX_PHYSMEM_BITS CONFIG_PA_BITS +#define SECTION_SIZE_BITS 27 +#endif /* CONFIG_SPARSEMEM */ + +#endif /* __ASM_SPARSEMEM_H */
arch/riscv/kernel/setup.c +3 -1
@@ -193,6 +193,9 @@ static void __init setup_bootmem(void)PFN_PHYS (end_pfn - start_pfn), &memblock.memory, 0); } + +memblocks_present (); +sparse_init (); }void __init setup_arch (char **cmdline_p)
@@ -224,4 +227,3 @@ void __init setup_arch(char **cmdline_p)riscv_fill_hwcap (); } -
arch/riscv/mm/init.c +8 -0
@@ -69,3 +69,11 @@ void free_initrd_mem(unsigned long start, unsigned long end) { }#endif /* CONFIG_BLK_DEV_INITRD */ + +#ifdef CONFIG_SPARSEMEM +int __meminit vmemmap_populate (unsigned long start,unsigned long end,int node, +struct vmem_altmap *altmap) +{ +return vmemmap_populate_basepages (start, end, node); +} +#endif
Signature · 1 line
2.19.0
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Looks fine,
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
In reply to Logan GunthorpeHi Logan,
Logan Gunthorpe <[email protected]> 於 2019年1月10日 週四 上午5:07寫道:
Quoted 141 lines
This patch implements sparsemem support for risc-v which helps pave the way for memory hotplug and eventually P2P support.
We introduce
Kconfigoptions for virtual and physical address bits which are used to calculate the size of the vmemmap and set theMAX_PHYSMEM_BITS.The vmemmap is located directly before the VMALLOC region and sized such that we can allocate enough pages to populate all the virtual address space in the system (similar to the way it's done in arm64).
During initialization, call
memblocks_present()andsparse_init(), and provide a stub forvmemmap_populate()(all of which is similar to arm64).Signed-off-by Logan Gunthorpe <[email protected]>Reviewed-by Palmer Dabbelt <[email protected]>Cc Albert Ou <[email protected]>Cc Andrew Waterman <[email protected]>Cc Olof Johansson <[email protected]>Cc Michael Clark <[email protected]>Cc Rob Herring <[email protected]>Cc Zong Li <[email protected]>--- arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/pgtable.h | 21 +++++++++++++++++---- arch/riscv/include/asm/sparsemem.h | 11 +++++++++++ arch/riscv/kernel/setup.c | 4 +++- arch/riscv/mm/init.c | 8 ++++++++ 5 files changed, 62 insertions(+), 5 deletions(-) create mode 100644 arch/riscv/include/asm/sparsemem.harch/riscv/Kconfig +23
-0@@ -54,12 +54,32 @@ config ZONE_DMA32 bool default y if 64BIT +config VA_BITS + int + default 32 if 32BIT + default 39 if 64BIT + +config PA_BITS + int + default 34 if 32BIT + default 56 if 64BIT + config PAGE_OFFSET hex default 0xC0000000 if 32BIT && MAXPHYSMEM_2GB default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB +config ARCH_FLATMEM_ENABLE + def_bool y + +config ARCH_SPARSEMEM_ENABLE + def_bool y + select SPARSEMEM_VMEMMAP_ENABLE + +config ARCH_SELECT_MEMORY_MODEL + def_bool ARCH_SPARSEMEM_ENABLE + config STACKTRACE_SUPPORT def_bool y@@ -94,6 +114,9 @@ config PGTABLE_LEVELS config HAVE_KPROBES def_bool n +config HAVE_ARCH_PFN_VALID + def_bool y + menu "Platform type" choicearch/riscv/include/asm/pgtable.h +17
-4@@ -89,6 +89,23 @@ extern pgd_t swapper_pg_dir[];#define __S110 PAGE_SHARED_EXEC#define __S111 PAGE_SHARED_EXEC +#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) +#define VMALLOC_END (PAGE_OFFSET - 1) +#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) + +/* +* Roughly size the vmemmap space to be large enough to fit enough +* struct pages to map half the virtual address space. Then +* position vmemmap directly below the VMALLOC region. +*/ +#define VMEMMAP_SHIFT \ + (CONFIG_VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT) +#define VMEMMAP_SIZE (1UL << VMEMMAP_SHIFT) +#define VMEMMAP_END (VMALLOC_START - 1) +#define VMEMMAP_START (VMALLOC_START - VMEMMAP_SIZE) + +#define vmemmap ((struct page *)VMEMMAP_START) +/* * ZERO_PAGE is a global shared page that is always zero, * used for zero-mapped memory areas, etc. @@ -411,10 +428,6 @@ static inline void pgtable_cache_init(void)/* No page table caches to initialize */ } -#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) -#define VMALLOC_END (PAGE_OFFSET - 1) -#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) -/* * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32. * Note that PGDIR_SIZE must evenly divide TASK_SIZE. arch/riscv/include/asm/sparsemem.h new file +11
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_SPARSEMEM_H +#define __ASM_SPARSEMEM_H + +#ifdef CONFIG_SPARSEMEM +#define MAX_PHYSMEM_BITS CONFIG_PA_BITS +#define SECTION_SIZE_BITS 27 +#endif /* CONFIG_SPARSEMEM */ + +#endif /* __ASM_SPARSEMEM_H */ arch/riscv/kernel/setup.c +0
-0
arch/riscv/kernel/setup.c static void __init setup_bootmem(void) · line 199
Quoted 7 lines
arch/riscv/kernel/setup.c +3
-0@@ -193,6 +193,9 @@ static void __init setup_bootmem(void)PFN_PHYS (end_pfn - start_pfn), &memblock.memory, 0); } + +memblocks_present (); +sparse_init (); }
I just applied this patch to Linux kernel 5.2. I used a dts with 2 memory nodes with hole int it.
memory@80000000 {
device_type = "memory";
reg = <0x0 0x80000000 0x0 0x40000000>;
};
memory@180000000 {
device_type = "memory";
reg = <0x1 0x80000000 0x0 0x40000000>;
};I found it will boot failure. Did I miss anything?
[ 0.000000] Sorting __ex_table... [ 0.000000] BUG: Bad page state in process swapper pfn:180001 [ 0.000000] page:ffffffcf05400038 refcount:0 mapcount:94371937 mapping:00000000ffffffff index:0x4000000000000000 [ 0.000000] anon [ 0.000000] flags: 0x0() [ 0.000000] raw: 0000000000000000 0000000000000000 0000000000000000 00000000ffffffff [ 0.000000] raw: 4000000000000000 ffffffcf05a00060 0000000005a00060 [ 0.000000] page dumped because: non-NULL mapping [ 0.000000] Modules linked in: [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.2.0-00001-g737d8214d9a9 #3 [ 0.000000] Call Trace: [ 0.000000] [<ffffffe00017759c>] walk_stackframe+0x0/0xa0 [ 0.000000] [<ffffffe00017769c>] show_stack+0x2a/0x34 [ 0.000000] [<ffffffe00070c53e>] dump_stack+0x62/0x7c [ 0.000000] [<ffffffe0002330ae>] bad_page+0xca/0x120 [ 0.000000] [<ffffffe00023313c>] free_pages_check_bad+0x38/0x7a [ 0.000000] [<ffffffe00023368a>] __free_pages_ok+0x496/0x4ba [ 0.000000] [<ffffffe000234a82>] __free_pages.part.4+0xe/0x22 [ 0.000000] [<ffffffe000234c9e>] __free_pages_core+0x9a/0xa6 [ 0.000000] [<ffffffe000009b0a>] memblock_free_pages+0x12/0x1a [ 0.000000] [<ffffffe00000b496>] memblock_free_all+0x144/0x1a8 [ 0.000000] [<ffffffe00000274a>] mem_init+0x28/0x36 [ 0.000000] [<ffffffe0000008a0>] start_kernel+0x1bc/0x360 [ 0.000000] [<ffffffe000000074>] clear_bss_done+0x34/0x38 [ 0.000000] Disabling lock debugging due to kernel taint [ 0.000000] BUG: Bad page state in process swapper pfn:180002 [ 0.000000] page:ffffffcf05400070 refcount:0 mapcount:94371993 mapping:00000000ffffffff index:0x4000000000000000 [ 0.000000] anon [ 0.000000] flags: 0x0() [ 0.000000] raw: 0000000000000000 0000000000000000 0000000000000000 00000000ffffffff [ 0.000000] raw: 4000000000000000 ffffffcf05a00098 0000000005a00098 [ 0.000000] page dumped because: non-NULL mapping [ 0.000000] Modules linked in: [ 0.000000] CPU: 0 PID: 0 Comm: swapper Tainted: G B 5.2.0-00001-g737d8214d9a9 #3 [ 0.000000] Call Trace: [ 0.000000] [<ffffffe00017759c>] walk_stackframe+0x0/0xa0 [ 0.000000] [<ffffffe00017769c>] show_stack+0x2a/0x34 [ 0.000000] [<ffffffe00070c53e>] dump_stack+0x62/0x7c [ 0.000000] [<ffffffe0002330ae>] bad_page+0xca/0x120 [ 0.000000] [<ffffffe00023313c>] free_pages_check_bad+0x38/0x7a [ 0.000000] [<ffffffe00023368a>] __free_pages_ok+0x496/0x4ba [ 0.000000] [<ffffffe000234a82>] __free_pages.part.4+0xe/0x22 [ 0.000000] [<ffffffe000234c9e>] __free_pages_core+0x9a/0xa6 [ 0.000000] [<ffffffe000009b0a>] memblock_free_pages+0x12/0x1a [ 0.000000] [<ffffffe00000b496>] memblock_free_all+0x144/0x1a8 [ 0.000000] [<ffffffe00000274a>] mem_init+0x28/0x36 [ 0.000000] [<ffffffe0000008a0>] start_kernel+0x1bc/0x360 [ 0.000000] [<ffffffe000000074>] clear_bss_done+0x34/0x38 [ 0.000000] BUG: Bad page state in process swapper pfn:180003 [ 0.000000] page:ffffffcf054000a8 refcount:0 mapcount:94372049 mapping:00000000ffffffff index:0x4000000000000000 [ 0.000000] anon [ 0.000000] flags: 0x0() [ 0.000000] raw: 0000000000000000 0000000000000000 0000000000000000 00000000ffffffff [ 0.000000] raw: 4000000000000000 ffffffcf05a000d0 0000000005a000d0 [ 0.000000] page dumped because: non-NULL mapping
I look this issue more closely.
I found it always sets each memblock region to node 0. Does this make sense? I am not sure if I understand this correctly. Do you have any idea for this? Thank you. :)
for_each_memblock(memory, reg) {
unsigned long start_pfn = memblock_region_memory_base_pfn(reg);
unsigned long end_pfn = memblock_region_memory_end_pfn(reg);
memblock_set_node(PFN_PHYS(start_pfn), PFN_PHYS(end_pfn -start_pfn), &memblock.memory, 0);
^^^ }
[ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080200000-0x00000000bfffffff] [ 0.000000] node 0: [mem 0x0000000180000000-0x00000001bfffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000001bfffffff]
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 4 lines · Greentime Hu
I look this issue more closely.
I found it always sets each memblock region to node 0. Does this make sense? I am not sure if I understand this correctly. Do you have any idea for this? Thank you. :)
Yes, I think this is normal. When we talk about memory nodes we're talking about NUMA nodes which is unrelated to device tree nodes.
I'm not really sure what's causing the crash. Have you verified it's this patch that causes it? Is it related to there being a hole in your memory, does it work if you only have one memory node?
Thanks,
Logan
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月1日 週四 上午1:08寫道:
Quoted 11 lines · Greentime Hu
Quoted 4 lines
I look this issue more closely.
I found it always sets each memblock region to node 0. Does this make sense? I am not sure if I understand this correctly. Do you have any idea for this? Thank you. :)Yes, I think this is normal. When we talk about memory nodes we're talking about NUMA nodes which is unrelated to device tree nodes.
Ok, but it seems the second memblock_region may overwrite the first memblock_region in for_each_memblock(memory, reg) loop. It always uses this API to set to node 0. memblock_set_node(PFN_PHYS(start_pfn),PFN_PHYS(end_pfn - start_pfn), &memblock.memory,0)
Quoted 4 lines
I'm not really sure what's causing the crash. Have you verified it's this patch that causes it? Is it related to there being a hole in your memory, does it work if you only have one memory node?
It works fine if there is only one memory node described in dts.
I think it is related to there being a hole in the device tree script. I don't actually have a platform with a hole in the memory region, so I use device tree script to describe it.
The physical address layout will be like this.
2GB-3GB-hole-6GB-7GB
memory@80000000 {
device_type = "memory";
reg = <0x0 0x80000000 0x0 0x40000000>;
};
memory@180000000 {
device_type = "memory";
reg = <0x1 0x80000000 0x0 0x40000000>;
};Thank you for the quick reply. :)
A message that is not in the archive (sent off-list or not yet received): CAEbi=3eZcgWevpX9VO9ohgxVDFVprk_t52Xbs3-TdtZ+js3NVA@mail.gmail.com
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 23 lines · Greentime Hu
arch/riscv/Kconfig +0
-0@@ -116,7 +116,8 @@ config PGTABLE_LEVELSdefault 2
config
HAVE_ARCH_PFN_VALID
-def_booly
+ bool
+ default !SPARSEMEM_VMEMMAPmenu "Platform type"
arch/riscv/include/asm/page.h +1
-0@@ -93,16 +93,20 @@ extern unsigned long min_low_pfn; #define virt_to_pfn(vaddr) (phys_to_pfn(__pa(vaddr))) #define pfn_to_virt(pfn) (__va(pfn_to_phys(pfn))) +#if !defined(CONFIG_SPARSEMEM_VMEMMAP )
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 106
Quoted 10 lines
arch/riscv/include/asm/page.h +6
-0+#define pfn_valid (pfn) \ + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) #define virt_to_page(vaddr) (pfn_to_page(virt_to_pfn(vaddr))) #define page_to_virt(page) (pfn_to_virt(page_to_pfn(page))) +#else +#define virt_to_page (vaddr) ((struct page *)((((u64)vaddr - va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) +#define page_to_virt (pg) ((void *)(((((u64)pg - VMEMMAP_START) / sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) +#endif
This doesn't make sense to me at all. It should always use pfn_to_page() for virt_to_page() and the generic pfn_to_page()/page_to_pfn() implementations essentially already do what you are doing in a cleaner way. So I'd be really surprised if this does anything at all.
Logan
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月9日 週五 下午11:47寫道:
Quoted 26 lines · Greentime Hu
Quoted 22 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +0
-0@@ -93,16 +93,20 @@ extern unsigned long min_low_pfn;#define virt_to_pfn (vaddr) (phys_to_pfn(__pa(vaddr)))#define pfn_to_virt (pfn) (__va(pfn_to_phys(pfn)))
Quoted 17 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 106
Quoted 11 lines
arch/riscv/include/asm/page.h +7
-0+#if !defined(CONFIG_SPARSEMEM_VMEMMAP ) +#define pfn_valid (pfn) \ + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))#define virt_to_page (vaddr) (pfn_to_page(virt_to_pfn(vaddr)))#define page_to_virt (page) (pfn_to_virt(page_to_pfn(page))) +#else +#define virt_to_page (vaddr) ((struct page *)((((u64)vaddr - va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) +#define page_to_virt (pg) ((void *)(((((u64)pg - VMEMMAP_START) / sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) +#endif This doesn't make sense to me at all. It should always use
pfn_to_page()forvirt_to_page()and the generic pfn_to_page()/page_to_pfn() implementations essentially already do what you are doing in a cleaner way. So I'd be really surprised if this does anything at all.
Thank you for point me out that. I just checked the generic implementation and I should use that one.
Sorry I didn't check the generic one and just implement it again. I think the only patch we need is the first part to use generic pfn_valid(). I just tested it and yes it can boot successfully in dts with hole.
It will fail in this check ((pfn)-pfn_base) < max_mapnr. In my test case it will be ((6GB>>PAGE_SHIFT)-(2GB>>PAGE_SHIFT)=(4GB>>PAGE_SHIFT) < (3GB>>PAGE_SHIFT) to return false.
memory@80000000 {
device_type = "memory";
reg = <0x0 0x80000000 0x0 0x80000000>;
};
memory@180000000 {
device_type = "memory";
reg = <0x1 0x80000000 0x0 0x40000000>;};
To cause the check here failed to initialize page struct.
for (pfn = start_pfn; pfn < end_pfn; pfn++) {
/*
* There can be holes in boot-time mem_map[]s handed to this
* function. They do not exist on hotplugged memory.
*/
if (context == MEMMAP_EARLY) {
if (!early_pfn_valid(pfn))
continue;
if (!early_pfn_in_nid(pfn, nid))
continue;
if (overlap_memmap_init(zone, &pfn))
continue;
if (defer_init(nid, pfn, end_pfn))
break;
}
page = pfn_to_page(pfn);
__init_single_page(page, pfn, zone, nid);------------------------------------------------------------------------------
arch/riscv/Kconfig +2 -1
@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"
arch/riscv/include/asm/page.h +2 -0
@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if !defined(CONFIG_SPARSEMEM_VMEMMAP )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif #define ARCH_PFN_OFFSET (pfn_base)
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 47 lines · Greentime Hu
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月9日 週五 下午11:47寫道:
Quoted 26 lines · Greentime Hu
Quoted 22 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +0
-0@@ -93,16 +93,20 @@ extern unsigned long min_low_pfn;#define virt_to_pfn (vaddr) (phys_to_pfn(__pa(vaddr)))#define pfn_to_virt (pfn) (__va(pfn_to_phys(pfn)))Quoted 17 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 106
Quoted 11 lines
arch/riscv/include/asm/page.h +7
-0+#if !defined(CONFIG_SPARSEMEM_VMEMMAP ) +#define pfn_valid (pfn) \ + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))#define virt_to_page (vaddr) (pfn_to_page(virt_to_pfn(vaddr)))#define page_to_virt (page) (pfn_to_virt(page_to_pfn(page))) +#else +#define virt_to_page (vaddr) ((struct page *)((((u64)vaddr - va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) +#define page_to_virt (pg) ((void *)(((((u64)pg - VMEMMAP_START) / sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) +#endif This doesn't make sense to me at all. It should always use
pfn_to_page()forvirt_to_page()and the generic pfn_to_page()/page_to_pfn() implementations essentially already do what you are doing in a cleaner way. So I'd be really surprised if this does anything at all.
Quoted 8 lines
Thank you for point me out that. I just checked the generic implementation and I should use that one.
Sorry I didn't check the generic one and just implement it again. I think the only patch we need is the first part to use genericpfn_valid(). I just tested it and yes it can boot successfully in dts with hole.It will fail in this check ((pfn)-pfn_base) <
max_mapnr.
Sounds to me like max_mapnr is not set correctly. See the code in setup_bootmem(). Seems like 'mem_size' should be set to the largest memory block, not just the one that contains the kernel...
Quoted 19 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +0
-0
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 108
Quoted 9 lines
arch/riscv/include/asm/page.h +2
-0@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if !defined(CONFIG_SPARSEMEM_VMEMMAP )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif #define ARCH_PFN_OFFSET (pfn_base)
This patch still makes no sense. I'm not sure why we have an arch specific pfn_valid() because it's very similar to the generic one. But my guess is there's a reason for it and it's not doing what it is supposed when you remove it for the sparsemem case.
Logan
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月10日 週六 上午3:03寫道:
Quoted 92 lines · Greentime Hu
Quoted 47 lines
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月9日 週五 下午11:47寫道:
Quoted 26 lines · Greentime Hu
Quoted 22 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +0
-0@@ -93,16 +93,20 @@ extern unsigned long min_low_pfn;#define virt_to_pfn (vaddr) (phys_to_pfn(__pa(vaddr)))#define pfn_to_virt (pfn) (__va(pfn_to_phys(pfn)))Quoted 17 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 106
Quoted 11 lines
arch/riscv/include/asm/page.h +7
-0+#if !defined(CONFIG_SPARSEMEM_VMEMMAP ) +#define pfn_valid (pfn) \ + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))#define virt_to_page (vaddr) (pfn_to_page(virt_to_pfn(vaddr)))#define page_to_virt (page) (pfn_to_virt(page_to_pfn(page))) +#else +#define virt_to_page (vaddr) ((struct page *)((((u64)vaddr - va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) +#define page_to_virt (pg) ((void *)(((((u64)pg - VMEMMAP_START) / sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) +#endif This doesn't make sense to me at all. It should always use
pfn_to_page()forvirt_to_page()and the generic pfn_to_page()/page_to_pfn() implementations essentially already do what you are doing in a cleaner way. So I'd be really surprised if this does anything at all.Quoted 8 lines
Thank you for point me out that. I just checked the generic implementation and I should use that one.
Sorry I didn't check the generic one and just implement it again. I think the only patch we need is the first part to use genericpfn_valid(). I just tested it and yes it can boot successfully in dts with hole.It will fail in this check ((pfn)-pfn_base) <
max_mapnr.Sounds to me like
max_mapnris not set correctly. See the code insetup_bootmem(). Seems like 'mem_size' should be set to the largest memory block, not just the one that contains the kernel...Quoted 27 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +2
-0@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if !defined(CONFIG_SPARSEMEM_VMEMMAP )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif
Quoted 7 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 108
Quoted 1 line
arch/riscv/include/asm/page.h +0
-0#define ARCH_PFN_OFFSET (pfn_base)This patch still makes no sense. I'm not sure why we have an arch specific
pfn_valid()because it's very similar to the generic one. But my guess is there's a reason for it and it's not doing what it is supposed when you remove it for the sparsemem case.
It will use another pfn_valid() implementation ininclude/linux/mmzone.h if CONFIG_SPARSEMEM and
!CONFIG_HAVE_ARCH_PFN_VALID
It will be this one.
static inline int pfn_valid(unsigned long pfn)
{
if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS)
return 0;
return valid_section(__nr_to_section(pfn_to_section_nr(pfn)));
}This generic pfn_valid() API can check the pfn is valid or not even if there a hole in the memory.
For example:
A hole is between 0x100000000 to 0x180000000 (4GB-6GB) in my dts test case.
[ 0.000000] In setup_bootmem, pfn_valid(0x180000)=1 [ 0.000000] In setup_bootmem, pfn_valid(0x80000)=1 [ 0.000000] In setup_bootmem, pfn_valid(0x80200)=1 [ 0.000000] In setup_bootmem, pfn_valid(0x80300)=1 [ 0.000000] In setup_bootmem, pfn_valid(0x160000)=0 [ 0.000000] In setup_bootmem, pfn_valid(0x17ffff)=0 [ 0.000000] In setup_bootmem, pfn_valid(0x120000)=0 [ 0.000000] In setup_bootmem, pfn_valid(0x100000)=0 [ 0.000000] In setup_bootmem, pfn_valid(0xfffff)=1
This generic pfn_valid() could tell the pfn is valid or not.
I think this one is only available for flatmem.
#define pfn_valid(pfn) (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 108 lines · Greentime Hu
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月10日 週六 上午3:03寫道:
Quoted 92 lines · Greentime Hu
Quoted 47 lines
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月9日 週五 下午11:47寫道:
Quoted 26 lines · Greentime Hu
Quoted 22 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +0
-0@@ -93,16 +93,20 @@ extern unsigned long min_low_pfn;#define virt_to_pfn (vaddr) (phys_to_pfn(__pa(vaddr)))#define pfn_to_virt (pfn) (__va(pfn_to_phys(pfn)))Quoted 17 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 106
Quoted 11 lines
arch/riscv/include/asm/page.h +7
-0+#if !defined(CONFIG_SPARSEMEM_VMEMMAP ) +#define pfn_valid (pfn) \ + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))#define virt_to_page (vaddr) (pfn_to_page(virt_to_pfn(vaddr)))#define page_to_virt (page) (pfn_to_virt(page_to_pfn(page))) +#else +#define virt_to_page (vaddr) ((struct page *)((((u64)vaddr - va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) +#define page_to_virt (pg) ((void *)(((((u64)pg - VMEMMAP_START) / sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) +#endif This doesn't make sense to me at all. It should always use
pfn_to_page()forvirt_to_page()and the generic pfn_to_page()/page_to_pfn() implementations essentially already do what you are doing in a cleaner way. So I'd be really surprised if this does anything at all.Quoted 8 lines
Thank you for point me out that. I just checked the generic implementation and I should use that one.
Sorry I didn't check the generic one and just implement it again. I think the only patch we need is the first part to use genericpfn_valid(). I just tested it and yes it can boot successfully in dts with hole.It will fail in this check ((pfn)-pfn_base) <
max_mapnr.Sounds to me like
max_mapnris not set correctly. See the code insetup_bootmem(). Seems like 'mem_size' should be set to the largest memory block, not just the one that contains the kernel...Quoted 27 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +2
-0@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if !defined(CONFIG_SPARSEMEM_VMEMMAP )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif Quoted 7 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 108
Quoted 1 line
arch/riscv/include/asm/page.h +0
-0#define ARCH_PFN_OFFSET (pfn_base)This patch still makes no sense. I'm not sure why we have an arch specific
pfn_valid()because it's very similar to the generic one. But my guess is there's a reason for it and it's not doing what it is supposed when you remove it for the sparsemem case.It will use another
pfn_valid()implementation ininclude/linux/mmzone.hifCONFIG_SPARSEMEMand
!CONFIG_HAVE_ARCH_PFN_VALID
It will be this one.
Quoted 6 lines
static inline int
pfn_valid(unsigned long pfn){ if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) return 0; return valid_section(__nr_to_section(pfn_to_section_nr(pfn))); }
Ah, ok I see. "page.h" is only included in no-mmu arches. Which explains why riscv re-implements that macro. Couple follow up questions then:
* Did you test the memory-with-hole scenario without the sparsemem patches? It seems pfn_valid() will be wrong regardless of sparse/flat mem.
* Any chance we can just use the generic pfn_valid() function in all cases not just sparsemem? Can you test that?
Thanks,
Logan
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月12日 週一 下午11:52寫道:
Quoted 119 lines · Greentime Hu
Quoted 108 lines
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月10日 週六 上午3:03寫道:
Quoted 92 lines · Greentime Hu
Quoted 47 lines
Hi Logan,
Logan Gunthorpe <[email protected]> 於 2019年8月9日 週五 下午11:47寫道:
Quoted 26 lines · Greentime Hu
Quoted 22 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +0
-0@@ -93,16 +93,20 @@ extern unsigned long min_low_pfn;#define virt_to_pfn (vaddr) (phys_to_pfn(__pa(vaddr)))#define pfn_to_virt (pfn) (__va(pfn_to_phys(pfn)))Quoted 17 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 106
Quoted 11 lines
arch/riscv/include/asm/page.h +7
-0+#if !defined(CONFIG_SPARSEMEM_VMEMMAP ) +#define pfn_valid (pfn) \ + (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))#define virt_to_page (vaddr) (pfn_to_page(virt_to_pfn(vaddr)))#define page_to_virt (page) (pfn_to_virt(page_to_pfn(page))) +#else +#define virt_to_page (vaddr) ((struct page *)((((u64)vaddr - va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START)) +#define page_to_virt (pg) ((void *)(((((u64)pg - VMEMMAP_START) / sizeof(struct page)) * PAGE_SIZE) + va_pa_offset)) +#endif This doesn't make sense to me at all. It should always use
pfn_to_page()forvirt_to_page()and the generic pfn_to_page()/page_to_pfn() implementations essentially already do what you are doing in a cleaner way. So I'd be really surprised if this does anything at all.Quoted 8 lines
Thank you for point me out that. I just checked the generic implementation and I should use that one.
Sorry I didn't check the generic one and just implement it again. I think the only patch we need is the first part to use genericpfn_valid(). I just tested it and yes it can boot successfully in dts with hole.It will fail in this check ((pfn)-pfn_base) <
max_mapnr.Sounds to me like
max_mapnris not set correctly. See the code insetup_bootmem(). Seems like 'mem_size' should be set to the largest memory block, not just the one that contains the kernel...Quoted 27 lines
arch/riscv/Kconfig +2
-1@@ -116,7 +116,8 @@ config PGTABLE_LEVELS default 2 config HAVE_ARCH_PFN_VALID - def_bool y + bool + default !SPARSEMEM_VMEMMAP menu "Platform type"arch/riscv/include/asm/page.h +2
-0@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if !defined(CONFIG_SPARSEMEM_VMEMMAP )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif Quoted 7 lines
arch/riscv/include/asm/page.h extern unsigned long min_low_pfn; · line 108
Quoted 1 line
arch/riscv/include/asm/page.h +0
-0#define ARCH_PFN_OFFSET (pfn_base)This patch still makes no sense. I'm not sure why we have an arch specific
pfn_valid()because it's very similar to the generic one. But my guess is there's a reason for it and it's not doing what it is supposed when you remove it for the sparsemem case.It will use another
pfn_valid()implementation ininclude/linux/mmzone.hifCONFIG_SPARSEMEMand
!CONFIG_HAVE_ARCH_PFN_VALID
It will be this one.Quoted 6 lines
static inline int
pfn_valid(unsigned long pfn){ if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) return 0; return valid_section(__nr_to_section(pfn_to_section_nr(pfn))); }
Quoted 9 lines
Ah, ok I see. "
page.h" is only included in no-mmu arches. Which explains why riscv re-implements that macro. Couple follow up questions then:* Did you test the memory-with-hole scenario without the sparsemem patches? It seems
pfn_valid()will be wrong regardless of sparse/flat mem.* Any chance we can just use the generic
pfn_valid()function in all cases not just sparsemem? Can you test that?
I think flat mem doesn't support memory-with-hole scenario. In mm/Kconfig, it says
"
For systems that have holes in their physical address spaces and for features like NUMA and memory hotplug, choose "Sparse Memory"
"
IMHO, the memory-with-hole scenario should only be tested for sparse mem but flat mem.
The generic pfn_valid() is just for non-mmu arches. Every architecture with mmu defines their own pfn_valid().
This is supposed to be another separate patch that do we need to implement a generic pfn_valid().
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 9 lines · Greentime Hu
I think flat mem doesn't support memory-with-hole scenario. In mm/Kconfig, it says
"
For systems that have holes in their physical address spaces and for features like NUMA and memory hotplug, choose "Sparse Memory"
"
IMHO, the memory-with-hole scenario should only be tested for sparse mem but flat mem.
Fair enough.
Quoted 1 line
The generic
pfn_valid()is just for non-mmu arches.
The generic pfn_valid() in asm-generic is only for non-mmu arches.
Quoted 2 lines
Every architecture
with mmu defines their ownpfn_valid().
Not true. Arm64, for example just uses the generic implementation in mmzone.h. My main question is whether we can just do that. If we can't we should probably structure it like powerpc where they only use the arch-specific helper for CONFIG_FLATMEM instead of when CONFIG_SPARSEMEM isn't set.
Logan
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 6 lines · Logan Gunthorpe
Quoted 1 line · Greentime Hu
Every architecture with mmu defines their own
pfn_valid().Not true. Arm64, for example just uses the generic implementation in
mmzone.h.
arm64 seems to define their own:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/Kconfig#n899
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/mm/init.c#n235
While there are many architectures which have their own pfn_valid(); oddly, almost none of them set HAVE_ARCH_PFN_VALID ?
- Paul
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 9 lines · Paul Walmsley
Quoted 6 lines · Logan Gunthorpe
Quoted 1 line · Greentime Hu
Every architecture with mmu defines their own
pfn_valid().Not true. Arm64, for example just uses the generic implementation in
mmzone.h.
Quoted 8 lines
arm64 seems to define their own:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/Kconfig#n899
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/mm/init.c#n235
While there are many architectures which have their own
pfn_valid(); oddly, almost none of them setHAVE_ARCH_PFN_VALID?
(fixed the linux-mm@ address)
- Paul
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
In reply to Paul WalmsleyQuoted 4 lines · Paul Walmsley
Quoted 2 lines · Logan Gunthorpe
On 2019-08-13 12:04 a.m., Greentime Hu wrote:
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Every architecture with mmu defines their own
pfn_valid().Not true. Arm64, for example just uses the generic implementation in
mmzone.h.arm64 seems to define their own:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/Kconfig#n899
Oh, yup. My mistake.
Quoted 4 lines
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/mm/init.c#n235
While there are many architectures which have their own
pfn_valid(); oddly, almost none of them setHAVE_ARCH_PFN_VALID?
Yes, much of this is super confusing. Seems HAVE_ARCH_PFN_VALID only matters if SPARSEMEM is set. So risc-v probably doesn't need to set it and we just need a #ifdef !CONFIG_FLATMEM around the pfn_valid definition like other arches.
Logan
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Logan Gunthorpe <[email protected]> 於 2019年8月14日 週三 上午12:50寫道:
Quoted 19 lines · Paul Walmsley
Quoted 12 lines · Logan Gunthorpe
Quoted 7 lines · Greentime Hu
Quoted 2 lines
Every architecture with mmu defines their own
pfn_valid().Not true. Arm64, for example just uses the generic implementation in
mmzone.h.arm64 seems to define their own:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/Kconfig#n899
Oh, yup. My mistake.
Quoted 8 lines
Quoted 2 lines
While there are many architectures which have their own
pfn_valid(); oddly, almost none of them setHAVE_ARCH_PFN_VALID?Yes, much of this is super confusing. Seems
HAVE_ARCH_PFN_VALIDonly matters if SPARSEMEM is set. So risc-v probably doesn't need to set it and we just need a #ifdef !CONFIG_FLATMEM around thepfn_validdefinition like other arches.
Maybe this commit explains why it used HAVE_ARCH_PFN_VALID instead of SPARSEMEM.
https://github.com/torvalds/linux/commit/7b7bf499f79de3f6c85a340c8453a78789523f85
BTW, I found another issue here.
#define FIXADDR_TOP (VMALLOC_START)
#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
#define VMEMMAP_END (VMALLOC_START - 1)
#define VMEMMAP_START (VMALLOC_START - VMEMMAP_SIZE) These 2 regions are overlapped.
How about this fix? Not sure if it is good for everyone.
arch/riscv/Kconfig +0 -3
@@ -115,9 +115,6 @@ config PGTABLE_LEVELS default 3 if 64BIT default 2 -config HAVE_ARCH_PFN_VALID - def_bool y - menu "Platform type" choice
arch/riscv/include/asm/fixmap.h +1 -1
@@ -26,7 +26,7 @@ enum fixed_addresses { };#define FIXADDR_SIZE (__end_of_fixed_addresses * PAGE_SIZE) -#define FIXADDR_TOP (VMALLOC_START) +#define FIXADDR_TOP (VMEMMAP_START)#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)#define FIXMAP_PAGE_IO PAGE_KERNEL
arch/riscv/include/asm/page.h +2 -0
@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if defined(CONFIG_FLATMEM )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif #define ARCH_PFN_OFFSET (pfn_base)
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 32 lines · Greentime Hu
Logan Gunthorpe <[email protected]> 於 2019年8月14日 週三 上午12:50寫道:
Quoted 19 lines · Paul Walmsley
Quoted 12 lines · Logan Gunthorpe
Quoted 7 lines · Greentime Hu
Quoted 2 lines
Every architecture with mmu defines their own
pfn_valid().Not true. Arm64, for example just uses the generic implementation in
mmzone.h.arm64 seems to define their own:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/Kconfig#n899
Oh, yup. My mistake.
Quoted 8 lines
Quoted 2 lines
While there are many architectures which have their own
pfn_valid(); oddly, almost none of them setHAVE_ARCH_PFN_VALID?Yes, much of this is super confusing. Seems
HAVE_ARCH_PFN_VALIDonly matters if SPARSEMEM is set. So risc-v probably doesn't need to set it and we just need a #ifdef !CONFIG_FLATMEM around thepfn_validdefinition like other arches.Maybe this commit explains why it used
HAVE_ARCH_PFN_VALIDinstead of SPARSEMEM.
https://github.com/torvalds/linux/commit/7b7bf499f79de3f6c85a340c8453a78789523f85
Quoted 8 lines
BTW, I found another issue here.
#defineFIXADDR_TOP(VMALLOC_START)
#defineFIXADDR_START(FIXADDR_TOP-FIXADDR_SIZE)
#defineVMEMMAP_END(VMALLOC_START- 1)
#defineVMEMMAP_START(VMALLOC_START-VMEMMAP_SIZE) These 2 regions are overlapped.How about this fix? Not sure if it is good for everyone.
Yes, this looks good to me. I can fold these changes into my patch and send a v5 to the list.
Thanks!
Logan
Quoted 41 lines
arch/riscv/Kconfig +0
-3@@ -115,9 +115,6 @@ config PGTABLE_LEVELS default 3 if 64BIT default 2 -config HAVE_ARCH_PFN_VALID - def_bool y - menu "Platform type" choicearch/riscv/include/asm/fixmap.h +1
-1@@ -26,7 +26,7 @@ enum fixed_addresses { };#define FIXADDR_SIZE (__end_of_fixed_addresses * PAGE_SIZE) -#define FIXADDR_TOP (VMALLOC_START) +#define FIXADDR_TOP (VMEMMAP_START)#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)#define FIXMAP_PAGE_IO PAGE_KERNELarch/riscv/include/asm/page.h +2
-0@@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;#define page_to_bus (page) (page_to_phys(page))#define phys_to_page (paddr) (pfn_to_page(phys_to_pfn(paddr))) +#if defined(CONFIG_FLATMEM )#define pfn_valid (pfn) \ (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr)) +#endif #define ARCH_PFN_OFFSET (pfn_base)
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 12 lines · Logan Gunthorpe
Quoted 10 lines · Greentime Hu
Maybe this commit explains why it used
HAVE_ARCH_PFN_VALIDinstead of SPARSEMEM.
https://github.com/torvalds/linux/commit/7b7bf499f79de3f6c85a340c8453a78789523f85BTW, I found another issue here.
#defineFIXADDR_TOP(VMALLOC_START)
#defineFIXADDR_START(FIXADDR_TOP-FIXADDR_SIZE)
#defineVMEMMAP_END(VMALLOC_START- 1)
#defineVMEMMAP_START(VMALLOC_START-VMEMMAP_SIZE) These 2 regions are overlapped.
Quoted 4 lines
Quoted 1 line
How about this fix? Not sure if it is good for everyone.
Yes, this looks good to me. I can fold these changes into my patch and send a v5 to the list.
The change to FIXADDR_TOP should be separated out into its own patch - it probably needs to go up as a fix.
- Paul
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 14 lines · Paul Walmsley
Quoted 12 lines · Logan Gunthorpe
Quoted 10 lines · Greentime Hu
Maybe this commit explains why it used
HAVE_ARCH_PFN_VALIDinstead of SPARSEMEM.
https://github.com/torvalds/linux/commit/7b7bf499f79de3f6c85a340c8453a78789523f85BTW, I found another issue here.
#defineFIXADDR_TOP(VMALLOC_START)
#defineFIXADDR_START(FIXADDR_TOP-FIXADDR_SIZE)
#defineVMEMMAP_END(VMALLOC_START- 1)
#defineVMEMMAP_START(VMALLOC_START-VMEMMAP_SIZE) These 2 regions are overlapped.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
How about this fix? Not sure if it is good for everyone.
Yes, this looks good to me. I can fold these changes into my patch and send a v5 to the list.
The change to
FIXADDR_TOPshould be separated out into its own patch - it probably needs to go up as a fix.
I don't think so... VMEMMAP_START doesn't exist until the sparsemem patch so it can't be changed until after the sparsemem patch and no sense adding a bug in the sparsemem patch...
Logan
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 20 lines · Logan Gunthorpe
Quoted 19 lines · Paul Walmsley
Quoted 16 lines · Logan Gunthorpe
Quoted 11 lines · Greentime Hu
Maybe this commit explains why it used
HAVE_ARCH_PFN_VALIDinstead of SPARSEMEM.
https://github.com/torvalds/linux/commit/7b7bf499f79de3f6c85a340c8453a78789523f85BTW, I found another issue here.
#defineFIXADDR_TOP(VMALLOC_START)
#defineFIXADDR_START(FIXADDR_TOP-FIXADDR_SIZE)
#defineVMEMMAP_END(VMALLOC_START- 1)
#defineVMEMMAP_START(VMALLOC_START-VMEMMAP_SIZE) These 2 regions are overlapped.How about this fix? Not sure if it is good for everyone.
Yes, this looks good to me. I can fold these changes into my patch and send a v5 to the list.
Quoted 6 lines
Quoted 2 lines
The change to
FIXADDR_TOPshould be separated out into its own patch - it probably needs to go up as a fix.I don't think so...
VMEMMAP_STARTdoesn't exist until the sparsemem patch so it can't be changed until after the sparsemem patch and no sense adding a bug in the sparsemem patch...
OK, that's fine then.
- Paul
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
In reply to Greentime HuHey,
Quoted 1 line · Greentime Hu
How about this fix? Not sure if it is good for everyone.
I applied your fix to the patch and it seems ok. But it doesn't seem to work on a recent version of the kernel. Have you got it working on v5.3? It seems the following patch breaks things:
671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages")
I don't really have time right now to debug this any further.
Logan
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Hi Logan,
Quoted 6 lines · Logan Gunthorpe
Hey,
Quoted 1 line · Greentime Hu
How about this fix? Not sure if it is good for everyone.
Quoted 8 lines
I applied your fix to the patch and it seems ok. But it doesn't seem to work on a recent version of the kernel. Have you got it working on v5.3? It seems the following patch breaks things:
671f9a3e2e24("RISC-V: Setup initial page tables in two stages")I don't really have time right now to debug this any further.
I just tried v5.3-rc4 and it failed. I try to debug this case. I found it failed might be because of an unmapped virtual address is used in memblocks_present() -> memblock_alloc ().
In this commit 671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages"), it finishes all the VA/PA mapping after setup_vm_final() is called.
So we have to call memblocks_present() and sparse_init() right after setup_vm_final().
Here is my patch and I tested with memory-with-hole.
It can boot normally in Unleashed board after applying this patch.
arch/riscv/mm/init.c +2 -2
@@ -138,8 +138,6 @@ void __init setup_bootmem(void)PFN_PHYS (end_pfn - start_pfn), &memblock.memory, 0); } -memblocks_present (); -sparse_init (); }unsigned long va_pa_offset;
@@ -452,6 +450,8 @@ static void __init setup_vm_final(void)void __init paging_init (void ) {setup_vm_final (); +memblocks_present (); +sparse_init ();setup_zero_page ();zone_sizes_init (); }
Test case:
memory@80000000 {
device_type = "memory";
reg = <0x0 0x80000000 0x0 0x80000000>;
};
memory@180000000 {
device_type = "memory";
reg = <0x1 0x80000000 0x0 0x40000000>;
};# cat /proc/meminfo MemTotal: 3003496 kB MemFree: 2986584 kB MemAvailable: 2970176 kB Buffers: 0 kB Cached: 3540 kB SwapCached: 0 kB Active: 3920 kB Inactive: 68 kB Active(anon): 3920 kB Inactive(anon): 68 kB Active(file): 0 kB Inactive(file): 0 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 528 kB Mapped: 1984 kB Shmem: 3540 kB KReclaimable: 688 kB Slab: 5700 kB SReclaimable: 688 kB SUnreclaim: 5012 kB KernelStack: 424 kB PageTables: 80 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 1501748 kB Committed_AS: 3200 kB VmallocTotal: 67108863 kB VmallocUsed: 12 kB VmallocChunk: 0 kB Percpu: 272 kB # uname -a Linux buildroot 5.3.0-rc4-00001-g44404421c481-dirty #10 SMP Thu Aug 15 16:28:24 DST 2019 riscv64 GNU/Lin[ 2352.443621] random: fast init done ux # zcat /proc/config.gz |grep SPARSE CONFIG_SPARSE_IRQ=y CONFIG_ARCH_SPARSEMEM_ENABLE=y CONFIG_SPARSEMEM_MANUAL=y CONFIG_SPARSEMEM=y CONFIG_SPARSEMEM_EXTREME=y CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y CONFIG_SPARSEMEM_VMEMMAP=y # CONFIG_INPUT_SPARSEKMAP is not set
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 22 lines · Greentime Hu
Hi Logan,
Quoted 6 lines · Logan Gunthorpe
Hey,
Quoted 1 line · Greentime Hu
How about this fix? Not sure if it is good for everyone.
Quoted 8 lines
I applied your fix to the patch and it seems ok. But it doesn't seem to work on a recent version of the kernel. Have you got it working on v5.3? It seems the following patch breaks things:
671f9a3e2e24("RISC-V: Setup initial page tables in two stages")I don't really have time right now to debug this any further.
I just tried v5.3-rc4 and it failed. I try to debug this case. I found it failed might be because of an unmapped virtual address is used in
memblocks_present()->memblock_alloc().
Quoted 8 lines
In this commit
671f9a3e2e24("RISC-V: Setup initial page tables in two stages"), it finishes all the VA/PA mapping aftersetup_vm_final()is called.
So we have to callmemblocks_present()andsparse_init()right aftersetup_vm_final().Here is my patch and I tested with memory-with-hole.
It can boot normally in Unleashed board after applying this patch.
Great, thanks! I'll roll this into my patch and send v5 out when I have a moment. Can I add your Signed-off-by for the bits you've contributed to give you credit for your work?
Logan
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v4 2/2] RISC-V: Implement sparsemem
Quoted 32 lines · Logan Gunthorpe
Quoted 28 lines · Greentime Hu
Hi Logan,
Quoted 6 lines · Logan Gunthorpe
Hey,
Quoted 1 line · Greentime Hu
How about this fix? Not sure if it is good for everyone.
Quoted 8 lines
I applied your fix to the patch and it seems ok. But it doesn't seem to work on a recent version of the kernel. Have you got it working on v5.3? It seems the following patch breaks things:
671f9a3e2e24("RISC-V: Setup initial page tables in two stages")I don't really have time right now to debug this any further.
I just tried v5.3-rc4 and it failed. I try to debug this case. I found it failed might be because of an unmapped virtual address is used in
memblocks_present()->memblock_alloc().In this commit
671f9a3e2e24("RISC-V: Setup initial page tables in two stages"), it finishes all the VA/PA mapping aftersetup_vm_final()is called.
So we have to callmemblocks_present()andsparse_init()right aftersetup_vm_final().
Quoted 6 lines
Quoted 2 lines
Here is my patch and I tested with memory-with-hole.
It can boot normally in Unleashed board after applying this patch.Great, thanks! I'll roll this into my patch and send v5 out when I have a moment. Can I add your Signed-off-by for the bits you've contributed to give you credit for your work?
Sure. Please use my Signed-off-by: Greentime Hu <[email protected]>