remove size limit on XIP kernel
v2 cover remove size limit on XIP kernel
Hi,
For XIP kernel, the writable data section is always at offset specified in XIP_OFFSET, which is hard-coded to 32MB.
Unfortunately, this means the read-only section (placed before the writable section) is restricted in size. This causes build failure if the kernel gets too large.
This series remove the use of XIP_OFFSET one by one, then remove this macro entirely at the end, with the goal of lifting this size restriction.
Also some cleanup and documentation along the way.
This series depends on
https://lore.kernel.org/linux-riscv/[email protected]/
v2: address all Alex's comments (thanks Alex!). This includes the addition of a new patch (riscv: don't export va_kernel_pa_offset in vmcoreinfo for XIP kernel), which fix a build failure if CONFIG_VMCORE_INFO=y
Best regards,
Nam
Nam Cao (8):
riscv: cleanup XIP_FIXUP macro
riscv: don't export va_kernel_pa_offset in vmcoreinfo for XIP kernel
riscv: replace misleading va_kernel_pa_offset on XIP kernel
riscv: drop the use of XIP_OFFSET in XIP_FIXUP_OFFSET
riscv: drop the use of XIP_OFFSET in XIP_FIXUP_FLASH_OFFSET
riscv: drop the use of XIP_OFFSET in kernel_mapping_va_to_pa()
riscv: drop the use of XIP_OFFSET in create_kernel_page_table()
riscv: remove limit on the size of read-only section for XIP kernel
arch/riscv/include/asm/page.h | 29 ++++++++++++++++++++-------- arch/riscv/include/asm/pgtable.h | 18 +++++++---------- arch/riscv/include/asm/set_memory.h | 2 +- arch/riscv/include/asm/xip_fixup.h | 30 +++++++++++++++++++++++------ arch/riscv/kernel/vmcore_info.c | 7 +++++++ arch/riscv/kernel/vmlinux-xip.lds.S | 5 +++-- arch/riscv/mm/init.c | 13 ++++++++----- 7 files changed, 71 insertions(+), 33 deletions(-)
Signature · 1 line
2.39.2
PATCH v2 1/8 riscv: cleanup XIP_FIXUP macro
The XIP_FIXUP macro is used to fix addresses early during boot before MMU:
generated code "thinks" the data section is in ROM while it is actually in RAM. So this macro corrects the addresses in the data section.
This macro determines if the address needs to be fixed by checking if it is within the range starting from ROM address up to the size of (2 * XIP_OFFSET).
This means if the kernel size is bigger than (2 * XIP_OFFSET), some addresses would not be fixed up.
XIP kernel can still work if the above scenario does not happen. But this macro is obviously incorrect.
Rewrite this macro to only fix up addresses within the data section.
--- arch/riscv/include/asm/pgtable.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
arch/riscv/include/asm/pgtable.h +7 -4
@@ -142,11 +142,14 @@#ifdef CONFIG_XIP_KERNEL #define XIP_FIXUP (addr) ({ \ + extern char _sdata[], _start[], _end[]; \ + uintptr_t __rom_start_data = CONFIG_XIP_PHYS_ADDR \ + + (uintptr_t)&_sdata - (uintptr_t)&_start; \ + uintptr_t __rom_end_data = CONFIG_XIP_PHYS_ADDR \ + + (uintptr_t)&_end - (uintptr_t)&_start; \ uintptr_t __a = (uintptr_t)(addr); \ - (__a >= CONFIG_XIP_PHYS_ADDR && \ - __a < CONFIG_XIP_PHYS_ADDR + XIP_OFFSET * 2) ? \ - __a - CONFIG_XIP_PHYS_ADDR + CONFIG_PHYS_RAM_BASE - XIP_OFFSET :\ - __a; \ + (__a >= __rom_start_data && __a < __rom_end_data) ? \ + __a - __rom_start_data + CONFIG_PHYS_RAM_BASE : __a; \ })#else #define XIP_FIXUP (addr) (addr)
Signature · 1 line
2.39.2
PATCH v2 2/8 riscv: don't export va_kernel_pa_offset in vmcoreinfo for XIP kernel
In reply to Nam CaoThe crash utility uses va_kernel_pa_offset to translate virtual addresses. This is incorrect in the case of XIP kernel, because va_kernel_pa_offset is not the virtual-physical address offset (yes, the name is misleading; this variable will be removed for XIP in a following commit).
Stop exporting this variable for XIP kernel. The replacement is to be determined, note it as a TODO for now.
--- arch/riscv/kernel/vmcore_info.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/riscv/kernel/vmcore_info.c +7 -0
@@ -19,6 +19,13 @@ void arch_crash_save_vmcoreinfo(void)#endif #endif vmcoreinfo_append_str ("NUMBER(KERNEL_LINK_ADDR)=0x%lx\n" ,KERNEL_LINK_ADDR ); +#ifdef CONFIG_XIP_KERNEL +/* TODO: Communicate with crash-utility developers on the information to +* export. The XIP case is more complicated, because the virtual-physical +* address offset depends on whether the address is in ROM or in RAM. +*/ +#else vmcoreinfo_append_str ("NUMBER(va_kernel_pa_offset)=0x%lx\n" , kernel_map.va_kernel_pa_offset); +#endif }
Signature · 1 line
2.39.2
Re: [PATCH v2 2/8] riscv: don't export va_kernel_pa_offset in vmcoreinfo for XIP kernel
Hi Nam,
Quoted 21 lines · Nam Cao
The crash utility uses
va_kernel_pa_offsetto translate virtual addresses. This is incorrect in the case of XIP kernel, becauseva_kernel_pa_offsetis not the virtual-physical address offset (yes, the name is misleading; this variable will be removed for XIP in a following commit).Stop exporting this variable for XIP kernel. The replacement is to be determined, note it as a TODO for now.
Signed-off-by Nam Cao <[email protected]>--- arch/riscv/kernel/vmcore_info.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/riscv/kernel/vmcore_info.c +0
-0@@ -19,6 +19,13 @@ void arch_crash_save_vmcoreinfo(void)#endif #endif vmcoreinfo_append_str ("NUMBER(KERNEL_LINK_ADDR)=0x%lx\n" ,KERNEL_LINK_ADDR );
arch/riscv/kernel/vmcore_info.c void arch_crash_save_vmcoreinfo(void) · line 31
Quoted 10 lines
arch/riscv/kernel/vmcore_info.c +7
-0+#ifdef CONFIG_XIP_KERNEL +/* TODO: Communicate with crash-utility developers on the information to +* export. The XIP case is more complicated, because the virtual-physical +* address offset depends on whether the address is in ROM or in RAM. +*/ +#else vmcoreinfo_append_str ("NUMBER(va_kernel_pa_offset)=0x%lx\n" , kernel_map.va_kernel_pa_offset); +#endif }
So that will regress crash on XIP kernels, but on the other hand this was wrong in the first place. And not sure anyone ever used crash on a XIP platform.
At least people will know that using va_kernel_pa_offset was wrong, so:
Thanks for all your changes in v2,
Alex
PATCH v2 3/8 riscv: replace misleading va_kernel_pa_offset on XIP kernel
In reply to Nam CaoOn XIP kernel, the name "va_kernel_pa_offset" is misleading: unlike "normal" kernel, it is not the virtual-physical address offset of kernel mapping, it is the offset of kernel mapping's first virtual address to first physical address in DRAM, which is not meaningful because the kernel's first physical address is not in DRAM.
For XIP kernel, there are 2 different offsets because the read-only part of the kernel resides in ROM while the rest is in RAM. The offset to ROM is in kernel_map.va_kernel_xip_pa_offset, while the offset to RAM is not stored
anywhere: it is calculated on-the-fly.
Remove this confusing "va_kernel_pa_offset" and add "va_kernel_xip_data_pa_offset" as its replacement. This new variable is the offset of virtual mapping of the kernel's data portion to the corresponding physical addresses.
With the introduction of this new variable, also rename va_kernel_xip_pa_offset -> va_kernel_xip_text_pa_offset to make it clear that this one is about the .text section.
--- arch/riscv/include/asm/page.h | 29 +++++++++++++++++++++-------- arch/riscv/mm/init.c | 6 ++++-- 2 files changed, 25 insertions(+), 10 deletions(-)
arch/riscv/include/asm/page.h +21 -8
@@ -112,11 +112,13 @@ struct kernel_mapping {/* Offset between linear mapping virtual address and kernel load address */ unsigned long va_pa_offset;/* Offset between kernel mapping virtual address and kernel load address */ -unsigned long va_kernel_pa_offset; -unsigned long va_kernel_xip_pa_offset;#ifdef CONFIG_XIP_KERNEL +unsigned long va_kernel_xip_text_pa_offset; +unsigned long va_kernel_xip_data_pa_offset;uintptr_t xiprom;uintptr_t xiprom_sz; +#else +unsigned long va_kernel_pa_offset;#endif };
@@ -134,12 +136,18 @@ extern phys_addr_t phys_ram_base;#else void *linear_mapping_pa_to_va (unsigned long x);#endif + +#ifdef CONFIG_XIP_KERNEL #define kernel_mapping_pa_to_va (y) ({ \ unsigned long _y = (unsigned long)(y); \ - (IS_ENABLED(CONFIG_XIP_KERNEL) && _y < phys_ram_base) ? \ - (void *)(_y + kernel_map.va_kernel_xip_pa_offset) : \ - (void *)(_y + kernel_map.va_kernel_pa_offset + XIP_OFFSET); \ + (_y < phys_ram_base) ? \ + (void *)(_y + kernel_map.va_kernel_xip_text_pa_offset) : \ + (void *)(_y + kernel_map.va_kernel_xip_data_pa_offset); \ }) +#else +#define kernel_mapping_pa_to_va (y) ((void *)((unsigned long)(y) + kernel_map.va_kernel_pa_offset)) +#endif +#define __pa_to_va_nodebug (x) linear_mapping_pa_to_va(x)#ifndef CONFIG_DEBUG_VIRTUAL
@@ -147,12 +155,17 @@ void *linear_mapping_pa_to_va(unsigned long x);#else phys_addr_t linear_mapping_va_to_pa (unsigned long x);#endif + +#ifdef CONFIG_XIP_KERNEL #define kernel_mapping_va_to_pa (y) ({ \ unsigned long _y = (unsigned long)(y); \ - (IS_ENABLED(CONFIG_XIP_KERNEL) && _y < kernel_map.virt_addr + XIP_OFFSET) ? \ - (_y - kernel_map.va_kernel_xip_pa_offset) : \ - (_y - kernel_map.va_kernel_pa_offset - XIP_OFFSET); \ + (_y < kernel_map.virt_addr + XIP_OFFSET) ? \ + (_y - kernel_map.va_kernel_xip_text_pa_offset) : \ + (_y - kernel_map.va_kernel_xip_data_pa_offset); \ }) +#else +#define kernel_mapping_va_to_pa (y) ((unsigned long)(y) - kernel_map.va_kernel_pa_offset) +#endif #define __va_to_pa_nodebug (x) ({ \ unsigned long _x = x; \
arch/riscv/mm/init.c +4 -2
@@ -1092,11 +1092,14 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) kernel_map.phys_addr = (uintptr_t )CONFIG_PHYS_RAM_BASE ; kernel_map.size = (uintptr_t )(&_end) - (uintptr_t )(&_start); - kernel_map.va_kernel_xip_pa_offset = kernel_map.virt_addr - kernel_map.xiprom; + kernel_map.va_kernel_xip_text_pa_offset = kernel_map.virt_addr - kernel_map.xiprom; + kernel_map.va_kernel_xip_data_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr + + (uintptr_t )&_sdata - (uintptr_t )&_start;#else kernel_map.page_offset =_AC (CONFIG_PAGE_OFFSET ,UL ); kernel_map.phys_addr = (uintptr_t )(&_start); kernel_map.size = (uintptr_t )(&_end) - kernel_map.phys_addr; + kernel_map.va_kernel_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr;#endif #if defined(CONFIG_64BIT ) && !defined(CONFIG_XIP_KERNEL )
@@ -1118,7 +1121,6 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)*/ kernel_map.va_pa_offset =IS_ENABLED (CONFIG_64BIT ) ?0UL :PAGE_OFFSET - kernel_map.phys_addr; - kernel_map.va_kernel_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr;/* * The default maximal physical memory size is KERN_VIRT_SIZE for 32-bit
Signature · 1 line
2.39.2
PATCH v2 4/8 riscv: drop the use of XIP_OFFSET in XIP_FIXUP_OFFSET
In reply to Nam CaoXIP_OFFSET is the hard-coded offset of writable data section within the kernel.
By hard-coding this value, the read-only section of the kernel (which is placed before the writable data section) is restricted in size.
As a preparation to remove this hard-coded macro XIP_OFFSET entirely, stop using XIP_OFFSET in XIP_FIXUP_OFFSET. Instead, use CONFIG_PHYS_RAM_BASE and _sdata to do the same thing.
While at it, also add a description for XIP_FIXUP_OFFSET.
--- arch/riscv/include/asm/xip_fixup.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
arch/riscv/include/asm/xip_fixup.h +12 -2
@@ -9,8 +9,19 @@#ifdef CONFIG_XIP_KERNEL .macro XIP_FIXUP_OFFSET reg - REG_L t0, _xip_fixup +/* Fix-up address in Flash into address in RAM early during boot before +* MMU is up. Because generated code "thinks" data is in Flash, but it +* is actually in RAM (actually data is also in Flash, but Flash is +* read-only, thus we need to use the data residing in RAM). +* +* The start of data in Flash is _sdata and the start of data in RAM is +* CONFIG_PHYS_RAM_BASE. So this fix-up essentially does this: +* reg += CONFIG_PHYS_RAM_BASE - _start +*/ + li t0, CONFIG_PHYS_RAM_BASE add \reg, \reg, t0 + la t0, _sdata + sub \reg, \reg, t0 .endm .macro XIP_FIXUP_FLASH_OFFSET reg la t0, __data_loc
@@ -19,7 +30,6 @@ add \reg, \reg, t0 .endm -_xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_OFFSET _xip_phys_offset: .dword CONFIG_XIP_PHYS_ADDR + XIP_OFFSET #else .macro XIP_FIXUP_OFFSET reg
Signature · 1 line
2.39.2
PATCH v2 5/8 riscv: drop the use of XIP_OFFSET in XIP_FIXUP_FLASH_OFFSET
In reply to Nam CaoXIP_OFFSET is the hard-coded offset of writable data section within the kernel.
By hard-coding this value, the read-only section of the kernel (which is placed before the writable data section) is restricted in size.
As a preparation to remove this hard-coded macro XIP_OFFSET entirely, stop using XIP_OFFSET in XIP_FIXUP_FLASH_OFFSET. Instead, use __data_loc and _sdata to do the same thing.
While at it, also add a description for XIP_FIXUP_FLASH_OFFSET.
--- arch/riscv/include/asm/xip_fixup.h | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
arch/riscv/include/asm/xip_fixup.h +12 -4
@@ -24,13 +24,21 @@ sub \reg, \reg, t0 .endm .macro XIP_FIXUP_FLASH_OFFSET reg +/* In linker script, at the transition from read-only section to +* writable section, the VMA is increased while LMA remains the same. +* (See in linker script how _sdata, __data_loc and LOAD_OFFSET is +* changed) +* +* Consequently, early during boot before MMU is up, the generated code +* reads the "writable" section at wrong addresses, because VMA is used +* by compiler to generate code, but the data is located in Flash using +* LMA. +*/ + la t0, _sdata + sub \reg, \reg, t0 la t0, __data_loc - REG_L t1, _xip_phys_offset - sub \reg, \reg, t1 add \reg, \reg, t0 .endm - -_xip_phys_offset: .dword CONFIG_XIP_PHYS_ADDR + XIP_OFFSET #else .macro XIP_FIXUP_OFFSET reg .endm
Signature · 1 line
2.39.2
PATCH v2 6/8 riscv: drop the use of XIP_OFFSET in kernel_mapping_va_to_pa()
In reply to Nam CaoXIP_OFFSET is the hard-coded offset of writable data section within the kernel.
By hard-coding this value, the read-only section of the kernel (which is placed before the writable data section) is restricted in size.
As a preparation to remove this hard-coded macro XIP_OFFSET entirely, remove the use of XIP_OFFSET in kernel_mapping_va_to_pa(). The macro XIP_OFFSET is used in this case to check if the virtual address is mapped to Flash or to RAM. The same check can be done with kernel_map.xiprom_sz.
--- arch/riscv/include/asm/page.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/riscv/include/asm/page.h +1 -1
@@ -159,7 +159,7 @@ phys_addr_t linear_mapping_va_to_pa(unsigned long x);#ifdef CONFIG_XIP_KERNEL #define kernel_mapping_va_to_pa (y) ({ \ unsigned long _y = (unsigned long)(y); \ - (_y < kernel_map.virt_addr + XIP_OFFSET) ? \ + (_y < kernel_map.virt_addr + kernel_map.xiprom_sz) ? \ (_y - kernel_map.va_kernel_xip_text_pa_offset) : \ (_y - kernel_map.va_kernel_xip_data_pa_offset); \ })
Signature · 1 line
2.39.2
PATCH v2 7/8 riscv: drop the use of XIP_OFFSET in create_kernel_page_table()
In reply to Nam CaoXIP_OFFSET is the hard-coded offset of writable data section within the kernel.
By hard-coding this value, the read-only section of the kernel (which is placed before the writable data section) is restricted in size.
As a preparation to remove this hard-coded value entirely, stop using XIP_OFFSET in create_kernel_page_table(). Instead use _sdata and _start to do the same thing.
--- arch/riscv/mm/init.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
arch/riscv/mm/init.c +4 -3
@@ -911,7 +911,7 @@ static void __init relocate_kernel(void)static void __init create_kernel_page_table (pgd_t *pgdir,__always_unused bool early) { -uintptr_t va, end_va; +uintptr_t va, start_va, end_va;/* Map the flash resident part */ end_va = kernel_map.virt_addr + kernel_map.xiprom_sz;
@@ -921,10 +921,11 @@ static void __init create_kernel_page_table(pgd_t *pgdir, PMD_SIZE, PAGE_KERNEL_EXEC);/* Map the data in RAM */ + start_va = kernel_map.virt_addr + (uintptr_t )&_sdata - (uintptr_t )&_start; end_va = kernel_map.virt_addr + kernel_map.size; -for (va = kernel_map.virt_addr +XIP_OFFSET ; va < end_va; va +=PMD_SIZE ) +for (va = start_va; va < end_va; va +=PMD_SIZE )create_pgd_mapping (pgdir, va, - kernel_map.phys_addr + (va - (kernel_map.virt_addr +XIP_OFFSET )), + kernel_map.phys_addr + (va - start_va), PMD_SIZE, PAGE_KERNEL); }#else
Signature · 1 line
2.39.2
PATCH v2 8/8 riscv: remove limit on the size of read-only section for XIP kernel
In reply to Nam CaoXIP_OFFSET is the hard-coded offset of writable data section within the kernel.
By hard-coding this value, the read-only section of the kernel (which is placed before the writable data section) is restricted in size. This causes build failures if the kernel gets too big [1].
Remove this limit.
--- arch/riscv/include/asm/pgtable.h | 7 ------- arch/riscv/include/asm/set_memory.h | 2 +- arch/riscv/kernel/vmlinux-xip.lds.S | 5 +++-- 3 files changed, 4 insertions(+), 10 deletions(-)
arch/riscv/include/asm/pgtable.h +0 -7
@@ -107,13 +107,6 @@#endif -#ifdef CONFIG_XIP_KERNEL -#define XIP_OFFSET SZ_32M -#define XIP_OFFSET_MASK (SZ_32M - 1) -#else -#define XIP_OFFSET 0 -#endif -#ifndef __ASSEMBLY__#include <asm/page.h>
arch/riscv/include/asm/set_memory.h +1 -1
@@ -46,7 +46,7 @@ bool kernel_page_present(struct page *page);#endif /* __ASSEMBLY__ */ -#ifdef CONFIG_STRICT_KERNEL_RWX +#if defined(CONFIG_STRICT_KERNEL_RWX ) || defined(CONFIG_XIP_KERNEL )#ifdef CONFIG_64BIT #define SECTION_ALIGN (1 << 21)#else
arch/riscv/kernel/vmlinux-xip.lds.S +3 -2
@@ -14,6 +14,7 @@ #include <asm/page.h> #include <asm/cache.h> #include <asm/thread_info.h> +#include <asm/set_memory.h> OUTPUT_ARCH(riscv) ENTRY(_start)
@@ -65,10 +66,10 @@ SECTIONS * From this point, stuff is considered writable and will be copied to RAM */ __data_loc = ALIGN(PAGE_SIZE); /* location in file */ - . = KERNEL_LINK_ADDR + XIP_OFFSET; /* location in memory */ + . = ALIGN(SECTION_ALIGN); /* location in memory */ #undef LOAD_OFFSET -#define LOAD_OFFSET (KERNEL_LINK_ADDR + XIP_OFFSET - (__data_loc & XIP_OFFSET_MASK)) +#define LOAD_OFFSET (KERNEL_LINK_ADDR + _sdata - __data_loc) _sdata = .; /* Start of data section */ _data = .;
Signature · 1 line
2.39.2
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:
Hello:
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:
Quoted 10 lines · On Fri, 7 Jun 2024 22:22:05 +0200 you wrote:
Hi,
For XIP kernel, the writable data section is always at offset specified in
XIP_OFFSET, which is hard-coded to 32MB.Unfortunately, this means the read-only section (placed before the writable section) is restricted in size. This causes build failure if the kernel gets too large.
[...]
Here is the summary with links:
- [v2,1/8] riscv: cleanup XIP_FIXUP macro
https://git.kernel.org/riscv/c/aa3457f22f00 - [v2,2/8] riscv: don't export va_kernel_pa_offset in vmcoreinfo for XIP kernel https://git.kernel.org/riscv/c/f2df5b4fdd74 - [v2,3/8] riscv: replace misleading va_kernel_pa_offset on XIP kernel https://git.kernel.org/riscv/c/5cf089672119 - [v2,4/8] riscv: drop the use of XIP_OFFSET in XIP_FIXUP_OFFSET https://git.kernel.org/riscv/c/e4eac34feda4 - [v2,5/8] riscv: drop the use of XIP_OFFSET in XIP_FIXUP_FLASH_OFFSET https://git.kernel.org/riscv/c/23311f57ee13 - [v2,6/8] riscv: drop the use of XIP_OFFSET in kernel_mapping_va_to_pa() https://git.kernel.org/riscv/c/75fdf791dff0 - [v2,7/8] riscv: drop the use of XIP_OFFSET in create_kernel_page_table() https://git.kernel.org/riscv/c/a7cfb999433a - [v2,8/8] riscv: remove limit on the size of read-only section for XIP kernel https://git.kernel.org/riscv/c/b635a84bde6f
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html