RISCV64: Use va_kernel_pa_offset in VTOP()
Subsystems (3)
Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes the phys_ram_base from the kernel_map.phys_addr to the start of DRAM.
The Crash's VTOP() still uses phys_ram_base and kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.
Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "va_kernel_pa_offset" to translate the kernel virtual address in VTOP() correctly.
---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crash
And I'll send the Linux changes to riscv/for-next If you're ok with this patch.
--- defs.h | 4 ++-- riscv64.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-)
defs.h +2 -2
@@ -3662,8 +3662,7 @@ typedef signed int s32;ulong _X =X ; \ (THIS_KERNEL_VERSION >=LINUX (5 ,13 ,0 ) && \ (_X) >= machdep->machspec->kernel_link_addr) ? \ - (((unsigned long )(_X)-(machdep->machspec->kernel_link_addr)) + \ - machdep->machspec->phys_base): \ + ((unsigned long )(_X)-(machdep->machspec->va_kernel_pa_offset)): \ (((unsigned long )(_X)-(machdep->kvbase)) + \ machdep->machspec->phys_base); \ })
@@ -7021,6 +7020,7 @@ struct machine_specific {ulong modules_vaddr;ulong modules_end;ulong kernel_link_addr; +ulong va_kernel_pa_offset;ulong _page_present;ulong _page_read;
riscv64.c +22 -0
@@ -418,6 +418,27 @@ error:error (FATAL ,"cannot get vm layout\n" ); } +static void +riscv64_get_va_kernel_pa_offset (struct machine_specific *ms) +{ +unsigned long kernel_version =riscv64_get_kernel_version (); + +/* +* va_kernel_pa_offset is defined in Linux kernel since 6.5. +*/ +if (kernel_version >=LINUX (6 ,5 ,0 )) { +char *string; +if ((string = pc->read_vmcoreinfo ("NUMBER(va_kernel_pa_offset)" ))) { + ms->va_kernel_pa_offset =htol (string,QUIET ,NULL ); +free (string); + }else +error (FATAL ,"cannot read va_kernel_pa_offset\n" ); + }else if (kernel_version >=LINUX (6 ,4 ,0 )) +error (FATAL ,"cannot determine va_kernel_pa_offset since Linux 6.4\n" ); +else + ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base; +} +static int riscv64_is_kvaddr (ulong vaddr) {
@@ -1352,6 +1373,7 @@ riscv64_init(int when)riscv64_get_struct_page_size (machdep->machspec);riscv64_get_va_bits (machdep->machspec);riscv64_get_va_range (machdep->machspec); +riscv64_get_va_kernel_pa_offset (machdep->machspec);pt_level_alloc (&machdep->pgd,"cannot malloc pgd space." );pt_level_alloc (&machdep->machspec->p4d,"cannot malloc p4d space." );
Signature · 7 lines
2.20.1 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Hey,
Quoted 12 lines · Song Shuai
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.
Quoted 6 lines
Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crashAnd I'll send the Linux changes to riscv/for-next If you're ok with this patch.
If you want this to go into 6.5, you'll need to send it for riscv/fixes instead. It sounds like a fix for this would need to go into 6.4 too, no?
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 20 lines · Conor Dooley
Hey,
Quoted 17 lines · Song Shuai
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crash
Quoted 5 lines
Quoted 1 line
And I'll send the Linux changes to riscv/for-next If you're ok with this patch.
If you want this to go into 6.5, you'll need to send it for riscv/fixes instead. It sounds like a fix for this would need to go into 6.4 too, no?
You're right, that should be riscv/fixes for 6.5 and this issue also need to be fixed in 6.4 stable.
How about waiting for Crash guys' comments on the introduction of the "va_kernel_pa_offset" in vmcoreinfo and then determine which stable version should be taken in the first "if" of kernel_version.
Quoted 1 line
Signature · 7 lines
Thanks Song Shuai _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 27 lines · Song Shuai
Quoted 20 lines · Conor Dooley
Hey,
Quoted 17 lines · Song Shuai
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crashQuoted 6 lines
Quoted 2 lines
And I'll send the Linux changes to riscv/for-next If you're ok with this patch.
If you want this to go into 6.5, you'll need to send it for riscv/fixes instead. It sounds like a fix for this would need to go into 6.4 too, no?
Quoted 7 lines
You're right, that should be riscv/fixes for 6.5 and this issue also need to be fixed in 6.4 stable.
How about waiting for Crash guys' comments on the introduction of the "
va_kernel_pa_offset" in vmcoreinfo
and then determine which stable version should be taken in the first "if" ofkernel_version.
I don't have any specific comment on this, it looks necessary and if it's accepted in vmcoreinfo, then we can accept a crash patch for it.
Thanks,
Kazu
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Alexandre Ghiti
reviewerHi Song,
Quoted 3 lines · Song Shuai
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.
Maybe we could be more explicit here, kernel_map.phys_addr actually points to the physical start of the kernel so maybe something like that:
"changes phys_ram_base from the physical start of the kernel to the actual start of the DRAM."
Quoted 75 lines
The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crashAnd I'll send the Linux changes to riscv/for-next If you're ok with this patch.
--- defs.h | 4 ++-- riscv64.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-)defs.h +2
-2@@ -3662,8 +3662,7 @@ typedef signed int s32;ulong _X =X ; \ (THIS_KERNEL_VERSION >=LINUX (5 ,13 ,0 ) && \ (_X) >= machdep->machspec->kernel_link_addr) ? \ - (((unsigned long )(_X)-(machdep->machspec->kernel_link_addr)) + \ - machdep->machspec->phys_base): \ + ((unsigned long )(_X)-(machdep->machspec->va_kernel_pa_offset)): \ (((unsigned long )(_X)-(machdep->kvbase)) + \ machdep->machspec->phys_base); \ })@@ -7021,6 +7020,7 @@ struct machine_specific {ulong modules_vaddr;ulong modules_end;ulong kernel_link_addr; +ulong va_kernel_pa_offset;ulong _page_present;ulong _page_read;riscv64.c +21
-0@@ -418,6 +418,27 @@ error:error (FATAL ,"cannot get vm layout\n" ); } +static void +riscv64_get_va_kernel_pa_offset (struct machine_specific *ms) +{ +unsigned long kernel_version =riscv64_get_kernel_version (); + +/* +* va_kernel_pa_offset is defined in Linux kernel since 6.5. +*/ +if (kernel_version >=LINUX (6 ,5 ,0 )) { +char *string; +if ((string = pc->read_vmcoreinfo ("NUMBER(va_kernel_pa_offset)" ))) { + ms->va_kernel_pa_offset =htol (string,QUIET ,NULL ); +free (string); + }else +error (FATAL ,"cannot read va_kernel_pa_offset\n" ); + }else if (kernel_version >=LINUX (6 ,4 ,0 )) +error (FATAL ,"cannot determine va_kernel_pa_offset since Linux 6.4\n" ); +else + ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base; +} +static int riscv64_is_kvaddr (ulong vaddr) {
riscv64.c riscv64_init(int when) · line 1379
Quoted 7 lines
riscv64.c +1
-0@@ -1352,6 +1373,7 @@ riscv64_init(int when)riscv64_get_struct_page_size (machdep->machspec);riscv64_get_va_bits (machdep->machspec);riscv64_get_va_range (machdep->machspec); +riscv64_get_va_kernel_pa_offset (machdep->machspec);pt_level_alloc (&machdep->pgd,"cannot malloc pgd space." );pt_level_alloc (&machdep->machspec->p4d,"cannot malloc p4d space." );
Would you mind giving me the instructions on how to reproduce the issue please? So that I can add that to our internal CI and avoid this type of breakage in the future.
Thanks,
Alex
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 4 lines · Alexandre Ghiti
Hi Song,
On 24/07/2023 06:06, Song Shuai wrote:
Quoted 9 lines
Quoted 3 lines
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.Maybe we could be more explicit here, kernel_map.phys_addr actually points to the physical start of the kernel so maybe something like that:
"changes
phys_ram_basefrom the physical start of the kernel to the actual start of the DRAM."
ok,
Quoted 79 lines
Quoted 79 lines
The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crashAnd I'll send the Linux changes to riscv/for-next If you're ok with this patch.
---
defs.h | 4 ++--
riscv64.c| 22 ++++++++++++++++++++++2 files changed, 24 insertions(+), 2 deletions(-)defs.h +2
-2@@ -3662,8 +3662,7 @@ typedef signed int s32; ulong _X = X; \ (THIS_KERNEL_VERSION >= LINUX(5,13,0) && \ (_X) >= machdep->machspec->kernel_link_addr) ? \ - (((unsigned long)(_X)-(machdep->machspec->kernel_link_addr)) + \ - machdep->machspec->phys_base): \ + ((unsigned long)(_X)-(machdep->machspec->va_kernel_pa_offset)): \ (((unsigned long)(_X)-(machdep->kvbase)) + \machdep->machspec->phys_base); \
})defs.h +1
-0@@ -7021,6 +7020,7 @@ struct machine_specific { ulong modules_vaddr; ulong modules_end; ulong kernel_link_addr; + ulong va_kernel_pa_offset; ulong _page_present; ulong _page_read;riscv64.c +21
-0@@ -418,6 +418,27 @@ error: error(FATAL, "cannot get vm layout\n"); } +static void +riscv64_get_va_kernel_pa_offset (struct machine_specific *ms) +{ + unsigned long kernel_version = riscv64_get_kernel_version(); + + /* + * va_kernel_pa_offset is defined in Linux kernel since 6.5. + */ + if (kernel_version >= LINUX(6,5,0)) { + char *string; + if ((string = pc->read_vmcoreinfo ("NUMBER(va_kernel_pa_offset)" ))) { + ms->va_kernel_pa_offset = htol(string, QUIET, NULL); + free(string); + } else + error(FATAL, "cannot read va_kernel_pa_offset\n"); + } else if (kernel_version >= LINUX(6,4,0)) + error(FATAL, "cannot determine va_kernel_pa_offset since Linux 6.4\n"); + else + ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base; +} +static int
riscv64_is_kvaddr(ulong vaddr)
{riscv64.c +0
-0@@ -1352,6 +1373,7 @@ riscv64_init(int when) riscv64_get_struct_page_size(machdep->machspec);
Quoted 9 lines
riscv64.c riscv64_init(int when) · line 1379
Quoted 5 lines
riscv64.c +1
-0riscv64_get_va_bits(machdep->machspec); riscv64_get_va_range(machdep->machspec); + riscv64_get_va_kernel_pa_offset(machdep->machspec); pt_level_alloc(&machdep->pgd, "cannot malloc pgd space."); pt_level_alloc(&machdep->machspec->p4d, "cannot malloc p4d space.");Would you mind giving me the instructions on how to reproduce the issue please? So that I can add that to our internal CI and avoid this type of breakage in the future.
You can reproduce this issue via :
1. compile the Linux v6.4 or later version with Kdump support
2. generate the vmcore file via sysrq-trigger
3. start the Crash (crash-utility/crash:master) with namelist(vmlinux) and vmcore with optional "-d" option
Crash would boot failed with some incorrect infos (like: empty cpu_*_mask,utsname ) and some error like:crash: read error: kernel virtual address: ffffffff80ecb498 type: "linux_banner"
Quoted 4 lines
Thanks,
Alex
Signature · 7 lines
Thanks Song Shuai _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 52 lines · Song Shuai
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crashAnd I'll send the Linux changes to riscv/for-next If you're ok with this patch.
--- defs.h | 4 ++-- riscv64.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-)defs.h +2
-2@@ -3662,8 +3662,7 @@ typedef signed int s32;ulong _X =X ; \ (THIS_KERNEL_VERSION >=LINUX (5 ,13 ,0 ) && \ (_X) >= machdep->machspec->kernel_link_addr) ? \ - (((unsigned long )(_X)-(machdep->machspec->kernel_link_addr)) + \ - machdep->machspec->phys_base): \ + ((unsigned long )(_X)-(machdep->machspec->va_kernel_pa_offset)): \ (((unsigned long )(_X)-(machdep->kvbase)) + \ machdep->machspec->phys_base); \ })@@ -7021,6 +7020,7 @@ struct machine_specific {ulong modules_vaddr;ulong modules_end;ulong kernel_link_addr; +ulong va_kernel_pa_offset;ulong _page_present;ulong _page_read;riscv64.c +0
-0@@ -418,6 +418,27 @@ error:error (FATAL ,"cannot get vm layout\n" ); }
riscv64.c error: · line 429
Quoted 10 lines
riscv64.c +9
-0+static void +riscv64_get_va_kernel_pa_offset (struct machine_specific *ms) +{ +unsigned long kernel_version =riscv64_get_kernel_version (); + +/* +* va_kernel_pa_offset is defined in Linux kernel since 6.5. +*/ +if (kernel_version >=LINUX (6 ,5 ,0 )) {
The kernel patches look accepted, so for the crash patch detail,
I think this first version check is not necessary, we can just use the vmcoreinfo entry if available. With it, backporting the kernel patches to e.g. 6.4.0 will also be supported.
Thanks,
Kazu
Quoted 16 lines
riscv64.c +12
-0+char *string; +if ((string = pc->read_vmcoreinfo ("NUMBER(va_kernel_pa_offset)" ))) { + ms->va_kernel_pa_offset =htol (string,QUIET ,NULL ); +free (string); + }else +error (FATAL ,"cannot read va_kernel_pa_offset\n" ); + }else if (kernel_version >=LINUX (6 ,4 ,0 )) +error (FATAL ,"cannot determine va_kernel_pa_offset since Linux 6.4\n" ); +else + ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base; +} +static int riscv64_is_kvaddr (ulong vaddr) {
riscv64.c riscv64_init(int when) · line 1379
Quoted 7 lines
riscv64.c +1
-0@@ -1352,6 +1373,7 @@ riscv64_init(int when)riscv64_get_struct_page_size (machdep->machspec);riscv64_get_va_bits (machdep->machspec);riscv64_get_va_range (machdep->machspec); +riscv64_get_va_kernel_pa_offset (machdep->machspec);pt_level_alloc (&machdep->pgd,"cannot malloc pgd space." );pt_level_alloc (&machdep->machspec->p4d,"cannot malloc p4d space." );
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 66 lines · HAGIO KAZUHITO(萩尾 一仁)
Quoted 52 lines · Song Shuai
Since RISC-V Linux v6.4, the commit
3335068f8721("riscv: Use PUD/P4D/PGD pages for the linear mapping") changes thephys_ram_basefrom the kernel_map.phys_addr to the start of DRAM.The Crash's
VTOP()still usesphys_ram_baseand kernel_map.virt_addr to translate kernel virtual address, that made Crash boot failed with Linux v6.4 and later version.Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can use "
va_kernel_pa_offset" to translate the kernel virtual address inVTOP()correctly.Signed-off-by Song Shuai <[email protected]>---
You can check/test the Linux changes from this link:
https://github.com/sugarfillet/linux/commits/6.5-rc3-crashAnd I'll send the Linux changes to riscv/for-next If you're ok with this patch.
--- defs.h | 4 ++-- riscv64.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-)defs.h +2
-2@@ -3662,8 +3662,7 @@ typedef signed int s32;ulong _X =X ; \ (THIS_KERNEL_VERSION >=LINUX (5 ,13 ,0 ) && \ (_X) >= machdep->machspec->kernel_link_addr) ? \ - (((unsigned long )(_X)-(machdep->machspec->kernel_link_addr)) + \ - machdep->machspec->phys_base): \ + ((unsigned long )(_X)-(machdep->machspec->va_kernel_pa_offset)): \ (((unsigned long )(_X)-(machdep->kvbase)) + \ machdep->machspec->phys_base); \ })@@ -7021,6 +7020,7 @@ struct machine_specific {ulong modules_vaddr;ulong modules_end;ulong kernel_link_addr; +ulong va_kernel_pa_offset;ulong _page_present;ulong _page_read;riscv64.c +0
-0@@ -418,6 +418,27 @@ error:error (FATAL ,"cannot get vm layout\n" ); }riscv64.c error: · line 429
Quoted 10 lines
riscv64.c +9
-0+static void +riscv64_get_va_kernel_pa_offset (struct machine_specific *ms) +{ +unsigned long kernel_version =riscv64_get_kernel_version (); + +/* +* va_kernel_pa_offset is defined in Linux kernel since 6.5. +*/ +if (kernel_version >=LINUX (6 ,5 ,0 )) {The kernel patches look accepted, so for the crash patch detail,
Quoted 4 lines
I think this first version check is not necessary, we can just use the vmcoreinfo entry if available. With it, backporting the kernel patches to e.g. 6.4.0 will also be supported.
ok, V2 is here:
https://lore.kernel.org/linux-riscv/[email protected]/T/#u
Quoted 26 lines
Thanks,
KazuQuoted 23 lines
riscv64.c +13
-0+char *string; +if ((string = pc->read_vmcoreinfo ("NUMBER(va_kernel_pa_offset)" ))) { + ms->va_kernel_pa_offset =htol (string,QUIET ,NULL ); +free (string); + }else +error (FATAL ,"cannot read va_kernel_pa_offset\n" ); + }else if (kernel_version >=LINUX (6 ,4 ,0 )) +error (FATAL ,"cannot determine va_kernel_pa_offset since Linux 6.4\n" ); +else + ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base; +} +static int riscv64_is_kvaddr (ulong vaddr) {@@ -1352,6 +1373,7 @@ riscv64_init(int when)riscv64_get_struct_page_size (machdep->machspec);riscv64_get_va_bits (machdep->machspec);riscv64_get_va_range (machdep->machspec); +riscv64_get_va_kernel_pa_offset (machdep->machspec);pt_level_alloc (&machdep->pgd,"cannot malloc pgd space." );pt_level_alloc (&machdep->machspec->p4d,"cannot malloc p4d space." );
Signature · 7 lines
Thanks Song Shuai _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec