x86, mm: Reserver some memory for bootmem allocator for NO_BOOTMEM
Subsystems (3)
PATCH 1/1 x86, mm: Reserver some memory for bootmem allocator for NO_BOOTMEM
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44e
The root cause is that when CONFIG_NO_BOOTMEM=y, before e820__memblock_setup() is called there is no memory for bootmem to allocate, and any alloc_bootmem() in this time window will trigger a panic. Seems this is a known issue as already mentioned in arch/x86/kernel/setup.c:
" /* after early param, so could get panic from serial */ memblock_x86_reserve_range_setup_data(); "
By reserving some memory in the linker script, small memory request could be fulfilled by bootmem allocator. And this memory region is not wasted, but usable after kernel boot.
--- arch/x86/include/asm/sections.h | 5 +++++ arch/x86/kernel/setup.c | 12 +++++++++++- arch/x86/kernel/vmlinux.lds.S | 9 +++++++++ 3 files changed, 25 insertions(+), 1 deletion(-)
arch/x86/include/asm/sections.h +5 -0
@@ -9,6 +9,11 @@ extern char __brk_base[], __brk_limit[];extern struct exception_table_entry __stop___ex_table[];extern char __end_rodata_aligned[]; +#ifdef CONFIG_NO_BOOTMEM +extern char __bootmem_start[], __bootmem_end[]; +#endif + +#if defined(CONFIG_X86_64 )extern char __end_rodata_hpage_align[];extern char __entry_trampoline_start[], __entry_trampoline_end[];
arch/x86/kernel/setup.c +11 -1
@@ -829,6 +829,17 @@ void __init setup_arch(char **cmdline_p)*/ memblock_reserve (0 ,PAGE_SIZE ); +#ifdef CONFIG_NO_BOOTMEM +/* +* There is small time window till e820__memblock_setup() is +* called, in which __bootmem_alloc() has no available memory +* to allocate and will trigger panic. Adding this revered bootmem +* can alleviate the situation. +*/ +memblock_add (__pa_symbol (__bootmem_start), + __bootmem_end - __bootmem_start); +#endif +early_reserve_initrd ();/*
@@ -989,7 +1000,6 @@ void __init setup_arch(char **cmdline_p)x86_report_nx (); -/* after early param, so could get panic from serial */ memblock_x86_reserve_range_setup_data ();if (acpi_mps_check ()) {
arch/x86/kernel/vmlinux.lds.S +9 -0
@@ -367,6 +367,15 @@ SECTIONS __brk_limit = .; } +#ifdef CONFIG_NO_BOOTMEM + . = ALIGN(PAGE_SIZE); + .bootmem : AT(ADDR(.bootmem) - LOAD_OFFSET) { + __bootmem_start = .; + . += 128 * 1024; /* 128 KB for early boot mem */ + __bootmem_end = .; + } +#endif + . = ALIGN(PAGE_SIZE); /* keep VO_INIT_SIZE page aligned */ _end = .;
Signature · 1 line
2.14.1
Peter Zijlstra
maintainerQuoted 20 lines · Feng Tang
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44e
Quoted 3 lines
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,
Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Quoted 25 lines · Peter Zijlstra
Quoted 20 lines · Feng Tang
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44eQuoted 3 lines
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,
Quoted 7 lines
Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to me. I am not familiar with this code path but why cannot we postpone the allocation to later or use a statically allocated storage?
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 8 lines · Michal Hocko
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Feng Tang
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Which you bloody well asked for by using NO_BOOTMEM=y.
Quoted 8 lines
Quoted 5 lines
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to
If we want to remove bootmem, then reintroducing it with a static bootmem section doesn't make any sense at all.
Thanks,
tglx
Quoted 12 lines · Thomas Gleixner
Quoted 11 lines · Michal Hocko
Quoted 10 lines · Peter Zijlstra
Quoted 3 lines · Feng Tang
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to
If we want to remove bootmem, then reintroducing it with a static bootmem section doesn't make any sense at all.
I have actually checked the code now and see what you mean. I thought it would be a single allocation that is needed but that is not the case so the static buffer is not going to fly.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 19 lines · Michal Hocko
Quoted 18 lines · Thomas Gleixner
Quoted 16 lines · Michal Hocko
Quoted 5 lines · Peter Zijlstra
Quoted 3 lines · Feng Tang
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Quoted 7 lines
Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to
Quoted 6 lines
Quoted 2 lines
If we want to remove bootmem, then reintroducing it with a static bootmem section doesn't make any sense at all.
I have actually checked the code now and see what you mean. I thought it would be a single allocation that is needed but that is not the case so the static buffer is not going to fly.
Exactly! I tried several ways for the static allocation, like in data, in bss section, but they all failed, as in the very start of setup_arch():
memblock_reserve(__pa_symbol(_text), (unsigned long)__bss_stop - (unsigned long)_text);
Those [_text, __bss_stop] is not able to be used by alloc_bootmem(). And I only got this patch, and really appreciate any good suggestions.
Thanks,
Feng
Quoted 27 lines · Feng Tang
Quoted 19 lines · Michal Hocko
Quoted 18 lines · Thomas Gleixner
Quoted 16 lines · Michal Hocko
Quoted 5 lines · Peter Zijlstra
Quoted 3 lines · Feng Tang
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Quoted 7 lines
Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to
Quoted 6 lines
Quoted 2 lines
If we want to remove bootmem, then reintroducing it with a static bootmem section doesn't make any sense at all.
I have actually checked the code now and see what you mean. I thought it would be a single allocation that is needed but that is not the case so the static buffer is not going to fly.
Quoted 8 lines
Exactly! I tried several ways for the static allocation, like in data, in bss section, but they all failed, as in the very start of
setup_arch():memblock_reserve(__pa_symbol(_text), (unsigned long)__bss_stop - (unsigned long)_text);Those [
_text,__bss_stop] is not able to be used byalloc_bootmem(). And I only got this patch, and really appreciate any good suggestions.
And why do you want to use bootmem in the first place? If boot mem is going away, why are you not fixing the early console crap to NOT use bootmem at all?
Thanks,
tglx
Hi Thomas,
Quoted 34 lines · Thomas Gleixner
Quoted 33 lines · Feng Tang
Quoted 19 lines · Michal Hocko
Quoted 18 lines · Thomas Gleixner
Quoted 16 lines · Michal Hocko
Quoted 5 lines · Peter Zijlstra
Quoted 3 lines · Feng Tang
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Quoted 7 lines
Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to
Quoted 6 lines
Quoted 2 lines
If we want to remove bootmem, then reintroducing it with a static bootmem section doesn't make any sense at all.
I have actually checked the code now and see what you mean. I thought it would be a single allocation that is needed but that is not the case so the static buffer is not going to fly.
Exactly! I tried several ways for the static allocation, like in data, in bss section, but they all failed, as in the very start of
setup_arch():memblock_reserve(__pa_symbol(_text), (unsigned long)__bss_stop - (unsigned long)_text);
Quoted 6 lines
Quoted 2 lines
Those [
_text,__bss_stop] is not able to be used byalloc_bootmem(). And I only got this patch, and really appreciate any good suggestions.And why do you want to use bootmem in the first place? If boot mem is going away, why are you not fixing the early console crap to NOT use bootmem at all?
The earlycon need use fixmap to map the mmio register, while fixmap will need a new page for page table if the pmd/pte is not already their, as shown in the commit log:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44e
inside the spp_getpage():
if (after_bootmem) ptr = (void *) get_zeroed_page(GFP_ATOMIC); else ptr = alloc_bootmem_pages(PAGE_SIZE);
Thanks,
Feng
Quoted 29 lines · Feng Tang
Quoted 6 lines · Thomas Gleixner
Quoted 2 lines
Those [
_text,__bss_stop] is not able to be used byalloc_bootmem(). And I only got this patch, and really appreciate any good suggestions.And why do you want to use bootmem in the first place? If boot mem is going away, why are you not fixing the early console crap to NOT use bootmem at all?
The earlycon need use fixmap to map the mmio register, while fixmap will need a new page for page table if the pmd/pte is not already their, as shown in the commit log:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44e
Quoted 6 lines
inside the
spp_getpage():if (after_bootmem) ptr = (void *) get_zeroed_page(GFP_ATOMIC); else ptr = alloc_bootmem_pages(PAGE_SIZE);
And where is the problem? We know that we need the fixmap during early boot anyway, so allocating the PTE page statically and setting it up early enough is not really rocket science. No allocator required.
Thanks,
tglx
Quoted 32 lines · Thomas Gleixner
Quoted 31 lines · Feng Tang
Quoted 6 lines · Thomas Gleixner
Quoted 2 lines
Those [
_text,__bss_stop] is not able to be used byalloc_bootmem(). And I only got this patch, and really appreciate any good suggestions.And why do you want to use bootmem in the first place? If boot mem is going away, why are you not fixing the early console crap to NOT use bootmem at all?
The earlycon need use fixmap to map the mmio register, while fixmap will need a new page for page table if the pmd/pte is not already their, as shown in the commit log:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44einside the
spp_getpage():
Quoted 8 lines
Quoted 4 lines
if (after_bootmem) ptr = (void *) get_zeroed_page(GFP_ATOMIC); else ptr = alloc_bootmem_pages(PAGE_SIZE);And where is the problem? We know that we need the fixmap during early boot anyway, so allocating the PTE page statically and setting it up early enough is not really rocket science. No allocator required.
Yes, if the bootmem itslef is going way, setting up the page table for fixmap is a cleaner solution.
Thanks,
Feng
Quoted 3 lines · Thomas Gleixner
And where is the problem? We know that we need the fixmap during early boot anyway, so allocating the PTE page statically and setting it up early enough is not really rocket science. No allocator required.
Further check shows the page table for fixmap is already setup in [-12M, -10M], but fixmap's address space breaks the expection under certain kernel config. So here is another fix, please review, thanks!
----------------------------------------------------------------
From f63e4e5c6166a2f8d13ad3142b7a49b2fd1f8edf Mon Sep 17 00:00:00 2001
From: Feng Tang <[email protected]>
Date: Fri, 31 Aug 2018 13:48:21 +0800
Subject: [PATCH] x86, mm: Make the real fixmap address consistent
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44e
This panic happens as the earlycon's fixmap address has no pmd/pte ready, and __set_fixmap will try to allocate memory to setup the page table, and trigger panic due to no memory.
x86 kernel actually prepares the page table for fixmap in head_64.S:
NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ .fill 5,8,0
and it expects the fixmap address is in [-12M, -10M] range, but current code in fixmap.h will break the expectation when X86_VSYSCALL_EMULATION=n
#ifdef CONFIG_X86_VSYSCALL_EMULATION VSYSCALL_PAGE = (FIXADDR_TOP - VSYSCALL_ADDR) >> PAGE_SHIFT, #endif
So removing the "#ifdef" will make the fixmap address space stable in [-12M, -10M] and fix the issue.
--- arch/x86/include/asm/fixmap.h | 2 -- 1 file changed, 2 deletions(-)
arch/x86/include/asm/fixmap.h +0 -2
@@ -67,9 +67,7 @@ enum fixed_addresses {#ifdef CONFIG_X86_32 FIX_HOLE , #else -#ifdef CONFIG_X86_VSYSCALL_EMULATIONVSYSCALL_PAGE = (FIXADDR_TOP -VSYSCALL_ADDR ) >>PAGE_SHIFT , -#endif #endifFIX_DBGP_BASE , FIX_EARLYCON_MEM_BASE,
Signature · 1 line
2.14.1
Quoted 17 lines · Feng Tang
On Thu, Aug 30, 2018 at 03:25:42PM +0200, Thomas Gleixner wrote:
This panic happens as the earlycon's fixmap address has no pmd/pte ready, and__set_fixmapwill try to allocate memory to setup the page table, and trigger panic due to no memory.x86 kernel actually prepares the page table for fixmap in
head_64.S:NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ .fill 5,8,0and it expects the fixmap address is in [-12M, -10M] range, but current code in
fixmap.hwill break the expectation when X86_VSYSCALL_EMULATION=n
Quoted 6 lines
#ifdef CONFIG_X86_VSYSCALL_EMULATION VSYSCALL_PAGE = (FIXADDR_TOP - VSYSCALL_ADDR) >> PAGE_SHIFT, #endifSo removing the "#ifdef" will make the fixmap address space stable in [-12M, -10M] and fix the issue.
Why on earth are you not fixing the damned PTE setup which is the obvious and correct thing to do?
Thanks,
tglx
Quoted 22 lines · Thomas Gleixner
Quoted 21 lines · Feng Tang
On Thu, Aug 30, 2018 at 03:25:42PM +0200, Thomas Gleixner wrote:
This panic happens as the earlycon's fixmap address has no pmd/pte ready, and__set_fixmapwill try to allocate memory to setup the page table, and trigger panic due to no memory.x86 kernel actually prepares the page table for fixmap in
head_64.S:NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ .fill 5,8,0and it expects the fixmap address is in [-12M, -10M] range, but current code in
fixmap.hwill break the expectation when X86_VSYSCALL_EMULATION=n#ifdef CONFIG_X86_VSYSCALL_EMULATION VSYSCALL_PAGE = (FIXADDR_TOP - VSYSCALL_ADDR) >> PAGE_SHIFT, #endif
Quoted 5 lines
Quoted 2 lines
So removing the "#ifdef" will make the fixmap address space stable in [-12M, -10M] and fix the issue.
Why on earth are you not fixing the damned PTE setup which is the obvious and correct thing to do?
Any sugestion? I can only have patch like this:
---
arch/x86/kernel/head_64.S +7 -3
@@ -446,11 +446,15 @@ NEXT_PAGE(level2_kernel_pgt) NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 - .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC + .quad level1_fixmap_pgt0 - __START_KERNEL_map + _PAGE_TABLE_NOENC + .quad level1_fixmap_pgt1 - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .fill 4,8,0 -NEXT_PAGE(level1_fixmap_pgt) +NEXT_PAGE(level1_fixmap_pgt0) + .fill 512,8,0 + +NEXT_PAGE(level1_fixmap_pgt1) .fill 512,8,0 #undef PMDS
Thanks,
Feng
Hi Thomas,
Quoted 23 lines · Feng Tang
Quoted 22 lines · Thomas Gleixner
Quoted 21 lines · Feng Tang
On Thu, Aug 30, 2018 at 03:25:42PM +0200, Thomas Gleixner wrote:
This panic happens as the earlycon's fixmap address has no pmd/pte ready, and__set_fixmapwill try to allocate memory to setup the page table, and trigger panic due to no memory.x86 kernel actually prepares the page table for fixmap in
head_64.S:NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ .fill 5,8,0and it expects the fixmap address is in [-12M, -10M] range, but current code in
fixmap.hwill break the expectation when X86_VSYSCALL_EMULATION=n#ifdef CONFIG_X86_VSYSCALL_EMULATION VSYSCALL_PAGE = (FIXADDR_TOP - VSYSCALL_ADDR) >> PAGE_SHIFT, #endif
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
So removing the "#ifdef" will make the fixmap address space stable in [-12M, -10M] and fix the issue.
Why on earth are you not fixing the damned PTE setup which is the obvious and correct thing to do?
Any sugestion? I can only have patch like this:
Could you review this patch, as at this time window there is no usable memory block or other memory allocators that I know, so I follow the exisitng static fixmap page table code and add one more.
Thanks,
Feng
Quoted 30 lines
---
arch/x86/kernel/head_64.S +7
-3@@ -446,11 +446,15 @@ NEXT_PAGE(level2_kernel_pgt) NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 - .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC + .quad level1_fixmap_pgt0 - __START_KERNEL_map + _PAGE_TABLE_NOENC + .quad level1_fixmap_pgt1 - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .fill 4,8,0 -NEXT_PAGE(level1_fixmap_pgt) +NEXT_PAGE(level1_fixmap_pgt0) + .fill 512,8,0 + +NEXT_PAGE(level1_fixmap_pgt1) .fill 512,8,0 #undef PMDSThanks,
Feng
Quoted 6 lines · Feng Tang
Quoted 1 line · Feng Tang
Any sugestion? I can only have patch like this:
Could you review this patch, as at this time window there is no usable memory block or other memory allocators that I know, so I follow the exisitng static fixmap page table code and add one more.
You really want to add a proper sanity check for this. The static stuff has to cover the fixmap. Just making it work for this particular case is sloppy at best as the next larger change of the fixmap will just bring the problem back.
Thanks,
tglx
Quoted 4 lines · Thomas Gleixner
Quoted 3 lines · Feng Tang
Quoted 1 line · Feng Tang
Any sugestion? I can only have patch like this:
Quoted 8 lines
Quoted 3 lines
Could you review this patch, as at this time window there is no usable memory block or other memory allocators that I know, so I follow the exisitng static fixmap page table code and add one more.
You really want to add a proper sanity check for this. The static stuff has to cover the fixmap. Just making it work for this particular case is sloppy at best as the next larger change of the fixmap will just bring the problem back.
Yes, agree.
Since the issue happens very early before any serial console, I can only think of a build time check.
Thanks,
Feng
--- arch/x86/kernel/head_64.S | 6 +++++- arch/x86/mm/pgtable.c | 13 +++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-)
arch/x86/kernel/head_64.S +5 -1
@@ -448,11 +448,15 @@ NEXT_PAGE(level2_fixmap_pgt) .fill 506,8,0 .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .quad level1_fixmap_pgt1 - __START_KERNEL_map + _PAGE_TABLE_NOENC + .fill 4,8,0 NEXT_PAGE(level1_fixmap_pgt) .fill 512,8,0 +NEXT_PAGE(level1_fixmap_pgt1) + .fill 512,8,0 + #undef PMDS .data
arch/x86/mm/pgtable.c +13 -0
@@ -637,6 +637,19 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx); +#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table +* has been setup for 4M space [-12M, -8M] +* 0xFFFFFFFFFF400000 ~ 0xFFFFFFFFFF7FFFFF +* Add a sanity check whether fixed_address crosses +* the boundary. +*/ +#define FIXMAP_STATIC_PGTABLE_START 0xFFFFFFFFFF400000 +BUILD_BUG_ON (__fix_to_virt (__end_of_permanent_fixed_addresses) + <FIXMAP_STATIC_PGTABLE_START ); +#endif +if (idx >= __end_of_fixed_addresses) {BUG ();return ;
Signature · 1 line
2.14.1
Quoted 4 lines
Thanks,
tglx
Quoted 9 lines · Feng Tang
Quoted 4 lines · Thomas Gleixner
You really want to add a proper sanity check for this. The static stuff has to cover the fixmap. Just making it work for this particular case is sloppy at best as the next larger change of the fixmap will just bring the problem back.
Yes, agree.
Since the issue happens very early before any serial console, I can only think of a build time check.
Of course. That's obvious. The size _IS_ known at build time.
Quoted 7 lines
arch/x86/mm/pgtable.c +0
-0@@ -637,6 +637,19 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx);
arch/x86/mm/pgtable.c void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) · line 648
Quoted 10 lines
arch/x86/mm/pgtable.c +9
-0+#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table +* has been setup for 4M space [-12M, -8M] +* 0xFFFFFFFFFF400000 ~ 0xFFFFFFFFFF7FFFFF +* Add a sanity check whether fixed_address crosses +* the boundary. +*/ +#define FIXMAP_STATIC_PGTABLE_START 0xFFFFFFFFFF400000
Errm what? This is beyond hillarious, really. What helps that if I remove that second FIXMAP entry in head_64.S?
The size of the fixmap is known at compile time and it is known when building head_64.S. So the obvious check is to check right there where the static pagetable is set up and where you know how many entries you set up whether it's covering the full size or not and err out there.
Thanks,
tglx
Hi Thomas,
Quoted 22 lines · Thomas Gleixner
Quoted 7 lines · Feng Tang
arch/x86/mm/pgtable.c +0
-0@@ -637,6 +637,19 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx);arch/x86/mm/pgtable.c void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) · line 648
Quoted 10 lines
arch/x86/mm/pgtable.c +9
-0+#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table +* has been setup for 4M space [-12M, -8M] +* 0xFFFFFFFFFF400000 ~ 0xFFFFFFFFFF7FFFFF +* Add a sanity check whether fixed_address crosses +* the boundary. +*/ +#define FIXMAP_STATIC_PGTABLE_START 0xFFFFFFFFFF400000Errm what? This is beyond hillarious, really. What helps that if I remove that second FIXMAP entry in
head_64.S?
Quoted 4 lines
The size of the fixmap is known at compile time and it is known when building
head_64.S. So the obvious check is to check right there where the static pagetable is set up and where you know how many entries you set up whether it's covering the full size or not and err out there.
Thanks for the suggestion, and I have 2 patches: one adds a build warning, the other prepares fixmap page table on demand and doesn't need warning.
But I met a problem, that the "__end_of_permanent_fixed_addresses" is defined in fixmap.h, which is protected by #ifndef __ASSEMBLY__, also fixmap.h reference many other header file, which makes it harder to extract the definition out. Any suggestion on this? thanks!
- Feng
patch1:
---
arch/x86/kernel/head_64.S +8 -0
@@ -451,12 +451,20 @@ NEXT_PAGE(level2_fixmap_pgt) .quad level1_fixmap_pgt1 - __START_KERNEL_map + _PAGE_TABLE_NOENC .fill 4,8,0 +/* + * Next are static page table for 4M fixmap space: + * [-12M, -10M], [-10M, -8M] + */ NEXT_PAGE(level1_fixmap_pgt) .fill 512,8,0 NEXT_PAGE(level1_fixmap_pgt1) .fill 512,8,0 +#if __end_of_permanent_fixed_addresses > 1024 +.error "Total fixmap space exceeds the static page table capacity, please expand the page table!" +#endif + #undef PMDS .data
Patch2:
---
arch/x86/kernel/head_64.S +11 -3
@@ -24,6 +24,7 @@ #include "../entry/calling.h" #include <asm/export.h> #include <asm/nospec-branch.h> +#include <asm/fixmap.h> #ifdef CONFIG_PARAVIRT #include <asm/asm-offsets.h>
@@ -444,14 +445,21 @@ NEXT_PAGE(level2_kernel_pgt) PMDS(0, __PAGE_KERNEL_LARGE_EXEC, KERNEL_IMAGE_SIZE/PMD_SIZE) +#define FIXMAP_PMD_NUM ((__end_of_permanent_fixed_addresses + 511) / 512) NEXT_PAGE(level2_fixmap_pgt) - .fill 506,8,0 - .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC + .fill (512 - 4 - FIXMAP_PMD_NUM),8,0 + pgtno = 0 + .rept (FIXMAP_PMD_NUM) + .quad level1_fixmap_pgt + (pgtno << PAGE_SHIFT) - __START_KERNEL_map + _PAGE_TABLE_NOENC + pgtno = pgtno + 1 + .endr /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .fill 4,8,0 NEXT_PAGE(level1_fixmap_pgt) + .rept (FIXMAP_PMD_NUM) .fill 512,8,0 + .endr #undef PMDS
Quoted 2 lines · Feng Tang
Thanks for the suggestion, and I have 2 patches: one adds a build warning, the other prepares fixmap page table on demand and doesn't need warning.
The latter please.
Quoted 4 lines
But I met a problem, that the "
__end_of_permanent_fixed_addresses" is defined infixmap.h, which is protected by #ifndef__ASSEMBLY__, alsofixmap.hreference many other header file, which makes it harder to extract the definition out. Any suggestion on this? thanks!
What prevents you from moving the enum out of the __ASSEMBLY__ protected section aside of a bit of careful work?
Thanks,
tglx
Hi Thomas,
Quoted 5 lines · Thomas Gleixner
Quoted 2 lines · Feng Tang
Thanks for the suggestion, and I have 2 patches: one adds a build warning, the other prepares fixmap page table on demand and doesn't need warning.
The latter please.
Okay.
Quoted 8 lines
Quoted 5 lines
But I met a problem, that the "
__end_of_permanent_fixed_addresses" is defined infixmap.h, which is protected by #ifndef__ASSEMBLY__, alsofixmap.hreference many other header file, which makes it harder to extract the definition out. Any suggestion on this? thanks!What prevents you from moving the enum out of the
__ASSEMBLY__protected section aside of a bit of careful work?
I have tried to change some header files incluing fixmap.h/apicdef.h/ vsyscall.h, and most of the .c files compile fine now, but I can not use the "__end_of_permanent_fixed_addresses" in head_64.S as it is a enum type, and could not be recognized by assembly code.
Thanks,
Feng
Quoted 4 lines
Thanks,
tglx
Quoted 4 lines · Feng Tang
I have tried to change some header files incluing fixmap.h/apicdef.h/
vsyscall.h, and most of the .c files compile fine now, but I can not use the "__end_of_permanent_fixed_addresses" inhead_64.Sas it is a enum type, and could not be recognized by assembly code.
Hrmm. I did not think about the enum. So we have two possibilities:
1) Have some preprocessing which provides the info for the assembler
2) Use a constant for the number of PMDs which is defined in a header and then compile time checked against the size of the fixmap in a C-file.
#1 would be preferred, but for a quick fix #2 is okay as well.
Thanks,
tglx
Quoted 6 lines · Thomas Gleixner
Quoted 4 lines · Feng Tang
I have tried to change some header files incluing fixmap.h/apicdef.h/
vsyscall.h, and most of the .c files compile fine now, but I can not use the "__end_of_permanent_fixed_addresses" inhead_64.Sas it is a enum type, and could not be recognized by assembly code.
Quoted 6 lines
Hrmm. I did not think about the enum. So we have two possibilities:
1) Have some preprocessing which provides the info for the assembler
2) Use a constant for the number of PMDs which is defined in a header and then compile time checked against the size of the fixmap in a C-file.
As there is no fixmap.c, I put the chck into __native_set_fixmap temply, please review, thanks
- Feng
arch/x86/include/asm/fixmap.h +3 -0
@@ -14,6 +14,9 @@#ifndef _ASM_X86_FIXMAP_H#define _ASM_X86_FIXMAP_H +/* Put here to be accessble for assembly code like head_64.S */ +#define FIXMAP_PMD_NUM 2 +#ifndef __ASSEMBLY__#include <linux/kernel.h> #include <asm/acpi.h>
arch/x86/include/asm/pgtable_64.h +2 -1
@@ -14,6 +14,7 @@#include <asm/processor.h> #include <linux/bitops.h> #include <linux/threads.h> +#include <asm/fixmap.h> extern p4d_t level4_kernel_pgt[512 ];extern p4d_t level4_ident_pgt[512 ];
@@ -22,7 +23,7 @@ extern pud_t level3_ident_pgt[512];extern pmd_t level2_kernel_pgt[512 ];extern pmd_t level2_fixmap_pgt[512 ];extern pmd_t level2_ident_pgt[512 ]; -extern pte_t level1_fixmap_pgt[512 ]; +extern pte_t level1_fixmap_pgt[512 *FIXMAP_PMD_NUM ];extern pgd_t init_top_pgt[];#define swapper_pg_dir init_top_pgt
arch/x86/kernel/head_64.S +11 -4
@@ -24,6 +24,7 @@ #include "../entry/calling.h" #include <asm/export.h> #include <asm/nospec-branch.h> +#include <asm/fixmap.h> #ifdef CONFIG_PARAVIRT #include <asm/asm-offsets.h>
@@ -445,13 +446,19 @@ NEXT_PAGE(level2_kernel_pgt) KERNEL_IMAGE_SIZE/PMD_SIZE) NEXT_PAGE(level2_fixmap_pgt) - .fill 506,8,0 - .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC - /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .fill (512 - 4 - FIXMAP_PMD_NUM),8,0 + pgtno = 0 + .rept (FIXMAP_PMD_NUM) + .quad level1_fixmap_pgt + (pgtno << PAGE_SHIFT) - __START_KERNEL_map + _PAGE_TABLE_NOENC + pgtno = pgtno + 1 + .endr + /* 6 MB reserved space + a 2MB hole */ + .fill 4,8,0 NEXT_PAGE(level1_fixmap_pgt) + .rept (FIXMAP_PMD_NUM) .fill 512,8,0 + .endr #undef PMDS
arch/x86/mm/pgtable.c +10 -0
@@ -637,6 +637,16 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx); +#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table has +* been setup for fixmap. Add a sanity compiling check +* whether fixmap space has exceeded the capacity. +*/ +BUILD_BUG_ON ((__end_of_permanent_fixed_addresses +511 )/512 + >FIXMAP_PMD_NUM ); +#endif +if (idx >= __end_of_fixed_addresses) {BUG ();return ;
arch/x86/xen/mmu_pv.c +4 -1
@@ -1953,7 +1953,10 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)set_page_prot (level2_ident_pgt,PAGE_KERNEL_RO );set_page_prot (level2_kernel_pgt,PAGE_KERNEL_RO );set_page_prot (level2_fixmap_pgt,PAGE_KERNEL_RO ); -set_page_prot (level1_fixmap_pgt,PAGE_KERNEL_RO ); + +for (i =0 ; i <FIXMAP_PMD_NUM ; i++) +set_page_prot (level1_fixmap_pgt + i *512 , +PAGE_KERNEL_RO );/* Pin down new L4 */ pin_pagetable_pfn(MMUEXT_PIN_L4_TABLE,
Quoted 7 lines
#1 would be preferred, but for a quick fix #2 is okay as well.
Thanks,
tglx
Quoted 6 lines · Feng Tang
arch/x86/mm/pgtable.c +0
-0@@ -637,6 +637,16 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {
arch/x86/mm/pgtable.c void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) · line 647
Quoted 10 lines
arch/x86/mm/pgtable.c +8
-0unsigned long address =__fix_to_virt (idx); +#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table has +* been setup for fixmap. Add a sanity compiling check +* whether fixmap space has exceeded the capacity. +*/ +BUILD_BUG_ON ((__end_of_permanent_fixed_addresses +511 )/512 + >FIXMAP_PMD_NUM );
There exist macros for rounding up and doing this in the header file is perfectly fine.
Quoted 10 lines
arch/x86/mm/pgtable.c +2
-0+#endif +if (idx >= __end_of_fixed_addresses) {BUG ();return ;arch/x86/xen/mmu_pv.c +0
-0
arch/x86/xen/mmu_pv.c void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) · line 1959
Quoted 8 lines
arch/x86/xen/mmu_pv.c +4
-1@@ -1953,7 +1953,10 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)set_page_prot (level2_ident_pgt,PAGE_KERNEL_RO );set_page_prot (level2_kernel_pgt,PAGE_KERNEL_RO );set_page_prot (level2_fixmap_pgt,PAGE_KERNEL_RO ); -set_page_prot (level1_fixmap_pgt,PAGE_KERNEL_RO ); + +for (i =0 ; i <FIXMAP_PMD_NUM ; i++) +set_page_prot (level1_fixmap_pgt + i *512 , +PAGE_KERNEL_RO );
The line break is there because?
Thanks,
tglx
Hi Thomas,
Quoted 9 lines · Thomas Gleixner
Quoted 8 lines · Feng Tang
arch/x86/mm/pgtable.c +0
-0@@ -637,6 +637,16 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx);
Quoted 11 lines
arch/x86/mm/pgtable.c void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) · line 647
Quoted 8 lines
arch/x86/mm/pgtable.c +8
-0+#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table has +* been setup for fixmap. Add a sanity compiling check +* whether fixmap space has exceeded the capacity. +*/ +BUILD_BUG_ON ((__end_of_permanent_fixed_addresses +511 )/512 + >FIXMAP_PMD_NUM );There exist macros for rounding up and doing this in the header file is perfectly fine.
I'll find the sample for the round up.
And for the header file check, I initially put this check directly in the fixmap.h like:
#if ((__end_of_permanent_fixed_addresses + 511)/512 > FIXMAP_PMD_NUM) #erro "some warning...."
#endif
But failed to compile, I got the warning
warning: "__end_of_permanent_fixed_addresses" is not defined
I guess the header file preprocessing can't handle it.
Quoted 13 lines
Quoted 13 lines
arch/x86/mm/pgtable.c +2
-0+#endif +if (idx >= __end_of_fixed_addresses) {BUG ();return ;arch/x86/xen/mmu_pv.c +0
-0@@ -1953,7 +1953,10 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)set_page_prot (level2_ident_pgt,PAGE_KERNEL_RO );set_page_prot (level2_kernel_pgt,PAGE_KERNEL_RO );
Quoted 8 lines
arch/x86/xen/mmu_pv.c void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) · line 1959
Quoted 6 lines
arch/x86/xen/mmu_pv.c +4
-1set_page_prot (level2_fixmap_pgt,PAGE_KERNEL_RO ); -set_page_prot (level1_fixmap_pgt,PAGE_KERNEL_RO ); + +for (i =0 ; i <FIXMAP_PMD_NUM ; i++) +set_page_prot (level1_fixmap_pgt + i *512 , +PAGE_KERNEL_RO );The line break is there because?
Will fix it. I used a MACRO before changing to 512 and forgot to remove the line break :)
Thanks,
Feng
Hi Thomas,
Quoted 29 lines · Feng Tang
Hi Thomas,
Quoted 9 lines · Thomas Gleixner
Quoted 8 lines · Feng Tang
arch/x86/mm/pgtable.c +0
-0@@ -637,6 +637,16 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx);Quoted 11 lines
arch/x86/mm/pgtable.c void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) · line 647
Quoted 8 lines
arch/x86/mm/pgtable.c +8
-0+#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table has +* been setup for fixmap. Add a sanity compiling check +* whether fixmap space has exceeded the capacity. +*/ +BUILD_BUG_ON ((__end_of_permanent_fixed_addresses +511 )/512 + >FIXMAP_PMD_NUM );There exist macros for rounding up and doing this in the header file is perfectly fine.
I'll find the sample for the round up.
And for the header file check, I initially put this check directly in the
fixmap.hlike:
Quoted 8 lines
#if ((
__end_of_permanent_fixed_addresses+ 511)/512 >FIXMAP_PMD_NUM) #erro "some warning...."
#endifBut failed to compile, I got the warning
warning: "__end_of_permanent_fixed_addresses" is not definedI guess the header file preprocessing can't handle it.
Below is a patch against 4.19-rc4, verified with earlycon working fine, please review, thanks!
- Feng
---
From 2710b48eb29a86c8ee568d00d5b04820c3fe1f3a Mon Sep 17 00:00:00 2001
From: Feng Tang <[email protected]>
Date: Mon, 17 Sep 2018 12:12:08 +0800
Subject: [PATCH] x86/mm: Expand static page table for fixmap space
We met a kernel panic when enabling earlycon, which is due to the fixmap address of earlycon is not statically setup.
Currently the static fixmap setup in head_64.S only covers 2M virtual address space, while it acutually could be in 4M space with different kernel configurations.
So increase the static space to 4M for now by defining FIXMAP_PMD_NUM to 2, and add a build time check for future possible overflow so that the macro could be increased accordingly.
--- arch/x86/include/asm/fixmap.h | 3 +++ arch/x86/include/asm/pgtable_64.h | 3 ++- arch/x86/kernel/head_64.S | 15 +++++++++++---- arch/x86/mm/pgtable.c | 10 ++++++++++ arch/x86/xen/mmu_pv.c | 4 +++- 5 files changed, 29 insertions(+), 6 deletions(-)
arch/x86/include/asm/fixmap.h +3 -0
@@ -14,6 +14,9 @@#ifndef _ASM_X86_FIXMAP_H#define _ASM_X86_FIXMAP_H +/* Put here to be accessble for assembly code like head_64.S */ +#define FIXMAP_PMD_NUM 2 +#ifndef __ASSEMBLY__#include <linux/kernel.h> #include <asm/acpi.h>
arch/x86/include/asm/pgtable_64.h +2 -1
@@ -14,6 +14,7 @@#include <asm/processor.h> #include <linux/bitops.h> #include <linux/threads.h> +#include <asm/fixmap.h> extern p4d_t level4_kernel_pgt[512 ];extern p4d_t level4_ident_pgt[512 ];
@@ -22,7 +23,7 @@ extern pud_t level3_ident_pgt[512];extern pmd_t level2_kernel_pgt[512 ];extern pmd_t level2_fixmap_pgt[512 ];extern pmd_t level2_ident_pgt[512 ]; -extern pte_t level1_fixmap_pgt[512 ]; +extern pte_t level1_fixmap_pgt[512 *FIXMAP_PMD_NUM ];extern pgd_t init_top_pgt[];#define swapper_pg_dir init_top_pgt
arch/x86/kernel/head_64.S +11 -4
@@ -24,6 +24,7 @@ #include "../entry/calling.h" #include <asm/export.h> #include <asm/nospec-branch.h> +#include <asm/fixmap.h> #ifdef CONFIG_PARAVIRT #include <asm/asm-offsets.h>
@@ -445,13 +446,19 @@ NEXT_PAGE(level2_kernel_pgt) KERNEL_IMAGE_SIZE/PMD_SIZE) NEXT_PAGE(level2_fixmap_pgt) - .fill 506,8,0 - .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC - /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .fill (512 - 4 - FIXMAP_PMD_NUM),8,0 + pgtno = 0 + .rept (FIXMAP_PMD_NUM) + .quad level1_fixmap_pgt + (pgtno << PAGE_SHIFT) - __START_KERNEL_map + _PAGE_TABLE_NOENC + pgtno = pgtno + 1 + .endr + /* 6 MB reserved space + a 2MB hole */ + .fill 4,8,0 NEXT_PAGE(level1_fixmap_pgt) + .rept (FIXMAP_PMD_NUM) .fill 512,8,0 + .endr #undef PMDS
arch/x86/mm/pgtable.c +10 -0
@@ -637,6 +637,16 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) {unsigned long address =__fix_to_virt (idx); +#ifdef CONFIG_X86_64 +/* +* In arch/x86/kernel/head_64.S, the static page table has +* been setup for fixmap. Add a sanity compiling check +* whether fixmap space has exceeded the capacity. +*/ +BUILD_BUG_ON (__end_of_permanent_fixed_addresses + > (FIXMAP_PMD_NUM *PTRS_PER_PMD )); +#endif +if (idx >= __end_of_fixed_addresses) {BUG ();return ;
arch/x86/xen/mmu_pv.c +3 -1
@@ -1952,7 +1952,9 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)set_page_prot (level2_ident_pgt,PAGE_KERNEL_RO );set_page_prot (level2_kernel_pgt,PAGE_KERNEL_RO );set_page_prot (level2_fixmap_pgt,PAGE_KERNEL_RO ); -set_page_prot (level1_fixmap_pgt,PAGE_KERNEL_RO ); + +for (i =0 ; i <FIXMAP_PMD_NUM ; i++) +set_page_prot (level1_fixmap_pgt + i *512 ,PAGE_KERNEL_RO );/* Pin down new L4 */ pin_pagetable_pfn(MMUEXT_PIN_L4_TABLE,
Signature · 1 line
2.7.4
Quoted 3 lines · Feng Tang
Below is a patch against 4.19-rc4, verified with earlycon working fine, please review, thanks!
Aside of the whitespace damage this looks about right. Care to send a proper patch?
Thanks,
tglx
Peter Zijlstra
maintainerQuoted 28 lines · Michal Hocko
Quoted 27 lines · Peter Zijlstra
Quoted 20 lines · Feng Tang
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44eQuoted 3 lines
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Which you bloody well asked for by using NO_BOOTMEM=y.
Quoted 7 lines
Quoted 5 lines
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So
wasn't aware of that. why?
Quoted 3 lines
making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to me. I am not familiar with this code path but why cannot we postpone the allocation to later or use a statically allocated storage?
because 'early'... I suppose. Youu really want the early con up and running asap.
Quoted 32 lines · Peter Zijlstra
Quoted 31 lines · Michal Hocko
Quoted 30 lines · Peter Zijlstra
Quoted 20 lines · Feng Tang
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44eQuoted 3 lines
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
Well, there is endeavor to remove bootmem allocator altogether. So
wasn't aware of that. why?
Because it is yet another allocator that we have to maintain and that is not really needed.
Quoted 6 lines
Quoted 3 lines
making earlycon depend on NO_BOOTMEM=n doesn't sound like a good fit to me. I am not familiar with this code path but why cannot we postpone the allocation to later or use a statically allocated storage?
because 'early'... I suppose. Youu really want the early con up and running asap.
Well, the memblock allocator is initialized quite early as well... If that is not early enough then we have to play tricks.
Signature · 2 lines
Michal Hocko SUSE Labs
Hi Peter,
Quoted 25 lines · Peter Zijlstra
Quoted 20 lines · Feng Tang
We hit a kernel panic when enabling earlycon for a platform, the call trace is:
panic+0xd2/0x220 __alloc_bootmem+0x31/0x34 spp_getpage+0x60/0x8a fill_pte+0x71/0x130 __set_pte_vaddr+0x1d/0x50 set_pte_vaddr+0x3c/0x60 __native_set_fixmap+0x23/0x30 native_set_fixmap+0x30/0x40 setup_earlycon+0x1e0/0x32f param_setup_earlycon+0x13/0x22 do_early_param+0x5b/0x90 parse_args+0x1f7/0x300 parse_early_options+0x24/0x28 parse_early_param+0x65/0x73 setup_arch+0x31e/0x9f1 start_kernel+0x58/0x44eQuoted 3 lines
The root cause is that when
CONFIG_NO_BOOTMEM=y, beforee820__memblock_setup()is called there is no memory for bootmem to allocate,
Quoted 7 lines
Which you bloody well asked for by using NO_BOOTMEM=y.
Going down this route; adding hacks for every little thing that does want bootmem, completely defeats the purpose.
If anything, make the earlycon thing depend on NO_BOOTMEM=n. That also solves your problem. No earlycon, no panic.
In current x86, the NO_BOOTMEM is "y" by default, and almost all the servers/desktops I could access have CONFIG_BOOTMEM=y, while earlycon is important for enabling new platforms, as well as kernel debugging. IMHO, it's better to keep earlycon capability. Also any alloc_bootmem() call in that time window will trigger panic.
Thanks,
Feng