mm/memory_hotplug: full support for add_memory_driver_managed() with CONFIG_ARCH_KEEP_MEMBLOCK
Subsystems (12)
v2 cover mm/memory_hotplug: full support for add_memory_driver_managed() with CONFIG_ARCH_KEEP_MEMBLOCK
Architectures that require CONFIG_ARCH_KEEP_MEMBLOCK=y, such as arm64, don't cleanly support add_memory_driver_managed() yet. Most prominently, kexec_file can still end up placing kexec images on such driver-managed memory, resulting in undesired behavior, for example, having kexec images located on memory not part of the firmware-provided memory map.
Teaching kexec to not place images on driver-managed memory is especially relevant for virtio-mem. Details can be found in commit 7b7b27214bba ("mm/memory_hotplug: introduce add_memory_driver_managed()").
Extend memblock with a new flag and set it from memory hotplug code when applicable. This is required to fully support virtio-mem on arm64, making also kexec_file behave like on x86-64.
v1 -> v2:
- "memblock: improve MEMBLOCK_HOTPLUG documentation" -- Added
- "memblock: add MEMBLOCK_DRIVER_MANAGED to mimic IORESOURCE_SYSRAM_DRIVER_MANAGED"
-- Improve documentation of MEMBLOCK_DRIVER_MANAGED
- Refine patch descriptions
David Hildenbrand (5):
mm/memory_hotplug: handle memblock_add_node() failures in
add_memory_resource()
memblock: improve MEMBLOCK_HOTPLUG documentation
memblock: allow to specify flags with memblock_add_node()
memblock: add MEMBLOCK_DRIVER_MANAGED to mimic
IORESOURCE_SYSRAM_DRIVER_MANAGED
mm/memory_hotplug: indicate MEMBLOCK_DRIVER_MANAGED with
IORESOURCE_SYSRAM_DRIVER_MANAGED
arch/arc/mm/init.c | 4 ++-- arch/ia64/mm/contig.c | 2 +- arch/ia64/mm/init.c | 2 +- arch/m68k/mm/mcfmmu.c | 3 ++- arch/m68k/mm/motorola.c | 6 ++++-- arch/mips/loongson64/init.c | 4 +++- arch/mips/sgi-ip27/ip27-memory.c | 3 ++- arch/s390/kernel/setup.c | 3 ++- include/linux/memblock.h | 25 +++++++++++++++++++++---- include/linux/mm.h | 2 +- kernel/kexec_file.c | 5 +++++ mm/memblock.c | 13 +++++++++---- mm/memory_hotplug.c | 11 +++++++++-- 13 files changed, 62 insertions(+), 21 deletions(-)
base-commit: 9e1ff307c779ce1f0f810c7ecce3d95bbae40896
Signature · 1 line
2.31.1
PATCH v2 1/5 mm/memory_hotplug: handle memblock_add_node() failures in add_memory_resource()
If memblock_add_node() fails, we're most probably running out of memory. While this is unlikely to happen, it can happen and having memory added without a memblock can be problematic for architectures that use memblock to detect valid memory. Let's fail in a nice way instead of silently ignoring the error.
--- mm/memory_hotplug.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
mm/memory_hotplug.c +6 -2
@@ -1384,8 +1384,11 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)mem_hotplug_begin (); -if (IS_ENABLED (CONFIG_ARCH_KEEP_MEMBLOCK )) -memblock_add_node (start, size, nid); +if (IS_ENABLED (CONFIG_ARCH_KEEP_MEMBLOCK )) { + ret =memblock_add_node (start, size, nid); +if (ret) + goto error_mem_hotplug_end; + } ret =__try_online_node (nid, false);if (ret <0 )
@@ -1458,6 +1461,7 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)rollback_node_hotadd (nid);if (IS_ENABLED (CONFIG_ARCH_KEEP_MEMBLOCK ))memblock_remove (start, size); +error_mem_hotplug_end:mem_hotplug_done ();return ret; }
Signature · 1 line
2.31.1
PATCH v2 2/5 memblock: improve MEMBLOCK_HOTPLUG documentation
In reply to David HildenbrandThe description of MEMBLOCK_HOTPLUG is currently short and consequently
misleading: we're actually dealing with a memory region that might get hotunplugged later (i.e., the platform+firmware supports it), yet it is indicated in the firmware-provided memory map as system ram that will just get used by the system for any purpose when not taking special care. The firmware marked this memory region as a hot(un)plugged (e.g., hotplugged before reboot), implying that it might get hotunplugged again later.
Whether we consider this information depends on the "movable_node" kernel commandline parameter: only with "movable_node" set, we'll try keeping this memory hotunpluggable, for example, by not serving early allocations from this memory region and by letting the buddy manage it using the ZONE_MOVABLE.
Let's make this clearer by extending the documentation.
Note: kexec *has to* indicate this memory to the second kernel. With "movable_node" set, we don't want to place kexec-images on this memory. Without "movable_node" set, we don't care and can place kexec-images on this memory. In both cases, after successful memory hotunplug, kexec has to be re-armed to update the memory map for the second kernel and to place the kexec-images somewhere else.
--- include/linux/memblock.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
include/linux/memblock.h +5 -1
@@ -28,7 +28,11 @@ extern unsigned long long max_possible_pfn;/** * enum memblock_flags - definition of memory region attributes * @MEMBLOCK_NONE: no special request -* @MEMBLOCK_HOTPLUG: hotpluggable region +* @MEMBLOCK_HOTPLUG: memory region indicated in the firmware-provided memory +* map during early boot as hot(un)pluggable system RAM (e.g., memory range +* that might get hotunplugged later). With "movable_node" set on the kernel +* commandline, try keeping this memory region hotunpluggable. Does not apply +* to memblocks added ("hotplugged") after early boot. * @MEMBLOCK_MIRROR: mirrored region * @MEMBLOCK_NOMAP: don't add to kernel direct mapping and treat as * reserved in the memory map; refer to memblock_mark_nomap() description
Signature · 1 line
2.31.1
Re: [PATCH v2 2/5] memblock: improve MEMBLOCK_HOTPLUG documentation
Quoted 16 lines · David Hildenbrand
The description of
MEMBLOCK_HOTPLUGis currently short and consequently
misleading: we're actually dealing with a memory region that might get hotunplugged later (i.e., the platform+firmware supports it), yet it is indicated in the firmware-provided memory map as system ram that will just get used by the system for any purpose when not taking special care. The firmware marked this memory region as a hot(un)plugged (e.g., hotplugged before reboot), implying that it might get hotunplugged again later.Whether we consider this information depends on the "
movable_node" kernel commandline parameter: only with "movable_node" set, we'll try keeping this memory hotunpluggable, for example, by not serving early allocations from this memory region and by letting the buddy manage it using theZONE_MOVABLE.Let's make this clearer by extending the documentation.
Quoted 8 lines
Note: kexec *has to* indicate this memory to the second kernel. With "
movable_node" set, we don't want to place kexec-images on this memory. Without "movable_node" set, we don't care and can place kexec-images on this memory. In both cases, after successful memory hotunplug, kexec has to be re-armed to update the memory map for the second kernel and to place the kexec-images somewhere else.Signed-off-by David Hildenbrand <[email protected]>
Quoted 24 lines
--- include/linux/memblock.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)include/linux/memblock.h +5
-1@@ -28,7 +28,11 @@ extern unsigned long long max_possible_pfn;/** * enum memblock_flags - definition of memory region attributes * @MEMBLOCK_NONE: no special request -* @MEMBLOCK_HOTPLUG: hotpluggable region +* @MEMBLOCK_HOTPLUG: memory region indicated in the firmware-provided memory +* map during early boot as hot(un)pluggable system RAM (e.g., memory range +* that might get hotunplugged later). With "movable_node" set on the kernel +* commandline, try keeping this memory region hotunpluggable. Does not apply +* to memblocks added ("hotplugged") after early boot. * @MEMBLOCK_MIRROR: mirrored region * @MEMBLOCK_NOMAP: don't add to kernel direct mapping and treat as * reserved in the memory map; refer to memblock_mark_nomap() description Signature · 1 line
2.31.1
Signature · 7 lines
Sincerely yours, Mike. _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 3/5 memblock: allow to specify flags with memblock_add_node()
In reply to David HildenbrandWe want to specify flags when hotplugging memory. Let's prepare to pass flags to memblock_add_node() by adjusting all existing users.
Note that when hotplugging memory the system is already up and running and we might have concurrent memblock users: for example, while we're hotplugging memory, kexec_file code might search for suitable memory regions to place kexec images. It's important to add the memory directly to memblock via a single call with the right flags, instead of adding the memory first and apply flags later: otherwise, concurrent memblock users might temporarily stumble over memblocks with wrong flags, which will be important in a follow-up patch that introduces a new flag to properly handle add_memory_driver_managed().
--- arch/arc/mm/init.c | 4 ++-- arch/ia64/mm/contig.c | 2 +- arch/ia64/mm/init.c | 2 +- arch/m68k/mm/mcfmmu.c | 3 ++- arch/m68k/mm/motorola.c | 6 ++++-- arch/mips/loongson64/init.c | 4 +++- arch/mips/sgi-ip27/ip27-memory.c | 3 ++- arch/s390/kernel/setup.c | 3 ++- include/linux/memblock.h | 3 ++- include/linux/mm.h | 2 +- mm/memblock.c | 9 +++++---- mm/memory_hotplug.c | 2 +- 12 files changed, 26 insertions(+), 17 deletions(-)
arch/arc/mm/init.c +2 -2
@@ -59,13 +59,13 @@ void __init early_init_dt_add_memory_arch(u64 base, u64 size) low_mem_sz = size; in_use =1 ; -memblock_add_node (base, size,0 ); +memblock_add_node (base, size,0 ,MEMBLOCK_NONE ); }else {#ifdef CONFIG_HIGHMEM high_mem_start = base; high_mem_sz = size; in_use =1 ; -memblock_add_node (base, size,1 ); +memblock_add_node (base, size,1 ,MEMBLOCK_NONE );memblock_reserve (base, size);#endif }
arch/ia64/mm/contig.c +1 -1
@@ -153,7 +153,7 @@ find_memory (void)efi_memmap_walk (find_max_min_low_pfn,NULL ); max_pfn = max_low_pfn; -memblock_add_node (0 ,PFN_PHYS (max_low_pfn),0 ); +memblock_add_node (0 ,PFN_PHYS (max_low_pfn),0 ,MEMBLOCK_NONE );find_initrd ();
arch/ia64/mm/init.c +1 -1
@@ -378,7 +378,7 @@ int __init register_active_ranges(u64 start, u64 len, int nid)#endif if (start < end) -memblock_add_node (__pa (start), end - start, nid); +memblock_add_node (__pa (start), end - start, nid,MEMBLOCK_NONE );return 0 ; }
arch/m68k/mm/mcfmmu.c +2 -1
@@ -174,7 +174,8 @@ void __init cf_bootmem_alloc(void) m68k_memory[0 ].addr = _rambase; m68k_memory[0 ].size = _ramend - _rambase; -memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 ); +memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 , +MEMBLOCK_NONE );/* compute total pages in system */ num_pages =PFN_DOWN (_ramend - _rambase);
arch/m68k/mm/motorola.c +4 -2
@@ -410,7 +410,8 @@ void __init paging_init(void) min_addr = m68k_memory[0 ].addr; max_addr = min_addr + m68k_memory[0 ].size; -memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 ); +memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 , +MEMBLOCK_NONE );for (i =1 ; i < m68k_num_memory;) {if (m68k_memory[i].addr < min_addr) { printk("Ignoring memory chunk at 0x%lx:0x%lx before the first chunk\n",
@@ -421,7 +422,8 @@ void __init paging_init(void) (m68k_num_memory - i) * sizeof(struct m68k_mem_info));continue ; } -memblock_add_node (m68k_memory[i].addr, m68k_memory[i].size, i); +memblock_add_node (m68k_memory[i].addr, m68k_memory[i].size, i, +MEMBLOCK_NONE ); addr = m68k_memory[i].addr + m68k_memory[i].size;if (addr > max_addr) max_addr = addr;
arch/mips/loongson64/init.c +3 -1
@@ -77,7 +77,9 @@ void __init szmem(unsigned int node) (u32)node_id, mem_type, mem_start, mem_size);pr_info (" start_pfn:0x%llx, end_pfn:0x%llx, num_physpages:0x%lx\n" , start_pfn, end_pfn, num_physpages); -memblock_add_node (PFN_PHYS (start_pfn),PFN_PHYS (node_psize), node); +memblock_add_node (PFN_PHYS (start_pfn), +PFN_PHYS (node_psize), node, +MEMBLOCK_NONE );break ;case SYSTEM_RAM_RESERVED : pr_info("Node%d: mem_type:%d, mem_start:0x%llx, mem_size:0x%llx MB\n",
arch/mips/sgi-ip27/ip27-memory.c +2 -1
@@ -341,7 +341,8 @@ static void __init szmem(void)continue ; }memblock_add_node (PFN_PHYS (slot_getbasepfn (node, slot)), -PFN_PHYS (slot_psize), node); +PFN_PHYS (slot_psize), node, +MEMBLOCK_NONE ); } } }
arch/s390/kernel/setup.c +2 -1
@@ -593,7 +593,8 @@ static void __init setup_resources(void)* part of the System RAM resource. */ if (crashk_res.end) { -memblock_add_node (crashk_res.start,resource_size (&crashk_res),0 ); +memblock_add_node (crashk_res.start,resource_size (&crashk_res), +0 ,MEMBLOCK_NONE );memblock_reserve (crashk_res.start,resource_size (&crashk_res));insert_resource (&iomem_resource, &crashk_res); }
include/linux/memblock.h +2 -1
@@ -104,7 +104,8 @@ static inline void memblock_discard(void) {}#endif void memblock_allow_resize (void ); -int memblock_add_node (phys_addr_t base,phys_addr_t size,int nid); +int memblock_add_node (phys_addr_t base,phys_addr_t size,int nid, +enum memblock_flags flags);int memblock_add (phys_addr_t base,phys_addr_t size);int memblock_remove (phys_addr_t base,phys_addr_t size);int memblock_free (phys_addr_t base,phys_addr_t size);
include/linux/mm.h +1 -1
@@ -2447,7 +2447,7 @@ static inline unsigned long get_num_physpages(void)* unsigned long max_zone_pfns[MAX_NR_ZONES] = {max_dma, max_normal_pfn, * max_highmem_pfn}; * for_each_valid_physical_page_range() -* memblock_add_node(base, size, nid) +* memblock_add_node(base, size, nid, MEMBLOCK_NONE) * free_area_init(max_zone_pfns); */ void free_area_init (unsigned long *max_zone_pfn);
mm/memblock.c +5 -4
@@ -655,6 +655,7 @@ static int __init_memblock memblock_add_range(struct memblock_type *type,* @base: base address of the new region * @size: size of the new region * @nid: nid of the new region +* @flags: flags of the new region * * Add new memblock region [@base, @base + @size) to the "memory" * type. See memblock_add_range() description for mode details
@@ -663,14 +664,14 @@ static int __init_memblock memblock_add_range(struct memblock_type *type,* 0 on success, -errno on failure. */ int __init_memblock memblock_add_node(phys_addr_t base, phys_addr_t size, -int nid) +int nid,enum memblock_flags flags) {phys_addr_t end = base + size -1 ; -memblock_dbg ("%s: [%pa-%pa] nid=%d %pS\n" , __func__, - &base, &end, nid, (void *)_RET_IP_); +memblock_dbg ("%s: [%pa-%pa] nid=%d flags=%x %pS\n" , __func__, + &base, &end, nid, flags, (void *)_RET_IP_); -return memblock_add_range (&memblock.memory, base, size, nid,0 ); +return memblock_add_range (&memblock.memory, base, size, nid, flags); }/**
mm/memory_hotplug.c +1 -1
@@ -1385,7 +1385,7 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)mem_hotplug_begin ();if (IS_ENABLED (CONFIG_ARCH_KEEP_MEMBLOCK )) { - ret =memblock_add_node (start, size, nid); + ret =memblock_add_node (start, size, nid,MEMBLOCK_NONE );if (ret) goto error_mem_hotplug_end; }
Signature · 1 line
2.31.1
Re: [PATCH v2 3/5] memblock: allow to specify flags with memblock_add_node()
Quoted 40 lines · David Hildenbrand
We want to specify flags when hotplugging memory. Let's prepare to pass flags to
memblock_add_node()by adjusting all existing users.Note that when hotplugging memory the system is already up and running and we might have concurrent memblock users: for example, while we're hotplugging memory,
kexec_filecode might search for suitable memory regions to place kexec images. It's important to add the memory directly to memblock via a single call with the right flags, instead of adding the memory first and apply flags later: otherwise, concurrent memblock users might temporarily stumble over memblocks with wrong flags, which will be important in a follow-up patch that introduces a new flag to properly handleadd_memory_driver_managed().Acked-by Heiko Carstens <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/arc/mm/init.c | 4 ++-- arch/ia64/mm/contig.c | 2 +- arch/ia64/mm/init.c | 2 +- arch/m68k/mm/mcfmmu.c | 3 ++- arch/m68k/mm/motorola.c | 6 ++++-- arch/mips/loongson64/init.c | 4 +++- arch/mips/sgi-ip27/ip27-memory.c | 3 ++- arch/s390/kernel/setup.c | 3 ++- include/linux/memblock.h | 3 ++- include/linux/mm.h | 2 +- mm/memblock.c | 9 +++++---- mm/memory_hotplug.c | 2 +- 12 files changed, 26 insertions(+), 17 deletions(-)arch/arc/mm/init.c +0
-1@@ -59,13 +59,13 @@ void __init early_init_dt_add_memory_arch(u64 base, u64 size) low_mem_sz = size; in_use =1 ; -memblock_add_node (base, size,0 );
arch/arc/mm/init.c void __init early_init_dt_add_memory_arch(u64 base, u64 size) · line 70
Quoted 10 lines
arch/arc/mm/init.c +2
-1+memblock_add_node (base, size,0 ,MEMBLOCK_NONE ); }else {#ifdef CONFIG_HIGHMEM high_mem_start = base; high_mem_sz = size; in_use =1 ; -memblock_add_node (base, size,1 ); +memblock_add_node (base, size,1 ,MEMBLOCK_NONE );memblock_reserve (base, size);#endif
arch/arc part: Acked-by: Shahab Vahedi <[email protected]>
Signature · 5 lines
Shahab _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 3/5] memblock: allow to specify flags with memblock_add_node()
In reply to David HildenbrandQuoted 13 lines · David Hildenbrand
We want to specify flags when hotplugging memory. Let's prepare to pass flags to
memblock_add_node()by adjusting all existing users.Note that when hotplugging memory the system is already up and running and we might have concurrent memblock users: for example, while we're hotplugging memory,
kexec_filecode might search for suitable memory regions to place kexec images. It's important to add the memory directly to memblock via a single call with the right flags, instead of adding the memory first and apply flags later: otherwise, concurrent memblock users might temporarily stumble over memblocks with wrong flags, which will be important in a follow-up patch that introduces a new flag to properly handleadd_memory_driver_managed().
Quoted 3 lines
Acked-by Heiko Carstens <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 216 lines
--- arch/arc/mm/init.c | 4 ++-- arch/ia64/mm/contig.c | 2 +- arch/ia64/mm/init.c | 2 +- arch/m68k/mm/mcfmmu.c | 3 ++- arch/m68k/mm/motorola.c | 6 ++++-- arch/mips/loongson64/init.c | 4 +++- arch/mips/sgi-ip27/ip27-memory.c | 3 ++- arch/s390/kernel/setup.c | 3 ++- include/linux/memblock.h | 3 ++- include/linux/mm.h | 2 +- mm/memblock.c | 9 +++++---- mm/memory_hotplug.c | 2 +- 12 files changed, 26 insertions(+), 17 deletions(-)arch/arc/mm/init.c +2
-2@@ -59,13 +59,13 @@ void __init early_init_dt_add_memory_arch(u64 base, u64 size) low_mem_sz = size; in_use =1 ; -memblock_add_node (base, size,0 ); +memblock_add_node (base, size,0 ,MEMBLOCK_NONE ); }else {#ifdef CONFIG_HIGHMEM high_mem_start = base; high_mem_sz = size; in_use =1 ; -memblock_add_node (base, size,1 ); +memblock_add_node (base, size,1 ,MEMBLOCK_NONE );memblock_reserve (base, size);#endif }arch/ia64/mm/contig.c +1
-1@@ -153,7 +153,7 @@ find_memory (void)efi_memmap_walk (find_max_min_low_pfn,NULL ); max_pfn = max_low_pfn; -memblock_add_node (0 ,PFN_PHYS (max_low_pfn),0 ); +memblock_add_node (0 ,PFN_PHYS (max_low_pfn),0 ,MEMBLOCK_NONE );find_initrd ();arch/ia64/mm/init.c +1
-1@@ -378,7 +378,7 @@ int __init register_active_ranges(u64 start, u64 len, int nid)#endif if (start < end) -memblock_add_node (__pa (start), end - start, nid); +memblock_add_node (__pa (start), end - start, nid,MEMBLOCK_NONE );return 0 ; }arch/m68k/mm/mcfmmu.c +2
-1@@ -174,7 +174,8 @@ void __init cf_bootmem_alloc(void) m68k_memory[0 ].addr = _rambase; m68k_memory[0 ].size = _ramend - _rambase; -memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 ); +memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 , +MEMBLOCK_NONE );/* compute total pages in system */ num_pages =PFN_DOWN (_ramend - _rambase);arch/m68k/mm/motorola.c +4
-2@@ -410,7 +410,8 @@ void __init paging_init(void) min_addr = m68k_memory[0 ].addr; max_addr = min_addr + m68k_memory[0 ].size; -memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 ); +memblock_add_node (m68k_memory[0 ].addr, m68k_memory[0 ].size,0 , +MEMBLOCK_NONE );for (i =1 ; i < m68k_num_memory;) {if (m68k_memory[i].addr < min_addr) { printk("Ignoring memory chunk at 0x%lx:0x%lx before the first chunk\n",@@ -421,7 +422,8 @@ void __init paging_init(void) (m68k_num_memory - i) * sizeof(struct m68k_mem_info));continue ; } -memblock_add_node (m68k_memory[i].addr, m68k_memory[i].size, i); +memblock_add_node (m68k_memory[i].addr, m68k_memory[i].size, i, +MEMBLOCK_NONE ); addr = m68k_memory[i].addr + m68k_memory[i].size;if (addr > max_addr) max_addr = addr;arch/mips/loongson64/init.c +3
-1@@ -77,7 +77,9 @@ void __init szmem(unsigned int node) (u32)node_id, mem_type, mem_start, mem_size);pr_info (" start_pfn:0x%llx, end_pfn:0x%llx, num_physpages:0x%lx\n" , start_pfn, end_pfn, num_physpages); -memblock_add_node (PFN_PHYS (start_pfn),PFN_PHYS (node_psize), node); +memblock_add_node (PFN_PHYS (start_pfn), +PFN_PHYS (node_psize), node, +MEMBLOCK_NONE );break ;case SYSTEM_RAM_RESERVED : pr_info("Node%d: mem_type:%d, mem_start:0x%llx, mem_size:0x%llx MB\n",arch/mips/sgi-ip27/ip27-memory.c +2
-1@@ -341,7 +341,8 @@ static void __init szmem(void)continue ; }memblock_add_node (PFN_PHYS (slot_getbasepfn (node, slot)), -PFN_PHYS (slot_psize), node); +PFN_PHYS (slot_psize), node, +MEMBLOCK_NONE ); } } }arch/s390/kernel/setup.c +2
-1@@ -593,7 +593,8 @@ static void __init setup_resources(void)* part of the System RAM resource. */ if (crashk_res.end) { -memblock_add_node (crashk_res.start,resource_size (&crashk_res),0 ); +memblock_add_node (crashk_res.start,resource_size (&crashk_res), +0 ,MEMBLOCK_NONE );memblock_reserve (crashk_res.start,resource_size (&crashk_res));insert_resource (&iomem_resource, &crashk_res); }include/linux/memblock.h +2
-1@@ -104,7 +104,8 @@ static inline void memblock_discard(void) {}#endif void memblock_allow_resize (void ); -int memblock_add_node (phys_addr_t base,phys_addr_t size,int nid); +int memblock_add_node (phys_addr_t base,phys_addr_t size,int nid, +enum memblock_flags flags);int memblock_add (phys_addr_t base,phys_addr_t size);int memblock_remove (phys_addr_t base,phys_addr_t size);int memblock_free (phys_addr_t base,phys_addr_t size);include/linux/mm.h +1
-1@@ -2447,7 +2447,7 @@ static inline unsigned long get_num_physpages(void)* unsigned long max_zone_pfns[MAX_NR_ZONES] = {max_dma, max_normal_pfn, * max_highmem_pfn}; * for_each_valid_physical_page_range() -* memblock_add_node(base, size, nid) +* memblock_add_node(base, size, nid, MEMBLOCK_NONE) * free_area_init(max_zone_pfns); */ void free_area_init (unsigned long *max_zone_pfn);mm/memblock.c +5
-4@@ -655,6 +655,7 @@ static int __init_memblock memblock_add_range(struct memblock_type *type,* @base: base address of the new region * @size: size of the new region * @nid: nid of the new region +* @flags: flags of the new region * * Add new memblock region [@base, @base + @size) to the "memory" * type. See memblock_add_range() description for mode details @@ -663,14 +664,14 @@ static int __init_memblock memblock_add_range(struct memblock_type *type,* 0 on success, -errno on failure. */ int __init_memblock memblock_add_node(phys_addr_t base, phys_addr_t size, -int nid) +int nid,enum memblock_flags flags) {phys_addr_t end = base + size -1 ; -memblock_dbg ("%s: [%pa-%pa] nid=%d %pS\n" , __func__, - &base, &end, nid, (void *)_RET_IP_); +memblock_dbg ("%s: [%pa-%pa] nid=%d flags=%x %pS\n" , __func__, + &base, &end, nid, flags, (void *)_RET_IP_); -return memblock_add_range (&memblock.memory, base, size, nid,0 ); +return memblock_add_range (&memblock.memory, base, size, nid, flags); }/** mm/memory_hotplug.c +1
-1@@ -1385,7 +1385,7 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)mem_hotplug_begin ();if (IS_ENABLED (CONFIG_ARCH_KEEP_MEMBLOCK )) { - ret =memblock_add_node (start, size, nid); + ret =memblock_add_node (start, size, nid,MEMBLOCK_NONE );if (ret) goto error_mem_hotplug_end; }Signature · 1 line
2.31.1
Signature · 7 lines
Sincerely yours, Mike. _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 4/5 memblock: add MEMBLOCK_DRIVER_MANAGED to mimic IORESOURCE_SYSRAM_DRIVER_MANAGED
In reply to David HildenbrandLet's add a flag that corresponds to IORESOURCE_SYSRAM_DRIVER_MANAGED, indicating that we're dealing with a memory region that is never indicated in the firmware-provided memory map, but always detected and added by a driver.
Similar to MEMBLOCK_HOTPLUG, most infrastructure has to treat such memory regions like ordinary MEMBLOCK_NONE memory regions -- for example, when selecting memory regions to add to the vmcore for dumping in the crashkernel via for_each_mem_range().
However, especially kexec_file is not supposed to select such memblocks via for_each_free_mem_range() / for_each_free_mem_range_reverse() to place kexec images, similar to how we handle IORESOURCE_SYSRAM_DRIVER_MANAGED without CONFIG_ARCH_KEEP_MEMBLOCK.
We'll make sure that memory hotplug code sets the flag where applicable (IORESOURCE_SYSRAM_DRIVER_MANAGED) next. This prepares architectures that need CONFIG_ARCH_KEEP_MEMBLOCK, such as arm64, for virtio-mem support.
Note that kexec *must not* indicate this memory to the second kernel and *must not* place kexec-images on this memory. Let's add a comment to kexec_walk_memblock(), documenting how we handle MEMBLOCK_DRIVER_MANAGED now just like using IORESOURCE_SYSRAM_DRIVER_MANAGED in locate_mem_hole_callback() for kexec_walk_resources().
Also note that MEMBLOCK_HOTPLUG cannot be reused due to different semantics:
MEMBLOCK_HOTPLUG: memory is indicated as "System RAM" in the firmware-provided memory map and added to the system early during boot; kexec *has to* indicate this memory to the second kernel and can place kexec-images on this memory. After memory hotunplug, kexec has to be re-armed. We mostly ignore this flag when "movable_node" is not set on the kernel command line, because then we're told to not care about hotunpluggability of such memory regions.
MEMBLOCK_DRIVER_MANAGED: memory is not indicated as "System RAM" in the firmware-provided memory map; this memory is always detected and added to the system by a driver; memory might not actually be physically hotunpluggable. kexec *must not* indicate this memory to the second kernel and *must not* place kexec-images on this memory.
--- include/linux/memblock.h | 16 ++++++++++++++-- kernel/kexec_file.c | 5 +++++ mm/memblock.c | 4 ++++ 3 files changed, 23 insertions(+), 2 deletions(-)
include/linux/memblock.h +14 -2
@@ -37,12 +37,17 @@ extern unsigned long long max_possible_pfn;* @MEMBLOCK_NOMAP: don't add to kernel direct mapping and treat as * reserved in the memory map; refer to memblock_mark_nomap() description * for further details +* @MEMBLOCK_DRIVER_MANAGED: memory region that is always detected and added +* via a driver, and never indicated in the firmware-provided memory map as +* system RAM. This corresponds to IORESOURCE_SYSRAM_DRIVER_MANAGED in the +* kernel resource tree. */ enum memblock_flags {MEMBLOCK_NONE =0x0 ,/* No special request */ MEMBLOCK_HOTPLUG =0x1 ,/* hotpluggable region */ MEMBLOCK_MIRROR =0x2 ,/* mirrored region */ MEMBLOCK_NOMAP =0x4 ,/* don't add to kernel direct mapping */ +MEMBLOCK_DRIVER_MANAGED =0x8 ,/* always detected via a driver */ };/**
@@ -213,7 +218,8 @@ static inline void __next_physmem_range(u64 *idx, struct memblock_type *type,*/ #define for_each_mem_range (i, p_start, p_end) \ __for_each_mem_range(i, &memblock.memory, NULL, NUMA_NO_NODE, \ - MEMBLOCK_HOTPLUG, p_start, p_end, NULL) + MEMBLOCK_HOTPLUG | MEMBLOCK_DRIVER_MANAGED, \ + p_start, p_end, NULL)/** * for_each_mem_range_rev - reverse iterate through memblock areas from
@@ -224,7 +230,8 @@ static inline void __next_physmem_range(u64 *idx, struct memblock_type *type,*/ #define for_each_mem_range_rev (i, p_start, p_end) \ __for_each_mem_range_rev(i, &memblock.memory, NULL, NUMA_NO_NODE, \ - MEMBLOCK_HOTPLUG, p_start, p_end, NULL) + MEMBLOCK_HOTPLUG | MEMBLOCK_DRIVER_MANAGED,\ + p_start, p_end, NULL)/** * for_each_reserved_mem_range - iterate over all reserved memblock areas
@@ -254,6 +261,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m)return m->flags &MEMBLOCK_NOMAP ; } +static inline bool memblock_is_driver_managed (struct memblock_region *m) +{ +return m->flags &MEMBLOCK_DRIVER_MANAGED ; +} +int memblock_search_pfn_nid (unsigned long pfn,unsigned long *start_pfn,unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
kernel/kexec_file.c +5 -0
@@ -556,6 +556,11 @@ static int kexec_walk_memblock(struct kexec_buf *kbuf,if (kbuf->image->type =KEXEC_TYPE_CRASH )return func (&crashk_res, kbuf); +/* +* Using MEMBLOCK_NONE will properly skip MEMBLOCK_DRIVER_MANAGED. See +* IORESOURCE_SYSRAM_DRIVER_MANAGED handling in +* locate_mem_hole_callback(). +*/ if (kbuf->top_down) {for_each_free_mem_range_reverse (i,NUMA_NO_NODE ,MEMBLOCK_NONE , &mstart, &mend,NULL ) {
mm/memblock.c +4 -0
@@ -979,6 +979,10 @@ static bool should_skip_region(struct memblock_type *type,if (!(flags &MEMBLOCK_NOMAP ) &&memblock_is_nomap (m))return true; +/* skip driver-managed memory unless we were asked for it explicitly */ +if (!(flags &MEMBLOCK_DRIVER_MANAGED ) &&memblock_is_driver_managed (m)) +return true; +return false; }
Signature · 1 line
2.31.1
Re: [PATCH v2 4/5] memblock: add MEMBLOCK_DRIVER_MANAGED to mimic IORESOURCE_SYSRAM_DRIVER_MANAGED
Quoted 37 lines · David Hildenbrand
Let's add a flag that corresponds to
IORESOURCE_SYSRAM_DRIVER_MANAGED, indicating that we're dealing with a memory region that is never indicated in the firmware-provided memory map, but always detected and added by a driver.Similar to
MEMBLOCK_HOTPLUG, most infrastructure has to treat such memory regions like ordinaryMEMBLOCK_NONEmemory regions -- for example, when selecting memory regions to add to the vmcore for dumping in the crashkernel viafor_each_mem_range().However, especially
kexec_fileis not supposed to select such memblocks viafor_each_free_mem_range()/for_each_free_mem_range_reverse()to place kexec images, similar to how we handleIORESOURCE_SYSRAM_DRIVER_MANAGEDwithoutCONFIG_ARCH_KEEP_MEMBLOCK.We'll make sure that memory hotplug code sets the flag where applicable (
IORESOURCE_SYSRAM_DRIVER_MANAGED) next. This prepares architectures that needCONFIG_ARCH_KEEP_MEMBLOCK, such as arm64, for virtio-mem support.Note that kexec *must not* indicate this memory to the second kernel and *must not* place kexec-images on this memory. Let's add a comment to
kexec_walk_memblock(), documenting how we handleMEMBLOCK_DRIVER_MANAGEDnow just like usingIORESOURCE_SYSRAM_DRIVER_MANAGEDinlocate_mem_hole_callback()forkexec_walk_resources().Also note that
MEMBLOCK_HOTPLUGcannot be reused due to different semantics:
MEMBLOCK_HOTPLUG: memory is indicated as "System RAM" in the firmware-provided memory map and added to the system early during boot; kexec *has to* indicate this memory to the second kernel and can place kexec-images on this memory. After memory hotunplug, kexec has to be re-armed. We mostly ignore this flag when "movable_node" is not set on the kernel command line, because then we're told to not care about hotunpluggability of such memory regions.
Quoted 7 lines
MEMBLOCK_DRIVER_MANAGED: memory is not indicated as "System RAM" in the firmware-provided memory map; this memory is always detected and added to the system by a driver; memory might not actually be physically hotunpluggable. kexec *must not* indicate this memory to the second kernel and *must not* place kexec-images on this memory.Signed-off-by David Hildenbrand <[email protected]>
Quoted 94 lines
--- include/linux/memblock.h | 16 ++++++++++++++-- kernel/kexec_file.c | 5 +++++ mm/memblock.c | 4 ++++ 3 files changed, 23 insertions(+), 2 deletions(-)include/linux/memblock.h +14
-2@@ -37,12 +37,17 @@ extern unsigned long long max_possible_pfn;* @MEMBLOCK_NOMAP: don't add to kernel direct mapping and treat as * reserved in the memory map; refer to memblock_mark_nomap() description * for further details +* @MEMBLOCK_DRIVER_MANAGED: memory region that is always detected and added +* via a driver, and never indicated in the firmware-provided memory map as +* system RAM. This corresponds to IORESOURCE_SYSRAM_DRIVER_MANAGED in the +* kernel resource tree. */ enum memblock_flags {MEMBLOCK_NONE =0x0 ,/* No special request */ MEMBLOCK_HOTPLUG =0x1 ,/* hotpluggable region */ MEMBLOCK_MIRROR =0x2 ,/* mirrored region */ MEMBLOCK_NOMAP =0x4 ,/* don't add to kernel direct mapping */ +MEMBLOCK_DRIVER_MANAGED =0x8 ,/* always detected via a driver */ };/** @@ -213,7 +218,8 @@ static inline void __next_physmem_range(u64 *idx, struct memblock_type *type,*/ #define for_each_mem_range (i, p_start, p_end) \ __for_each_mem_range(i, &memblock.memory, NULL, NUMA_NO_NODE, \ - MEMBLOCK_HOTPLUG, p_start, p_end, NULL) + MEMBLOCK_HOTPLUG | MEMBLOCK_DRIVER_MANAGED, \ + p_start, p_end, NULL)/** * for_each_mem_range_rev - reverse iterate through memblock areas from @@ -224,7 +230,8 @@ static inline void __next_physmem_range(u64 *idx, struct memblock_type *type,*/ #define for_each_mem_range_rev (i, p_start, p_end) \ __for_each_mem_range_rev(i, &memblock.memory, NULL, NUMA_NO_NODE, \ - MEMBLOCK_HOTPLUG, p_start, p_end, NULL) + MEMBLOCK_HOTPLUG | MEMBLOCK_DRIVER_MANAGED,\ + p_start, p_end, NULL)/** * for_each_reserved_mem_range - iterate over all reserved memblock areas @@ -254,6 +261,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m)return m->flags &MEMBLOCK_NOMAP ; } +static inline bool memblock_is_driver_managed (struct memblock_region *m) +{ +return m->flags &MEMBLOCK_DRIVER_MANAGED ; +} +int memblock_search_pfn_nid (unsigned long pfn,unsigned long *start_pfn,unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,kernel/kexec_file.c +5
-0@@ -556,6 +556,11 @@ static int kexec_walk_memblock(struct kexec_buf *kbuf,if (kbuf->image->type ==KEXEC_TYPE_CRASH )return func (&crashk_res, kbuf); +/* +* Using MEMBLOCK_NONE will properly skip MEMBLOCK_DRIVER_MANAGED. See +* IORESOURCE_SYSRAM_DRIVER_MANAGED handling in +* locate_mem_hole_callback(). +*/ if (kbuf->top_down) {for_each_free_mem_range_reverse (i,NUMA_NO_NODE ,MEMBLOCK_NONE , &mstart, &mend,NULL ) {mm/memblock.c +4
-0@@ -979,6 +979,10 @@ static bool should_skip_region(struct memblock_type *type,if (!(flags &MEMBLOCK_NOMAP ) &&memblock_is_nomap (m))return true; +/* skip driver-managed memory unless we were asked for it explicitly */ +if (!(flags &MEMBLOCK_DRIVER_MANAGED ) &&memblock_is_driver_managed (m)) +return true; +return false; }Signature · 1 line
2.31.1
Signature · 7 lines
Sincerely yours, Mike. _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 5/5 mm/memory_hotplug: indicate MEMBLOCK_DRIVER_MANAGED with IORESOURCE_SYSRAM_DRIVER_MANAGED
In reply to David HildenbrandLet's communicate driver-managed regions to memblock, to properly teach kexec_file with CONFIG_ARCH_KEEP_MEMBLOCK to not place images on these memory regions.
--- mm/memory_hotplug.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
mm/memory_hotplug.c +4 -1
@@ -1357,6 +1357,7 @@ bool mhp_supports_memmap_on_memory(unsigned long size)int __ref add_memory_resource (int nid,struct resource *res,mhp_t mhp_flags) {struct mhp_params params = { .pgprot =pgprot_mhp (PAGE_KERNEL ) }; +enum memblock_flags memblock_flags =MEMBLOCK_NONE ;struct vmem_altmap mhp_altmap = {};struct memory_group *group =NULL ;u64 start, size;
@@ -1385,7 +1386,9 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)mem_hotplug_begin ();if (IS_ENABLED (CONFIG_ARCH_KEEP_MEMBLOCK )) { - ret =memblock_add_node (start, size, nid,MEMBLOCK_NONE ); +if (res->flags &IORESOURCE_SYSRAM_DRIVER_MANAGED ) + memblock_flags =MEMBLOCK_DRIVER_MANAGED ; + ret =memblock_add_node (start, size, nid, memblock_flags);if (ret) goto error_mem_hotplug_end; }
Signature · 1 line
2.31.1