mm: introduce numa_memblks
Subsystems (24)
Mike Rapoport
maintainerv4 cover mm: introduce numa_memblks
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following the discussion about handling of CXL fixed memory windows on arm64 [1] I decided to bite the bullet and move numa_memblks from x86 to the generic code so they will be available on arm64/riscv and maybe on loongarch sometime later.
While it could be possible to use memblock to describe CXL memory windows, it currently lacks notion of unpopulated memory ranges and numa_memblks does implement this.
Another reason to make numa_memblks generic is that both arch_numa (arm64 and riscv) and loongarch use trimmed copy of x86 code although there is no fundamental reason why the same code cannot be used on all these platforms. Having numa_memblks in mm/ will make it's interaction with ACPI and FDT more consistent and I believe will reduce maintenance burden.
And with generic numa_memblks it is (almost) straightforward to enable NUMA emulation on arm64 and riscv.
The first 9 commits in this series are cleanups that are not strictly related to numa_memblks.
Commits 10-16 slightly reorder code in x86 to allow extracting numa_memblks and NUMA emulation to the generic code.
Commits 17-19 actually move the code from arch/x86/ to mm/ and commits 20-22 does some aftermath cleanups.
Commit 23 updates of_numa_init() to return error of no NUMA nodes were found in the device tree.
Commit 24 switches arch_numa to numa_memblks.
Commit 25 enables usage of phys_to_target_node() and memory_add_physaddr_to_nid() with numa_memblks.
Commit 26 moves the description for numa=fake from x86 to admin-guide.
[1] https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]
* update allocation of offline node, thanks Jonathan
* add comment about dependency of get_pfn_range_for_nid on memblock_set_node(), per Dan
* fix build errros with 32-bit phys_address_t reported by kbuild
* add Acked- and Reviewed-by, thanks Dan and David
v2: https://lore.kernel.org/all/[email protected]
* rebase on v6.11-rc1
* fix dummy_numa_init() in arch_numa, thanks Zi Yan
* update of_numa_init() to return error of no NUMA nodes were
* add Tested-by, thanks Zi Yan
v1: https://lore.kernel.org/all/[email protected]
* add cleanup for arch_alloc_nodedata and HAVE_ARCH_NODEDATA_EXTENSION
* add patch that moves description of numa=fake kernel parameter from x86 to admin-guide
* reduce rounding up of node_data allocations from PAGE_SIZE to SMP_CACHE_BYTES
* restore single allocation attempt of numa_distance
* fix several comments
* added review tags
Mike Rapoport (Microsoft) (26):
mm: move kernel/numa.c to mm/
MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
MIPS: loongson64: rename __node_data to node_data
MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
arch, mm: move definition of node_data to generic code
mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
arch, mm: pull out allocation of NODE_DATA to generic code x86/numa: simplify numa_distance allocation
x86/numa: use get_pfn_range_for_nid to verify that node spans memory x86/numa: move FAKE_NODE_* defines to numa_emu x86/numa_emu: simplify allocation of phys_dist x86/numa_emu: split __apicid_to_node update to a helper function x86/numa_emu: use a helper function to get MAX_DMA32_PFN x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
mm: introduce numa_memblks
mm: move numa_distance and related code from x86 to numa_memblks
mm: introduce numa_emulation
mm: numa_memblks: introduce numa_memblks_init
mm: numa_memblks: make several functions and variables static
mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing
meminfo
of, numa: return -EINVAL when no numa-node-id is found
arch_numa: switch over to numa_memblks
mm: make range-to-target_node lookup facility a part of numa_memblks
docs: move numa=fake description to kernel-parameters.txt
.../admin-guide/kernel-parameters.txt | 15 +
.../arch/x86/x86_64/boot-options.rst | 12 -
arch/arm64/include/asm/Kbuild | 1 +
arch/arm64/include/asm/mmzone.h | 13 -
arch/arm64/include/asm/topology.h | 1 +
arch/loongarch/include/asm/Kbuild | 1 +
arch/loongarch/include/asm/mmzone.h | 16 -
arch/loongarch/include/asm/topology.h | 1 +
arch/loongarch/kernel/numa.c | 21 -
arch/mips/Kconfig | 5 -
arch/mips/include/asm/mach-ip27/mmzone.h | 1 -
.../mips/include/asm/mach-loongson64/mmzone.h | 4 -
arch/mips/loongson64/numa.c | 28 +-
arch/mips/sgi-ip27/ip27-memory.c | 12 +-
arch/mips/sgi-ip27/ip27-smp.c | 2 +
arch/powerpc/include/asm/mmzone.h | 6 -
arch/powerpc/mm/numa.c | 26 +-
arch/riscv/include/asm/Kbuild | 1 +
arch/riscv/include/asm/mmzone.h | 13 -
arch/riscv/include/asm/topology.h | 4 +
arch/s390/include/asm/Kbuild | 1 +
arch/s390/include/asm/mmzone.h | 17 -
arch/s390/kernel/numa.c | 3 -
arch/sh/include/asm/mmzone.h | 3 -
arch/sh/mm/init.c | 7 +-
arch/sh/mm/numa.c | 3 -
arch/sparc/include/asm/mmzone.h | 4 -
arch/sparc/mm/init_64.c | 11 +-
arch/x86/Kconfig | 9 +-
arch/x86/include/asm/Kbuild | 1 +
arch/x86/include/asm/mmzone.h | 6 -
arch/x86/include/asm/mmzone_32.h | 17 -
arch/x86/include/asm/mmzone_64.h | 18 -
arch/x86/include/asm/numa.h | 26 +-
arch/x86/include/asm/sparsemem.h | 9 -
arch/x86/mm/Makefile | 1 -
arch/x86/mm/amdtopology.c | 1 +
arch/x86/mm/numa.c | 622 +-----------------
arch/x86/mm/numa_internal.h | 24 -
drivers/acpi/numa/srat.c | 1 +
drivers/base/Kconfig | 1 +
drivers/base/arch_numa.c | 224 ++-----
drivers/cxl/Kconfig | 2 +-
drivers/dax/Kconfig | 2 +-
drivers/of/of_numa.c | 5 +-
include/asm-generic/mmzone.h | 5 +
include/asm-generic/numa.h | 6 +-
include/linux/memory_hotplug.h | 48 --
include/linux/numa.h | 8 +
include/linux/numa_memblks.h | 58 ++
kernel/Makefile | 1 -
kernel/numa.c | 26 -
mm/Kconfig | 11 +
mm/Makefile | 3 +
mm/mm_init.c | 10 +-
mm/numa.c | 69 ++
{arch/x86/mm => mm}/numa_emulation.c | 42 +-
mm/numa_memblks.c | 571 ++++++++++++++++
58 files changed, 893 insertions(+), 1166 deletions(-)
delete mode 100644 arch/arm64/include/asm/mmzone.h
delete mode 100644 arch/loongarch/include/asm/mmzone.h
delete mode 100644 arch/riscv/include/asm/mmzone.h
delete mode 100644 arch/s390/include/asm/mmzone.h
delete mode 100644 arch/x86/include/asm/mmzone.h
delete mode 100644 arch/x86/include/asm/mmzone_32.h
delete mode 100644 arch/x86/include/asm/mmzone_64.h
create mode 100644 include/asm-generic/mmzone.h
create mode 100644 include/linux/numa_memblks.h
delete mode 100644 kernel/numa.c
create mode 100644 mm/numa.c
rename {arch/x86/mm => mm}/numa_emulation.c (94%)
create mode 100644 mm/numa_memblks.cbase-commit: 8400291e289ee6b2bf9779ff1c83a291501f017b
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 1/26 mm: move kernel/numa.c to mm/
From: "Mike Rapoport (Microsoft)" <[email protected]>
The stub functions in kernel/numa.c belong to mm/ rather than to kernel/
---
kernel/Makefile | 1 -
mm/Makefile | 1 +
{kernel => mm}/numa.c | 0
3 files changed, 1 insertion(+), 1 deletion(-)
rename {kernel => mm}/numa.c (100%)kernel/Makefile +0 -1
@@ -116,7 +116,6 @@ obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o obj-$(CONFIG_HAVE_STATIC_CALL) += static_call.o obj-$(CONFIG_HAVE_STATIC_CALL_INLINE) += static_call_inline.o obj-$(CONFIG_CFI_CLANG) += cfi.o -obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_PERF_EVENTS) += events/
mm/Makefile +1 -0
@@ -141,3 +141,4 @@ obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o +obj-$(CONFIG_NUMA) += numa.o
mm/numa.c renamed from kernel/numa.c +0 -0
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 2/26 MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
sgi-ip27 is the only system that defines NODE_DATA() differently than the rest of NUMA machines.
Add node_data array of struct pglist pointers that will point to __node_data[node]->pglist and redefine NODE_DATA() to use node_data array.
This will allow pulling declaration of node_data to the generic mm code in the next commit.
--- arch/mips/include/asm/mach-ip27/mmzone.h | 5 ++++- arch/mips/sgi-ip27/ip27-memory.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)
arch/mips/include/asm/mach-ip27/mmzone.h +4 -1
@@ -22,7 +22,10 @@ struct node_data {extern struct node_data *__node_data[]; -#define NODE_DATA (n) (&__node_data[(n)]->pglist)#define hub_data (n) (&__node_data[(n)]->hub) +extern struct pglist_data *node_data[]; + +#define NODE_DATA (nid) (node_data[nid]) +#endif /* _ASM_MACH_MMZONE_H */
arch/mips/sgi-ip27/ip27-memory.c +4 -1
@@ -34,8 +34,10 @@#define SLOT_PFNSHIFT (SLOT_SHIFT - PAGE_SHIFT)#define PFN_NASIDSHFT (NASID_SHFT - PAGE_SHIFT) -struct node_data *__node_data[MAX_NUMNODES ]; +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data); +struct node_data *__node_data[MAX_NUMNODES ];EXPORT_SYMBOL (__node_data);static u64 gen_region_mask (void )
@@ -361,6 +363,7 @@ static void __init node_mem_init(nasid_t node)*/ __node_data[node] =__va (slot_freepfn <<PAGE_SHIFT );memset (__node_data[node],0 ,PAGE_SIZE ); + node_data[node] = &__node_data[node]->pglist;NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 3/26 MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
For SGI IP27 machines node_possible_map is statically set to NODE_MASK_ALL and it is not updated during NUMA initialization.
Ensure that it only contains nodes present in the system.
--- arch/mips/sgi-ip27/ip27-smp.c | 2 ++ 1 file changed, 2 insertions(+)
arch/mips/sgi-ip27/ip27-smp.c +2 -0
@@ -70,11 +70,13 @@ void cpu_node_probe(void)gda_t *gdap =GDA ;nodes_clear (node_online_map); +nodes_clear (node_possible_map);for (i =0 ; i <MAX_NUMNODES ; i++) {nasid_t nasid = gdap->g_nasidtable[i];if (nasid ==INVALID_NASID )break ;node_set_online (nasid); +node_set (nasid, node_possible_map); highest =node_scan_cpus (nasid, highest); }
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 4/26 MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Commit f8f9f21c7848 ("MIPS: Fix build error for loongson64 and sgi-ip27") added HAVE_ARCH_NODEDATA_EXTENSION to sgi-ip27 to silence a compilation error that happened because sgi-ip27 didn't define array of pg_data_t as node_data like most other architectures did.
After addition of node_data array that matches other architectures and after ensuring that offline nodes do not appear on node_possible_map, it is safe to drop arch_alloc_nodedata() and HAVE_ARCH_NODEDATA_EXTENSION from sgi-ip27.
--- arch/mips/Kconfig | 1 - arch/mips/sgi-ip27/ip27-memory.c | 10 ---------- 2 files changed, 11 deletions(-)
arch/mips/Kconfig +0 -1
@@ -735,7 +735,6 @@ config SGI_IP27 select WAR_R10000_LLSC select MIPS_L1_CACHE_SHIFT_7 select NUMA - select HAVE_ARCH_NODEDATA_EXTENSION help This are the SGI Origin 200, Origin 2000 and Onyx 2 Graphics workstations. To compile a Linux kernel that runs on these, say Y
arch/mips/sgi-ip27/ip27-memory.c +0 -10
@@ -426,13 +426,3 @@ void __init mem_init(void)memblock_free_all ();setup_zero_pages ();/* This comes from node 0 */ } - -pg_data_t *__init arch_alloc_nodedata (int nid) -{ -return memblock_alloc (sizeof (pg_data_t),SMP_CACHE_BYTES ); -} - -void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - __node_data[nid] = (struct node_data *)pgdat; -}
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 5/26 MIPS: loongson64: rename __node_data to node_data
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Make definition of node_data match other architectures.
This will allow pulling declaration of node_data to the generic mm code in the following commit.
--- arch/mips/include/asm/mach-loongson64/mmzone.h | 4 ++-- arch/mips/loongson64/numa.c | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-)
arch/mips/include/asm/mach-loongson64/mmzone.h +2 -2
@@ -14,9 +14,9 @@#define pa_to_nid (addr) (((addr) & 0xf00000000000) >> NODE_ADDRSPACE_SHIFT)#define nid_to_addrbase (nid) ((unsigned long)(nid) << NODE_ADDRSPACE_SHIFT) -extern struct pglist_data *__node_data[]; +extern struct pglist_data *node_data[]; -#define NODE_DATA (n) (__node_data[n]) +#define NODE_DATA (n) (node_data[n])extern void __init prom_init_numa_memory (void );
arch/mips/loongson64/numa.c +4 -4
@@ -29,8 +29,8 @@unsigned char __node_distances[MAX_NUMNODES ][MAX_NUMNODES ];EXPORT_SYMBOL (__node_distances); -struct pglist_data *__node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (__node_data); +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data);cpumask_t __node_cpumask[MAX_NUMNODES ];EXPORT_SYMBOL (__node_cpumask);
@@ -107,7 +107,7 @@ static void __init node_mem_init(unsigned int node) tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT );if (tnid != node)pr_info ("NODE_DATA(%d) on node %d\n" , node, tnid); - __node_data[node] = nd; + node_data[node] = nd;NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
@@ -206,5 +206,5 @@ pg_data_t * __init arch_alloc_nodedata(int nid)void arch_refresh_nodedata (int nid,pg_data_t *pgdat) { - __node_data[nid] = pgdat; + node_data[nid] = pgdat; }
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 6/26 MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Commit f8f9f21c7848 ("MIPS: Fix build error for loongson64 and sgi-ip27") added HAVE_ARCH_NODEDATA_EXTENSION to loongson64 to silence a compilation error that happened because loongson64 didn't define array of pg_data_t as node_data like most other architectures did.
After rename of __node_data to node_data arch_alloc_nodedata() and HAVE_ARCH_NODEDATA_EXTENSION can be dropped from loongson64.
Since it was the only user of HAVE_ARCH_NODEDATA_EXTENSION config option also remove this option from arch/mips/Kconfig.
--- arch/mips/Kconfig | 4 ---- arch/mips/loongson64/numa.c | 10 ---------- 2 files changed, 14 deletions(-)
arch/mips/Kconfig +0 -4
@@ -502,7 +502,6 @@ config MACH_LOONGSON64 select USE_OF select BUILTIN_DTB select PCI_HOST_GENERIC - select HAVE_ARCH_NODEDATA_EXTENSION if NUMA help This enables the support of Loongson-2/3 family of machines.
@@ -2612,9 +2611,6 @@ config NUMA config SYS_SUPPORTS_NUMA bool -config HAVE_ARCH_NODEDATA_EXTENSION - bool - config RELOCATABLE bool "Relocatable kernel" depends on SYS_SUPPORTS_RELOCATABLE
arch/mips/loongson64/numa.c +0 -10
@@ -198,13 +198,3 @@ void __init prom_init_numa_memory(void)pr_info ("CP0_PageGrain: CP0 5.1 (0x%x)\n" ,read_c0_pagegrain ());prom_meminit (); } - -pg_data_t *__init arch_alloc_nodedata (int nid) -{ -return memblock_alloc (sizeof (pg_data_t),SMP_CACHE_BYTES ); -} - -void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - node_data[nid] = pgdat; -}
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 7/26 arch, mm: move definition of node_data to generic code
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Every architecture that supports NUMA defines node_data in the same way:
struct pglist_data *node_data[MAX_NUMNODES];
No reason to keep multiple copies of this definition and its forward declarations, especially when such forward declaration is the only thing in include/asm/mmzone.h for many architectures.
Add definition and declaration of node_data to generic code and drop architecture-specific versions.
--- arch/arm64/include/asm/Kbuild | 1 + arch/arm64/include/asm/mmzone.h | 13 ------------- arch/arm64/include/asm/topology.h | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/asm/mmzone.h | 16 ---------------- arch/loongarch/include/asm/topology.h | 1 + arch/loongarch/kernel/numa.c | 3 --- arch/mips/include/asm/mach-ip27/mmzone.h | 4 ---- arch/mips/include/asm/mach-loongson64/mmzone.h | 4 ---- arch/mips/loongson64/numa.c | 2 -- arch/mips/sgi-ip27/ip27-memory.c | 3 --- arch/powerpc/include/asm/mmzone.h | 6 ------ arch/powerpc/mm/numa.c | 2 -- arch/riscv/include/asm/Kbuild | 1 + arch/riscv/include/asm/mmzone.h | 13 ------------- arch/riscv/include/asm/topology.h | 4 ++++ arch/s390/include/asm/Kbuild | 1 + arch/s390/include/asm/mmzone.h | 17 ----------------- arch/s390/kernel/numa.c | 3 --- arch/sh/include/asm/mmzone.h | 3 --- arch/sh/mm/numa.c | 3 --- arch/sparc/include/asm/mmzone.h | 4 ---- arch/sparc/mm/init_64.c | 2 -- arch/x86/include/asm/Kbuild | 1 + arch/x86/include/asm/mmzone.h | 6 ------ arch/x86/include/asm/mmzone_32.h | 17 ----------------- arch/x86/include/asm/mmzone_64.h | 18 ------------------ arch/x86/mm/numa.c | 3 --- drivers/base/arch_numa.c | 2 -- include/asm-generic/mmzone.h | 5 +++++ include/linux/numa.h | 3 +++ mm/numa.c | 3 +++ 32 files changed, 22 insertions(+), 144 deletions(-) delete mode 100644 arch/arm64/include/asm/mmzone.h delete mode 100644 arch/loongarch/include/asm/mmzone.h delete mode 100644 arch/riscv/include/asm/mmzone.h delete mode 100644 arch/s390/include/asm/mmzone.h delete mode 100644 arch/x86/include/asm/mmzone.h delete mode 100644 arch/x86/include/asm/mmzone_32.h delete mode 100644 arch/x86/include/asm/mmzone_64.h create mode 100644 include/asm-generic/mmzone.h
arch/arm64/include/asm/Kbuild +1 -0
@@ -9,6 +9,7 @@ syscall-y += unistd_compat_32.h generic-y += early_ioremap.h generic-y += mcs_spinlock.h +generic-y += mmzone.h generic-y += qrwlock.h generic-y += qspinlock.h generic-y += parport.h
arch/arm64/include/asm/mmzone.h deleted +0 -13
@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __ASM_MMZONE_H -#define __ASM_MMZONE_H - -#ifdef CONFIG_NUMA - -#include <asm/numa.h> - -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[(nid)]) - -#endif /* CONFIG_NUMA */ -#endif /* __ASM_MMZONE_H */
arch/arm64/include/asm/topology.h +1 -0
@@ -5,6 +5,7 @@#include <linux/cpumask.h> #ifdef CONFIG_NUMA +#include <asm/numa.h> struct pci_bus ;int pcibus_to_node (struct pci_bus *bus);
arch/loongarch/include/asm/Kbuild +1 -0
@@ -9,5 +9,6 @@ generic-y += qrwlock.h generic-y += qspinlock.h generic-y += user.h generic-y += ioctl.h +generic-y += mmzone.h generic-y += statfs.h generic-y += param.h
arch/loongarch/include/asm/mmzone.h deleted +0 -16
@@ -1,16 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* -* Author: Huacai Chen ([email protected]) -* Copyright (C) 2020-2022 Loongson Technology Corporation Limited -*/ -#ifndef _ASM_MMZONE_H_ -#define _ASM_MMZONE_H_ - -#include <asm/page.h> -#include <asm/numa.h> - -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[(nid)]) - -#endif /* _ASM_MMZONE_H_ */
arch/loongarch/include/asm/topology.h +1 -0
@@ -8,6 +8,7 @@#include <linux/smp.h> #ifdef CONFIG_NUMA +#include <asm/numa.h> extern cpumask_t cpus_on_node[];
arch/loongarch/kernel/numa.c +0 -3
@@ -27,10 +27,7 @@#include <asm/time.h> int numa_off; -struct pglist_data *node_data[MAX_NUMNODES ];unsigned char node_distances[MAX_NUMNODES ][MAX_NUMNODES ]; - -EXPORT_SYMBOL (node_data);EXPORT_SYMBOL (node_distances);static struct numa_meminfo numa_meminfo;
arch/mips/include/asm/mach-ip27/mmzone.h +0 -4
@@ -24,8 +24,4 @@ extern struct node_data *__node_data[];#define hub_data (n) (&__node_data[(n)]->hub) -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[nid]) -#endif /* _ASM_MACH_MMZONE_H */
arch/mips/include/asm/mach-loongson64/mmzone.h +0 -4
@@ -14,10 +14,6 @@#define pa_to_nid (addr) (((addr) & 0xf00000000000) >> NODE_ADDRSPACE_SHIFT)#define nid_to_addrbase (nid) ((unsigned long)(nid) << NODE_ADDRSPACE_SHIFT) -extern struct pglist_data *node_data[]; - -#define NODE_DATA (n) (node_data[n]) -extern void __init prom_init_numa_memory (void );#endif /* _ASM_MACH_MMZONE_H */
arch/mips/loongson64/numa.c +0 -2
@@ -29,8 +29,6 @@unsigned char __node_distances[MAX_NUMNODES ][MAX_NUMNODES ];EXPORT_SYMBOL (__node_distances); -struct pglist_data *node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (node_data);cpumask_t __node_cpumask[MAX_NUMNODES ];EXPORT_SYMBOL (__node_cpumask);
arch/mips/sgi-ip27/ip27-memory.c +0 -3
@@ -34,9 +34,6 @@#define SLOT_PFNSHIFT (SLOT_SHIFT - PAGE_SHIFT)#define PFN_NASIDSHFT (NASID_SHFT - PAGE_SHIFT) -struct pglist_data *node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (node_data); -struct node_data *__node_data[MAX_NUMNODES ];EXPORT_SYMBOL (__node_data);
arch/powerpc/include/asm/mmzone.h +0 -6
@@ -20,12 +20,6 @@#ifdef CONFIG_NUMA -extern struct pglist_data *node_data[]; -/* -* Return a pointer to the node data for node n. -*/ -#define NODE_DATA (nid) (node_data[nid]) -/* * Following are specific to this numa platform. */
arch/powerpc/mm/numa.c +0 -2
@@ -43,11 +43,9 @@ static char *cmdline __initdata;int numa_cpu_lookup_table[NR_CPUS ];cpumask_var_t node_to_cpumask_map[MAX_NUMNODES ]; -struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (numa_cpu_lookup_table);EXPORT_SYMBOL (node_to_cpumask_map); -EXPORT_SYMBOL (node_data);static int primary_domain_index;static int n_mem_addr_cells, n_mem_size_cells;
arch/riscv/include/asm/Kbuild +1 -0
@@ -5,6 +5,7 @@ syscall-y += syscall_table_64.h generic-y += early_ioremap.h generic-y += flat.h generic-y += kvm_para.h +generic-y += mmzone.h generic-y += parport.h generic-y += spinlock.h generic-y += spinlock_types.h
arch/riscv/include/asm/mmzone.h deleted +0 -13
@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __ASM_MMZONE_H -#define __ASM_MMZONE_H - -#ifdef CONFIG_NUMA - -#include <asm/numa.h> - -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[(nid)]) - -#endif /* CONFIG_NUMA */ -#endif /* __ASM_MMZONE_H */
arch/riscv/include/asm/topology.h +4 -0
@@ -4,6 +4,10 @@#include <linux/arch_topology.h> +#ifdef CONFIG_NUMA +#include <asm/numa.h> +#endif +/* Replace task scheduler's default frequency-invariant accounting */ #define arch_scale_freq_tick topology_scale_freq_tick#define arch_set_freq_scale topology_set_freq_scale
arch/s390/include/asm/Kbuild +1 -0
@@ -7,3 +7,4 @@ generated-y += unistd_nr.h generic-y += asm-offsets.h generic-y += kvm_types.h generic-y += mcs_spinlock.h +generic-y += mmzone.h
arch/s390/include/asm/mmzone.h deleted +0 -17
@@ -1,17 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* -* NUMA support for s390 -* -* Copyright IBM Corp. 2015 -*/ - -#ifndef _ASM_S390_MMZONE_H -#define _ASM_S390_MMZONE_H - -#ifdef CONFIG_NUMA - -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[nid]) - -#endif /* CONFIG_NUMA */ -#endif /* _ASM_S390_MMZONE_H */
arch/s390/kernel/numa.c +0 -3
@@ -14,9 +14,6 @@#include <linux/node.h> #include <asm/numa.h> -struct pglist_data *node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (node_data); -void __init numa_setup (void ) {int nid;
arch/sh/include/asm/mmzone.h +0 -3
@@ -5,9 +5,6 @@#ifdef CONFIG_NUMA #include <linux/numa.h> -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[nid]) -static inline int pfn_to_nid (unsigned long pfn) {int nid;
arch/sh/mm/numa.c +0 -3
@@ -14,9 +14,6 @@#include <linux/pfn.h> #include <asm/sections.h> -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL_GPL (node_data); -/* * On SH machines the conventional approach is to stash system RAM * in node 0, and other memory blocks in to node 1 and up, ordered by
arch/sparc/include/asm/mmzone.h +0 -4
@@ -6,10 +6,6 @@#include <linux/cpumask.h> -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[nid]) -extern int numa_cpu_lookup_table[];extern cpumask_t numa_cpumask_lookup_table[];
arch/sparc/mm/init_64.c +0 -2
@@ -1115,11 +1115,9 @@ static void init_node_masks_nonnuma(void) }#ifdef CONFIG_NUMA -struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (numa_cpu_lookup_table);EXPORT_SYMBOL (numa_cpumask_lookup_table); -EXPORT_SYMBOL (node_data);static int scan_pio_for_cfg_handle (struct mdesc_handle *md,u64 pio,u32 cfg_handle)
arch/x86/include/asm/Kbuild +1 -0
@@ -11,3 +11,4 @@ generated-y += xen-hypercalls.h generic-y += early_ioremap.h generic-y += mcs_spinlock.h +generic-y += mmzone.h
arch/x86/include/asm/mmzone.h deleted +0 -6
@@ -1,6 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifdef CONFIG_X86_32 -# include <asm/mmzone_32.h> -#else -# include <asm/mmzone_64.h> -#endif
arch/x86/include/asm/mmzone_32.h deleted +0 -17
@@ -1,17 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* -* Written by Pat Gaughen ([email protected]) Mar 2002 -* -*/ - -#ifndef _ASM_X86_MMZONE_32_H -#define _ASM_X86_MMZONE_32_H - -#include <asm/smp.h> - -#ifdef CONFIG_NUMA -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[nid]) -#endif /* CONFIG_NUMA */ - -#endif /* _ASM_X86_MMZONE_32_H */
arch/x86/include/asm/mmzone_64.h deleted +0 -18
@@ -1,18 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* K8 NUMA support */ -/* Copyright 2002,2003 by Andi Kleen, SuSE Labs */ -/* 2.5 Version loosely based on the NUMAQ Code by Pat Gaughen. */ -#ifndef _ASM_X86_MMZONE_64_H -#define _ASM_X86_MMZONE_64_H - -#ifdef CONFIG_NUMA - -#include <linux/mmdebug.h> -#include <asm/smp.h> - -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[nid]) - -#endif -#endif /* _ASM_X86_MMZONE_64_H */
arch/x86/mm/numa.c +0 -3
@@ -24,9 +24,6 @@int numa_off;nodemask_t numa_nodes_parsed__initdata ; -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL (node_data); - static struct numa_meminfo numa_meminfo __initdata_or_meminfo; static struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo;
drivers/base/arch_numa.c +0 -2
@@ -15,8 +15,6 @@#include <asm/sections.h> -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL (node_data);nodemask_t numa_nodes_parsed__initdata ;static int cpu_to_node_map[NR_CPUS ] = { [0 ...NR_CPUS -1 ] =NUMA_NO_NODE };
include/asm-generic/mmzone.h new file +5 -0
@@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_MMZONE_H +#define _ASM_GENERIC_MMZONE_H + +#endif
include/linux/numa.h +3 -0
@@ -30,6 +30,9 @@ static inline bool numa_valid_node(int nid)#ifdef CONFIG_NUMA #include <asm/sparsemem.h> +extern struct pglist_data *node_data[]; +#define NODE_DATA (nid) (node_data[nid]) +/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);
mm/numa.c +3 -0
@@ -3,6 +3,9 @@#include <linux/printk.h> #include <linux/numa.h> +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data); +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 8/26 mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
There are no users of HAVE_ARCH_NODEDATA_EXTENSION left, so arch_alloc_nodedata() and arch_refresh_nodedata() are not needed anymore.
Replace the call to arch_alloc_nodedata() in free_area_init() with a new helper alloc_offline_node_data(), remove arch_refresh_nodedata() and cleanup include/linux/memory_hotplug.h from the associated ifdefery.
--- include/linux/memory_hotplug.h | 48 ---------------------------------- include/linux/numa.h | 4 +++ mm/mm_init.c | 10 ++----- mm/numa.c | 12 +++++++++ 4 files changed, 18 insertions(+), 56 deletions(-)
include/linux/memory_hotplug.h +0 -48
@@ -16,54 +16,6 @@ struct resource;struct vmem_altmap ;struct dev_pagemap ; -#ifdef CONFIG_HAVE_ARCH_NODEDATA_EXTENSION -/* -* For supporting node-hotadd, we have to allocate a new pgdat. -* -* If an arch has generic style NODE_DATA(), -* node_data[nid] = kzalloc() works well. But it depends on the architecture. -* -* In general, generic_alloc_nodedata() is used. -* -*/ -extern pg_data_t *arch_alloc_nodedata (int nid); -extern void arch_refresh_nodedata (int nid,pg_data_t *pgdat); - -#else /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ - -#define arch_alloc_nodedata (nid) generic_alloc_nodedata(nid) - -#ifdef CONFIG_NUMA -/* -* XXX: node aware allocation can't work well to get new node's memory at this time. -* Because, pgdat for the new node is not allocated/initialized yet itself. -* To use new node's memory, more consideration will be necessary. -*/ -#define generic_alloc_nodedata (nid) \ -({ \ - memblock_alloc(sizeof(*pgdat), SMP_CACHE_BYTES); \ -}) - -extern pg_data_t *node_data[]; -static inline void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - node_data[nid] = pgdat; -} - -#else /* !CONFIG_NUMA */ - -/* never called */ -static inline pg_data_t *generic_alloc_nodedata (int nid) -{ -BUG (); -return NULL ; -} -static inline void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ -} -#endif /* CONFIG_NUMA */ -#endif /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ -#ifdef CONFIG_MEMORY_HOTPLUG struct page *pfn_to_online_page (unsigned long pfn);
include/linux/numa.h +4 -0
@@ -33,6 +33,8 @@ static inline bool numa_valid_node(int nid)extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid]) +void __init alloc_offline_node_data (int nid); +/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);
@@ -60,6 +62,8 @@ static inline int phys_to_target_node(u64 start) {return 0 ; } + +static inline void alloc_offline_node_data (int nid) {}#endif #define numa_map_to_online_node (node) numa_nearest_node(node, N_ONLINE)
mm/mm_init.c +2 -8
@@ -1836,14 +1836,8 @@ void __init free_area_init(unsigned long *max_zone_pfn)for_each_node (nid) {pg_data_t *pgdat; -if (!node_online (nid)) { -/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); -if (!pgdat) -panic ("Cannot allocate %zuB for node %d.\n" , -sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat); - } +if (!node_online (nid)) +alloc_offline_node_data (nid); pgdat =NODE_DATA (nid);free_area_init_node (nid);
mm/numa.c +12 -0
@@ -6,6 +6,18 @@struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (node_data); +void __init alloc_offline_node_data (int nid) +{ +pg_data_t *pgdat; + + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); +if (!pgdat) +panic ("Cannot allocate %zuB for node %d.\n" , +sizeof (*pgdat), nid); + + node_data[nid] = pgdat; +} +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 9/26 arch, mm: pull out allocation of NODE_DATA to generic code
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Architectures that support NUMA duplicate the code that allocates NODE_DATA on the node-local memory with slight variations in reporting of the addresses where the memory was allocated.
Use x86 version as the basis for the generic alloc_node_data() function and call this function in architecture specific numa initialization.
Round up node data size to SMP_CACHE_BYTES rather than to PAGE_SIZE like x86 used to do since the bootmem era when allocation granularity was PAGE_SIZE anyway.
--- arch/loongarch/kernel/numa.c | 18 ------------------ arch/mips/loongson64/numa.c | 16 ++-------------- arch/powerpc/mm/numa.c | 24 +++--------------------- arch/sh/mm/init.c | 7 +------ arch/sparc/mm/init_64.c | 9 ++------- arch/x86/mm/numa.c | 34 +--------------------------------- drivers/base/arch_numa.c | 21 +-------------------- include/linux/numa.h | 1 + mm/numa.c | 27 +++++++++++++++++++++++++++ 9 files changed, 38 insertions(+), 119 deletions(-)
arch/loongarch/kernel/numa.c +0 -18
@@ -187,24 +187,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)return numa_add_memblk_to (nid, start, end, &numa_meminfo); } -static void __init alloc_node_data (int nid) -{ -void *nd; -unsigned long nd_pa; -size_t nd_sz =roundup (sizeof (pg_data_t),PAGE_SIZE ); - - nd_pa =memblock_phys_alloc_try_nid (nd_sz,SMP_CACHE_BYTES , nid); -if (!nd_pa) { -pr_err ("Cannot find %zu Byte for node_data (initial node: %d)\n" , nd_sz, nid); -return ; - } - - nd =__va (nd_pa); - - node_data[nid] = nd; -memset (nd,0 ,sizeof (pg_data_t)); -} -static void __init node_mem_init (unsigned int node) {unsigned long start_pfn, end_pfn;
arch/mips/loongson64/numa.c +2 -14
@@ -81,12 +81,8 @@ static void __init init_topology_matrix(void)static void __init node_mem_init (unsigned int node) { -struct pglist_data *nd;unsigned long node_addrspace_offset;unsigned long start_pfn, end_pfn; -unsigned long nd_pa; -int tnid; -const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); node_addrspace_offset =nid_to_addrbase (node); pr_info("Node%d's addrspace_offset is 0x%lx\n",
@@ -96,16 +92,8 @@ static void __init node_mem_init(unsigned int node)pr_info ("Node%d: start_pfn=0x%lx, end_pfn=0x%lx\n" , node, start_pfn, end_pfn); - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , node); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, node); - nd =__va (nd_pa); -memset (nd,0 ,sizeof (struct pglist_data )); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != node) -pr_info ("NODE_DATA(%d) on node %d\n" , node, tnid); - node_data[node] = nd; +alloc_node_data (node); +NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
arch/powerpc/mm/numa.c +3 -21
@@ -1093,27 +1093,9 @@ void __init dump_numa_cpu_topology(void)static void __init setup_node_data (int nid,u64 start_pfn,u64 end_pfn) {u64 spanned_pages = end_pfn - start_pfn; -const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); -u64 nd_pa; -void *nd; -int tnid; - - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, nid); - - nd =__va (nd_pa); - -/* report and initialize */ -pr_info (" NODE_DATA [mem %#010Lx-%#010Lx]\n" , - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -pr_info (" NODE_DATA(%d) on node %d\n" , nid, tnid); - - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); + +alloc_node_data (nid); +NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start_pfn;NODE_DATA (nid)->node_spanned_pages = spanned_pages;
arch/sh/mm/init.c +1 -6
@@ -212,12 +212,7 @@ void __init allocate_pgdat(unsigned int nid)get_pfn_range_for_nid (nid, &start_pfn, &end_pfn);#ifdef CONFIG_NUMA -NODE_DATA (nid) =memblock_alloc_try_nid ( -sizeof (struct pglist_data ), -SMP_CACHE_BYTES ,MEMBLOCK_LOW_LIMIT , -MEMBLOCK_ALLOC_ACCESSIBLE , nid); -if (!NODE_DATA (nid)) -panic ("Can't allocate pgdat for node %d\n" , nid); +alloc_node_data (nid);#endif NODE_DATA (nid)->node_start_pfn = start_pfn;
arch/sparc/mm/init_64.c +2 -7
@@ -1075,14 +1075,9 @@ static void __init allocate_node_data(int nid) {struct pglist_data *p;unsigned long start_pfn, end_pfn; -#ifdef CONFIG_NUMA -NODE_DATA (nid) =memblock_alloc_node (sizeof (struct pglist_data ), -SMP_CACHE_BYTES , nid); -if (!NODE_DATA (nid)) { -prom_printf ("Cannot allocate pglist_data for nid[%d]\n" , nid); -prom_halt (); - } +#ifdef CONFIG_NUMA +alloc_node_data (nid);NODE_DATA (nid)->node_id = nid;#endif
arch/x86/mm/numa.c +1 -33
@@ -191,39 +191,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)return numa_add_memblk_to (nid, start, end, &numa_meminfo); } -/* Allocate NODE_DATA for a node on the local memory */ -static void __init alloc_node_data (int nid) -{ -const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE ); -u64 nd_pa; -void *nd; -int tnid; - -/* -* Allocate node data. Try node-local memory and then any node. -* Never allocate in DMA zone. -*/ - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) { -pr_err ("Cannot find %zu bytes in any node (initial node: %d)\n" , - nd_size, nid); -return ; - } - nd =__va (nd_pa); - -/* report and initialize */ -printk (KERN_INFO "NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n" , nid, - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nid, tnid); - - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); - -node_set_online (nid); -} -/** * numa_cleanup_meminfo - Cleanup a numa_meminfo * @mi: numa_meminfo to clean up
@@ -571,6 +538,7 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)continue ;alloc_node_data (nid); +node_set_online (nid); }/* Dump memblock with node info and return. */
drivers/base/arch_numa.c +1 -20
@@ -216,30 +216,11 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)*/ static void __init setup_node_data (int nid,u64 start_pfn,u64 end_pfn) { -const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); -u64 nd_pa; -void *nd; -int tnid; -if (start_pfn >= end_pfn)pr_info ("Initmem setup node %d [<memory-less node>]\n" , nid); - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, nid); - - nd =__va (nd_pa); - -/* report and initialize */ -pr_info ("NODE_DATA [mem %#010Lx-%#010Lx]\n" , - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -pr_info ("NODE_DATA(%d) on node %d\n" , nid, tnid); +alloc_node_data (nid); - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t));NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start_pfn;NODE_DATA (nid)->node_spanned_pages = end_pfn - start_pfn;
include/linux/numa.h +1 -0
@@ -33,6 +33,7 @@ static inline bool numa_valid_node(int nid)extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid]) +void __init alloc_node_data (int nid);void __init alloc_offline_node_data (int nid);/* Generic implementation available */
mm/numa.c +27 -0
@@ -1,11 +1,38 @@// SPDX-License-Identifier: GPL-2.0-or-later +#include <linux/memblock.h> #include <linux/printk.h> #include <linux/numa.h> struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (node_data); +/* Allocate NODE_DATA for a node on the local memory */ +void __init alloc_node_data (int nid) +{ +const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); +u64 nd_pa; +void *nd; +int tnid; + +/* Allocate node data. Try node-local memory and then any node. */ + nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); +if (!nd_pa) +panic ("Cannot allocate %zu bytes for node %d data\n" , + nd_size, nid); + nd =__va (nd_pa); + +/* report and initialize */ +pr_info ("NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n" , nid, + nd_pa, nd_pa + nd_size -1 ); + tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); +if (tnid != nid) +pr_info (" NODE_DATA(%d) on node %d\n" , nid, tnid); + + node_data[nid] = nd; +memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); +} +void __init alloc_offline_node_data (int nid) {pg_data_t *pgdat;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 10/26 x86/numa: simplify numa_distance allocation
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Allocation of numa_distance uses memblock_phys_alloc_range() to limit allocation to be below the last mapped page.
But NUMA initializaition runs after the direct map is populated and there is also code in setup_arch() that adjusts memblock limit to reflect how much memory is already mapped in the direct map.
Simplify the allocation of numa_distance and use plain memblock_alloc().
--- arch/x86/mm/numa.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
arch/x86/mm/numa.c +2 -5
@@ -331,7 +331,6 @@ static int __init numa_alloc_distance(void)nodemask_t nodes_parsed;size_t size;int i, j, cnt =0 ; -u64 phys;/* size the new table and allocate it */ nodes_parsed = numa_nodes_parsed;
@@ -342,16 +341,14 @@ static int __init numa_alloc_distance(void) cnt++; size = cnt * cnt *sizeof (numa_distance[0 ]); - phys =memblock_phys_alloc_range (size,PAGE_SIZE ,0 , -PFN_PHYS (max_pfn_mapped)); -if (!phys) { + numa_distance =memblock_alloc (size,PAGE_SIZE ); +if (!numa_distance) {pr_warn ("Warning: can't allocate distance table!\n" );/* don't retry until explicitly reset */ numa_distance = (void *)1LU ;return -ENOMEM ; } - numa_distance =__va (phys); numa_distance_cnt = cnt;/* fill with the default distances */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 11/26 x86/numa: use get_pfn_range_for_nid to verify that node spans memory
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of looping over numa_meminfo array to detect node's start and end addresses use get_pfn_range_for_init().
This is shorter and make it easier to lift numa_memblks to generic code.
--- arch/x86/mm/numa.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
arch/x86/mm/numa.c +7 -10
@@ -521,17 +521,14 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start =PFN_PHYS (max_pfn); -u64 end =0 ; +unsigned long start_pfn, end_pfn; -for (i =0 ; i < mi->nr_blks; i++) { -if (nid != mi->blk[i].nid) -continue ; - start =min (mi->blk[i].start, start); - end =max (mi->blk[i].end, end); - } - -if (start >= end) +/* +* Note, get_pfn_range_for_nid() depends on +* memblock_set_node() having already happened +*/ +get_pfn_range_for_nid (nid, &start_pfn, &end_pfn); +if (start_pfn >= end_pfn)continue ;alloc_node_data (nid);
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 12/26 x86/numa: move FAKE_NODE_* defines to numa_emu
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
The definitions of FAKE_NODE_MIN_SIZE and FAKE_NODE_MIN_HASH_MASK are only used by numa emulation code, make them local to arch/x86/mm/numa_emulation.c
--- arch/x86/include/asm/numa.h | 2 -- arch/x86/mm/numa_emulation.c | 3 +++ 2 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/numa.h +0 -2
@@ -71,8 +71,6 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);#endif #ifdef CONFIG_NUMA_EMU -#define FAKE_NODE_MIN_SIZE ((u64)32 << 20) -#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))int numa_emu_cmdline (char *str);#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str)
arch/x86/mm/numa_emulation.c +3 -0
@@ -10,6 +10,9 @@#include "numa_internal.h" +#define FAKE_NODE_MIN_SIZE ((u64)32 << 20) +#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL)) +static int emu_nid_to_phys[MAX_NUMNODES ];static char *emu_cmdline__initdata ;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 13/26 x86/numa_emu: simplify allocation of phys_dist
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
By the time numa_emulation() is called, all physical memory is already mapped in the direct map and there is no need to define limits for memblock allocation.
Replace memblock_phys_alloc_range() with memblock_alloc().
--- arch/x86/mm/numa_emulation.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/mm/numa_emulation.c +2 -6
@@ -448,15 +448,11 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)/* copy the physical distance table */ if (numa_dist_cnt) { -u64 phys; - - phys =memblock_phys_alloc_range (phys_size,PAGE_SIZE ,0 , -PFN_PHYS (max_pfn_mapped)); -if (!phys) { + phys_dist =memblock_alloc (phys_size,PAGE_SIZE ); +if (!phys_dist) {pr_warn ("NUMA: Warning: can't allocate copy of distance table, disabling emulation\n" ); goto no_emu; } - phys_dist =__va (phys);for (i =0 ; i < numa_dist_cnt; i++)for (j =0 ; j < numa_dist_cnt; j++)
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 14/26 x86/numa_emu: split __apicid_to_node update to a helper function
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
This is required to make numa emulation code architecture independent so that it can be moved to generic code in following commits.
--- arch/x86/include/asm/numa.h | 2 ++ arch/x86/mm/numa.c | 22 ++++++++++++++++++++++ arch/x86/mm/numa_emulation.c | 14 +------------- 3 files changed, 25 insertions(+), 13 deletions(-)
arch/x86/include/asm/numa.h +2 -0
@@ -72,6 +72,8 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);#ifdef CONFIG_NUMA_EMU int numa_emu_cmdline (char *str); +void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, +unsigned int nr_emu_nids);#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str) {
arch/x86/mm/numa.c +22 -0
@@ -852,6 +852,28 @@ EXPORT_SYMBOL(cpumask_of_node);#endif /* !CONFIG_DEBUG_PER_CPU_MAPS */ +#ifdef CONFIG_NUMA_EMU +void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, +unsigned int nr_emu_nids) +{ +int i, j; + +/* +* Transform __apicid_to_node table to use emulated nids by +* reverse-mapping phys_nid. The maps should always exist but fall +* back to zero just in case. +*/ +for (i =0 ; i <ARRAY_SIZE (__apicid_to_node); i++) { +if (__apicid_to_node[i] ==NUMA_NO_NODE ) +continue ; +for (j =0 ; j < nr_emu_nids; j++) +if (__apicid_to_node[i] == emu_nid_to_phys[j]) +break ; + __apicid_to_node[i] = j < nr_emu_nids ? j :0 ; + } +} +#endif /* CONFIG_NUMA_EMU */ +#ifdef CONFIG_NUMA_KEEP_MEMINFO static int meminfo_to_nid (struct numa_meminfo *mi,u64 start) {
arch/x86/mm/numa_emulation.c +1 -13
@@ -476,19 +476,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) ei.blk[i].nid != NUMA_NO_NODE)node_set (ei.blk[i].nid, numa_nodes_parsed); -/* -* Transform __apicid_to_node table to use emulated nids by -* reverse-mapping phys_nid. The maps should always exist but fall -* back to zero just in case. -*/ -for (i =0 ; i <ARRAY_SIZE (__apicid_to_node); i++) { -if (__apicid_to_node[i] ==NUMA_NO_NODE ) -continue ; -for (j =0 ; j <ARRAY_SIZE (emu_nid_to_phys); j++) -if (__apicid_to_node[i] == emu_nid_to_phys[j]) -break ; - __apicid_to_node[i] = j <ARRAY_SIZE (emu_nid_to_phys) ? j :0 ; - } +numa_emu_update_cpu_to_node (emu_nid_to_phys,ARRAY_SIZE (emu_nid_to_phys));/* make sure all emulated nodes are mapped to a physical node */ for (i =0 ; i <ARRAY_SIZE (emu_nid_to_phys); i++)
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 15/26 x86/numa_emu: use a helper function to get MAX_DMA32_PFN
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
This is required to make numa emulation code architecture independent so that it can be moved to generic code in following commits.
--- arch/x86/include/asm/numa.h | 1 + arch/x86/mm/numa.c | 5 +++++ arch/x86/mm/numa_emulation.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)
arch/x86/include/asm/numa.h +1 -0
@@ -74,6 +74,7 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);int numa_emu_cmdline (char *str);void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys,unsigned int nr_emu_nids); +u64 __init numa_emu_dma_end (void );#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str) {
arch/x86/mm/numa.c +5 -0
@@ -872,6 +872,11 @@ void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys, __apicid_to_node[i] = j < nr_emu_nids ? j : 0; } } + +u64 __init numa_emu_dma_end (void ) +{ +return PFN_PHYS (MAX_DMA32_PFN ); +}#endif /* CONFIG_NUMA_EMU */ #ifdef CONFIG_NUMA_KEEP_MEMINFO
arch/x86/mm/numa_emulation.c +2 -2
@@ -128,7 +128,7 @@ static int __init split_nodes_interleave(struct numa_meminfo *ei,*/ while (!nodes_empty (physnode_mask)) {for_each_node_mask (i, physnode_mask) { -u64 dma32_end =PFN_PHYS (MAX_DMA32_PFN ); +u64 dma32_end =numa_emu_dma_end ();u64 start, limit, end;int phys_blk;
@@ -275,7 +275,7 @@ static int __init split_nodes_size_interleave_uniform(struct numa_meminfo *ei,*/ while (!nodes_empty (physnode_mask)) {for_each_node_mask (i, physnode_mask) { -u64 dma32_end =PFN_PHYS (MAX_DMA32_PFN ); +u64 dma32_end =numa_emu_dma_end ();u64 start, limit, end;int phys_blk;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 16/26 x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
CPU id cannot be negative.
Making it unsigned also aligns with declarations in include/asm-generic/numa.h used by arm64 and riscv and allows sharing numa emulation code with these architectures.
--- arch/x86/include/asm/numa.h | 10 +++++----- arch/x86/mm/numa.c | 10 +++++----- arch/x86/mm/numa_emulation.c | 10 +++++----- 3 files changed, 15 insertions(+), 15 deletions(-)
arch/x86/include/asm/numa.h +5 -5
@@ -54,20 +54,20 @@ static inline int numa_cpu_node(int cpu)extern void numa_set_node (int cpu,int node);extern void numa_clear_node (int cpu);extern void __init init_cpu_to_node (void ); -extern void numa_add_cpu (int cpu); -extern void numa_remove_cpu (int cpu); +extern void numa_add_cpu (unsigned int cpu); +extern void numa_remove_cpu (unsigned int cpu);extern void init_gi_nodes (void );#else /* CONFIG_NUMA */ static inline void numa_set_node (int cpu,int node) { }static inline void numa_clear_node (int cpu) { }static inline void init_cpu_to_node (void ) { } -static inline void numa_add_cpu (int cpu) { } -static inline void numa_remove_cpu (int cpu) { } +static inline void numa_add_cpu (unsigned int cpu) { } +static inline void numa_remove_cpu (unsigned int cpu) { }static inline void init_gi_nodes (void ) { }#endif /* CONFIG_NUMA */ #ifdef CONFIG_DEBUG_PER_CPU_MAPS -void debug_cpumask_set_cpu (int cpu,int node,bool enable); +void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable);#endif #ifdef CONFIG_NUMA_EMU
arch/x86/mm/numa.c +5 -5
@@ -741,12 +741,12 @@ void __init init_cpu_to_node(void)#ifndef CONFIG_DEBUG_PER_CPU_MAPS # ifndef CONFIG_NUMA_EMU -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {cpumask_set_cpu (cpu, node_to_cpumask_map[early_cpu_to_node (cpu)]); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {cpumask_clear_cpu (cpu, node_to_cpumask_map[early_cpu_to_node (cpu)]); }
@@ -784,7 +784,7 @@ int early_cpu_to_node(int cpu)return per_cpu (x86_cpu_to_node_map, cpu); } -void debug_cpumask_set_cpu (int cpu,int node,bool enable) +void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable) {struct cpumask *mask;
@@ -816,12 +816,12 @@ static void numa_set_cpumask(int cpu, bool enable)debug_cpumask_set_cpu (cpu,early_cpu_to_node (cpu), enable); } -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {numa_set_cpumask (cpu, true); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {numa_set_cpumask (cpu, false); }
arch/x86/mm/numa_emulation.c +5 -5
@@ -514,7 +514,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) }#ifndef CONFIG_DEBUG_PER_CPU_MAPS -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {int physnid, nid;
@@ -532,7 +532,7 @@ void numa_add_cpu(int cpu)cpumask_set_cpu (cpu, node_to_cpumask_map[nid]); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {int i;
@@ -540,7 +540,7 @@ void numa_remove_cpu(int cpu) cpumask_clear_cpu(cpu, node_to_cpumask_map[i]); }#else /* !CONFIG_DEBUG_PER_CPU_MAPS */ -static void numa_set_cpumask (int cpu,bool enable) +static void numa_set_cpumask (unsigned int cpu,bool enable) {int nid, physnid;
@@ -560,12 +560,12 @@ static void numa_set_cpumask(int cpu, bool enable) } } -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {numa_set_cpumask (cpu, true); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {numa_set_cpumask (cpu, false); }
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 17/26 mm: introduce numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with numa_memblks from arch/x86 to mm/ and add Kconfig options to let x86 select it in its Kconfig.
This code will be later reused by arch_numa.
No functional changes.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/numa.h | 3 - arch/x86/mm/amdtopology.c | 1 + arch/x86/mm/numa.c | 372 +-------------------------------- arch/x86/mm/numa_emulation.c | 1 + arch/x86/mm/numa_internal.h | 15 +- drivers/acpi/numa/srat.c | 1 + drivers/of/of_numa.c | 1 + include/linux/numa_memblks.h | 35 ++++ mm/Kconfig | 3 + mm/Makefile | 1 + mm/numa_memblks.c | 385 +++++++++++++++++++++++++++++++++++ 12 files changed, 436 insertions(+), 383 deletions(-) create mode 100644 include/linux/numa_memblks.h create mode 100644 mm/numa_memblks.c
arch/x86/Kconfig +1 -0
@@ -296,6 +296,7 @@ config X86 select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK select NEED_SG_DMA_LENGTH + select NUMA_MEMBLKS if NUMA select PCI_DOMAINS if PCI select PCI_LOCKLESS_CONFIG if PCI select PERF_EVENTS
arch/x86/include/asm/numa.h +0 -3
@@ -10,8 +10,6 @@#ifdef CONFIG_NUMA -#define NR_NODE_MEMBLKS (MAX_NUMNODES*2) -extern int numa_off;/*
@@ -25,7 +23,6 @@ extern int numa_off;extern s16 __apicid_to_node[MAX_LOCAL_APIC ];extern nodemask_t numa_nodes_parsed__initdata ; -extern int __init numa_add_memblk (int nodeid,u64 start,u64 end);extern void __init numa_set_distance (int from,int to,int distance);static inline void set_apicid_to_node (int apicid,s16 node)
arch/x86/mm/amdtopology.c +1 -0
@@ -12,6 +12,7 @@#include <linux/string.h> #include <linux/nodemask.h> #include <linux/memblock.h> +#include <linux/numa_memblks.h> #include <asm/io.h> #include <linux/pci_ids.h>
arch/x86/mm/numa.c +5 -367
402 lines not on this page · raw message
arch/x86/mm/numa_emulation.c +1 -0
@@ -6,6 +6,7 @@#include <linux/errno.h> #include <linux/topology.h> #include <linux/memblock.h> +#include <linux/numa_memblks.h> #include <asm/dma.h> #include "numa_internal.h"
arch/x86/mm/numa_internal.h +2 -13
@@ -5,23 +5,12 @@#include <linux/types.h> #include <asm/numa.h> -struct numa_memblk { -u64 start; -u64 end; -int nid; -}; - -struct numa_meminfo { -int nr_blks; -struct numa_memblk blk[NR_NODE_MEMBLKS ]; -}; - -void __init numa_remove_memblk_from (int idx,struct numa_meminfo *mi); -int __init numa_cleanup_meminfo (struct numa_meminfo *mi);void __init numa_reset_distance (void );void __init x86_numa_init (void ); +struct numa_meminfo ; +#ifdef CONFIG_NUMA_EMU void __init numa_emulation (struct numa_meminfo *numa_meminfo,int numa_dist_cnt);
drivers/acpi/numa/srat.c +1 -0
@@ -17,6 +17,7 @@#include <linux/numa.h> #include <linux/nodemask.h> #include <linux/topology.h> +#include <linux/numa_memblks.h> static nodemask_t nodes_found_map =NODE_MASK_NONE ;
drivers/of/of_numa.c +1 -0
@@ -10,6 +10,7 @@#include <linux/of.h> #include <linux/of_address.h> #include <linux/nodemask.h> +#include <linux/numa_memblks.h> #include <asm/numa.h>
include/linux/numa_memblks.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __NUMA_MEMBLKS_H +#define __NUMA_MEMBLKS_H + +#ifdef CONFIG_NUMA_MEMBLKS +#include <linux/types.h> + +#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) + +struct numa_memblk { +u64 start; +u64 end; +int nid; +}; + +struct numa_meminfo { +int nr_blks; +struct numa_memblk blk[NR_NODE_MEMBLKS ]; +}; + +extern struct numa_meminfo numa_meminfo __initdata_or_meminfo; +extern struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; + +int __init numa_add_memblk (int nodeid,u64 start,u64 end); +void __init numa_remove_memblk_from (int idx,struct numa_meminfo *mi); + +int __init numa_cleanup_meminfo (struct numa_meminfo *mi); +int __init numa_register_meminfo (struct numa_meminfo *mi); + +void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, +const struct numa_meminfo *mi); + +#endif /* CONFIG_NUMA_MEMBLKS */ + +#endif /* __NUMA_MEMBLKS_H */
mm/Kconfig +3 -0
@@ -1263,6 +1263,9 @@ config IOMMU_MM_DATA config EXECMEM bool +config NUMA_MEMBLKS + bool + source "mm/damon/Kconfig" endmenu
mm/Makefile +1 -0
@@ -142,3 +142,4 @@ obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_NUMA) += numa.o +obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o
mm/numa_memblks.c new file +385 -0
385 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 18/26 mm: move numa_distance and related code from x86 to numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with numa_distance array from arch/x86 to mm/numa_memblks.c
This code will be later reused by arch_numa.
No functional changes.
--- arch/x86/include/asm/numa.h | 2 - arch/x86/mm/numa.c | 104 ----------------------------------- arch/x86/mm/numa_internal.h | 2 - include/linux/numa_memblks.h | 4 ++ mm/numa_memblks.c | 104 +++++++++++++++++++++++++++++++++++ 5 files changed, 108 insertions(+), 108 deletions(-)
arch/x86/include/asm/numa.h +0 -2
@@ -23,8 +23,6 @@ extern int numa_off;extern s16 __apicid_to_node[MAX_LOCAL_APIC ];extern nodemask_t numa_nodes_parsed__initdata ; -extern void __init numa_set_distance (int from,int to,int distance); -static inline void set_apicid_to_node (int apicid,s16 node) { __apicid_to_node[apicid] = node;
arch/x86/mm/numa.c +0 -104
116 lines not on this page · raw message
arch/x86/mm/numa_internal.h +0 -2
@@ -5,8 +5,6 @@#include <linux/types.h> #include <asm/numa.h> -void __init numa_reset_distance (void ); -void __init x86_numa_init (void );struct numa_meminfo ;
include/linux/numa_memblks.h +4 -0
@@ -7,6 +7,10 @@#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) +extern int numa_distance_cnt; +void __init numa_set_distance (int from,int to,int distance); +void __init numa_reset_distance (void ); +struct numa_memblk {u64 start;u64 end;
mm/numa_memblks.c +104 -0
115 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 19/26 mm: introduce numa_emulation
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move numa_emulation code from arch/x86 to mm/numa_emulation.c
This code will be later reused by arch_numa.
No functional changes.
---
arch/x86/Kconfig | 8 --------
arch/x86/include/asm/numa.h | 12 ------------
arch/x86/mm/Makefile | 1 -
arch/x86/mm/numa_internal.h | 11 -----------
include/linux/numa_memblks.h | 17 +++++++++++++++++
mm/Kconfig | 8 ++++++++
mm/Makefile | 1 +
{arch/x86/mm => mm}/numa_emulation.c | 4 +---
8 files changed, 27 insertions(+), 35 deletions(-)
rename {arch/x86/mm => mm}/numa_emulation.c (99%)arch/x86/Kconfig +0 -8
@@ -1600,14 +1600,6 @@ config X86_64_ACPI_NUMA help Enable ACPI SRAT based node topology detection. -config NUMA_EMU - bool "NUMA emulation" - depends on NUMA - help - Enable NUMA emulation. A flat machine will be split - into virtual nodes when booted with "numa=fake=N", where N is the - number of nodes. This is only useful for debugging. - config NODES_SHIFT int "Maximum NUMA Nodes (as a power of 2)" if !MAXSMP range 1 10
arch/x86/include/asm/numa.h +0 -12
@@ -65,16 +65,4 @@ static inline void init_gi_nodes(void) { }void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable);#endif -#ifdef CONFIG_NUMA_EMU -int numa_emu_cmdline (char *str); -void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, -unsigned int nr_emu_nids); -u64 __init numa_emu_dma_end (void ); -#else /* CONFIG_NUMA_EMU */ -static inline int numa_emu_cmdline (char *str) -{ -return -EINVAL ; -} -#endif /* CONFIG_NUMA_EMU */ -#endif /* _ASM_X86_NUMA_H */
arch/x86/mm/Makefile +0 -1
@@ -57,7 +57,6 @@ obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o obj-$(CONFIG_NUMA) += numa.o numa_$(BITS).o obj-$(CONFIG_AMD_NUMA) += amdtopology.o obj-$(CONFIG_ACPI_NUMA) += srat.o -obj-$(CONFIG_NUMA_EMU) += numa_emulation.o obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
arch/x86/mm/numa_internal.h +0 -11
@@ -7,15 +7,4 @@void __init x86_numa_init (void ); -struct numa_meminfo ; - -#ifdef CONFIG_NUMA_EMU -void __init numa_emulation (struct numa_meminfo *numa_meminfo, -int numa_dist_cnt); -#else -static inline void numa_emulation (struct numa_meminfo *numa_meminfo, -int numa_dist_cnt) -{ } -#endif -#endif /* __X86_MM_NUMA_INTERNAL_H */
include/linux/numa_memblks.h +17 -0
@@ -34,6 +34,23 @@ int __init numa_register_meminfo(struct numa_meminfo *mi);void __init numa_nodemask_from_meminfo (nodemask_t *nodemask,const struct numa_meminfo *mi); +#ifdef CONFIG_NUMA_EMU +int numa_emu_cmdline (char *str); +void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, +unsigned int nr_emu_nids); +u64 __init numa_emu_dma_end (void ); +void __init numa_emulation (struct numa_meminfo *numa_meminfo, +int numa_dist_cnt); +#else +static inline void numa_emulation (struct numa_meminfo *numa_meminfo, +int numa_dist_cnt) +{ } +static inline int numa_emu_cmdline (char *str) +{ +return -EINVAL ; +} +#endif /* CONFIG_NUMA_EMU */ +#endif /* CONFIG_NUMA_MEMBLKS */ #endif /* __NUMA_MEMBLKS_H */
mm/Kconfig +8 -0
@@ -1266,6 +1266,14 @@ config EXECMEM config NUMA_MEMBLKS bool +config NUMA_EMU + bool "NUMA emulation" + depends on NUMA_MEMBLKS + help + Enable NUMA emulation. A flat machine will be split + into virtual nodes when booted with "numa=fake=N", where N is the + number of nodes. This is only useful for debugging. + source "mm/damon/Kconfig" endmenu
mm/Makefile +1 -0
@@ -143,3 +143,4 @@ obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o +obj-$(CONFIG_NUMA_EMU) += numa_emulation.o
mm/numa_emulation.c renamed from arch/x86/mm/numa_emulation.c +1 -3
@@ -7,9 +7,7 @@#include <linux/topology.h> #include <linux/memblock.h> #include <linux/numa_memblks.h> -#include <asm/dma.h> - -#include "numa_internal.h" +#include <asm/numa.h> #define FAKE_NODE_MIN_SIZE ((u64)32 << 20)#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 20/26 mm: numa_memblks: introduce numa_memblks_init
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move most of x86::numa_init() to numa_memblks so that the latter will be more self-contained.
With this numa_memblk data structures should not be exposed to the architecture specific code.
--- arch/x86/mm/numa.c | 40 ++++------------------------------- include/linux/numa_memblks.h | 3 +++ mm/numa_memblks.c | 41 ++++++++++++++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 36 deletions(-)
arch/x86/mm/numa.c +4 -36
@@ -115,13 +115,9 @@ void __init setup_node_to_cpumask_map(void)pr_debug ("Node to cpumask map for %u nodes\n" , nr_node_ids); } -static int __init numa_register_memblks (struct numa_meminfo *mi) +static int __init numa_register_nodes (void ) { -int nid, err; - - err =numa_register_meminfo (mi); -if (err) -return err; +int nid;if (!memblock_validate_numa_coverage (SZ_1M ))return -EINVAL ;
@@ -175,39 +171,11 @@ static int __init numa_init(int (*init_func)(void))for (i =0 ; i <MAX_LOCAL_APIC ; i++)set_apicid_to_node (i,NUMA_NO_NODE ); -nodes_clear (numa_nodes_parsed); -nodes_clear (node_possible_map); -nodes_clear (node_online_map); -memset (&numa_meminfo,0 ,sizeof (numa_meminfo)); -WARN_ON (memblock_set_node (0 ,ULLONG_MAX , &memblock.memory, -NUMA_NO_NODE )); -WARN_ON (memblock_set_node (0 ,ULLONG_MAX , &memblock.reserved, -NUMA_NO_NODE )); -/* In case that parsing SRAT failed. */ -WARN_ON (memblock_clear_hotplug (0 ,ULLONG_MAX )); -numa_reset_distance (); - - ret =init_func (); + ret =numa_memblks_init (init_func,/* memblock_force_top_down */ true);if (ret <0 )return ret; -/* -* We reset memblock back to the top-down direction -* here because if we configured ACPI_NUMA, we have -* parsed SRAT in init_func(). It is ok to have the -* reset here even if we did't configure ACPI_NUMA -* or acpi numa init fails and fallbacks to dummy -* numa init. -*/ -memblock_set_bottom_up (false); - - ret =numa_cleanup_meminfo (&numa_meminfo); -if (ret <0 ) -return ret; - -numa_emulation (&numa_meminfo, numa_distance_cnt); - - ret =numa_register_memblks (&numa_meminfo); + ret =numa_register_nodes ();if (ret <0 )return ret;
include/linux/numa_memblks.h +3 -0
@@ -34,6 +34,9 @@ int __init numa_register_meminfo(struct numa_meminfo *mi);void __init numa_nodemask_from_meminfo (nodemask_t *nodemask,const struct numa_meminfo *mi); +int __init numa_memblks_init (int (*init_func)(void ), +bool memblock_force_top_down); +#ifdef CONFIG_NUMA_EMU int numa_emu_cmdline (char *str); void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys,
mm/numa_memblks.c +41 -0
47 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 21/26 mm: numa_memblks: make several functions and variables static
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Make functions and variables that are exclusively used by numa_memblks static.
Move numa_nodemask_from_meminfo() before its callers to avoid forward declaration.
--- include/linux/numa_memblks.h | 8 -------- mm/numa_memblks.c | 36 ++++++++++++++++++------------------ 2 files changed, 18 insertions(+), 26 deletions(-)
include/linux/numa_memblks.h +0 -8
@@ -7,7 +7,6 @@#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) -extern int numa_distance_cnt;void __init numa_set_distance (int from,int to,int distance);void __init numa_reset_distance (void );
@@ -22,17 +21,10 @@ struct numa_meminfo {struct numa_memblk blk[NR_NODE_MEMBLKS ]; }; -extern struct numa_meminfo numa_meminfo __initdata_or_meminfo; -extern struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; -int __init numa_add_memblk (int nodeid,u64 start,u64 end);void __init numa_remove_memblk_from (int idx,struct numa_meminfo *mi);int __init numa_cleanup_meminfo (struct numa_meminfo *mi); -int __init numa_register_meminfo (struct numa_meminfo *mi); - -void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, -const struct numa_meminfo *mi);int __init numa_memblks_init (int (*init_func)(void ),bool memblock_force_top_down);
mm/numa_memblks.c +18 -18
@@ -7,13 +7,27 @@#include <linux/numa.h> #include <linux/numa_memblks.h> -int numa_distance_cnt; +static int numa_distance_cnt;static u8 *numa_distance;nodemask_t numa_nodes_parsed__initdata ; -struct numa_meminfo numa_meminfo __initdata_or_meminfo; -struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; +static struct numa_meminfo numa_meminfo __initdata_or_meminfo; +static struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; + +/* +* Set nodes, which have memory in @mi, in *@nodemask. +*/ +static void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, +const struct numa_meminfo *mi) +{ +int i; + +for (i =0 ; i <ARRAY_SIZE (mi->blk); i++) +if (mi->blk[i].start != mi->blk[i].end && + mi->blk[i].nid !=NUMA_NO_NODE ) +node_set (mi->blk[i].nid, *nodemask); +}/** * numa_reset_distance - Reset NUMA distance table
@@ -290,20 +304,6 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi)return 0 ; } -/* -* Set nodes, which have memory in @mi, in *@nodemask. -*/ -void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, -const struct numa_meminfo *mi) -{ -int i; - -for (i =0 ; i <ARRAY_SIZE (mi->blk); i++) -if (mi->blk[i].start != mi->blk[i].end && - mi->blk[i].nid !=NUMA_NO_NODE ) -node_set (mi->blk[i].nid, *nodemask); -} -/* * Mark all currently memblock-reserved physical memory (which covers the * kernel's own memory ranges) as hot-unswappable.
@@ -371,7 +371,7 @@ static void __init numa_clear_kernel_node_hotplug(void) } } -int __init numa_register_meminfo (struct numa_meminfo *mi) +static int __init numa_register_meminfo (struct numa_meminfo *mi) {int i;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 22/26 mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
numa_cleanup_meminfo() moves blocks outside system RAM to numa_reserved_meminfo and it uses 0 and PFN_PHYS(max_pfn) to determine the memory boundaries.
Replace the memory range boundaries with more portable memblock_start_of_DRAM() and memblock_end_of_DRAM().
--- mm/numa_memblks.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/numa_memblks.c +2 -2
@@ -212,8 +212,8 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)*/ int __init numa_cleanup_meminfo (struct numa_meminfo *mi) { -const u64 low =0 ; -const u64 high =PFN_PHYS (max_pfn); +const u64 low =memblock_start_of_DRAM (); +const u64 high =memblock_end_of_DRAM ();int i, j, k;/* first, trim all entries */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 23/26 of, numa: return -EINVAL when no numa-node-id is found
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Currently of_numa_parse_memory_nodes() returns 0 if no "memory" node in device tree contains "numa-node-id" property. This makes of_numa_init() to return "success" despite no NUMA nodes were actually parsed and set up.
arch_numa workarounds this by returning an error if numa_nodes_parsed is empty.
numa_memblks however would WARN() in such case and since it will be used by arch_numa shortly, such warning is not desirable.
Make sure of_numa_init() returns -EINVAL when no NUMA node information was found in the device tree.
--- drivers/of/of_numa.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/of/of_numa.c +2 -2
@@ -45,7 +45,7 @@ static int __init of_numa_parse_memory_nodes(void)struct device_node *np =NULL ;struct resource rsrc;u32 nid; -int i, r; +int i, r = -EINVAL ;for_each_node_by_type (np,"memory" ) { r =of_property_read_u32 (np,"numa-node-id" , &nid);
@@ -72,7 +72,7 @@ static int __init of_numa_parse_memory_nodes(void) } } -return 0 ; +return r; }static int __init of_numa_parse_distance_map_v1 (struct device_node *map)
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 24/26 arch_numa: switch over to numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Until now arch_numa was directly translating firmware NUMA information to memblock.
Using numa_memblks as an intermediate step has a few advantages:
* alignment with more battle tested x86 implementation
* availability of NUMA emulation
* maintaining node information for not yet populated memory
Adjust a few places in numa_memblks to compile with 32-bit phys_addr_t and replace current functionality related to numa_add_memblk() and __node_distance() in arch_numa with the implementation based on numa_memblks and add functions required by numa_emulation.
--- drivers/base/Kconfig | 1 + drivers/base/arch_numa.c | 201 +++++++++++-------------------------- include/asm-generic/numa.h | 6 +- mm/numa_memblks.c | 17 ++-- 4 files changed, 75 insertions(+), 150 deletions(-)
drivers/base/Kconfig +1 -0
@@ -226,6 +226,7 @@ config GENERIC_ARCH_TOPOLOGY config GENERIC_ARCH_NUMA bool + select NUMA_MEMBLKS help Enable support for generic NUMA implementation. Currently, RISC-V and ARM64 use it.
drivers/base/arch_numa.c +60 -141
272 lines not on this page · raw message
include/asm-generic/numa.h +4 -2
@@ -32,8 +32,6 @@ static inline const struct cpumask *cpumask_of_node(int node)void __init arch_numa_init (void );int __init numa_add_memblk (int nodeid,u64 start,u64 end); -void __init numa_set_distance (int from,int to,int distance); -void __init numa_free_distance (void );void __init early_map_cpu_to_node (unsigned int cpu,int nid);int __init early_cpu_to_node (int cpu);void numa_store_cpu_info (unsigned int cpu);
@@ -51,4 +49,8 @@ static inline int early_cpu_to_node(int cpu) { return 0; }#endif /* CONFIG_NUMA */ +#ifdef CONFIG_NUMA_EMU +void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable); +#endif +#endif /* __ASM_GENERIC_NUMA_H */
mm/numa_memblks.c +10 -7
@@ -405,9 +405,12 @@ static int __init numa_register_meminfo(struct numa_meminfo *mi)unsigned long pfn_align =node_map_pfn_alignment ();if (pfn_align && pfn_align <PAGES_PER_SECTION ) { -pr_warn ("Node alignment %LuMB < min %LuMB, rejecting NUMA config\n" , -PFN_PHYS (pfn_align) >>20 , -PFN_PHYS (PAGES_PER_SECTION ) >>20 ); +unsigned long node_align_mb =PFN_PHYS (pfn_align) >>20 ; + +unsigned long sect_align_mb =PFN_PHYS (PAGES_PER_SECTION ) >>20 ; + +pr_warn ("Node alignment %luMB < min %luMB, rejecting NUMA config\n" , + node_align_mb, sect_align_mb);return -EINVAL ; } }
@@ -418,18 +421,18 @@ static int __init numa_register_meminfo(struct numa_meminfo *mi)int __init numa_memblks_init (int (*init_func)(void ),bool memblock_force_top_down) { +phys_addr_t max_addr = (phys_addr_t )ULLONG_MAX ;int ret;nodes_clear (numa_nodes_parsed);nodes_clear (node_possible_map);nodes_clear (node_online_map);memset (&numa_meminfo,0 ,sizeof (numa_meminfo)); -WARN_ON (memblock_set_node (0 ,ULLONG_MAX , &memblock.memory, -NUMA_NO_NODE )); -WARN_ON (memblock_set_node (0 ,ULLONG_MAX , &memblock.reserved, +WARN_ON (memblock_set_node (0 , max_addr, &memblock.memory,NUMA_NO_NODE )); +WARN_ON (memblock_set_node (0 , max_addr, &memblock.reserved,NUMA_NO_NODE ));/* In case that parsing SRAT failed. */ -WARN_ON (memblock_clear_hotplug (0 ,ULLONG_MAX )); +WARN_ON (memblock_clear_hotplug (0 , max_addr));numa_reset_distance (); ret =init_func ();
Signature · 1 line
2.43.0
Arnd Bergmann
maintainerRe: [PATCH v4 24/26] arch_numa: switch over to numa_memblks
Quoted 4 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Until now
arch_numawas directly translating firmware NUMA information to memblock.
I get a link time warning from this:
WARNING: modpost: vmlinux: section mismatch in reference: numa_set_cpumask+0x24 (section: .text.unlikely) -> early_cpu_to_node (section: .init.text)
drivers/base/arch_numa.c void __init early_map_cpu_to_node(unsigned int cpu, int nid) · line 150
Quoted 9 lines
drivers/base/arch_numa.c +1
-1@@ -142,7 +144,7 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid)unsigned long __per_cpu_offset[NR_CPUS ]__read_mostly ;EXPORT_SYMBOL (__per_cpu_offset); -int __init early_cpu_to_node (int cpu) +int early_cpu_to_node (int cpu) {return cpu_to_node_map[cpu]; }
early_cpu_to_node() can no longer be __init here
Quoted 6 lines
drivers/base/arch_numa.c +1
-0+#endif /* CONFIG_NUMA_EMU */ include/asm-generic/numa.h +0
-0
include/asm-generic/numa.h static inline const struct cpumask *cpumask_of_node(int node) · line 37
Quoted 8 lines
include/asm-generic/numa.h +0
-2@@ -32,8 +32,6 @@ static inline const struct cpumask *cpumask_of_node(int node)void __init arch_numa_init (void );int __init numa_add_memblk (int nodeid,u64 start,u64 end); -void __init numa_set_distance (int from,int to,int distance); -void __init numa_free_distance (void );void __init early_map_cpu_to_node (unsigned int cpu,int nid);int __init early_cpu_to_node (int cpu);void numa_store_cpu_info (unsigned int cpu);
but is still declared as __init in the header, so it is still put in that section and discarded after boot.
I was confused by this at first, since the 'early' name seems to imply that you shouldn't call it once the system is up, but now you do.
Arnd
Mike Rapoport
maintainerRe: [PATCH v4 24/26] arch_numa: switch over to numa_memblks
Quoted 9 lines · Arnd Bergmann
Quoted 4 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Until now
arch_numawas directly translating firmware NUMA information to memblock.I get a link time warning from this:
WARNING: modpost: vmlinux: section mismatch in reference: numa_set_cpumask+0x24 (section: .text.unlikely) -> early_cpu_to_node (section: .init.text)
I didn't see this neither in my build tests nor in kbuild reports :/
Quoted 19 lines
Review void __init early_map_cpu_to_node(unsigned int cpu, int nid) · line 150
Quoted 9 lines
(quoted diff) +1
-1@@ -142,7 +144,7 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid) unsigned long __per_cpu_offset[NR_CPUS] __read_mostly; EXPORT_SYMBOL(__per_cpu_offset); -int __init early_cpu_to_node(int cpu) +int early_cpu_to_node(int cpu) { return cpu_to_node_map[cpu]; }
early_cpu_to_node()can no longer be__inithereQuoted 7 lines
(quoted diff) +1
-0+#endif /* CONFIG_NUMA_EMU */include/asm-generic/numa.h +0
-0@@ -32,8 +32,6 @@ static inline const struct cpumask *cpumask_of_node(int node)
Quoted 10 lines
include/asm-generic/numa.h static inline const struct cpumask *cpumask_of_node(int node) · line 37
Quoted 7 lines
include/asm-generic/numa.h +0
-2void __init arch_numa_init (void );int __init numa_add_memblk (int nodeid,u64 start,u64 end); -void __init numa_set_distance (int from,int to,int distance); -void __init numa_free_distance (void );void __init early_map_cpu_to_node (unsigned int cpu,int nid);int __init early_cpu_to_node (int cpu);void numa_store_cpu_info (unsigned int cpu);but is still declared as
__initin the header, so it is still put in that section and discarded after boot.
I believe this should fix it
include/asm-generic/numa.h +1 -1
@@ -33,7 +33,7 @@ static inline const struct cpumask *cpumask_of_node(int node)void __init arch_numa_init (void );int __init numa_add_memblk (int nodeid,u64 start,u64 end);void __init early_map_cpu_to_node (unsigned int cpu,int nid); -int __init early_cpu_to_node (int cpu); +int early_cpu_to_node (int cpu);void numa_store_cpu_info (unsigned int cpu);void numa_add_cpu (unsigned int cpu);void numa_remove_cpu (unsigned int cpu);
Quoted 3 lines
I was confused by this at first, since the 'early' name seems to imply that you shouldn't call it once the system is up, but now you do.
I agree that this is confusing, but that's what x86 does and numa_emulation uses.
Quoted 2 lines
Arnd
Signature · 2 lines
Sincerely yours, Mike.
Arnd Bergmann
maintainerRe: [PATCH v4 24/26] arch_numa: switch over to numa_memblks
Quoted 14 lines · Mike Rapoport
Quoted 12 lines · Arnd Bergmann
Review
Quoted 8 lines · Mike Rapoport
void
__initarch_numa_init(void);
int__initnuma_add_memblk(int nodeid, u64 start, u64 end);(quoted diff) +0
-2-void __init numa_set_distance(int from, int to, int distance); -void __init numa_free_distance(void); void __init early_map_cpu_to_node(unsigned int cpu, int nid); int __init early_cpu_to_node(int cpu); void numa_store_cpu_info(unsigned int cpu);but is still declared as
__initin the header, so it is still put in that section and discarded after boot.
Quoted 1 line
I believe this should fix it
Yes, sorry I should have posted the patch as well, this is what I tested with locally.
Arnd
Re: [PATCH v4 24/26] arch_numa: switch over to numa_memblks
In reply to Mike RapoportHi Mike,
Sorry for reviving a rather old thread.
Quoted 29 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Until now
arch_numawas directly translating firmware NUMA information to memblock.Using
numa_memblksas an intermediate step has a few advantages:
* alignment with more battle tested x86 implementation
* availability of NUMA emulation
* maintaining node information for not yet populated memoryAdjust a few places in
numa_memblksto compile with 32-bitphys_addr_tand replace current functionality related tonuma_add_memblk()and__node_distance()inarch_numawith the implementation based onnuma_memblksand add functions required bynuma_emulation.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Acked-by Dan Williams <[email protected]>--- drivers/base/Kconfig | 1 + drivers/base/arch_numa.c | 201 +++++++++++-------------------------- include/asm-generic/numa.h | 6 +- mm/numa_memblks.c | 17 ++-- 4 files changed, 75 insertions(+), 150 deletions(-)
[...]
Quoted 8 lines
static int
__initnuma_register_nodes(void)
{drivers/base/arch_numa.c +0
-4int nid; -struct memblock_region *mblk; - -/* Check that valid nid is set to memblks */ -for_each_mem_region (mblk) {
drivers/base/arch_numa.c static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) · line 338
Quoted 10 lines
drivers/base/arch_numa.c +0
-10-int mblk_nid =memblock_get_region_node (mblk); -phys_addr_t start = mblk->base; -phys_addr_t end = mblk->base + mblk->size -1 ; - -if (mblk_nid ==NUMA_NO_NODE || mblk_nid >=MAX_NUMNODES ) { -pr_warn ("Warning: invalid memblk node %d [mem %pap-%pap]\n" , - mblk_nid, &start, &end); -return -EINVAL ; - } - }
This hunk has the unfortunate side effect of killing my ThunderX extremely early at boot time, as this sorry excuse for a machine really relies on the kernel recognising that whatever NUMA information the FW offers is BS.
Reverting this hunk restores happiness (sort of).
FWIW, I've posted a patch with such revert at [1].
Thanks,
M.
[1] https://lore.kernel.org/r/[email protected]
Signature · 1 line
Without deviation from the norm, progress is not possible.
Mike Rapoport
maintainerPATCH v4 25/26 mm: make range-to-target_node lookup facility a part of numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
The x86 implementation of range-to-target_node lookup (i.e. phys_to_target_node() and memory_add_physaddr_to_nid()) relies on numa_memblks.
Since numa_memblks are now part of the generic code, move these functions from x86 to mm/numa_memblks.c and select CONFIG_NUMA_KEEP_MEMINFO when CONFIG_NUMA_MEMBLKS=y for dax and cxl.
--- arch/x86/include/asm/sparsemem.h | 9 -------- arch/x86/mm/numa.c | 38 -------------------------------- drivers/cxl/Kconfig | 2 +- drivers/dax/Kconfig | 2 +- include/linux/numa_memblks.h | 7 ++++++ mm/numa.c | 1 + mm/numa_memblks.c | 38 ++++++++++++++++++++++++++++++++ 7 files changed, 48 insertions(+), 49 deletions(-)
arch/x86/include/asm/sparsemem.h +0 -9
@@ -31,13 +31,4 @@#endif /* CONFIG_SPARSEMEM */ -#ifndef __ASSEMBLY__ -#ifdef CONFIG_NUMA_KEEP_MEMINFO -extern int phys_to_target_node (phys_addr_t start); -#define phys_to_target_node phys_to_target_node -extern int memory_add_physaddr_to_nid (u64 start); -#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid -#endif -#endif /* __ASSEMBLY__ */ -#endif /* _ASM_X86_SPARSEMEM_H */
arch/x86/mm/numa.c +0 -38
@@ -453,41 +453,3 @@ u64 __init numa_emu_dma_end(void) return PFN_PHYS(MAX_DMA32_PFN); }#endif /* CONFIG_NUMA_EMU */ - -#ifdef CONFIG_NUMA_KEEP_MEMINFO -static int meminfo_to_nid (struct numa_meminfo *mi,u64 start) -{ -int i; - -for (i =0 ; i < mi->nr_blks; i++) -if (mi->blk[i].start <= start && mi->blk[i].end > start) -return mi->blk[i].nid; -return NUMA_NO_NODE ; -} - -int phys_to_target_node (phys_addr_t start) -{ -int nid =meminfo_to_nid (&numa_meminfo, start); - -/* -* Prefer online nodes, but if reserved memory might be -* hot-added continue the search with reserved ranges. -*/ -if (nid !=NUMA_NO_NODE ) -return nid; - -return meminfo_to_nid (&numa_reserved_meminfo, start); -} -EXPORT_SYMBOL_GPL (phys_to_target_node); - -int memory_add_physaddr_to_nid (u64 start) -{ -int nid =meminfo_to_nid (&numa_meminfo, start); - -if (nid ==NUMA_NO_NODE ) - nid = numa_meminfo.blk[0 ].nid; -return nid; -} -EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid); - -#endif
drivers/cxl/Kconfig +1 -1
@@ -6,7 +6,7 @@ menuconfig CXL_BUS select FW_UPLOAD select PCI_DOE select FIRMWARE_TABLE - select NUMA_KEEP_MEMINFO if (NUMA && X86) + select NUMA_KEEP_MEMINFO if NUMA_MEMBLKS help CXL is a bus that is electrically compatible with PCI Express, but layers three protocols on that signalling (CXL.io, CXL.cache, and
drivers/dax/Kconfig +1 -1
@@ -30,7 +30,7 @@ config DEV_DAX_PMEM config DEV_DAX_HMEM tristate "HMEM DAX: direct access to 'specific purpose' memory" depends on EFI_SOFT_RESERVE - select NUMA_KEEP_MEMINFO if (NUMA && X86) + select NUMA_KEEP_MEMINFO if NUMA_MEMBLKS default DEV_DAX help EFI 2.8 platforms, and others, may advertise 'specific purpose'
include/linux/numa_memblks.h +7 -0
@@ -46,6 +46,13 @@ static inline int numa_emu_cmdline(char *str) }#endif /* CONFIG_NUMA_EMU */ +#ifdef CONFIG_NUMA_KEEP_MEMINFO +extern int phys_to_target_node (phys_addr_t start); +#define phys_to_target_node phys_to_target_node +extern int memory_add_physaddr_to_nid (u64 start); +#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid +#endif /* CONFIG_NUMA_KEEP_MEMINFO */ +#endif /* CONFIG_NUMA_MEMBLKS */ #endif /* __NUMA_MEMBLKS_H */
mm/numa.c +1 -0
@@ -3,6 +3,7 @@#include <linux/memblock.h> #include <linux/printk.h> #include <linux/numa.h> +#include <linux/numa_memblks.h> struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (node_data);
mm/numa_memblks.c +38 -0
@@ -531,3 +531,41 @@ int __init numa_fill_memblks(u64 start, u64 end) }return 0 ; } + +#ifdef CONFIG_NUMA_KEEP_MEMINFO +static int meminfo_to_nid (struct numa_meminfo *mi,u64 start) +{ +int i; + +for (i =0 ; i < mi->nr_blks; i++) +if (mi->blk[i].start <= start && mi->blk[i].end > start) +return mi->blk[i].nid; +return NUMA_NO_NODE ; +} + +int phys_to_target_node (phys_addr_t start) +{ +int nid =meminfo_to_nid (&numa_meminfo, start); + +/* +* Prefer online nodes, but if reserved memory might be +* hot-added continue the search with reserved ranges. +*/ +if (nid !=NUMA_NO_NODE ) +return nid; + +return meminfo_to_nid (&numa_reserved_meminfo, start); +} +EXPORT_SYMBOL_GPL (phys_to_target_node); + +int memory_add_physaddr_to_nid (u64 start) +{ +int nid =meminfo_to_nid (&numa_meminfo, start); + +if (nid ==NUMA_NO_NODE ) + nid = numa_meminfo.blk[0 ].nid; +return nid; +} +EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid); + +#endif /* CONFIG_NUMA_KEEP_MEMINFO */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v4 26/26 docs: move numa=fake description to kernel-parameters.txt
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
NUMA emulation can be now enabled on arm64 and riscv in addition to x86.
Move description of numa=fake parameters from x86 documentation of admin-guide/kernel-parameters.txt
--- Documentation/admin-guide/kernel-parameters.txt | 15 +++++++++++++++ Documentation/arch/x86/x86_64/boot-options.rst | 12 ------------ 2 files changed, 15 insertions(+), 12 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +15 -0
@@ -4123,6 +4123,21 @@ Disable NUMA, Only set up a single NUMA node spanning all memory. + numa=fake=<size>[MG] + [KNL, ARM64, RISCV, X86, EARLY] + If given as a memory unit, fills all system RAM with + nodes of size interleaved over physical nodes. + + numa=fake=<N> + [KNL, ARM64, RISCV, X86, EARLY] + If given as an integer, fills all system RAM with N + fake nodes interleaved over physical nodes. + + numa=fake=<N>U + [KNL, ARM64, RISCV, X86, EARLY] + If given as an integer followed by 'U', it will + divide each physical node into N emulated nodes. + numa_balancing= [KNL,ARM64,PPC,RISCV,S390,X86] Enable or disable automatic NUMA balancing. Allowed values are enable and disable
Documentation/arch/x86/x86_64/boot-options.rst +0 -12
@@ -170,18 +170,6 @@ NUMA Don't parse the HMAT table for NUMA setup, or soft-reserved memory partitioning. - numa=fake=<size>[MG] - If given as a memory unit, fills all system RAM with nodes of - size interleaved over physical nodes. - - numa=fake=<N> - If given as an integer, fills all system RAM with N fake nodes - interleaved over physical nodes. - - numa=fake=<N>U - If given as an integer followed by 'U', it will divide each - physical node into N emulated nodes. - ACPI ====
Signature · 1 line
2.43.0