mm: introduce numa_memblks
Subsystems (24)
Mike Rapoport
maintainerv3 cover mm: introduce numa_memblks
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following the discussion about handling of CXL fixed memory windows on arm64 [1] I decided to bite the bullet and move numa_memblks from x86 to the generic code so they will be available on arm64/riscv and maybe on loongarch sometime later.
While it could be possible to use memblock to describe CXL memory windows, it currently lacks notion of unpopulated memory ranges and numa_memblks does implement this.
Another reason to make numa_memblks generic is that both arch_numa (arm64 and riscv) and loongarch use trimmed copy of x86 code although there is no fundamental reason why the same code cannot be used on all these platforms. Having numa_memblks in mm/ will make it's interaction with ACPI and FDT more consistent and I believe will reduce maintenance burden.
And with generic numa_memblks it is (almost) straightforward to enable NUMA emulation on arm64 and riscv.
The first 9 commits in this series are cleanups that are not strictly related to numa_memblks.
Commits 10-16 slightly reorder code in x86 to allow extracting numa_memblks and NUMA emulation to the generic code.
Commits 17-19 actually move the code from arch/x86/ to mm/ and commits 20-22 does some aftermath cleanups.
Commit 23 updates of_numa_init() to return error of no NUMA nodes were found in the device tree.
Commit 24 switches arch_numa to numa_memblks.
Commit 25 enables usage of phys_to_target_node() and memory_add_physaddr_to_nid() with numa_memblks.
Commit 26 moves the description for numa=fake from x86 to admin-guide.
[1] https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]
* rebase on v6.11-rc1
* fix dummy_numa_init() in arch_numa, thanks Zi Yan
* update of_numa_init() to return error of no NUMA nodes were
* add Tested-by, thanks Zi Yan
v1: https://lore.kernel.org/all/[email protected]
* add cleanup for arch_alloc_nodedata and HAVE_ARCH_NODEDATA_EXTENSION
* add patch that moves description of numa=fake kernel parameter from x86 to admin-guide
* reduce rounding up of node_data allocations from PAGE_SIZE to SMP_CACHE_BYTES
* restore single allocation attempt of numa_distance
* fix several comments
* added review tags
Mike Rapoport (Microsoft) (26):
mm: move kernel/numa.c to mm/
MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
MIPS: loongson64: rename __node_data to node_data
MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
arch, mm: move definition of node_data to generic code arch, mm: pull out allocation of NODE_DATA to generic code x86/numa: simplify numa_distance allocation
x86/numa: use get_pfn_range_for_nid to verify that node spans memory x86/numa: move FAKE_NODE_* defines to numa_emu x86/numa_emu: simplify allocation of phys_dist x86/numa_emu: split __apicid_to_node update to a helper function x86/numa_emu: use a helper function to get MAX_DMA32_PFN x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
mm: introduce numa_memblks
mm: move numa_distance and related code from x86 to numa_memblks
mm: introduce numa_emulation
mm: numa_memblks: introduce numa_memblks_init
mm: numa_memblks: make several functions and variables static
mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing
meminfo
of, numa: return -EINVAL when no numa-node-id is found
arch_numa: switch over to numa_memblks
mm: make range-to-target_node lookup facility a part of numa_memblks
docs: move numa=fake description to kernel-parameters.txt
.../admin-guide/kernel-parameters.txt | 15 +
.../arch/x86/x86_64/boot-options.rst | 12 -
arch/arm64/include/asm/Kbuild | 1 +
arch/arm64/include/asm/mmzone.h | 13 -
arch/arm64/include/asm/topology.h | 1 +
arch/loongarch/include/asm/Kbuild | 1 +
arch/loongarch/include/asm/mmzone.h | 16 -
arch/loongarch/include/asm/topology.h | 1 +
arch/loongarch/kernel/numa.c | 21 -
arch/mips/Kconfig | 5 -
arch/mips/include/asm/mach-ip27/mmzone.h | 1 -
.../mips/include/asm/mach-loongson64/mmzone.h | 4 -
arch/mips/loongson64/numa.c | 28 +-
arch/mips/sgi-ip27/ip27-memory.c | 12 +-
arch/mips/sgi-ip27/ip27-smp.c | 2 +
arch/powerpc/include/asm/mmzone.h | 6 -
arch/powerpc/mm/numa.c | 26 +-
arch/riscv/include/asm/Kbuild | 1 +
arch/riscv/include/asm/mmzone.h | 13 -
arch/riscv/include/asm/topology.h | 4 +
arch/s390/include/asm/Kbuild | 1 +
arch/s390/include/asm/mmzone.h | 17 -
arch/s390/kernel/numa.c | 3 -
arch/sh/include/asm/mmzone.h | 3 -
arch/sh/mm/init.c | 7 +-
arch/sh/mm/numa.c | 3 -
arch/sparc/include/asm/mmzone.h | 4 -
arch/sparc/mm/init_64.c | 11 +-
arch/x86/Kconfig | 9 +-
arch/x86/include/asm/Kbuild | 1 +
arch/x86/include/asm/mmzone.h | 6 -
arch/x86/include/asm/mmzone_32.h | 17 -
arch/x86/include/asm/mmzone_64.h | 18 -
arch/x86/include/asm/numa.h | 26 +-
arch/x86/include/asm/sparsemem.h | 9 -
arch/x86/mm/Makefile | 1 -
arch/x86/mm/amdtopology.c | 1 +
arch/x86/mm/numa.c | 618 +-----------------
arch/x86/mm/numa_internal.h | 24 -
drivers/acpi/numa/srat.c | 1 +
drivers/base/Kconfig | 1 +
drivers/base/arch_numa.c | 224 ++-----
drivers/cxl/Kconfig | 2 +-
drivers/dax/Kconfig | 2 +-
drivers/of/of_numa.c | 5 +-
include/asm-generic/mmzone.h | 5 +
include/asm-generic/numa.h | 6 +-
include/linux/memory_hotplug.h | 48 --
include/linux/numa.h | 5 +
include/linux/numa_memblks.h | 58 ++
kernel/Makefile | 1 -
kernel/numa.c | 26 -
mm/Kconfig | 11 +
mm/Makefile | 3 +
mm/mm_init.c | 3 +-
mm/numa.c | 57 ++
{arch/x86/mm => mm}/numa_emulation.c | 42 +-
mm/numa_memblks.c | 568 ++++++++++++++++
58 files changed, 870 insertions(+), 1160 deletions(-)
delete mode 100644 arch/arm64/include/asm/mmzone.h
delete mode 100644 arch/loongarch/include/asm/mmzone.h
delete mode 100644 arch/riscv/include/asm/mmzone.h
delete mode 100644 arch/s390/include/asm/mmzone.h
delete mode 100644 arch/x86/include/asm/mmzone.h
delete mode 100644 arch/x86/include/asm/mmzone_32.h
delete mode 100644 arch/x86/include/asm/mmzone_64.h
create mode 100644 include/asm-generic/mmzone.h
create mode 100644 include/linux/numa_memblks.h
delete mode 100644 kernel/numa.c
create mode 100644 mm/numa.c
rename {arch/x86/mm => mm}/numa_emulation.c (94%)
create mode 100644 mm/numa_memblks.cbase-commit: 8400291e289ee6b2bf9779ff1c83a291501f017b
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 1/26 mm: move kernel/numa.c to mm/
From: "Mike Rapoport (Microsoft)" <[email protected]>
The stub functions in kernel/numa.c belong to mm/ rather than to kernel/
---
kernel/Makefile | 1 -
mm/Makefile | 1 +
{kernel => mm}/numa.c | 0
3 files changed, 1 insertion(+), 1 deletion(-)
rename {kernel => mm}/numa.c (100%)kernel/Makefile +0 -1
@@ -116,7 +116,6 @@ obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o obj-$(CONFIG_HAVE_STATIC_CALL) += static_call.o obj-$(CONFIG_HAVE_STATIC_CALL_INLINE) += static_call_inline.o obj-$(CONFIG_CFI_CLANG) += cfi.o -obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_PERF_EVENTS) += events/
mm/Makefile +1 -0
@@ -141,3 +141,4 @@ obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o +obj-$(CONFIG_NUMA) += numa.o
mm/numa.c renamed from kernel/numa.c +0 -0
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 2/26 MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
sgi-ip27 is the only system that defines NODE_DATA() differently than the rest of NUMA machines.
Add node_data array of struct pglist pointers that will point to __node_data[node]->pglist and redefine NODE_DATA() to use node_data array.
This will allow pulling declaration of node_data to the generic mm code in the next commit.
--- arch/mips/include/asm/mach-ip27/mmzone.h | 5 ++++- arch/mips/sgi-ip27/ip27-memory.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)
arch/mips/include/asm/mach-ip27/mmzone.h +4 -1
@@ -22,7 +22,10 @@ struct node_data {extern struct node_data *__node_data[]; -#define NODE_DATA (n) (&__node_data[(n)]->pglist)#define hub_data (n) (&__node_data[(n)]->hub) +extern struct pglist_data *node_data[]; + +#define NODE_DATA (nid) (node_data[nid]) +#endif /* _ASM_MACH_MMZONE_H */
arch/mips/sgi-ip27/ip27-memory.c +4 -1
@@ -34,8 +34,10 @@#define SLOT_PFNSHIFT (SLOT_SHIFT - PAGE_SHIFT)#define PFN_NASIDSHFT (NASID_SHFT - PAGE_SHIFT) -struct node_data *__node_data[MAX_NUMNODES ]; +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data); +struct node_data *__node_data[MAX_NUMNODES ];EXPORT_SYMBOL (__node_data);static u64 gen_region_mask (void )
@@ -361,6 +363,7 @@ static void __init node_mem_init(nasid_t node)*/ __node_data[node] =__va (slot_freepfn <<PAGE_SHIFT );memset (__node_data[node],0 ,PAGE_SIZE ); + node_data[node] = &__node_data[node]->pglist;NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 3/26 MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
For SGI IP27 machines node_possible_map is statically set to NODE_MASK_ALL and it is not updated during NUMA initialization.
Ensure that it only contains nodes present in the system.
--- arch/mips/sgi-ip27/ip27-smp.c | 2 ++ 1 file changed, 2 insertions(+)
arch/mips/sgi-ip27/ip27-smp.c +2 -0
@@ -70,11 +70,13 @@ void cpu_node_probe(void)gda_t *gdap =GDA ;nodes_clear (node_online_map); +nodes_clear (node_possible_map);for (i =0 ; i <MAX_NUMNODES ; i++) {nasid_t nasid = gdap->g_nasidtable[i];if (nasid ==INVALID_NASID )break ;node_set_online (nasid); +node_set (nasid, node_possible_map); highest =node_scan_cpus (nasid, highest); }
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 4/26 MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Commit f8f9f21c7848 ("MIPS: Fix build error for loongson64 and sgi-ip27") added HAVE_ARCH_NODEDATA_EXTENSION to sgi-ip27 to silence a compilation error that happened because sgi-ip27 didn't define array of pg_data_t as node_data like most other architectures did.
After addition of node_data array that matches other architectures and after ensuring that offline nodes do not appear on node_possible_map, it is safe to drop arch_alloc_nodedata() and HAVE_ARCH_NODEDATA_EXTENSION from sgi-ip27.
--- arch/mips/Kconfig | 1 - arch/mips/sgi-ip27/ip27-memory.c | 10 ---------- 2 files changed, 11 deletions(-)
arch/mips/Kconfig +0 -1
@@ -735,7 +735,6 @@ config SGI_IP27 select WAR_R10000_LLSC select MIPS_L1_CACHE_SHIFT_7 select NUMA - select HAVE_ARCH_NODEDATA_EXTENSION help This are the SGI Origin 200, Origin 2000 and Onyx 2 Graphics workstations. To compile a Linux kernel that runs on these, say Y
arch/mips/sgi-ip27/ip27-memory.c +0 -10
@@ -426,13 +426,3 @@ void __init mem_init(void)memblock_free_all ();setup_zero_pages ();/* This comes from node 0 */ } - -pg_data_t *__init arch_alloc_nodedata (int nid) -{ -return memblock_alloc (sizeof (pg_data_t),SMP_CACHE_BYTES ); -} - -void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - __node_data[nid] = (struct node_data *)pgdat; -}
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 5/26 MIPS: loongson64: rename __node_data to node_data
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Make definition of node_data match other architectures.
This will allow pulling declaration of node_data to the generic mm code in the following commit.
--- arch/mips/include/asm/mach-loongson64/mmzone.h | 4 ++-- arch/mips/loongson64/numa.c | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-)
arch/mips/include/asm/mach-loongson64/mmzone.h +2 -2
@@ -14,9 +14,9 @@#define pa_to_nid (addr) (((addr) & 0xf00000000000) >> NODE_ADDRSPACE_SHIFT)#define nid_to_addrbase (nid) ((unsigned long)(nid) << NODE_ADDRSPACE_SHIFT) -extern struct pglist_data *__node_data[]; +extern struct pglist_data *node_data[]; -#define NODE_DATA (n) (__node_data[n]) +#define NODE_DATA (n) (node_data[n])extern void __init prom_init_numa_memory (void );
arch/mips/loongson64/numa.c +4 -4
@@ -29,8 +29,8 @@unsigned char __node_distances[MAX_NUMNODES ][MAX_NUMNODES ];EXPORT_SYMBOL (__node_distances); -struct pglist_data *__node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (__node_data); +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data);cpumask_t __node_cpumask[MAX_NUMNODES ];EXPORT_SYMBOL (__node_cpumask);
@@ -107,7 +107,7 @@ static void __init node_mem_init(unsigned int node) tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT );if (tnid != node)pr_info ("NODE_DATA(%d) on node %d\n" , node, tnid); - __node_data[node] = nd; + node_data[node] = nd;NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
@@ -206,5 +206,5 @@ pg_data_t * __init arch_alloc_nodedata(int nid)void arch_refresh_nodedata (int nid,pg_data_t *pgdat) { - __node_data[nid] = pgdat; + node_data[nid] = pgdat; }
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 6/26 MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Commit f8f9f21c7848 ("MIPS: Fix build error for loongson64 and sgi-ip27") added HAVE_ARCH_NODEDATA_EXTENSION to loongson64 to silence a compilation error that happened because loongson64 didn't define array of pg_data_t as node_data like most other architectures did.
After rename of __node_data to node_data arch_alloc_nodedata() and HAVE_ARCH_NODEDATA_EXTENSION can be dropped from loongson64.
Since it was the only user of HAVE_ARCH_NODEDATA_EXTENSION config option also remove this option from arch/mips/Kconfig.
--- arch/mips/Kconfig | 4 ---- arch/mips/loongson64/numa.c | 10 ---------- 2 files changed, 14 deletions(-)
arch/mips/Kconfig +0 -4
@@ -502,7 +502,6 @@ config MACH_LOONGSON64 select USE_OF select BUILTIN_DTB select PCI_HOST_GENERIC - select HAVE_ARCH_NODEDATA_EXTENSION if NUMA help This enables the support of Loongson-2/3 family of machines.
@@ -2612,9 +2611,6 @@ config NUMA config SYS_SUPPORTS_NUMA bool -config HAVE_ARCH_NODEDATA_EXTENSION - bool - config RELOCATABLE bool "Relocatable kernel" depends on SYS_SUPPORTS_RELOCATABLE
arch/mips/loongson64/numa.c +0 -10
@@ -198,13 +198,3 @@ void __init prom_init_numa_memory(void)pr_info ("CP0_PageGrain: CP0 5.1 (0x%x)\n" ,read_c0_pagegrain ());prom_meminit (); } - -pg_data_t *__init arch_alloc_nodedata (int nid) -{ -return memblock_alloc (sizeof (pg_data_t),SMP_CACHE_BYTES ); -} - -void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - node_data[nid] = pgdat; -}
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 7/26 mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
There are no users of HAVE_ARCH_NODEDATA_EXTENSION left, so arch_alloc_nodedata() and arch_refresh_nodedata() are not needed anymore.
Replace the call to arch_alloc_nodedata() in free_area_init() with memblock_alloc(), remove arch_refresh_nodedata() and cleanup include/linux/memory_hotplug.h from the associated ifdefery.
--- include/linux/memory_hotplug.h | 48 ---------------------------------- mm/mm_init.c | 3 +-- 2 files changed, 1 insertion(+), 50 deletions(-)
include/linux/memory_hotplug.h +0 -48
@@ -16,54 +16,6 @@ struct resource;struct vmem_altmap ;struct dev_pagemap ; -#ifdef CONFIG_HAVE_ARCH_NODEDATA_EXTENSION -/* -* For supporting node-hotadd, we have to allocate a new pgdat. -* -* If an arch has generic style NODE_DATA(), -* node_data[nid] = kzalloc() works well. But it depends on the architecture. -* -* In general, generic_alloc_nodedata() is used. -* -*/ -extern pg_data_t *arch_alloc_nodedata (int nid); -extern void arch_refresh_nodedata (int nid,pg_data_t *pgdat); - -#else /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ - -#define arch_alloc_nodedata (nid) generic_alloc_nodedata(nid) - -#ifdef CONFIG_NUMA -/* -* XXX: node aware allocation can't work well to get new node's memory at this time. -* Because, pgdat for the new node is not allocated/initialized yet itself. -* To use new node's memory, more consideration will be necessary. -*/ -#define generic_alloc_nodedata (nid) \ -({ \ - memblock_alloc(sizeof(*pgdat), SMP_CACHE_BYTES); \ -}) - -extern pg_data_t *node_data[]; -static inline void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - node_data[nid] = pgdat; -} - -#else /* !CONFIG_NUMA */ - -/* never called */ -static inline pg_data_t *generic_alloc_nodedata (int nid) -{ -BUG (); -return NULL ; -} -static inline void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ -} -#endif /* CONFIG_NUMA */ -#endif /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ -#ifdef CONFIG_MEMORY_HOTPLUG struct page *pfn_to_online_page (unsigned long pfn);
mm/mm_init.c +1 -2
@@ -1838,11 +1838,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)if (!node_online (nid)) {/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES );if (!pgdat)panic ("Cannot allocate %zuB for node %d.\n" ,sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat); } pgdat =NODE_DATA (nid);
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 8/26 arch, mm: move definition of node_data to generic code
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Every architecture that supports NUMA defines node_data in the same way:
struct pglist_data *node_data[MAX_NUMNODES];
No reason to keep multiple copies of this definition and its forward declarations, especially when such forward declaration is the only thing in include/asm/mmzone.h for many architectures.
Add definition and declaration of node_data to generic code and drop architecture-specific versions.
--- arch/arm64/include/asm/Kbuild | 1 + arch/arm64/include/asm/mmzone.h | 13 ------------- arch/arm64/include/asm/topology.h | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/asm/mmzone.h | 16 ---------------- arch/loongarch/include/asm/topology.h | 1 + arch/loongarch/kernel/numa.c | 3 --- arch/mips/include/asm/mach-ip27/mmzone.h | 4 ---- arch/mips/include/asm/mach-loongson64/mmzone.h | 4 ---- arch/mips/loongson64/numa.c | 2 -- arch/mips/sgi-ip27/ip27-memory.c | 3 --- arch/powerpc/include/asm/mmzone.h | 6 ------ arch/powerpc/mm/numa.c | 2 -- arch/riscv/include/asm/Kbuild | 1 + arch/riscv/include/asm/mmzone.h | 13 ------------- arch/riscv/include/asm/topology.h | 4 ++++ arch/s390/include/asm/Kbuild | 1 + arch/s390/include/asm/mmzone.h | 17 ----------------- arch/s390/kernel/numa.c | 3 --- arch/sh/include/asm/mmzone.h | 3 --- arch/sh/mm/numa.c | 3 --- arch/sparc/include/asm/mmzone.h | 4 ---- arch/sparc/mm/init_64.c | 2 -- arch/x86/include/asm/Kbuild | 1 + arch/x86/include/asm/mmzone.h | 6 ------ arch/x86/include/asm/mmzone_32.h | 17 ----------------- arch/x86/include/asm/mmzone_64.h | 18 ------------------ arch/x86/mm/numa.c | 3 --- drivers/base/arch_numa.c | 2 -- include/asm-generic/mmzone.h | 5 +++++ include/linux/numa.h | 3 +++ mm/numa.c | 3 +++ 32 files changed, 22 insertions(+), 144 deletions(-) delete mode 100644 arch/arm64/include/asm/mmzone.h delete mode 100644 arch/loongarch/include/asm/mmzone.h delete mode 100644 arch/riscv/include/asm/mmzone.h delete mode 100644 arch/s390/include/asm/mmzone.h delete mode 100644 arch/x86/include/asm/mmzone.h delete mode 100644 arch/x86/include/asm/mmzone_32.h delete mode 100644 arch/x86/include/asm/mmzone_64.h create mode 100644 include/asm-generic/mmzone.h
arch/arm64/include/asm/Kbuild +1 -0
@@ -9,6 +9,7 @@ syscall-y += unistd_compat_32.h generic-y += early_ioremap.h generic-y += mcs_spinlock.h +generic-y += mmzone.h generic-y += qrwlock.h generic-y += qspinlock.h generic-y += parport.h
arch/arm64/include/asm/mmzone.h deleted +0 -13
@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __ASM_MMZONE_H -#define __ASM_MMZONE_H - -#ifdef CONFIG_NUMA - -#include <asm/numa.h> - -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[(nid)]) - -#endif /* CONFIG_NUMA */ -#endif /* __ASM_MMZONE_H */
arch/arm64/include/asm/topology.h +1 -0
@@ -5,6 +5,7 @@#include <linux/cpumask.h> #ifdef CONFIG_NUMA +#include <asm/numa.h> struct pci_bus ;int pcibus_to_node (struct pci_bus *bus);
arch/loongarch/include/asm/Kbuild +1 -0
@@ -9,5 +9,6 @@ generic-y += qrwlock.h generic-y += qspinlock.h generic-y += user.h generic-y += ioctl.h +generic-y += mmzone.h generic-y += statfs.h generic-y += param.h
arch/loongarch/include/asm/mmzone.h deleted +0 -16
@@ -1,16 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* -* Author: Huacai Chen ([email protected]) -* Copyright (C) 2020-2022 Loongson Technology Corporation Limited -*/ -#ifndef _ASM_MMZONE_H_ -#define _ASM_MMZONE_H_ - -#include <asm/page.h> -#include <asm/numa.h> - -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[(nid)]) - -#endif /* _ASM_MMZONE_H_ */
arch/loongarch/include/asm/topology.h +1 -0
@@ -8,6 +8,7 @@#include <linux/smp.h> #ifdef CONFIG_NUMA +#include <asm/numa.h> extern cpumask_t cpus_on_node[];
arch/loongarch/kernel/numa.c +0 -3
@@ -27,10 +27,7 @@#include <asm/time.h> int numa_off; -struct pglist_data *node_data[MAX_NUMNODES ];unsigned char node_distances[MAX_NUMNODES ][MAX_NUMNODES ]; - -EXPORT_SYMBOL (node_data);EXPORT_SYMBOL (node_distances);static struct numa_meminfo numa_meminfo;
arch/mips/include/asm/mach-ip27/mmzone.h +0 -4
@@ -24,8 +24,4 @@ extern struct node_data *__node_data[];#define hub_data (n) (&__node_data[(n)]->hub) -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[nid]) -#endif /* _ASM_MACH_MMZONE_H */
arch/mips/include/asm/mach-loongson64/mmzone.h +0 -4
@@ -14,10 +14,6 @@#define pa_to_nid (addr) (((addr) & 0xf00000000000) >> NODE_ADDRSPACE_SHIFT)#define nid_to_addrbase (nid) ((unsigned long)(nid) << NODE_ADDRSPACE_SHIFT) -extern struct pglist_data *node_data[]; - -#define NODE_DATA (n) (node_data[n]) -extern void __init prom_init_numa_memory (void );#endif /* _ASM_MACH_MMZONE_H */
arch/mips/loongson64/numa.c +0 -2
@@ -29,8 +29,6 @@unsigned char __node_distances[MAX_NUMNODES ][MAX_NUMNODES ];EXPORT_SYMBOL (__node_distances); -struct pglist_data *node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (node_data);cpumask_t __node_cpumask[MAX_NUMNODES ];EXPORT_SYMBOL (__node_cpumask);
arch/mips/sgi-ip27/ip27-memory.c +0 -3
@@ -34,9 +34,6 @@#define SLOT_PFNSHIFT (SLOT_SHIFT - PAGE_SHIFT)#define PFN_NASIDSHFT (NASID_SHFT - PAGE_SHIFT) -struct pglist_data *node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (node_data); -struct node_data *__node_data[MAX_NUMNODES ];EXPORT_SYMBOL (__node_data);
arch/powerpc/include/asm/mmzone.h +0 -6
@@ -20,12 +20,6 @@#ifdef CONFIG_NUMA -extern struct pglist_data *node_data[]; -/* -* Return a pointer to the node data for node n. -*/ -#define NODE_DATA (nid) (node_data[nid]) -/* * Following are specific to this numa platform. */
arch/powerpc/mm/numa.c +0 -2
@@ -43,11 +43,9 @@ static char *cmdline __initdata;int numa_cpu_lookup_table[NR_CPUS ];cpumask_var_t node_to_cpumask_map[MAX_NUMNODES ]; -struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (numa_cpu_lookup_table);EXPORT_SYMBOL (node_to_cpumask_map); -EXPORT_SYMBOL (node_data);static int primary_domain_index;static int n_mem_addr_cells, n_mem_size_cells;
arch/riscv/include/asm/Kbuild +1 -0
@@ -5,6 +5,7 @@ syscall-y += syscall_table_64.h generic-y += early_ioremap.h generic-y += flat.h generic-y += kvm_para.h +generic-y += mmzone.h generic-y += parport.h generic-y += spinlock.h generic-y += spinlock_types.h
arch/riscv/include/asm/mmzone.h deleted +0 -13
@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __ASM_MMZONE_H -#define __ASM_MMZONE_H - -#ifdef CONFIG_NUMA - -#include <asm/numa.h> - -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[(nid)]) - -#endif /* CONFIG_NUMA */ -#endif /* __ASM_MMZONE_H */
arch/riscv/include/asm/topology.h +4 -0
@@ -4,6 +4,10 @@#include <linux/arch_topology.h> +#ifdef CONFIG_NUMA +#include <asm/numa.h> +#endif +/* Replace task scheduler's default frequency-invariant accounting */ #define arch_scale_freq_tick topology_scale_freq_tick#define arch_set_freq_scale topology_set_freq_scale
arch/s390/include/asm/Kbuild +1 -0
@@ -7,3 +7,4 @@ generated-y += unistd_nr.h generic-y += asm-offsets.h generic-y += kvm_types.h generic-y += mcs_spinlock.h +generic-y += mmzone.h
arch/s390/include/asm/mmzone.h deleted +0 -17
@@ -1,17 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* -* NUMA support for s390 -* -* Copyright IBM Corp. 2015 -*/ - -#ifndef _ASM_S390_MMZONE_H -#define _ASM_S390_MMZONE_H - -#ifdef CONFIG_NUMA - -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[nid]) - -#endif /* CONFIG_NUMA */ -#endif /* _ASM_S390_MMZONE_H */
arch/s390/kernel/numa.c +0 -3
@@ -14,9 +14,6 @@#include <linux/node.h> #include <asm/numa.h> -struct pglist_data *node_data[MAX_NUMNODES ]; -EXPORT_SYMBOL (node_data); -void __init numa_setup (void ) {int nid;
arch/sh/include/asm/mmzone.h +0 -3
@@ -5,9 +5,6 @@#ifdef CONFIG_NUMA #include <linux/numa.h> -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[nid]) -static inline int pfn_to_nid (unsigned long pfn) {int nid;
arch/sh/mm/numa.c +0 -3
@@ -14,9 +14,6 @@#include <linux/pfn.h> #include <asm/sections.h> -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL_GPL (node_data); -/* * On SH machines the conventional approach is to stash system RAM * in node 0, and other memory blocks in to node 1 and up, ordered by
arch/sparc/include/asm/mmzone.h +0 -4
@@ -6,10 +6,6 @@#include <linux/cpumask.h> -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[nid]) -extern int numa_cpu_lookup_table[];extern cpumask_t numa_cpumask_lookup_table[];
arch/sparc/mm/init_64.c +0 -2
@@ -1115,11 +1115,9 @@ static void init_node_masks_nonnuma(void) }#ifdef CONFIG_NUMA -struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (numa_cpu_lookup_table);EXPORT_SYMBOL (numa_cpumask_lookup_table); -EXPORT_SYMBOL (node_data);static int scan_pio_for_cfg_handle (struct mdesc_handle *md,u64 pio,u32 cfg_handle)
arch/x86/include/asm/Kbuild +1 -0
@@ -11,3 +11,4 @@ generated-y += xen-hypercalls.h generic-y += early_ioremap.h generic-y += mcs_spinlock.h +generic-y += mmzone.h
arch/x86/include/asm/mmzone.h deleted +0 -6
@@ -1,6 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifdef CONFIG_X86_32 -# include <asm/mmzone_32.h> -#else -# include <asm/mmzone_64.h> -#endif
arch/x86/include/asm/mmzone_32.h deleted +0 -17
@@ -1,17 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* -* Written by Pat Gaughen ([email protected]) Mar 2002 -* -*/ - -#ifndef _ASM_X86_MMZONE_32_H -#define _ASM_X86_MMZONE_32_H - -#include <asm/smp.h> - -#ifdef CONFIG_NUMA -extern struct pglist_data *node_data[]; -#define NODE_DATA (nid) (node_data[nid]) -#endif /* CONFIG_NUMA */ - -#endif /* _ASM_X86_MMZONE_32_H */
arch/x86/include/asm/mmzone_64.h deleted +0 -18
@@ -1,18 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* K8 NUMA support */ -/* Copyright 2002,2003 by Andi Kleen, SuSE Labs */ -/* 2.5 Version loosely based on the NUMAQ Code by Pat Gaughen. */ -#ifndef _ASM_X86_MMZONE_64_H -#define _ASM_X86_MMZONE_64_H - -#ifdef CONFIG_NUMA - -#include <linux/mmdebug.h> -#include <asm/smp.h> - -extern struct pglist_data *node_data[]; - -#define NODE_DATA (nid) (node_data[nid]) - -#endif -#endif /* _ASM_X86_MMZONE_64_H */
arch/x86/mm/numa.c +0 -3
@@ -24,9 +24,6 @@int numa_off;nodemask_t numa_nodes_parsed__initdata ; -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL (node_data); - static struct numa_meminfo numa_meminfo __initdata_or_meminfo; static struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo;
drivers/base/arch_numa.c +0 -2
@@ -15,8 +15,6 @@#include <asm/sections.h> -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL (node_data);nodemask_t numa_nodes_parsed__initdata ;static int cpu_to_node_map[NR_CPUS ] = { [0 ...NR_CPUS -1 ] =NUMA_NO_NODE };
include/asm-generic/mmzone.h new file +5 -0
@@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_MMZONE_H +#define _ASM_GENERIC_MMZONE_H + +#endif
include/linux/numa.h +3 -0
@@ -30,6 +30,9 @@ static inline bool numa_valid_node(int nid)#ifdef CONFIG_NUMA #include <asm/sparsemem.h> +extern struct pglist_data *node_data[]; +#define NODE_DATA (nid) (node_data[nid]) +/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);
mm/numa.c +3 -0
@@ -3,6 +3,9 @@#include <linux/printk.h> #include <linux/numa.h> +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data); +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 9/26 arch, mm: pull out allocation of NODE_DATA to generic code
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Architectures that support NUMA duplicate the code that allocates NODE_DATA on the node-local memory with slight variations in reporting of the addresses where the memory was allocated.
Use x86 version as the basis for the generic alloc_node_data() function and call this function in architecture specific numa initialization.
Round up node data size to SMP_CACHE_BYTES rather than to PAGE_SIZE like x86 used to do since the bootmem era when allocation granularity was PAGE_SIZE anyway.
--- arch/loongarch/kernel/numa.c | 18 ------------------ arch/mips/loongson64/numa.c | 16 ++-------------- arch/powerpc/mm/numa.c | 24 +++--------------------- arch/sh/mm/init.c | 7 +------ arch/sparc/mm/init_64.c | 9 ++------- arch/x86/mm/numa.c | 34 +--------------------------------- drivers/base/arch_numa.c | 21 +-------------------- include/linux/numa.h | 2 ++ mm/numa.c | 27 +++++++++++++++++++++++++++ 9 files changed, 39 insertions(+), 119 deletions(-)
arch/loongarch/kernel/numa.c +0 -18
@@ -187,24 +187,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)return numa_add_memblk_to (nid, start, end, &numa_meminfo); } -static void __init alloc_node_data (int nid) -{ -void *nd; -unsigned long nd_pa; -size_t nd_sz =roundup (sizeof (pg_data_t),PAGE_SIZE ); - - nd_pa =memblock_phys_alloc_try_nid (nd_sz,SMP_CACHE_BYTES , nid); -if (!nd_pa) { -pr_err ("Cannot find %zu Byte for node_data (initial node: %d)\n" , nd_sz, nid); -return ; - } - - nd =__va (nd_pa); - - node_data[nid] = nd; -memset (nd,0 ,sizeof (pg_data_t)); -} -static void __init node_mem_init (unsigned int node) {unsigned long start_pfn, end_pfn;
arch/mips/loongson64/numa.c +2 -14
@@ -81,12 +81,8 @@ static void __init init_topology_matrix(void)static void __init node_mem_init (unsigned int node) { -struct pglist_data *nd;unsigned long node_addrspace_offset;unsigned long start_pfn, end_pfn; -unsigned long nd_pa; -int tnid; -const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); node_addrspace_offset =nid_to_addrbase (node); pr_info("Node%d's addrspace_offset is 0x%lx\n",
@@ -96,16 +92,8 @@ static void __init node_mem_init(unsigned int node)pr_info ("Node%d: start_pfn=0x%lx, end_pfn=0x%lx\n" , node, start_pfn, end_pfn); - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , node); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, node); - nd =__va (nd_pa); -memset (nd,0 ,sizeof (struct pglist_data )); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != node) -pr_info ("NODE_DATA(%d) on node %d\n" , node, tnid); - node_data[node] = nd; +alloc_node_data (node); +NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
arch/powerpc/mm/numa.c +3 -21
@@ -1093,27 +1093,9 @@ void __init dump_numa_cpu_topology(void)static void __init setup_node_data (int nid,u64 start_pfn,u64 end_pfn) {u64 spanned_pages = end_pfn - start_pfn; -const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); -u64 nd_pa; -void *nd; -int tnid; - - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, nid); - - nd =__va (nd_pa); - -/* report and initialize */ -pr_info (" NODE_DATA [mem %#010Lx-%#010Lx]\n" , - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -pr_info (" NODE_DATA(%d) on node %d\n" , nid, tnid); - - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); + +alloc_node_data (nid); +NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start_pfn;NODE_DATA (nid)->node_spanned_pages = spanned_pages;
arch/sh/mm/init.c +1 -6
@@ -212,12 +212,7 @@ void __init allocate_pgdat(unsigned int nid)get_pfn_range_for_nid (nid, &start_pfn, &end_pfn);#ifdef CONFIG_NUMA -NODE_DATA (nid) =memblock_alloc_try_nid ( -sizeof (struct pglist_data ), -SMP_CACHE_BYTES ,MEMBLOCK_LOW_LIMIT , -MEMBLOCK_ALLOC_ACCESSIBLE , nid); -if (!NODE_DATA (nid)) -panic ("Can't allocate pgdat for node %d\n" , nid); +alloc_node_data (nid);#endif NODE_DATA (nid)->node_start_pfn = start_pfn;
arch/sparc/mm/init_64.c +2 -7
@@ -1075,14 +1075,9 @@ static void __init allocate_node_data(int nid) {struct pglist_data *p;unsigned long start_pfn, end_pfn; -#ifdef CONFIG_NUMA -NODE_DATA (nid) =memblock_alloc_node (sizeof (struct pglist_data ), -SMP_CACHE_BYTES , nid); -if (!NODE_DATA (nid)) { -prom_printf ("Cannot allocate pglist_data for nid[%d]\n" , nid); -prom_halt (); - } +#ifdef CONFIG_NUMA +alloc_node_data (nid);NODE_DATA (nid)->node_id = nid;#endif
arch/x86/mm/numa.c +1 -33
@@ -191,39 +191,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)return numa_add_memblk_to (nid, start, end, &numa_meminfo); } -/* Allocate NODE_DATA for a node on the local memory */ -static void __init alloc_node_data (int nid) -{ -const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE ); -u64 nd_pa; -void *nd; -int tnid; - -/* -* Allocate node data. Try node-local memory and then any node. -* Never allocate in DMA zone. -*/ - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) { -pr_err ("Cannot find %zu bytes in any node (initial node: %d)\n" , - nd_size, nid); -return ; - } - nd =__va (nd_pa); - -/* report and initialize */ -printk (KERN_INFO "NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n" , nid, - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nid, tnid); - - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); - -node_set_online (nid); -} -/** * numa_cleanup_meminfo - Cleanup a numa_meminfo * @mi: numa_meminfo to clean up
@@ -571,6 +538,7 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)continue ;alloc_node_data (nid); +node_set_online (nid); }/* Dump memblock with node info and return. */
drivers/base/arch_numa.c +1 -20
@@ -216,30 +216,11 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)*/ static void __init setup_node_data (int nid,u64 start_pfn,u64 end_pfn) { -const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); -u64 nd_pa; -void *nd; -int tnid; -if (start_pfn >= end_pfn)pr_info ("Initmem setup node %d [<memory-less node>]\n" , nid); - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, nid); - - nd =__va (nd_pa); - -/* report and initialize */ -pr_info ("NODE_DATA [mem %#010Lx-%#010Lx]\n" , - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -pr_info ("NODE_DATA(%d) on node %d\n" , nid, tnid); +alloc_node_data (nid); - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t));NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start_pfn;NODE_DATA (nid)->node_spanned_pages = end_pfn - start_pfn;
include/linux/numa.h +2 -0
@@ -33,6 +33,8 @@ static inline bool numa_valid_node(int nid)extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid]) +void __init alloc_node_data (int nid); +/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);
mm/numa.c +27 -0
@@ -1,11 +1,38 @@// SPDX-License-Identifier: GPL-2.0-or-later +#include <linux/memblock.h> #include <linux/printk.h> #include <linux/numa.h> struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (node_data); +/* Allocate NODE_DATA for a node on the local memory */ +void __init alloc_node_data (int nid) +{ +const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); +u64 nd_pa; +void *nd; +int tnid; + +/* Allocate node data. Try node-local memory and then any node. */ + nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); +if (!nd_pa) +panic ("Cannot allocate %zu bytes for node %d data\n" , + nd_size, nid); + nd =__va (nd_pa); + +/* report and initialize */ +pr_info ("NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n" , nid, + nd_pa, nd_pa + nd_size -1 ); + tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); +if (tnid != nid) +pr_info (" NODE_DATA(%d) on node %d\n" , nid, tnid); + + node_data[nid] = nd; +memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); +} +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 10/26 x86/numa: simplify numa_distance allocation
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Allocation of numa_distance uses memblock_phys_alloc_range() to limit allocation to be below the last mapped page.
But NUMA initializaition runs after the direct map is populated and there is also code in setup_arch() that adjusts memblock limit to reflect how much memory is already mapped in the direct map.
Simplify the allocation of numa_distance and use plain memblock_alloc().
--- arch/x86/mm/numa.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
arch/x86/mm/numa.c +2 -5
@@ -331,7 +331,6 @@ static int __init numa_alloc_distance(void)nodemask_t nodes_parsed;size_t size;int i, j, cnt =0 ; -u64 phys;/* size the new table and allocate it */ nodes_parsed = numa_nodes_parsed;
@@ -342,16 +341,14 @@ static int __init numa_alloc_distance(void) cnt++; size = cnt * cnt *sizeof (numa_distance[0 ]); - phys =memblock_phys_alloc_range (size,PAGE_SIZE ,0 , -PFN_PHYS (max_pfn_mapped)); -if (!phys) { + numa_distance =memblock_alloc (size,PAGE_SIZE ); +if (!numa_distance) {pr_warn ("Warning: can't allocate distance table!\n" );/* don't retry until explicitly reset */ numa_distance = (void *)1LU ;return -ENOMEM ; } - numa_distance =__va (phys); numa_distance_cnt = cnt;/* fill with the default distances */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 11/26 x86/numa: use get_pfn_range_for_nid to verify that node spans memory
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of looping over numa_meminfo array to detect node's start and end addresses use get_pfn_range_for_init().
This is shorter and make it easier to lift numa_memblks to generic code.
--- arch/x86/mm/numa.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-)
arch/x86/mm/numa.c +3 -10
@@ -521,17 +521,10 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start =PFN_PHYS (max_pfn); -u64 end =0 ; +unsigned long start_pfn, end_pfn; -for (i =0 ; i < mi->nr_blks; i++) { -if (nid != mi->blk[i].nid) -continue ; - start =min (mi->blk[i].start, start); - end =max (mi->blk[i].end, end); - } - -if (start >= end) +get_pfn_range_for_nid (nid, &start_pfn, &end_pfn); +if (start_pfn >= end_pfn)continue ;alloc_node_data (nid);
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 12/26 x86/numa: move FAKE_NODE_* defines to numa_emu
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
The definitions of FAKE_NODE_MIN_SIZE and FAKE_NODE_MIN_HASH_MASK are only used by numa emulation code, make them local to arch/x86/mm/numa_emulation.c
--- arch/x86/include/asm/numa.h | 2 -- arch/x86/mm/numa_emulation.c | 3 +++ 2 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/numa.h +0 -2
@@ -71,8 +71,6 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);#endif #ifdef CONFIG_NUMA_EMU -#define FAKE_NODE_MIN_SIZE ((u64)32 << 20) -#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))int numa_emu_cmdline (char *str);#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str)
arch/x86/mm/numa_emulation.c +3 -0
@@ -10,6 +10,9 @@#include "numa_internal.h" +#define FAKE_NODE_MIN_SIZE ((u64)32 << 20) +#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL)) +static int emu_nid_to_phys[MAX_NUMNODES ];static char *emu_cmdline__initdata ;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 13/26 x86/numa_emu: simplify allocation of phys_dist
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
By the time numa_emulation() is called, all physical memory is already mapped in the direct map and there is no need to define limits for memblock allocation.
Replace memblock_phys_alloc_range() with memblock_alloc().
--- arch/x86/mm/numa_emulation.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/mm/numa_emulation.c +2 -6
@@ -448,15 +448,11 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)/* copy the physical distance table */ if (numa_dist_cnt) { -u64 phys; - - phys =memblock_phys_alloc_range (phys_size,PAGE_SIZE ,0 , -PFN_PHYS (max_pfn_mapped)); -if (!phys) { + phys_dist =memblock_alloc (phys_size,PAGE_SIZE ); +if (!phys_dist) {pr_warn ("NUMA: Warning: can't allocate copy of distance table, disabling emulation\n" ); goto no_emu; } - phys_dist =__va (phys);for (i =0 ; i < numa_dist_cnt; i++)for (j =0 ; j < numa_dist_cnt; j++)
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 14/26 x86/numa_emu: split __apicid_to_node update to a helper function
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
This is required to make numa emulation code architecture independent so that it can be moved to generic code in following commits.
--- arch/x86/include/asm/numa.h | 2 ++ arch/x86/mm/numa.c | 22 ++++++++++++++++++++++ arch/x86/mm/numa_emulation.c | 14 +------------- 3 files changed, 25 insertions(+), 13 deletions(-)
arch/x86/include/asm/numa.h +2 -0
@@ -72,6 +72,8 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);#ifdef CONFIG_NUMA_EMU int numa_emu_cmdline (char *str); +void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, +unsigned int nr_emu_nids);#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str) {
arch/x86/mm/numa.c +22 -0
@@ -848,6 +848,28 @@ EXPORT_SYMBOL(cpumask_of_node);#endif /* !CONFIG_DEBUG_PER_CPU_MAPS */ +#ifdef CONFIG_NUMA_EMU +void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, +unsigned int nr_emu_nids) +{ +int i, j; + +/* +* Transform __apicid_to_node table to use emulated nids by +* reverse-mapping phys_nid. The maps should always exist but fall +* back to zero just in case. +*/ +for (i =0 ; i <ARRAY_SIZE (__apicid_to_node); i++) { +if (__apicid_to_node[i] ==NUMA_NO_NODE ) +continue ; +for (j =0 ; j < nr_emu_nids; j++) +if (__apicid_to_node[i] == emu_nid_to_phys[j]) +break ; + __apicid_to_node[i] = j < nr_emu_nids ? j :0 ; + } +} +#endif /* CONFIG_NUMA_EMU */ +#ifdef CONFIG_NUMA_KEEP_MEMINFO static int meminfo_to_nid (struct numa_meminfo *mi,u64 start) {
arch/x86/mm/numa_emulation.c +1 -13
@@ -476,19 +476,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) ei.blk[i].nid != NUMA_NO_NODE)node_set (ei.blk[i].nid, numa_nodes_parsed); -/* -* Transform __apicid_to_node table to use emulated nids by -* reverse-mapping phys_nid. The maps should always exist but fall -* back to zero just in case. -*/ -for (i =0 ; i <ARRAY_SIZE (__apicid_to_node); i++) { -if (__apicid_to_node[i] ==NUMA_NO_NODE ) -continue ; -for (j =0 ; j <ARRAY_SIZE (emu_nid_to_phys); j++) -if (__apicid_to_node[i] == emu_nid_to_phys[j]) -break ; - __apicid_to_node[i] = j <ARRAY_SIZE (emu_nid_to_phys) ? j :0 ; - } +numa_emu_update_cpu_to_node (emu_nid_to_phys,ARRAY_SIZE (emu_nid_to_phys));/* make sure all emulated nodes are mapped to a physical node */ for (i =0 ; i <ARRAY_SIZE (emu_nid_to_phys); i++)
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 15/26 x86/numa_emu: use a helper function to get MAX_DMA32_PFN
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
This is required to make numa emulation code architecture independent so that it can be moved to generic code in following commits.
--- arch/x86/include/asm/numa.h | 1 + arch/x86/mm/numa.c | 5 +++++ arch/x86/mm/numa_emulation.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)
arch/x86/include/asm/numa.h +1 -0
@@ -74,6 +74,7 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);int numa_emu_cmdline (char *str);void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys,unsigned int nr_emu_nids); +u64 __init numa_emu_dma_end (void );#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str) {
arch/x86/mm/numa.c +5 -0
@@ -868,6 +868,11 @@ void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys, __apicid_to_node[i] = j < nr_emu_nids ? j : 0; } } + +u64 __init numa_emu_dma_end (void ) +{ +return PFN_PHYS (MAX_DMA32_PFN ); +}#endif /* CONFIG_NUMA_EMU */ #ifdef CONFIG_NUMA_KEEP_MEMINFO
arch/x86/mm/numa_emulation.c +2 -2
@@ -128,7 +128,7 @@ static int __init split_nodes_interleave(struct numa_meminfo *ei,*/ while (!nodes_empty (physnode_mask)) {for_each_node_mask (i, physnode_mask) { -u64 dma32_end =PFN_PHYS (MAX_DMA32_PFN ); +u64 dma32_end =numa_emu_dma_end ();u64 start, limit, end;int phys_blk;
@@ -275,7 +275,7 @@ static int __init split_nodes_size_interleave_uniform(struct numa_meminfo *ei,*/ while (!nodes_empty (physnode_mask)) {for_each_node_mask (i, physnode_mask) { -u64 dma32_end =PFN_PHYS (MAX_DMA32_PFN ); +u64 dma32_end =numa_emu_dma_end ();u64 start, limit, end;int phys_blk;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 16/26 x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
CPU id cannot be negative.
Making it unsigned also aligns with declarations in include/asm-generic/numa.h used by arm64 and riscv and allows sharing numa emulation code with these architectures.
--- arch/x86/include/asm/numa.h | 10 +++++----- arch/x86/mm/numa.c | 10 +++++----- arch/x86/mm/numa_emulation.c | 10 +++++----- 3 files changed, 15 insertions(+), 15 deletions(-)
arch/x86/include/asm/numa.h +5 -5
@@ -54,20 +54,20 @@ static inline int numa_cpu_node(int cpu)extern void numa_set_node (int cpu,int node);extern void numa_clear_node (int cpu);extern void __init init_cpu_to_node (void ); -extern void numa_add_cpu (int cpu); -extern void numa_remove_cpu (int cpu); +extern void numa_add_cpu (unsigned int cpu); +extern void numa_remove_cpu (unsigned int cpu);extern void init_gi_nodes (void );#else /* CONFIG_NUMA */ static inline void numa_set_node (int cpu,int node) { }static inline void numa_clear_node (int cpu) { }static inline void init_cpu_to_node (void ) { } -static inline void numa_add_cpu (int cpu) { } -static inline void numa_remove_cpu (int cpu) { } +static inline void numa_add_cpu (unsigned int cpu) { } +static inline void numa_remove_cpu (unsigned int cpu) { }static inline void init_gi_nodes (void ) { }#endif /* CONFIG_NUMA */ #ifdef CONFIG_DEBUG_PER_CPU_MAPS -void debug_cpumask_set_cpu (int cpu,int node,bool enable); +void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable);#endif #ifdef CONFIG_NUMA_EMU
arch/x86/mm/numa.c +5 -5
@@ -737,12 +737,12 @@ void __init init_cpu_to_node(void)#ifndef CONFIG_DEBUG_PER_CPU_MAPS # ifndef CONFIG_NUMA_EMU -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {cpumask_set_cpu (cpu, node_to_cpumask_map[early_cpu_to_node (cpu)]); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {cpumask_clear_cpu (cpu, node_to_cpumask_map[early_cpu_to_node (cpu)]); }
@@ -780,7 +780,7 @@ int early_cpu_to_node(int cpu)return per_cpu (x86_cpu_to_node_map, cpu); } -void debug_cpumask_set_cpu (int cpu,int node,bool enable) +void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable) {struct cpumask *mask;
@@ -812,12 +812,12 @@ static void numa_set_cpumask(int cpu, bool enable)debug_cpumask_set_cpu (cpu,early_cpu_to_node (cpu), enable); } -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {numa_set_cpumask (cpu, true); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {numa_set_cpumask (cpu, false); }
arch/x86/mm/numa_emulation.c +5 -5
@@ -514,7 +514,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) }#ifndef CONFIG_DEBUG_PER_CPU_MAPS -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {int physnid, nid;
@@ -532,7 +532,7 @@ void numa_add_cpu(int cpu)cpumask_set_cpu (cpu, node_to_cpumask_map[nid]); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {int i;
@@ -540,7 +540,7 @@ void numa_remove_cpu(int cpu) cpumask_clear_cpu(cpu, node_to_cpumask_map[i]); }#else /* !CONFIG_DEBUG_PER_CPU_MAPS */ -static void numa_set_cpumask (int cpu,bool enable) +static void numa_set_cpumask (unsigned int cpu,bool enable) {int nid, physnid;
@@ -560,12 +560,12 @@ static void numa_set_cpumask(int cpu, bool enable) } } -void numa_add_cpu (int cpu) +void numa_add_cpu (unsigned int cpu) {numa_set_cpumask (cpu, true); } -void numa_remove_cpu (int cpu) +void numa_remove_cpu (unsigned int cpu) {numa_set_cpumask (cpu, false); }
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 17/26 mm: introduce numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with numa_memblks from arch/x86 to mm/ and add Kconfig options to let x86 select it in its Kconfig.
This code will be later reused by arch_numa.
No functional changes.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/numa.h | 3 - arch/x86/mm/amdtopology.c | 1 + arch/x86/mm/numa.c | 372 +-------------------------------- arch/x86/mm/numa_emulation.c | 1 + arch/x86/mm/numa_internal.h | 15 +- drivers/acpi/numa/srat.c | 1 + drivers/of/of_numa.c | 1 + include/linux/numa_memblks.h | 35 ++++ mm/Kconfig | 3 + mm/Makefile | 1 + mm/numa_memblks.c | 385 +++++++++++++++++++++++++++++++++++ 12 files changed, 436 insertions(+), 383 deletions(-) create mode 100644 include/linux/numa_memblks.h create mode 100644 mm/numa_memblks.c
arch/x86/Kconfig +1 -0
@@ -296,6 +296,7 @@ config X86 select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK select NEED_SG_DMA_LENGTH + select NUMA_MEMBLKS if NUMA select PCI_DOMAINS if PCI select PCI_LOCKLESS_CONFIG if PCI select PERF_EVENTS
arch/x86/include/asm/numa.h +0 -3
@@ -10,8 +10,6 @@#ifdef CONFIG_NUMA -#define NR_NODE_MEMBLKS (MAX_NUMNODES*2) -extern int numa_off;/*
@@ -25,7 +23,6 @@ extern int numa_off;extern s16 __apicid_to_node[MAX_LOCAL_APIC ];extern nodemask_t numa_nodes_parsed__initdata ; -extern int __init numa_add_memblk (int nodeid,u64 start,u64 end);extern void __init numa_set_distance (int from,int to,int distance);static inline void set_apicid_to_node (int apicid,s16 node)
arch/x86/mm/amdtopology.c +1 -0
@@ -12,6 +12,7 @@#include <linux/string.h> #include <linux/nodemask.h> #include <linux/memblock.h> +#include <linux/numa_memblks.h> #include <asm/io.h> #include <linux/pci_ids.h>
arch/x86/mm/numa.c +5 -367
402 lines not on this page · raw message
arch/x86/mm/numa_emulation.c +1 -0
@@ -6,6 +6,7 @@#include <linux/errno.h> #include <linux/topology.h> #include <linux/memblock.h> +#include <linux/numa_memblks.h> #include <asm/dma.h> #include "numa_internal.h"
arch/x86/mm/numa_internal.h +2 -13
@@ -5,23 +5,12 @@#include <linux/types.h> #include <asm/numa.h> -struct numa_memblk { -u64 start; -u64 end; -int nid; -}; - -struct numa_meminfo { -int nr_blks; -struct numa_memblk blk[NR_NODE_MEMBLKS ]; -}; - -void __init numa_remove_memblk_from (int idx,struct numa_meminfo *mi); -int __init numa_cleanup_meminfo (struct numa_meminfo *mi);void __init numa_reset_distance (void );void __init x86_numa_init (void ); +struct numa_meminfo ; +#ifdef CONFIG_NUMA_EMU void __init numa_emulation (struct numa_meminfo *numa_meminfo,int numa_dist_cnt);
drivers/acpi/numa/srat.c +1 -0
@@ -17,6 +17,7 @@#include <linux/numa.h> #include <linux/nodemask.h> #include <linux/topology.h> +#include <linux/numa_memblks.h> static nodemask_t nodes_found_map =NODE_MASK_NONE ;
drivers/of/of_numa.c +1 -0
@@ -10,6 +10,7 @@#include <linux/of.h> #include <linux/of_address.h> #include <linux/nodemask.h> +#include <linux/numa_memblks.h> #include <asm/numa.h>
include/linux/numa_memblks.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __NUMA_MEMBLKS_H +#define __NUMA_MEMBLKS_H + +#ifdef CONFIG_NUMA_MEMBLKS +#include <linux/types.h> + +#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) + +struct numa_memblk { +u64 start; +u64 end; +int nid; +}; + +struct numa_meminfo { +int nr_blks; +struct numa_memblk blk[NR_NODE_MEMBLKS ]; +}; + +extern struct numa_meminfo numa_meminfo __initdata_or_meminfo; +extern struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; + +int __init numa_add_memblk (int nodeid,u64 start,u64 end); +void __init numa_remove_memblk_from (int idx,struct numa_meminfo *mi); + +int __init numa_cleanup_meminfo (struct numa_meminfo *mi); +int __init numa_register_meminfo (struct numa_meminfo *mi); + +void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, +const struct numa_meminfo *mi); + +#endif /* CONFIG_NUMA_MEMBLKS */ + +#endif /* __NUMA_MEMBLKS_H */
mm/Kconfig +3 -0
@@ -1263,6 +1263,9 @@ config IOMMU_MM_DATA config EXECMEM bool +config NUMA_MEMBLKS + bool + source "mm/damon/Kconfig" endmenu
mm/Makefile +1 -0
@@ -142,3 +142,4 @@ obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_NUMA) += numa.o +obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o
mm/numa_memblks.c new file +385 -0
385 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 18/26 mm: move numa_distance and related code from x86 to numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with numa_distance array from arch/x86 to mm/numa_memblks.c
This code will be later reused by arch_numa.
No functional changes.
--- arch/x86/include/asm/numa.h | 2 - arch/x86/mm/numa.c | 104 ----------------------------------- arch/x86/mm/numa_internal.h | 2 - include/linux/numa_memblks.h | 4 ++ mm/numa_memblks.c | 104 +++++++++++++++++++++++++++++++++++ 5 files changed, 108 insertions(+), 108 deletions(-)
arch/x86/include/asm/numa.h +0 -2
@@ -23,8 +23,6 @@ extern int numa_off;extern s16 __apicid_to_node[MAX_LOCAL_APIC ];extern nodemask_t numa_nodes_parsed__initdata ; -extern void __init numa_set_distance (int from,int to,int distance); -static inline void set_apicid_to_node (int apicid,s16 node) { __apicid_to_node[apicid] = node;
arch/x86/mm/numa.c +0 -104
116 lines not on this page · raw message
arch/x86/mm/numa_internal.h +0 -2
@@ -5,8 +5,6 @@#include <linux/types.h> #include <asm/numa.h> -void __init numa_reset_distance (void ); -void __init x86_numa_init (void );struct numa_meminfo ;
include/linux/numa_memblks.h +4 -0
@@ -7,6 +7,10 @@#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) +extern int numa_distance_cnt; +void __init numa_set_distance (int from,int to,int distance); +void __init numa_reset_distance (void ); +struct numa_memblk {u64 start;u64 end;
mm/numa_memblks.c +104 -0
115 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 19/26 mm: introduce numa_emulation
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move numa_emulation codfrom arch/x86 to mm/numa_emulation.c
This code will be later reused by arch_numa.
No functional changes.
---
arch/x86/Kconfig | 8 --------
arch/x86/include/asm/numa.h | 12 ------------
arch/x86/mm/Makefile | 1 -
arch/x86/mm/numa_internal.h | 11 -----------
include/linux/numa_memblks.h | 17 +++++++++++++++++
mm/Kconfig | 8 ++++++++
mm/Makefile | 1 +
{arch/x86/mm => mm}/numa_emulation.c | 4 +---
8 files changed, 27 insertions(+), 35 deletions(-)
rename {arch/x86/mm => mm}/numa_emulation.c (99%)arch/x86/Kconfig +0 -8
@@ -1600,14 +1600,6 @@ config X86_64_ACPI_NUMA help Enable ACPI SRAT based node topology detection. -config NUMA_EMU - bool "NUMA emulation" - depends on NUMA - help - Enable NUMA emulation. A flat machine will be split - into virtual nodes when booted with "numa=fake=N", where N is the - number of nodes. This is only useful for debugging. - config NODES_SHIFT int "Maximum NUMA Nodes (as a power of 2)" if !MAXSMP range 1 10
arch/x86/include/asm/numa.h +0 -12
@@ -65,16 +65,4 @@ static inline void init_gi_nodes(void) { }void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable);#endif -#ifdef CONFIG_NUMA_EMU -int numa_emu_cmdline (char *str); -void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, -unsigned int nr_emu_nids); -u64 __init numa_emu_dma_end (void ); -#else /* CONFIG_NUMA_EMU */ -static inline int numa_emu_cmdline (char *str) -{ -return -EINVAL ; -} -#endif /* CONFIG_NUMA_EMU */ -#endif /* _ASM_X86_NUMA_H */
arch/x86/mm/Makefile +0 -1
@@ -57,7 +57,6 @@ obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o obj-$(CONFIG_NUMA) += numa.o numa_$(BITS).o obj-$(CONFIG_AMD_NUMA) += amdtopology.o obj-$(CONFIG_ACPI_NUMA) += srat.o -obj-$(CONFIG_NUMA_EMU) += numa_emulation.o obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
arch/x86/mm/numa_internal.h +0 -11
@@ -7,15 +7,4 @@void __init x86_numa_init (void ); -struct numa_meminfo ; - -#ifdef CONFIG_NUMA_EMU -void __init numa_emulation (struct numa_meminfo *numa_meminfo, -int numa_dist_cnt); -#else -static inline void numa_emulation (struct numa_meminfo *numa_meminfo, -int numa_dist_cnt) -{ } -#endif -#endif /* __X86_MM_NUMA_INTERNAL_H */
include/linux/numa_memblks.h +17 -0
@@ -34,6 +34,23 @@ int __init numa_register_meminfo(struct numa_meminfo *mi);void __init numa_nodemask_from_meminfo (nodemask_t *nodemask,const struct numa_meminfo *mi); +#ifdef CONFIG_NUMA_EMU +int numa_emu_cmdline (char *str); +void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys, +unsigned int nr_emu_nids); +u64 __init numa_emu_dma_end (void ); +void __init numa_emulation (struct numa_meminfo *numa_meminfo, +int numa_dist_cnt); +#else +static inline void numa_emulation (struct numa_meminfo *numa_meminfo, +int numa_dist_cnt) +{ } +static inline int numa_emu_cmdline (char *str) +{ +return -EINVAL ; +} +#endif /* CONFIG_NUMA_EMU */ +#endif /* CONFIG_NUMA_MEMBLKS */ #endif /* __NUMA_MEMBLKS_H */
mm/Kconfig +8 -0
@@ -1266,6 +1266,14 @@ config EXECMEM config NUMA_MEMBLKS bool +config NUMA_EMU + bool "NUMA emulation" + depends on NUMA_MEMBLKS + help + Enable NUMA emulation. A flat machine will be split + into virtual nodes when booted with "numa=fake=N", where N is the + number of nodes. This is only useful for debugging. + source "mm/damon/Kconfig" endmenu
mm/Makefile +1 -0
@@ -143,3 +143,4 @@ obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o +obj-$(CONFIG_NUMA_EMU) += numa_emulation.o
mm/numa_emulation.c renamed from arch/x86/mm/numa_emulation.c +1 -3
@@ -7,9 +7,7 @@#include <linux/topology.h> #include <linux/memblock.h> #include <linux/numa_memblks.h> -#include <asm/dma.h> - -#include "numa_internal.h" +#include <asm/numa.h> #define FAKE_NODE_MIN_SIZE ((u64)32 << 20)#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 20/26 mm: numa_memblks: introduce numa_memblks_init
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move most of x86::numa_init() to numa_memblks so that the latter will be more self-contained.
With this numa_memblk data structures should not be exposed to the architecture specific code.
--- arch/x86/mm/numa.c | 40 ++++------------------------------- include/linux/numa_memblks.h | 3 +++ mm/numa_memblks.c | 41 ++++++++++++++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 36 deletions(-)
arch/x86/mm/numa.c +4 -36
@@ -115,13 +115,9 @@ void __init setup_node_to_cpumask_map(void)pr_debug ("Node to cpumask map for %u nodes\n" , nr_node_ids); } -static int __init numa_register_memblks (struct numa_meminfo *mi) +static int __init numa_register_nodes (void ) { -int nid, err; - - err =numa_register_meminfo (mi); -if (err) -return err; +int nid;if (!memblock_validate_numa_coverage (SZ_1M ))return -EINVAL ;
@@ -171,39 +167,11 @@ static int __init numa_init(int (*init_func)(void))for (i =0 ; i <MAX_LOCAL_APIC ; i++)set_apicid_to_node (i,NUMA_NO_NODE ); -nodes_clear (numa_nodes_parsed); -nodes_clear (node_possible_map); -nodes_clear (node_online_map); -memset (&numa_meminfo,0 ,sizeof (numa_meminfo)); -WARN_ON (memblock_set_node (0 ,ULLONG_MAX , &memblock.memory, -NUMA_NO_NODE )); -WARN_ON (memblock_set_node (0 ,ULLONG_MAX , &memblock.reserved, -NUMA_NO_NODE )); -/* In case that parsing SRAT failed. */ -WARN_ON (memblock_clear_hotplug (0 ,ULLONG_MAX )); -numa_reset_distance (); - - ret =init_func (); + ret =numa_memblks_init (init_func,/* memblock_force_top_down */ true);if (ret <0 )return ret; -/* -* We reset memblock back to the top-down direction -* here because if we configured ACPI_NUMA, we have -* parsed SRAT in init_func(). It is ok to have the -* reset here even if we did't configure ACPI_NUMA -* or acpi numa init fails and fallbacks to dummy -* numa init. -*/ -memblock_set_bottom_up (false); - - ret =numa_cleanup_meminfo (&numa_meminfo); -if (ret <0 ) -return ret; - -numa_emulation (&numa_meminfo, numa_distance_cnt); - - ret =numa_register_memblks (&numa_meminfo); + ret =numa_register_nodes ();if (ret <0 )return ret;
include/linux/numa_memblks.h +3 -0
@@ -34,6 +34,9 @@ int __init numa_register_meminfo(struct numa_meminfo *mi);void __init numa_nodemask_from_meminfo (nodemask_t *nodemask,const struct numa_meminfo *mi); +int __init numa_memblks_init (int (*init_func)(void ), +bool memblock_force_top_down); +#ifdef CONFIG_NUMA_EMU int numa_emu_cmdline (char *str); void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys,
mm/numa_memblks.c +41 -0
47 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 21/26 mm: numa_memblks: make several functions and variables static
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Make functions and variables that are exclusively used by numa_memblks static.
Move numa_nodemask_from_meminfo() before its callers to avoid forward declaration.
--- include/linux/numa_memblks.h | 8 -------- mm/numa_memblks.c | 36 ++++++++++++++++++------------------ 2 files changed, 18 insertions(+), 26 deletions(-)
include/linux/numa_memblks.h +0 -8
@@ -7,7 +7,6 @@#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) -extern int numa_distance_cnt;void __init numa_set_distance (int from,int to,int distance);void __init numa_reset_distance (void );
@@ -22,17 +21,10 @@ struct numa_meminfo {struct numa_memblk blk[NR_NODE_MEMBLKS ]; }; -extern struct numa_meminfo numa_meminfo __initdata_or_meminfo; -extern struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; -int __init numa_add_memblk (int nodeid,u64 start,u64 end);void __init numa_remove_memblk_from (int idx,struct numa_meminfo *mi);int __init numa_cleanup_meminfo (struct numa_meminfo *mi); -int __init numa_register_meminfo (struct numa_meminfo *mi); - -void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, -const struct numa_meminfo *mi);int __init numa_memblks_init (int (*init_func)(void ),bool memblock_force_top_down);
mm/numa_memblks.c +18 -18
@@ -7,13 +7,27 @@#include <linux/numa.h> #include <linux/numa_memblks.h> -int numa_distance_cnt; +static int numa_distance_cnt;static u8 *numa_distance;nodemask_t numa_nodes_parsed__initdata ; -struct numa_meminfo numa_meminfo __initdata_or_meminfo; -struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; +static struct numa_meminfo numa_meminfo __initdata_or_meminfo; +static struct numa_meminfo numa_reserved_meminfo __initdata_or_meminfo; + +/* +* Set nodes, which have memory in @mi, in *@nodemask. +*/ +static void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, +const struct numa_meminfo *mi) +{ +int i; + +for (i =0 ; i <ARRAY_SIZE (mi->blk); i++) +if (mi->blk[i].start != mi->blk[i].end && + mi->blk[i].nid !=NUMA_NO_NODE ) +node_set (mi->blk[i].nid, *nodemask); +}/** * numa_reset_distance - Reset NUMA distance table
@@ -290,20 +304,6 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi)return 0 ; } -/* -* Set nodes, which have memory in @mi, in *@nodemask. -*/ -void __init numa_nodemask_from_meminfo (nodemask_t *nodemask, -const struct numa_meminfo *mi) -{ -int i; - -for (i =0 ; i <ARRAY_SIZE (mi->blk); i++) -if (mi->blk[i].start != mi->blk[i].end && - mi->blk[i].nid !=NUMA_NO_NODE ) -node_set (mi->blk[i].nid, *nodemask); -} -/* * Mark all currently memblock-reserved physical memory (which covers the * kernel's own memory ranges) as hot-unswappable.
@@ -371,7 +371,7 @@ static void __init numa_clear_kernel_node_hotplug(void) } } -int __init numa_register_meminfo (struct numa_meminfo *mi) +static int __init numa_register_meminfo (struct numa_meminfo *mi) {int i;
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 22/26 mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
numa_cleanup_meminfo() moves blocks outside system RAM to numa_reserved_meminfo and it uses 0 and PFN_PHYS(max_pfn) to determine the memory boundaries.
Replace the memory range boundaries with more portable memblock_start_of_DRAM() and memblock_end_of_DRAM().
--- mm/numa_memblks.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/numa_memblks.c +2 -2
@@ -212,8 +212,8 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)*/ int __init numa_cleanup_meminfo (struct numa_meminfo *mi) { -const u64 low =0 ; -const u64 high =PFN_PHYS (max_pfn); +const u64 low =memblock_start_of_DRAM (); +const u64 high =memblock_end_of_DRAM ();int i, j, k;/* first, trim all entries */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 23/26 of, numa: return -EINVAL when no numa-node-id is found
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Currently of_numa_parse_memory_nodes() returns 0 if no "memory" node in device tree contains "numa-node-id" property. This makes of_numa_init() to return "success" despite no NUMA nodes were actually parsed and set up.
arch_numa workarounds this by returning an error if numa_nodes_parsed is empty.
numa_memblks however would WARN() in such case and since it will be used by arch_numa shortly, such warning is not desirable.
Make sure of_numa_init() returns -EINVAL when no NUMA node information was found in the device tree.
--- drivers/of/of_numa.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/of/of_numa.c +2 -2
@@ -45,7 +45,7 @@ static int __init of_numa_parse_memory_nodes(void)struct device_node *np =NULL ;struct resource rsrc;u32 nid; -int i, r; +int i, r = -EINVAL ;for_each_node_by_type (np,"memory" ) { r =of_property_read_u32 (np,"numa-node-id" , &nid);
@@ -72,7 +72,7 @@ static int __init of_numa_parse_memory_nodes(void) } } -return 0 ; +return r; }static int __init of_numa_parse_distance_map_v1 (struct device_node *map)
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 24/26 arch_numa: switch over to numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Until now arch_numa was directly translating firmware NUMA information to memblock.
Using numa_memblks as an intermediate step has a few advantages:
* alignment with more battle tested x86 implementation
* availability of NUMA emulation
* maintaining node information for not yet populated memory
Replace current functionality related to numa_add_memblk() and __node_distance() with the implementation based on numa_memblks and add functions required by numa_emulation.
--- drivers/base/Kconfig | 1 + drivers/base/arch_numa.c | 201 +++++++++++-------------------------- include/asm-generic/numa.h | 6 +- 3 files changed, 65 insertions(+), 143 deletions(-)
drivers/base/Kconfig +1 -0
@@ -226,6 +226,7 @@ config GENERIC_ARCH_TOPOLOGY config GENERIC_ARCH_NUMA bool + select NUMA_MEMBLKS help Enable support for generic NUMA implementation. Currently, RISC-V and ARM64 use it.
drivers/base/arch_numa.c +60 -141
272 lines not on this page · raw message
include/asm-generic/numa.h +4 -2
@@ -32,8 +32,6 @@ static inline const struct cpumask *cpumask_of_node(int node)void __init arch_numa_init (void );int __init numa_add_memblk (int nodeid,u64 start,u64 end); -void __init numa_set_distance (int from,int to,int distance); -void __init numa_free_distance (void );void __init early_map_cpu_to_node (unsigned int cpu,int nid);int __init early_cpu_to_node (int cpu);void numa_store_cpu_info (unsigned int cpu);
@@ -51,4 +49,8 @@ static inline int early_cpu_to_node(int cpu) { return 0; }#endif /* CONFIG_NUMA */ +#ifdef CONFIG_NUMA_EMU +void debug_cpumask_set_cpu (unsigned int cpu,int node,bool enable); +#endif +#endif /* __ASM_GENERIC_NUMA_H */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 25/26 mm: make range-to-target_node lookup facility a part of numa_memblks
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
The x86 implementation of range-to-target_node lookup (i.e. phys_to_target_node() and memory_add_physaddr_to_nid()) relies on numa_memblks.
Since numa_memblks are now part of the generic code, move these functions from x86 to mm/numa_memblks.c and select CONFIG_NUMA_KEEP_MEMINFO when CONFIG_NUMA_MEMBLKS=y for dax and cxl.
--- arch/x86/include/asm/sparsemem.h | 9 -------- arch/x86/mm/numa.c | 38 -------------------------------- drivers/cxl/Kconfig | 2 +- drivers/dax/Kconfig | 2 +- include/linux/numa_memblks.h | 7 ++++++ mm/numa.c | 1 + mm/numa_memblks.c | 38 ++++++++++++++++++++++++++++++++ 7 files changed, 48 insertions(+), 49 deletions(-)
arch/x86/include/asm/sparsemem.h +0 -9
@@ -31,13 +31,4 @@#endif /* CONFIG_SPARSEMEM */ -#ifndef __ASSEMBLY__ -#ifdef CONFIG_NUMA_KEEP_MEMINFO -extern int phys_to_target_node (phys_addr_t start); -#define phys_to_target_node phys_to_target_node -extern int memory_add_physaddr_to_nid (u64 start); -#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid -#endif -#endif /* __ASSEMBLY__ */ -#endif /* _ASM_X86_SPARSEMEM_H */
arch/x86/mm/numa.c +0 -38
@@ -449,41 +449,3 @@ u64 __init numa_emu_dma_end(void) return PFN_PHYS(MAX_DMA32_PFN); }#endif /* CONFIG_NUMA_EMU */ - -#ifdef CONFIG_NUMA_KEEP_MEMINFO -static int meminfo_to_nid (struct numa_meminfo *mi,u64 start) -{ -int i; - -for (i =0 ; i < mi->nr_blks; i++) -if (mi->blk[i].start <= start && mi->blk[i].end > start) -return mi->blk[i].nid; -return NUMA_NO_NODE ; -} - -int phys_to_target_node (phys_addr_t start) -{ -int nid =meminfo_to_nid (&numa_meminfo, start); - -/* -* Prefer online nodes, but if reserved memory might be -* hot-added continue the search with reserved ranges. -*/ -if (nid !=NUMA_NO_NODE ) -return nid; - -return meminfo_to_nid (&numa_reserved_meminfo, start); -} -EXPORT_SYMBOL_GPL (phys_to_target_node); - -int memory_add_physaddr_to_nid (u64 start) -{ -int nid =meminfo_to_nid (&numa_meminfo, start); - -if (nid ==NUMA_NO_NODE ) - nid = numa_meminfo.blk[0 ].nid; -return nid; -} -EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid); - -#endif
drivers/cxl/Kconfig +1 -1
@@ -6,7 +6,7 @@ menuconfig CXL_BUS select FW_UPLOAD select PCI_DOE select FIRMWARE_TABLE - select NUMA_KEEP_MEMINFO if (NUMA && X86) + select NUMA_KEEP_MEMINFO if NUMA_MEMBLKS help CXL is a bus that is electrically compatible with PCI Express, but layers three protocols on that signalling (CXL.io, CXL.cache, and
drivers/dax/Kconfig +1 -1
@@ -30,7 +30,7 @@ config DEV_DAX_PMEM config DEV_DAX_HMEM tristate "HMEM DAX: direct access to 'specific purpose' memory" depends on EFI_SOFT_RESERVE - select NUMA_KEEP_MEMINFO if (NUMA && X86) + select NUMA_KEEP_MEMINFO if NUMA_MEMBLKS default DEV_DAX help EFI 2.8 platforms, and others, may advertise 'specific purpose'
include/linux/numa_memblks.h +7 -0
@@ -46,6 +46,13 @@ static inline int numa_emu_cmdline(char *str) }#endif /* CONFIG_NUMA_EMU */ +#ifdef CONFIG_NUMA_KEEP_MEMINFO +extern int phys_to_target_node (phys_addr_t start); +#define phys_to_target_node phys_to_target_node +extern int memory_add_physaddr_to_nid (u64 start); +#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid +#endif /* CONFIG_NUMA_KEEP_MEMINFO */ +#endif /* CONFIG_NUMA_MEMBLKS */ #endif /* __NUMA_MEMBLKS_H */
mm/numa.c +1 -0
@@ -3,6 +3,7 @@#include <linux/memblock.h> #include <linux/printk.h> #include <linux/numa.h> +#include <linux/numa_memblks.h> struct pglist_data *node_data[MAX_NUMNODES ];EXPORT_SYMBOL (node_data);
mm/numa_memblks.c +38 -0
@@ -528,3 +528,41 @@ int __init numa_fill_memblks(u64 start, u64 end) }return 0 ; } + +#ifdef CONFIG_NUMA_KEEP_MEMINFO +static int meminfo_to_nid (struct numa_meminfo *mi,u64 start) +{ +int i; + +for (i =0 ; i < mi->nr_blks; i++) +if (mi->blk[i].start <= start && mi->blk[i].end > start) +return mi->blk[i].nid; +return NUMA_NO_NODE ; +} + +int phys_to_target_node (phys_addr_t start) +{ +int nid =meminfo_to_nid (&numa_meminfo, start); + +/* +* Prefer online nodes, but if reserved memory might be +* hot-added continue the search with reserved ranges. +*/ +if (nid !=NUMA_NO_NODE ) +return nid; + +return meminfo_to_nid (&numa_reserved_meminfo, start); +} +EXPORT_SYMBOL_GPL (phys_to_target_node); + +int memory_add_physaddr_to_nid (u64 start) +{ +int nid =meminfo_to_nid (&numa_meminfo, start); + +if (nid ==NUMA_NO_NODE ) + nid = numa_meminfo.blk[0 ].nid; +return nid; +} +EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid); + +#endif /* CONFIG_NUMA_KEEP_MEMINFO */
Signature · 1 line
2.43.0
Mike Rapoport
maintainerPATCH v3 26/26 docs: move numa=fake description to kernel-parameters.txt
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
NUMA emulation can be now enabled on arm64 and riscv in addition to x86.
Move description of numa=fake parameters from x86 documentation of admin-guide/kernel-parameters.txt
--- Documentation/admin-guide/kernel-parameters.txt | 15 +++++++++++++++ Documentation/arch/x86/x86_64/boot-options.rst | 12 ------------ 2 files changed, 15 insertions(+), 12 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +15 -0
@@ -4123,6 +4123,21 @@ Disable NUMA, Only set up a single NUMA node spanning all memory. + numa=fake=<size>[MG] + [KNL, ARM64, RISCV, X86, EARLY] + If given as a memory unit, fills all system RAM with + nodes of size interleaved over physical nodes. + + numa=fake=<N> + [KNL, ARM64, RISCV, X86, EARLY] + If given as an integer, fills all system RAM with N + fake nodes interleaved over physical nodes. + + numa=fake=<N>U + [KNL, ARM64, RISCV, X86, EARLY] + If given as an integer followed by 'U', it will + divide each physical node into N emulated nodes. + numa_balancing= [KNL,ARM64,PPC,RISCV,S390,X86] Enable or disable automatic NUMA balancing. Allowed values are enable and disable
Documentation/arch/x86/x86_64/boot-options.rst +0 -12
@@ -170,18 +170,6 @@ NUMA Don't parse the HMAT table for NUMA setup, or soft-reserved memory partitioning. - numa=fake=<size>[MG] - If given as a memory unit, fills all system RAM with nodes of - size interleaved over physical nodes. - - numa=fake=<N> - If given as an integer, fills all system RAM with N fake nodes - interleaved over physical nodes. - - numa=fake=<N>U - If given as an integer followed by 'U', it will divide each - physical node into N emulated nodes. - ACPI ====
Signature · 1 line
2.43.0
Re: [PATCH v3 02/26] MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
In reply to Mike RapoportQuoted 5 lines · On Thu, 1 Aug 2024 09:08:02 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
sgi-ip27 is the only system that defines
NODE_DATA()differently than the rest of NUMA machines.
Quoted 8 lines
Add
node_dataarray of struct pglist pointers that will point to __node_data[node]->pglist and redefineNODE_DATA()to usenode_dataarray.This will allow pulling declaration of
node_datato the generic mm code in the next commit.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
After staring for a while at the use made of the other part of the __node_data I think what you have in this an the next two patches is fine.
I'm far from convinced it was correct before though as arch_refresh_node_data() called on offline nodes in free_area_init() would have replaced __node_data with an allocation of size pg_data_t but the hub_data(), visible below, is after that. Maybe hub_data() as never called for offline nodes, but
I couldn't establish that.
After these patches the arch_refresh_node_data() generic version will only be replacing the pointer in node_data leaving the hub_data where it was in the first place and thus is fine.
So with that in mind (and it could be completely wrong ;)
Quoted 45 lines
--- arch/mips/include/asm/mach-ip27/mmzone.h | 5 ++++- arch/mips/sgi-ip27/ip27-memory.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)arch/mips/include/asm/mach-ip27/mmzone.h +4
-1@@ -22,7 +22,10 @@ struct node_data {extern struct node_data *__node_data[]; -#define NODE_DATA (n) (&__node_data[(n)]->pglist)#define hub_data (n) (&__node_data[(n)]->hub) +extern struct pglist_data *node_data[]; + +#define NODE_DATA (nid) (node_data[nid]) +#endif /* _ASM_MACH_MMZONE_H */ arch/mips/sgi-ip27/ip27-memory.c +4
-1@@ -34,8 +34,10 @@#define SLOT_PFNSHIFT (SLOT_SHIFT - PAGE_SHIFT)#define PFN_NASIDSHFT (NASID_SHFT - PAGE_SHIFT) -struct node_data *__node_data[MAX_NUMNODES ]; +struct pglist_data *node_data[MAX_NUMNODES ]; +EXPORT_SYMBOL (node_data); +struct node_data *__node_data[MAX_NUMNODES ];EXPORT_SYMBOL (__node_data);static u64 gen_region_mask (void )@@ -361,6 +363,7 @@ static void __init node_mem_init(nasid_t node)*/ __node_data[node] =__va (slot_freepfn <<PAGE_SHIFT );memset (__node_data[node],0 ,PAGE_SIZE ); + node_data[node] = &__node_data[node]->pglist;NODE_DATA (node)->node_start_pfn = start_pfn;NODE_DATA (node)->node_spanned_pages = end_pfn - start_pfn;
Re: [PATCH v3 03/26] MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
In reply to Mike RapoportQuoted 8 lines · On Thu, 1 Aug 2024 09:08:03 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
For SGI IP27 machines
node_possible_mapis statically set toNODE_MASK_ALLand it is not updated during NUMA initialization.Ensure that it only contains nodes present in the system.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 22 lines
--- arch/mips/sgi-ip27/ip27-smp.c | 2 ++ 1 file changed, 2 insertions(+)arch/mips/sgi-ip27/ip27-smp.c +2
-0@@ -70,11 +70,13 @@ void cpu_node_probe(void)gda_t *gdap =GDA ;nodes_clear (node_online_map); +nodes_clear (node_possible_map);for (i =0 ; i <MAX_NUMNODES ; i++) {nasid_t nasid = gdap->g_nasidtable[i];if (nasid ==INVALID_NASID )break ;node_set_online (nasid); +node_set (nasid, node_possible_map); highest =node_scan_cpus (nasid, highest); }
Re: [PATCH v3 04/26] MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportQuoted 7 lines · On Thu, 1 Aug 2024 09:08:04 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Commit
f8f9f21c7848("MIPS: Fix build error for loongson64 and sgi-ip27") addedHAVE_ARCH_NODEDATA_EXTENSIONto sgi-ip27 to silence a compilation error that happened because sgi-ip27 didn't define array ofpg_data_tasnode_datalike most other architectures did.
Quoted 6 lines
After addition of
node_dataarray that matches other architectures and after ensuring that offline nodes do not appear onnode_possible_map, it is safe to droparch_alloc_nodedata()andHAVE_ARCH_NODEDATA_EXTENSIONfrom sgi-ip27.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 35 lines
--- arch/mips/Kconfig | 1 - arch/mips/sgi-ip27/ip27-memory.c | 10 ---------- 2 files changed, 11 deletions(-)arch/mips/Kconfig +0
-1@@ -735,7 +735,6 @@ config SGI_IP27 select WAR_R10000_LLSC select MIPS_L1_CACHE_SHIFT_7 select NUMA - select HAVE_ARCH_NODEDATA_EXTENSION help This are the SGI Origin 200, Origin 2000 and Onyx 2 Graphics workstations. To compile a Linux kernel that runs on these, say Yarch/mips/sgi-ip27/ip27-memory.c +0
-10@@ -426,13 +426,3 @@ void __init mem_init(void)memblock_free_all ();setup_zero_pages ();/* This comes from node 0 */ } - -pg_data_t *__init arch_alloc_nodedata (int nid) -{ -return memblock_alloc (sizeof (pg_data_t),SMP_CACHE_BYTES ); -} - -void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - __node_data[nid] = (struct node_data *)pgdat; -}
Re: [PATCH v3 06/26] MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportQuoted 7 lines · On Thu, 1 Aug 2024 09:08:06 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Commit
f8f9f21c7848("MIPS: Fix build error for loongson64 and sgi-ip27") addedHAVE_ARCH_NODEDATA_EXTENSIONto loongson64 to silence a compilation error that happened because loongson64 didn't define array ofpg_data_tasnode_datalike most other architectures did.
Quoted 7 lines
After rename of
__node_datatonode_dataarch_alloc_nodedata()andHAVE_ARCH_NODEDATA_EXTENSIONcan be dropped from loongson64.Since it was the only user of
HAVE_ARCH_NODEDATA_EXTENSIONconfig option also remove this option fromarch/mips/Kconfig.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
These are as you say now identical to the generic form, so don't need a special version for any reason I can see.
Quoted 45 lines
--- arch/mips/Kconfig | 4 ---- arch/mips/loongson64/numa.c | 10 ---------- 2 files changed, 14 deletions(-)arch/mips/Kconfig +0
-4@@ -502,7 +502,6 @@ config MACH_LOONGSON64 select USE_OF select BUILTIN_DTB select PCI_HOST_GENERIC - select HAVE_ARCH_NODEDATA_EXTENSION if NUMA help This enables the support of Loongson-2/3 family of machines.@@ -2612,9 +2611,6 @@ config NUMA config SYS_SUPPORTS_NUMA bool -config HAVE_ARCH_NODEDATA_EXTENSION - bool - config RELOCATABLE bool "Relocatable kernel" depends on SYS_SUPPORTS_RELOCATABLEarch/mips/loongson64/numa.c +0
-10@@ -198,13 +198,3 @@ void __init prom_init_numa_memory(void)pr_info ("CP0_PageGrain: CP0 5.1 (0x%x)\n" ,read_c0_pagegrain ());prom_meminit (); } - -pg_data_t *__init arch_alloc_nodedata (int nid) -{ -return memblock_alloc (sizeof (pg_data_t),SMP_CACHE_BYTES ); -} - -void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - node_data[nid] = pgdat; -}
Re: [PATCH v3 07/26] mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportQuoted 6 lines · On Thu, 1 Aug 2024 09:08:07 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
There are no users of
HAVE_ARCH_NODEDATA_EXTENSIONleft, soarch_alloc_nodedata()andarch_refresh_nodedata()are not needed anymore.
Quoted 6 lines
Replace the call to
arch_alloc_nodedata()infree_area_init()withmemblock_alloc(), removearch_refresh_nodedata()and cleanupinclude/linux/memory_hotplug.hfrom the associated ifdefery.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Hi Mike,
This has an accidental (I assume) functional change and if you have an initially offline node it all goes wrong.
Quoted 69 lines
--- include/linux/memory_hotplug.h | 48 ---------------------------------- mm/mm_init.c | 3 +-- 2 files changed, 1 insertion(+), 50 deletions(-)include/linux/memory_hotplug.h +0
-48@@ -16,54 +16,6 @@ struct resource;struct vmem_altmap ;struct dev_pagemap ; -#ifdef CONFIG_HAVE_ARCH_NODEDATA_EXTENSION -/* -* For supporting node-hotadd, we have to allocate a new pgdat. -* -* If an arch has generic style NODE_DATA(), -* node_data[nid] = kzalloc() works well. But it depends on the architecture. -* -* In general, generic_alloc_nodedata() is used. -* -*/ -extern pg_data_t *arch_alloc_nodedata (int nid); -extern void arch_refresh_nodedata (int nid,pg_data_t *pgdat); - -#else /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ - -#define arch_alloc_nodedata (nid) generic_alloc_nodedata(nid) - -#ifdef CONFIG_NUMA -/* -* XXX: node aware allocation can't work well to get new node's memory at this time. -* Because, pgdat for the new node is not allocated/initialized yet itself. -* To use new node's memory, more consideration will be necessary. -*/ -#define generic_alloc_nodedata (nid) \ -({ \ - memblock_alloc(sizeof(*pgdat), SMP_CACHE_BYTES); \ -}) - -extern pg_data_t *node_data[]; -static inline void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ - node_data[nid] = pgdat; -} - -#else /* !CONFIG_NUMA */ - -/* never called */ -static inline pg_data_t *generic_alloc_nodedata (int nid) -{ -BUG (); -return NULL ; -} -static inline void arch_refresh_nodedata (int nid,pg_data_t *pgdat) -{ -} -#endif /* CONFIG_NUMA */ -#endif /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ -#ifdef CONFIG_MEMORY_HOTPLUG struct page *pfn_to_online_page (unsigned long pfn);mm/mm_init.c +0
-0
mm/mm_init.c void __init free_area_init(unsigned long *max_zone_pfn) · line 1845
Quoted 9 lines
mm/mm_init.c +1
-2@@ -1838,11 +1838,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)if (!node_online (nid)) {/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES );if (!pgdat)panic ("Cannot allocate %zuB for node %d.\n" ,sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat);
This allocates pgdat but never sets node_data[nid] to it and promptly leaks it on the line below.
Just to sanity check this I spun up a qemu machine with no memory initially present on some nodes and it went boom as you'd expect.
I tested with addition of
NODE_DATA(nid) = pgdat;
and it all seems to work as expected.
Jonathan
Quoted 3 lines
mm/mm_init.c +0
-0} pgdat =NODE_DATA (nid);
Re: [PATCH v3 09/26] arch, mm: pull out allocation of NODE_DATA to generic code
In reply to Mike RapoportQuoted 9 lines · On Thu, 1 Aug 2024 09:08:09 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Architectures that support NUMA duplicate the code that allocates
NODE_DATAon the node-local memory with slight variations in reporting of the addresses where the memory was allocated.Use x86 version as the basis for the generic
alloc_node_data()function and call this function in architecture specific numa initialization.
Quoted 8 lines
Round up node data size to
SMP_CACHE_BYTESrather than toPAGE_SIZElike x86 used to do since the bootmem era when allocation granularity wasPAGE_SIZEanyway.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>
One comment unrelated to this patch set as such, just made more obvious by it.
arch/powerpc/mm/numa.c void __init dump_numa_cpu_topology(void) · line 1095
Quoted 8 lines
arch/powerpc/mm/numa.c +0
-0@@ -1093,27 +1093,9 @@ void __init dump_numa_cpu_topology(void)static void __init setup_node_data (int nid,u64 start_pfn,u64 end_pfn) {u64 spanned_pages = end_pfn - start_pfn;
Trivial, but might as well squash this local variable into the single place it's used.
Quoted 27 lines
arch/powerpc/mm/numa.c +3
-21-const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES ); -u64 nd_pa; -void *nd; -int tnid; - - nd_pa =memblock_phys_alloc_try_nid (nd_size,SMP_CACHE_BYTES , nid); -if (!nd_pa) -panic ("Cannot allocate %zu bytes for node %d data\n" , - nd_size, nid); - - nd =__va (nd_pa); - -/* report and initialize */ -pr_info (" NODE_DATA [mem %#010Lx-%#010Lx]\n" , - nd_pa, nd_pa + nd_size -1 ); - tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) -pr_info (" NODE_DATA(%d) on node %d\n" , nid, tnid); - - node_data[nid] = nd; -memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); + +alloc_node_data (nid); +NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start_pfn;NODE_DATA (nid)->node_spanned_pages = spanned_pages;
Re: [PATCH v3 10/26] x86/numa: simplify numa_distance allocation
In reply to Mike RapoportQuoted 5 lines · On Thu, 1 Aug 2024 09:08:10 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Allocation of
numa_distanceusesmemblock_phys_alloc_range()to limit allocation to be below the last mapped page.
Quoted 8 lines
But NUMA initializaition runs after the direct map is populated and there is also code in
setup_arch()that adjusts memblock limit to reflect how much memory is already mapped in the direct map.Simplify the allocation of
numa_distanceand use plainmemblock_alloc().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Seems sensible. FWIW (which might just be me not bothering to read this one again ;)
Re: [PATCH v3 11/26] x86/numa: use get_pfn_range_for_nid to verify that node spans memory
In reply to Mike RapoportQuoted 9 lines · On Thu, 1 Aug 2024 09:08:11 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of looping over
numa_meminfoarray to detect node's start and end addresses useget_pfn_range_for_init().This is shorter and make it easier to lift
numa_memblksto generic code.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Fair enough given code a few lines up has set the node for all the memblocks so this should I think give the same effective result.
Quoted 29 lines
--- arch/x86/mm/numa.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-)arch/x86/mm/numa.c +3
-10@@ -521,17 +521,10 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start =PFN_PHYS (max_pfn); -u64 end =0 ; +unsigned long start_pfn, end_pfn; -for (i =0 ; i < mi->nr_blks; i++) { -if (nid != mi->blk[i].nid) -continue ; - start =min (mi->blk[i].start, start); - end =max (mi->blk[i].end, end); - } - -if (start >= end) +get_pfn_range_for_nid (nid, &start_pfn, &end_pfn); +if (start_pfn >= end_pfn)continue ;alloc_node_data (nid);
Quoted 11 lines · On Thu, 1 Aug 2024 09:08:17 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with
numa_memblksfromarch/x86tomm/and addKconfigoptions to let x86 select it in itsKconfig.This code will be later reused by
arch_numa.No functional changes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Re: [PATCH v3 18/26] mm: move numa_distance and related code from x86 to numa_memblks
In reply to Mike RapoportQuoted 11 lines · On Thu, 1 Aug 2024 09:08:18 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with
numa_distancearray fromarch/x86tomm/numa_memblks.cThis code will be later reused by
arch_numa.No functional changes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
As you say, simple code move and I'll cope with the confusion of stuff that isn't numa_memblk in that file given the convenience.
Re: [PATCH v3 20/26] mm: numa_memblks: introduce numa_memblks_init
In reply to Mike RapoportQuoted 10 lines · On Thu, 1 Aug 2024 09:08:20 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move most of
x86::numa_init()tonuma_memblksso that the latter will be more self-contained.With this
numa_memblkdata structures should not be exposed to the architecture specific code.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Just code motion as expected.
Re: [PATCH v3 21/26] mm: numa_memblks: make several functions and variables static
In reply to Mike RapoportQuoted 10 lines · On Thu, 1 Aug 2024 09:08:21 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Make functions and variables that are exclusively used by
numa_memblksstatic.Move
numa_nodemask_from_meminfo()before its callers to avoid forward declaration.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Good. I was wondering why some of this internal detail was in the header. Much nicer after this.
Re: [PATCH v3 22/26] mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo
In reply to Mike RapoportQuoted 11 lines · On Thu, 1 Aug 2024 09:08:22 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
numa_cleanup_meminfo()moves blocks outside system RAM tonuma_reserved_meminfoand it uses 0 andPFN_PHYS(max_pfn)to determine the memory boundaries.Replace the memory range boundaries with more portable
memblock_start_of_DRAM()andmemblock_end_of_DRAM().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Makes sense
Quoted 19 lines
--- mm/numa_memblks.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)mm/numa_memblks.c +2
-2@@ -212,8 +212,8 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)*/ int __init numa_cleanup_meminfo (struct numa_meminfo *mi) { -const u64 low =0 ; -const u64 high =PFN_PHYS (max_pfn); +const u64 low =memblock_start_of_DRAM (); +const u64 high =memblock_end_of_DRAM ();int i, j, k;/* first, trim all entries */
Re: [PATCH v3 19/26] mm: introduce numa_emulation
In reply to Mike RapoportQuoted 10 lines · On Thu, 1 Aug 2024 09:08:19 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move
numa_emulationcodfromarch/x86tomm/numa_emulation.cThis code will be later reused by
arch_numa.No functional changes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I ran some basic tests on ARM with this. Seems to do the job.
Works on both ACPI and dsdt boots.
Re: [PATCH v3 23/26] of, numa: return -EINVAL when no numa-node-id is found
In reply to Mike RapoportQuoted 10 lines · On Thu, 1 Aug 2024 09:08:23 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Currently
of_numa_parse_memory_nodes()returns 0 if no "memory" node in device tree contains "numa-node-id" property. This makesof_numa_init()to return "success" despite no NUMA nodes were actually parsed and set up.
arch_numaworkarounds this by returning an error ifnuma_nodes_parsedis empty.
Quoted 7 lines
numa_memblkshowever wouldWARN()in such case and since it will be used byarch_numashortly, such warning is not desirable.Make sure
of_numa_init()returns-EINVALwhen no NUMA node information was found in the device tree.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Re: [PATCH v3 24/26] arch_numa: switch over to numa_memblks
In reply to Mike RapoportQuoted 10 lines · On Thu, 1 Aug 2024 09:08:24 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Until now
arch_numawas directly translating firmware NUMA information to memblock.Using
numa_memblksas an intermediate step has a few advantages:
* alignment with more battle tested x86 implementation
* availability of NUMA emulation
* maintaining node information for not yet populated memory
Quoted 6 lines
Replace current functionality related to
numa_add_memblk()and__node_distance()with the implementation based onnuma_memblksand add functions required bynuma_emulation.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Re: [PATCH v3 26/26] docs: move numa=fake description to kernel-parameters.txt
In reply to Mike RapoportQuoted 9 lines · On Thu, 1 Aug 2024 09:08:26 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
NUMA emulation can be now enabled on arm64 and riscv in addition to x86.
Move description of numa=fake parameters from x86 documentation of
admin-guide/kernel-parameters.txtSuggested-by Zi Yan <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 13 lines · On Thu, 1 Aug 2024 09:08:00 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following the discussion about handling of CXL fixed memory windows on arm64 [1] I decided to bite the bullet and move
numa_memblksfrom x86 to the generic code so they will be available on arm64/riscv and maybe on loongarch sometime later.While it could be possible to use memblock to describe CXL memory windows, it currently lacks notion of unpopulated memory ranges and
numa_memblksdoes implement this.
Quoted 8 lines
Another reason to make
numa_memblksgeneric is that botharch_numa(arm64 and riscv) and loongarch use trimmed copy of x86 code although there is no fundamental reason why the same code cannot be used on all these platforms. Havingnuma_memblksinmm/will make it's interaction with ACPI and FDT more consistent and I believe will reduce maintenance burden.And with generic
numa_memblksit is (almost) straightforward to enable NUMA emulation on arm64 and riscv.
With that one fix in patch 7.
Feel free to figure out which patches actually got tested by that (or tag them all - I'll pretend I tested ip27 :)
Jonathan
Andrew Morton
maintainerRe: [PATCH v3 07/26] mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
In reply to Jonathan CameronQuoted 16 lines · On Fri, 2 Aug 2024 10:49:22 +0100 Jonathan Cameron…
mm/mm_init.c void __init free_area_init(unsigned long *max_zone_pfn) · line 1845
Quoted 12 lines
mm/mm_init.c +1
-2@@ -1838,11 +1838,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)if (!node_online (nid)) {/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES );if (!pgdat)panic ("Cannot allocate %zuB for node %d.\n" ,sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat);This allocates pgdat but never sets node_data[nid] to it and promptly leaks it on the line below.
Quoted 6 lines
Just to sanity check this I spun up a qemu machine with no memory initially present on some nodes and it went boom as you'd expect.
I tested with addition of
NODE_DATA(nid) = pgdat;and it all seems to work as expected.
Thanks, I added that. It blew up on x86_64 allnoconfig because node_data[] (and hence NODE_DATA()) isn't an lvalue when CONFIG_NUMA=n.
I'll put some #ifdef CONFIG_NUMAs in there for now but
a) NODE_DATA() is upper-case. Implies "constant". Shouldn't be assigned to.
b) NODE_DATA() should be non-lvalue when CONFIG_NUMA=y also. But no, we insist on implementing things in cpp instead of in C.
c) In fact assigning to anything which ends in "()" is nuts. Please clean up my tempfix.
c) Mike, generally I'm wondering if there's a bunch of code here which isn't needed on CONFIG_NUMA=n. Please check all of this for unneeded bloatiness.
Mike Rapoport
maintainerRe: [PATCH v3 07/26] mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
Quoted 28 lines · Andrew Morton
Quoted 16 lines · On Fri, 2 Aug 2024 10:49:22 +0100 Jonathan Cameron…
mm/mm_init.c void __init free_area_init(unsigned long *max_zone_pfn) · line 1845
Quoted 12 lines
mm/mm_init.c +1
-2@@ -1838,11 +1838,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)if (!node_online (nid)) {/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES );if (!pgdat)panic ("Cannot allocate %zuB for node %d.\n" ,sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat);This allocates pgdat but never sets node_data[nid] to it and promptly leaks it on the line below.
Quoted 6 lines
Just to sanity check this I spun up a qemu machine with no memory initially present on some nodes and it went boom as you'd expect.
I tested with addition of
NODE_DATA(nid) = pgdat;and it all seems to work as expected.
Thanks, I added that. It blew up on
x86_64allnoconfig because node_data[] (and henceNODE_DATA()) isn't an lvalue whenCONFIG_NUMA=n.
Quoted 6 lines
I'll put some #ifdef
CONFIG_NUMAsin there for now buta)
NODE_DATA()is upper-case. Implies "constant". Shouldn't be assigned to.b)
NODE_DATA()should be non-lvalue whenCONFIG_NUMA=yalso. But no, we insist on implementing things in cpp instead of in C.
This looks like a candidate for a separate tree-wide cleanup.
Quoted 6 lines
c) In fact assigning to anything which ends in "()" is nuts. Please clean up my tempfix.
c) Mike, generally I'm wondering if there's a bunch of code here which isn't needed on
CONFIG_NUMA=n. Please check all of this for unneeded bloatiness.
I believe the patch addresses your concerns, just with this the commit log needs update. Instead of
Replace the call to arch_alloc_nodedata() in free_area_init() with memblock_alloc(), remove arch_refresh_nodedata() and cleanup include/linux/memory_hotplug.h from the associated ifdefery.
it should be
Replace the call to arch_alloc_nodedata() in free_area_init() with a new helper alloc_offline_node_data(), remove arch_refresh_nodedata() and cleanup include/linux/memory_hotplug.h from the associated ifdefery.
I can send an updated patch if you prefer.
include/linux/numa.h +3 -0
@@ -34,6 +34,7 @@ extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid])void __init alloc_node_data (int nid); +void __init alloc_offline_node_data (int nit);/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);
@@ -62,6 +63,8 @@ static inline int phys_to_target_node(u64 start) {return 0 ; } + +static inline void alloc_offline_node_data (int nit) {}#endif #define numa_map_to_online_node (node) numa_nearest_node(node, N_ONLINE)
mm/mm_init.c +2 -7
@@ -1836,13 +1836,8 @@ void __init free_area_init(unsigned long *max_zone_pfn)for_each_node (nid) {pg_data_t *pgdat; -if (!node_online (nid)) { -/* Allocator not initialized yet */ - pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); -if (!pgdat) -panic ("Cannot allocate %zuB for node %d.\n" , -sizeof (*pgdat), nid); - } +if (!node_online (nid)) +alloc_offline_node_data (nid); pgdat =NODE_DATA (nid);free_area_init_node (nid);
mm/numa.c +12 -0
@@ -34,6 +34,18 @@ void __init alloc_node_data(int nid)memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); } +void __init alloc_offline_node_data (int nit) +{ +pg_data_t *pgdat; + + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); +if (!pgdat) +panic ("Cannot allocate %zuB for node %d.\n" , +sizeof (*pgdat), nid); + + node_data[nid] = pgdat; +} +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 07/26] mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
Quoted 52 lines · On Sun, 4 Aug 2024 10:24:15 +0300 Mike Rapoport <[email protected]> wrote:
Quoted 28 lines · Andrew Morton
Quoted 16 lines · On Fri, 2 Aug 2024 10:49:22 +0100 Jonathan Cameron…
mm/mm_init.c void __init free_area_init(unsigned long *max_zone_pfn) · line 1845
Quoted 12 lines
mm/mm_init.c +1
-2@@ -1838,11 +1838,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)if (!node_online (nid)) {/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES );if (!pgdat)panic ("Cannot allocate %zuB for node %d.\n" ,sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat);This allocates pgdat but never sets node_data[nid] to it and promptly leaks it on the line below.
Quoted 6 lines
Just to sanity check this I spun up a qemu machine with no memory initially present on some nodes and it went boom as you'd expect.
I tested with addition of
NODE_DATA(nid) = pgdat;and it all seems to work as expected.
Thanks, I added that. It blew up on
x86_64allnoconfig because node_data[] (and henceNODE_DATA()) isn't an lvalue whenCONFIG_NUMA=n.Quoted 6 lines
I'll put some #ifdef
CONFIG_NUMAsin there for now buta)
NODE_DATA()is upper-case. Implies "constant". Shouldn't be assigned to.b)
NODE_DATA()should be non-lvalue whenCONFIG_NUMA=yalso. But no, we insist on implementing things in cpp instead of in C.This looks like a candidate for a separate tree-wide cleanup.
Quoted 6 lines
c) In fact assigning to anything which ends in "()" is nuts. Please clean up my tempfix.
c) Mike, generally I'm wondering if there's a bunch of code here which isn't needed on
CONFIG_NUMA=n. Please check all of this for unneeded bloatiness.I believe the patch addresses your concerns, just with this the commit log needs update. Instead of
Replace the call to
arch_alloc_nodedata()infree_area_init()withmemblock_alloc(), removearch_refresh_nodedata()and cleanupinclude/linux/memory_hotplug.hfrom the associated ifdefery.
Quoted 8 lines
it should be
Replace the call to
arch_alloc_nodedata()infree_area_init()with a new helperalloc_offline_node_data(), removearch_refresh_nodedata()and cleanupinclude/linux/memory_hotplug.hfrom the associated ifdefery.I can send an updated patch if you prefer.
This solution looks good to me - except for a Freudian typo that means it won't compile :)
Jonathan
Quoted 14 lines
include/linux/numa.h +1
-0@@ -34,6 +34,7 @@ extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid])void __init alloc_node_data (int nid); +void __init alloc_offline_node_data (int nit);/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);
include/linux/numa.h static inline int phys_to_target_node(u64 start) · line 67
Quoted 5 lines
include/linux/numa.h +2
-0@@ -62,6 +63,8 @@ static inline int phys_to_target_node(u64 start) {return 0 ; } + +static inline void alloc_offline_node_data (int nit) {}
nid
Quoted 28 lines
include/linux/numa.h +0
-0#endif #define numa_map_to_online_node (node) numa_nearest_node(node, N_ONLINE)mm/mm_init.c +2
-7@@ -1836,13 +1836,8 @@ void __init free_area_init(unsigned long *max_zone_pfn)for_each_node (nid) {pg_data_t *pgdat; -if (!node_online (nid)) { -/* Allocator not initialized yet */ - pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); -if (!pgdat) -panic ("Cannot allocate %zuB for node %d.\n" , -sizeof (*pgdat), nid); - } +if (!node_online (nid)) +alloc_offline_node_data (nid); pgdat =NODE_DATA (nid);free_area_init_node (nid);mm/numa.c +0
-0
mm/numa.c void __init alloc_node_data(int nid) · line 37
Quoted 4 lines
mm/numa.c +1
-0@@ -34,6 +34,18 @@ void __init alloc_node_data(int nid)memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); } +void __init alloc_offline_node_data (int nit)
nid
Quoted 17 lines
mm/numa.c +11
-0+{ +pg_data_t *pgdat; + + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); +if (!pgdat) +panic ("Cannot allocate %zuB for node %d.\n" , +sizeof (*pgdat), nid); + + node_data[nid] = pgdat; +} +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Mike Rapoport
maintainerRe: [PATCH v3 07/26] mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
Quoted 57 lines · Jonathan Cameron
Quoted 54 lines · On Sun, 4 Aug 2024 10:24:15 +0300 Mike Rapoport <[email protected]> wrote:
Quoted 28 lines · Andrew Morton
Quoted 16 lines · On Fri, 2 Aug 2024 10:49:22 +0100 Jonathan Cameron…
mm/mm_init.c void __init free_area_init(unsigned long *max_zone_pfn) · line 1845
Quoted 12 lines
mm/mm_init.c +1
-2@@ -1838,11 +1838,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)if (!node_online (nid)) {/* Allocator not initialized yet */ - pgdat =arch_alloc_nodedata (nid); + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES );if (!pgdat)panic ("Cannot allocate %zuB for node %d.\n" ,sizeof (*pgdat), nid); -arch_refresh_nodedata (nid, pgdat);This allocates pgdat but never sets node_data[nid] to it and promptly leaks it on the line below.
Quoted 6 lines
Just to sanity check this I spun up a qemu machine with no memory initially present on some nodes and it went boom as you'd expect.
I tested with addition of
NODE_DATA(nid) = pgdat;and it all seems to work as expected.
Thanks, I added that. It blew up on
x86_64allnoconfig because node_data[] (and henceNODE_DATA()) isn't an lvalue whenCONFIG_NUMA=n.Quoted 6 lines
I'll put some #ifdef
CONFIG_NUMAsin there for now buta)
NODE_DATA()is upper-case. Implies "constant". Shouldn't be assigned to.b)
NODE_DATA()should be non-lvalue whenCONFIG_NUMA=yalso. But no, we insist on implementing things in cpp instead of in C.This looks like a candidate for a separate tree-wide cleanup.
Quoted 6 lines
c) In fact assigning to anything which ends in "()" is nuts. Please clean up my tempfix.
c) Mike, generally I'm wondering if there's a bunch of code here which isn't needed on
CONFIG_NUMA=n. Please check all of this for unneeded bloatiness.I believe the patch addresses your concerns, just with this the commit log needs update. Instead of
Replace the call to
arch_alloc_nodedata()infree_area_init()withmemblock_alloc(), removearch_refresh_nodedata()and cleanupinclude/linux/memory_hotplug.hfrom the associated ifdefery.it should be
Quoted 8 lines
Quoted 6 lines
Replace the call to
arch_alloc_nodedata()infree_area_init()with a new helperalloc_offline_node_data(), removearch_refresh_nodedata()and cleanupinclude/linux/memory_hotplug.hfrom the associated ifdefery.I can send an updated patch if you prefer.
This solution looks good to me - except for a Freudian typo that means it won't compile :)
Right :)
I'll post v4 after kbuild confirms it compiles :)
Quoted 76 lines
Jonathan
Quoted 14 lines
include/linux/numa.h +1
-0@@ -34,6 +34,7 @@ extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid])void __init alloc_node_data (int nid); +void __init alloc_offline_node_data (int nit);/* Generic implementation available */ int numa_nearest_node (int node,unsigned int state);include/linux/numa.h static inline int phys_to_target_node(u64 start) · line 67
Quoted 5 lines
include/linux/numa.h +2
-0@@ -62,6 +63,8 @@ static inline int phys_to_target_node(u64 start) {return 0 ; } + +static inline void alloc_offline_node_data (int nit) {}nid
Quoted 28 lines
include/linux/numa.h +0
-0#endif #define numa_map_to_online_node (node) numa_nearest_node(node, N_ONLINE)mm/mm_init.c +2
-7@@ -1836,13 +1836,8 @@ void __init free_area_init(unsigned long *max_zone_pfn)for_each_node (nid) {pg_data_t *pgdat; -if (!node_online (nid)) { -/* Allocator not initialized yet */ - pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); -if (!pgdat) -panic ("Cannot allocate %zuB for node %d.\n" , -sizeof (*pgdat), nid); - } +if (!node_online (nid)) +alloc_offline_node_data (nid); pgdat =NODE_DATA (nid);free_area_init_node (nid);mm/numa.c +0
-0mm/numa.c void __init alloc_node_data(int nid) · line 37
Quoted 4 lines
mm/numa.c +1
-0@@ -34,6 +34,18 @@ void __init alloc_node_data(int nid)memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); } +void __init alloc_offline_node_data (int nit)nid
Quoted 17 lines
mm/numa.c +11
-0+{ +pg_data_t *pgdat; + + pgdat =memblock_alloc (sizeof (*pgdat),SMP_CACHE_BYTES ); +if (!pgdat) +panic ("Cannot allocate %zuB for node %d.\n" , +sizeof (*pgdat), nid); + + node_data[nid] = pgdat; +} +/* Stub functions: */ #ifndef memory_add_physaddr_to_nid
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerRe: [PATCH v3 09/26] arch, mm: pull out allocation of NODE_DATA to generic code
In reply to Jonathan CameronQuoted 38 lines · Jonathan Cameron
Quoted 9 lines · On Thu, 1 Aug 2024 09:08:09 +0300 Mike Rapoport <[email protected]> wrote:
From: "Mike Rapoport (Microsoft)" <[email protected]>
Architectures that support NUMA duplicate the code that allocates
NODE_DATAon the node-local memory with slight variations in reporting of the addresses where the memory was allocated.Use x86 version as the basis for the generic
alloc_node_data()function and call this function in architecture specific numa initialization.Quoted 8 lines
Round up node data size to
SMP_CACHE_BYTESrather than toPAGE_SIZElike x86 used to do since the bootmem era when allocation granularity wasPAGE_SIZEanyway.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>One comment unrelated to this patch set as such, just made more obvious by it.
arch/powerpc/mm/numa.c void __init dump_numa_cpu_topology(void) · line 1095
Quoted 8 lines
arch/powerpc/mm/numa.c +0
-0@@ -1093,27 +1093,9 @@ void __init dump_numa_cpu_topology(void)static void __init setup_node_data (int nid,u64 start_pfn,u64 end_pfn) {u64 spanned_pages = end_pfn - start_pfn;Trivial, but might as well squash this local variable into the single place it's used.
Quoted 1 line
arch/powerpc/mm/numa.c +0
-1-const size_t nd_size =roundup (sizeof (pg_data_t),SMP_CACHE_BYTES );
...
Quoted 6 lines
arch/powerpc/mm/numa.c void __init dump_numa_cpu_topology(void) · line 1101
Quoted 6 lines
arch/powerpc/mm/numa.c +3
-0+ +alloc_node_data (nid); +NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start_pfn;NODE_DATA (nid)->node_spanned_pages = spanned_pages;
These are actually overridden later in free_area_init(), it would make sense to audit all arch-specific node setup functions and clean them up a bit.
Signature · 2 lines
Sincerely yours, Mike.
Quoted 13 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following the discussion about handling of CXL fixed memory windows on arm64 [1] I decided to bite the bullet and move
numa_memblksfrom x86 to the generic code so they will be available on arm64/riscv and maybe on loongarch sometime later.While it could be possible to use memblock to describe CXL memory windows, it currently lacks notion of unpopulated memory ranges and
numa_memblksdoes implement this.
Quoted 8 lines
Another reason to make
numa_memblksgeneric is that botharch_numa(arm64 and riscv) and loongarch use trimmed copy of x86 code although there is no fundamental reason why the same code cannot be used on all these platforms. Havingnuma_memblksinmm/will make it's interaction with ACPI and FDT more consistent and I believe will reduce maintenance burden.And with generic
numa_memblksit is (almost) straightforward to enable NUMA emulation on arm64 and riscv.
Hey Mike,
So interesting to see this come full circle and instead of moving numa_memblks to memblock, just uplevel numa_memblks. From the perspective of having numa_memblks enhancements work for more architectures this gets an enthusiastic thumbs up from me. Let me go look at the details...
Re: [PATCH v3 11/26] x86/numa: use get_pfn_range_for_nid to verify that node spans memory
In reply to Mike RapoportQuoted 25 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of looping over
numa_meminfoarray to detect node's start and end addresses useget_pfn_range_for_init().This is shorter and make it easier to lift
numa_memblksto generic code.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- arch/x86/mm/numa.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-)arch/x86/mm/numa.c +1
-2@@ -521,17 +521,10 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start =PFN_PHYS (max_pfn); -u64 end =0 ; +unsigned long start_pfn, end_pfn;
arch/x86/mm/numa.c static int __init numa_register_memblks(struct numa_meminfo *mi) · line 527
Quoted 10 lines
arch/x86/mm/numa.c +2
-8-for (i =0 ; i < mi->nr_blks; i++) { -if (nid != mi->blk[i].nid) -continue ; - start =min (mi->blk[i].start, start); - end =max (mi->blk[i].end, end); - } - -if (start >= end) +get_pfn_range_for_nid (nid, &start_pfn, &end_pfn); +if (start_pfn >= end_pfn)
Assuming I understand why this works, would it be worth a comment like:
"Note, get_pfn_range_for_nid() depends on memblock_set_node() having
already happened"
...at least that context was not part of the diff so took me second to figure out how this works.
Re: [PATCH v3 19/26] mm: introduce numa_emulation
In reply to Mike RapoportQuoted 3 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move
numa_emulationcodfromarch/x86tomm/numa_emulation.c
s/codfrom/code from/
I am surprised that numa-emulation stayed x86 only for so long. I think it is useful facility for debugging NUMA scaling and heterogenous memory topologies. So, glad to see it upleveled.
Re: [PATCH v3 22/26] mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo
In reply to Mike RapoportQuoted 8 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
numa_cleanup_meminfo()moves blocks outside system RAM tonuma_reserved_meminfoand it uses 0 andPFN_PHYS(max_pfn)to determine the memory boundaries.Replace the memory range boundaries with more portable
memblock_start_of_DRAM()andmemblock_end_of_DRAM().
Can you say a bit more about why this is more portable? Is there any scenario for which (0, max_pfn) does the wrong thing?
Re: [PATCH v3 25/26] mm: make range-to-target_node lookup facility a part of numa_memblks
In reply to Mike RapoportQuoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
The x86 implementation of range-to-target_node lookup (i.e.
phys_to_target_node()andmemory_add_physaddr_to_nid()) relies onnuma_memblks.
Quoted 7 lines
Since
numa_memblksare now part of the generic code, move these functions from x86 tomm/numa_memblks.cand selectCONFIG_NUMA_KEEP_MEMINFOwhenCONFIG_NUMA_MEMBLKS=yfor dax and cxl.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>
Looks good to me:
Dan Williams
Acked-byQuoted 23 lines · Dan Williams
Quoted 13 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following the discussion about handling of CXL fixed memory windows on arm64 [1] I decided to bite the bullet and move
numa_memblksfrom x86 to the generic code so they will be available on arm64/riscv and maybe on loongarch sometime later.While it could be possible to use memblock to describe CXL memory windows, it currently lacks notion of unpopulated memory ranges and
numa_memblksdoes implement this.Quoted 8 lines
Another reason to make
numa_memblksgeneric is that botharch_numa(arm64 and riscv) and loongarch use trimmed copy of x86 code although there is no fundamental reason why the same code cannot be used on all these platforms. Havingnuma_memblksinmm/will make it's interaction with ACPI and FDT more consistent and I believe will reduce maintenance burden.And with generic
numa_memblksit is (almost) straightforward to enable NUMA emulation on arm64 and riscv.
Quoted 7 lines
Hey Mike,
So interesting to see this come full circle and instead of moving
numa_memblksto memblock, just uplevelnuma_memblks. From the perspective of havingnuma_memblksenhancements work for more architectures this gets an enthusiastic thumbs up from me. Let me go look at the details...
For the series you can add:
Mike Rapoport
maintainerRe: [PATCH v3 22/26] mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo
In reply to Dan WilliamsQuoted 7 lines · Dan Williams
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
numa_cleanup_meminfo()moves blocks outside system RAM tonuma_reserved_meminfoand it uses 0 andPFN_PHYS(max_pfn)to determine the memory boundaries.
Quoted 5 lines
Quoted 2 lines
Replace the memory range boundaries with more portable
memblock_start_of_DRAM()andmemblock_end_of_DRAM().Can you say a bit more about why this is more portable? Is there any scenario for which (0,
max_pfn) does the wrong thing?
arm64 may have DRAM starting at addresses other than 0.
And max_pfn seems to me a redundant global variable that I'd love to see gone.
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerRe: [PATCH v3 11/26] x86/numa: use get_pfn_range_for_nid to verify that node spans memory
In reply to Dan WilliamsQuoted 39 lines · Dan Williams
Quoted 25 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of looping over
numa_meminfoarray to detect node's start and end addresses useget_pfn_range_for_init().This is shorter and make it easier to lift
numa_memblksto generic code.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- arch/x86/mm/numa.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-)arch/x86/mm/numa.c +1
-2@@ -521,17 +521,10 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start =PFN_PHYS (max_pfn); -u64 end =0 ; +unsigned long start_pfn, end_pfn;arch/x86/mm/numa.c static int __init numa_register_memblks(struct numa_meminfo *mi) · line 527
Quoted 10 lines
arch/x86/mm/numa.c +2
-8-for (i =0 ; i < mi->nr_blks; i++) { -if (nid != mi->blk[i].nid) -continue ; - start =min (mi->blk[i].start, start); - end =max (mi->blk[i].end, end); - } - -if (start >= end) +get_pfn_range_for_nid (nid, &start_pfn, &end_pfn); +if (start_pfn >= end_pfn)Assuming I understand why this works, would it be worth a comment like:
Quoted 2 lines
"Note,
get_pfn_range_for_nid()depends onmemblock_set_node()having
already happened"
Will add a comment, sure.
Quoted 3 lines
...at least that context was not part of the diff so took me second to figure out how this works.
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 02/26] MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
sgi-ip27 is the only system that defines
NODE_DATA()differently than the rest of NUMA machines.Add
node_dataarray of struct pglist pointers that will point to __node_data[node]->pglist and redefineNODE_DATA()to usenode_dataarray.
Quoted 5 lines
This will allow pulling declaration of
node_datato the generic mm code in the next commit.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 03/26] MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
For SGI IP27 machines
node_possible_mapis statically set toNODE_MASK_ALLand it is not updated during NUMA initialization.Ensure that it only contains nodes present in the system.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 04/26] MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportQuoted 7 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Commit
f8f9f21c7848("MIPS: Fix build error for loongson64 and sgi-ip27") addedHAVE_ARCH_NODEDATA_EXTENSIONto sgi-ip27 to silence a compilation error that happened because sgi-ip27 didn't define array ofpg_data_tasnode_datalike most other architectures did.
Quoted 7 lines
After addition of
node_dataarray that matches other architectures and after ensuring that offline nodes do not appear onnode_possible_map, it is safe to droparch_alloc_nodedata()andHAVE_ARCH_NODEDATA_EXTENSIONfrom sgi-ip27.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 06/26] MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
In reply to Mike RapoportQuoted 7 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Commit
f8f9f21c7848("MIPS: Fix build error for loongson64 and sgi-ip27") addedHAVE_ARCH_NODEDATA_EXTENSIONto loongson64 to silence a compilation error that happened because loongson64 didn't define array ofpg_data_tasnode_datalike most other architectures did.
Quoted 8 lines
After rename of
__node_datatonode_dataarch_alloc_nodedata()andHAVE_ARCH_NODEDATA_EXTENSIONcan be dropped from loongson64.Since it was the only user of
HAVE_ARCH_NODEDATA_EXTENSIONconfig option also remove this option fromarch/mips/Kconfig.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 10/26] x86/numa: simplify numa_distance allocation
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Allocation of
numa_distanceusesmemblock_phys_alloc_range()to limit allocation to be below the last mapped page.But NUMA initializaition runs after the direct map is populated and there is also code in
setup_arch()that adjusts memblock limit to reflect how much memory is already mapped in the direct map.
Quoted 5 lines
Simplify the allocation of
numa_distanceand use plainmemblock_alloc().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 11/26] x86/numa: use get_pfn_range_for_nid to verify that node spans memory
In reply to Mike RapoportQuoted 28 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of looping over
numa_meminfoarray to detect node's start and end addresses useget_pfn_range_for_init().This is shorter and make it easier to lift
numa_memblksto generic code.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- arch/x86/mm/numa.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-)arch/x86/mm/numa.c +1
-5@@ -521,17 +521,10 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start =PFN_PHYS (max_pfn); -u64 end =0 ; +unsigned long start_pfn, end_pfn; -for (i =0 ; i < mi->nr_blks; i++) { -if (nid != mi->blk[i].nid) -continue ;
arch/x86/mm/numa.c static int __init numa_register_memblks(struct numa_meminfo *mi) · line 530
Quoted 10 lines
arch/x86/mm/numa.c +2
-5- start =min (mi->blk[i].start, start); - end =max (mi->blk[i].end, end); - } - -if (start >= end) +get_pfn_range_for_nid (nid, &start_pfn, &end_pfn); +if (start_pfn >= end_pfn)continue ;alloc_node_data (nid);
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 12/26] x86/numa: move FAKE_NODE_* defines to numa_emu
In reply to Mike RapoportQuoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
The definitions of
FAKE_NODE_MIN_SIZEandFAKE_NODE_MIN_HASH_MASKare only used by numa emulation code, make them local toarch/x86/mm/numa_emulation.cSigned-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 13/26] x86/numa_emu: simplify allocation of phys_dist
In reply to Mike RapoportQuoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
By the time
numa_emulation()is called, all physical memory is already mapped in the direct map and there is no need to define limits for memblock allocation.
Quoted 6 lines
Replace
memblock_phys_alloc_range()withmemblock_alloc().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 14/26] x86/numa_emu: split __apicid_to_node update to a helper function
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
This is required to make numa emulation code architecture independent so that it can be moved to generic code in following commits.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 15/26] x86/numa_emu: use a helper function to get MAX_DMA32_PFN
In reply to Mike RapoportQuoted 31 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
This is required to make numa emulation code architecture independent so that it can be moved to generic code in following commits.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>--- arch/x86/include/asm/numa.h | 1 + arch/x86/mm/numa.c | 5 +++++ arch/x86/mm/numa_emulation.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)arch/x86/include/asm/numa.h +1
-0@@ -74,6 +74,7 @@ void debug_cpumask_set_cpu(int cpu, int node, bool enable);int numa_emu_cmdline (char *str);void __init numa_emu_update_cpu_to_node (int *emu_nid_to_phys,unsigned int nr_emu_nids); +u64 __init numa_emu_dma_end (void );#else /* CONFIG_NUMA_EMU */ static inline int numa_emu_cmdline (char *str) {arch/x86/mm/numa.c +0
-0
arch/x86/mm/numa.c void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys, · line 875
Quoted 8 lines
arch/x86/mm/numa.c +5
-0@@ -868,6 +868,11 @@ void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys, __apicid_to_node[i] = j < nr_emu_nids ? j :0 ; } } + +u64 __init numa_emu_dma_end (void ) +{ +return PFN_PHYS (MAX_DMA32_PFN ); +}
Inline function in header?
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 16/26] x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
In reply to Mike RapoportQuoted 4 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
CPU id cannot be negative.
Quoted 8 lines
Making it unsigned also aligns with declarations in
include/asm-generic/numa.hused by arm64 and riscv and allows sharing numa emulation code with these architectures.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Quoted 5 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with
numa_memblksfromarch/x86tomm/and addKconfigoptions to let x86 select it in itsKconfig.
Quoted 7 lines
This code will be later reused by
arch_numa.No functional changes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 18/26] mm: move numa_distance and related code from x86 to numa_memblks
In reply to Mike RapoportQuoted 5 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move code dealing with
numa_distancearray fromarch/x86tomm/numa_memblks.c
Quoted 7 lines
This code will be later reused by
arch_numa.No functional changes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 19/26] mm: introduce numa_emulation
In reply to Dan WilliamsQuoted 10 lines · Dan Williams
Quoted 3 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move
numa_emulationcodfromarch/x86tomm/numa_emulation.cs/codfrom/code from/
I am surprised that numa-emulation stayed x86 only for so long. I think it is useful facility for debugging NUMA scaling and heterogenous memory topologies. So, glad to see it upleveled.
I recall that s390x had NUMA emulation at some point (primarily to have multiple kthreads be responsible for different physical memory ranges), but it got removed a while ago.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 19/26] mm: introduce numa_emulation
In reply to Mike RapoportQuoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move
numa_emulationcodfromarch/x86tomm/numa_emulation.cThis code will be later reused by
arch_numa.
I'm confused why documentation lists for "numa=fake="
[KNL, ARM64, RISCV, X86, EARLY]
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 20/26] mm: numa_memblks: introduce numa_memblks_init
In reply to Mike RapoportQuoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move most of
x86::numa_init()tonuma_memblksso that the latter will be more self-contained.With this
numa_memblkdata structures should not be exposed to the architecture specific code.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 21/26] mm: numa_memblks: make several functions and variables static
In reply to Mike RapoportQuoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Make functions and variables that are exclusively used by
numa_memblksstatic.Move
numa_nodemask_from_meminfo()before its callers to avoid forward declaration.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 22/26] mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo
In reply to Mike RapoportQuoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
numa_cleanup_meminfo()moves blocks outside system RAM tonuma_reserved_meminfoand it uses 0 andPFN_PHYS(max_pfn)to determine the memory boundaries.
Quoted 6 lines
Replace the memory range boundaries with more portable
memblock_start_of_DRAM()andmemblock_end_of_DRAM().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 23/26] of, numa: return -EINVAL when no numa-node-id is found
In reply to Mike RapoportQuoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Currently
of_numa_parse_memory_nodes()returns 0 if no "memory" node in device tree contains "numa-node-id" property. This makesof_numa_init()to return "success" despite no NUMA nodes were actually parsed and set up.
arch_numaworkarounds this by returning an error ifnuma_nodes_parsedis empty.
Quoted 8 lines
numa_memblkshowever wouldWARN()in such case and since it will be used byarch_numashortly, such warning is not desirable.Make sure
of_numa_init()returns-EINVALwhen no NUMA node information was found in the device tree.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 19/26] mm: introduce numa_emulation
In reply to David HildenbrandQuoted 11 lines · David Hildenbrand
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move
numa_emulationcodfromarch/x86tomm/numa_emulation.cThis code will be later reused by
arch_numa.I'm confused why documentation lists for "numa=fake="
[KNL, ARM64, RISCV, X86, EARLY]
Ahh, I looked at mm-unstable which contain your patches (in particular the one that documents this :)) )
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 26/26] docs: move numa=fake description to kernel-parameters.txt
In reply to Mike RapoportQuoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
NUMA emulation can be now enabled on arm64 and riscv in addition to x86.
Move description of numa=fake parameters from x86 documentation of
admin-guide/kernel-parameters.txtSuggested-by Zi Yan <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 25/26] mm: make range-to-target_node lookup facility a part of numa_memblks
In reply to Mike RapoportQuoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
The x86 implementation of range-to-target_node lookup (i.e.
phys_to_target_node()andmemory_add_physaddr_to_nid()) relies onnuma_memblks.
Quoted 8 lines
Since
numa_memblksare now part of the generic code, move these functions from x86 tomm/numa_memblks.cand selectCONFIG_NUMA_KEEP_MEMINFOwhenCONFIG_NUMA_MEMBLKS=yfor dax and cxl.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>---
Nice
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v3 24/26] arch_numa: switch over to numa_memblks
In reply to Mike RapoportQuoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Until now
arch_numawas directly translating firmware NUMA information to memblock.Using
numa_memblksas an intermediate step has a few advantages:
* alignment with more battle tested x86 implementation
* availability of NUMA emulation
* maintaining node information for not yet populated memory
Quoted 7 lines
Replace current functionality related to
numa_add_memblk()and__node_distance()with the implementation based onnuma_memblksand add functions required bynuma_emulation.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb