Add interfaces for ACPI MRRM table
Subsystems (6)
cover Add interfaces for ACPI MRRM table
Memory used to be homogeneous. Then NUMA came along. Later different types of memory (persistent memory, on-package high bandwidth memory, CXL attached memory).
Each type of memory has its own performance characteristics, and users will need to monitor and control access by type.
The MRRM solution is to tag physical address ranges with "region IDs" so that platform firmware[1] can indicate the type of memory for each range (with separate tags available for local vs. remote access to each range).
The region IDs will be used to provide separate event counts for each region for "perf" and for the "resctrl" file system to monitor and control memory bandwidth in each region.
Users will need to know the address range(s) that are part of each region. This patch series adds /sys/devices/memory/rangeX directories to provide user space accessible enumeration.
-Tony
[1] MRRM definition allow for future expansion for the OS to assign these region IDs.
Fenghua Yu (1):
ACPICA: Define MRRM ACPI table
Tony Luck (3):
ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX ACPI/MRRM: ABI documentation for /sys/devices/system/memory/rangeX
include/linux/memory.h | 9 + include/acpi/actbl3.h | 40 ++++ drivers/acpi/acpi_mrrm.c | 188 ++++++++++++++++++ drivers/base/memory.c | 9 + .../ABI/testing/sysfs-devices-memory | 32 +++ arch/x86/Kconfig | 1 + drivers/acpi/Kconfig | 4 + drivers/acpi/Makefile | 1 + 8 files changed, 284 insertions(+) create mode 100644 drivers/acpi/acpi_mrrm.c
base-commit: a64dcfb451e254085a7daee5fe51bf22959d52d3
Signature · 1 line
2.48.1
PATCH 1/4 ACPICA: Define MRRM ACPI table
From: Fenghua Yu <[email protected]>
The MRRM table describes association between physical address ranges and "region numbers". This is used by:
1) The /sys/fs/resctrl filesystem to report memory traffic per-RMID for each region.
2) Perf subsystem to report memory related uncore events per region.
Structure defined in the Intel Resource Director Technology (RDT) Architecture specification downloadable from www.intel.com/sdm
--- include/acpi/actbl3.h | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
include/acpi/actbl3.h +40 -0
@@ -42,6 +42,7 @@#define ACPI_SIG_WSMT "WSMT"/* Windows SMM Security Mitigations Table */ #define ACPI_SIG_XENV "XENV"/* Xen Environment table */ #define ACPI_SIG_XXXX "XXXX"/* Intermediate AML header for ASL/ASL+ converter */ +#define ACPI_SIG_MRRM "MRRM"/* Memory Range and Region Mapping table */ /* * All tables must be byte-packed to match the ACPI specification, since
@@ -793,6 +794,45 @@ struct acpi_table_xenv {u8 event_flags; }; +/******************************************************************************* +* +* MRRM - Memory Range and Region Mapping (MRRM) table +* +******************************************************************************/ + +struct acpi_table_mrrm { +struct acpi_table_header header; +u8 max_mem_region;/* Max Memory Regions supported */ +u8 flags;/* Region assignment type */ +u8 reserved[26 ]; +/* Memory range entry array */ +}; +#define ACPI_MRRM_FLAGS_REGION_ASSIGNMENT_OS (1<<0) + +/******************************************************************************* +* +* Memory Range entry - Memory Range entry in MRRM table +* +******************************************************************************/ + +struct acpi_table_mrrm_mem_range_entry { +u16 type;/* Type 0="MRRM" */ +u16 length;/* 32B + sizeof(Region-ID Programming Reg[]) */ +u32 reserved;/* Reserved */ +u32 base_addr_low;/* Low 32 bits of base addr of the mem range */ +u32 base_addr_high;/* High 32 bits of base addr of the mem range */ +u32 len_low;/* Low 32 bits of length of the mem range */ +u32 len_high;/* High 32 bits of length of the mem range */ +u16 region_id_flags;/* Valid local or remote Region-ID */ +u8 local_region_id;/* Platform-assigned static local Region-ID */ +u8 remote_region_id;/* Platform-assigned static remote Region-ID */ +u32 reserved1;/* Reserved */ +/* Region-ID Programming Registers[] */ +}; + +#define ACPI_MRRM_VALID_REGION_ID_FLAGS_LOCAL (1<<0) +#define ACPI_MRRM_VALID_REGION_ID_FLAGS_REMOTE (1<<1) +/* Reset to default packing */ #pragma pack()
Signature · 1 line
2.48.1
Rafael J. Wysocki
maintainerRe: [PATCH 1/4] ACPICA: Define MRRM ACPI table
Quoted 10 lines · Tony Luck
From: Fenghua Yu <[email protected]>
The MRRM table describes association between physical address ranges and "region numbers". This is used by:
1) The
/sys/fs/resctrlfilesystem to report memory traffic per-RMID for each region.
2) Perf subsystem to report memory related uncore events per region.
Quoted 5 lines
Structure defined in the Intel Resource Director Technology (RDT) Architecture specification downloadable from www.intel.com/sdm
Signed-off-by Fenghua Yu <[email protected]>Signed-off-by Tony Luck <[email protected]>
So the process for ACPICA changes is that they first need to go into the upstream ACPICA project on GitHub.
Once merged there, you can submit a corresponding Linux patch pointing to the upstream commit, but submitting it is not mandatory because upstream material lands in the Linux kernel eventually automatically.
Quoted 66 lines
--- include/acpi/actbl3.h | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)include/acpi/actbl3.h +40
-1@@ -42,6 +42,7 @@#define ACPI_SIG_WSMT "WSMT"/* Windows SMM Security Mitigations Table */ #define ACPI_SIG_XENV "XENV"/* Xen Environment table */ #define ACPI_SIG_XXXX "XXXX"/* Intermediate AML header for ASL/ASL+ converter */ +#define ACPI_SIG_MRRM "MRRM"/* Memory Range and Region Mapping table */ /* * All tables must be byte-packed to match the ACPI specification, since @@ -793,6 +794,45 @@ struct acpi_table_xenv {u8 event_flags; }; +/******************************************************************************* +* +* MRRM - Memory Range and Region Mapping (MRRM) table +* +******************************************************************************/ + +struct acpi_table_mrrm { +struct acpi_table_header header; +u8 max_mem_region;/* Max Memory Regions supported */ +u8 flags;/* Region assignment type */ +u8 reserved[26 ]; +/* Memory range entry array */ +}; +#define ACPI_MRRM_FLAGS_REGION_ASSIGNMENT_OS (1<<0) + +/******************************************************************************* +* +* Memory Range entry - Memory Range entry in MRRM table +* +******************************************************************************/ + +struct acpi_table_mrrm_mem_range_entry { +u16 type;/* Type 0="MRRM" */ +u16 length;/* 32B + sizeof(Region-ID Programming Reg[]) */ +u32 reserved;/* Reserved */ +u32 base_addr_low;/* Low 32 bits of base addr of the mem range */ +u32 base_addr_high;/* High 32 bits of base addr of the mem range */ +u32 len_low;/* Low 32 bits of length of the mem range */ +u32 len_high;/* High 32 bits of length of the mem range */ +u16 region_id_flags;/* Valid local or remote Region-ID */ +u8 local_region_id;/* Platform-assigned static local Region-ID */ +u8 remote_region_id;/* Platform-assigned static remote Region-ID */ +u32 reserved1;/* Reserved */ +/* Region-ID Programming Registers[] */ +}; + +#define ACPI_MRRM_VALID_REGION_ID_FLAGS_LOCAL (1<<0) +#define ACPI_MRRM_VALID_REGION_ID_FLAGS_REMOTE (1<<1) +/* Reset to default packing */ #pragma pack() --2.48.1
PATCH 2/4 ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
In reply to Tony LuckPerf and resctrl users need an enumeration of which memory addresses are bound to which "region" tag.
Parse the ACPI MRRM table and add /sys entries for each memory range describing base address, length, and which region tags apply for same-socket and cross-socket access.
[Derived from code developed by Fenghua Yu <[email protected]>]
--- include/linux/memory.h | 9 +++ drivers/acpi/acpi_mrrm.c | 159 +++++++++++++++++++++++++++++++++++++++ drivers/base/memory.c | 9 +++ arch/x86/Kconfig | 1 + drivers/acpi/Kconfig | 4 + drivers/acpi/Makefile | 1 + 6 files changed, 183 insertions(+) create mode 100644 drivers/acpi/acpi_mrrm.c
include/linux/memory.h +9 -0
@@ -189,4 +189,13 @@ void memory_block_add_nid(struct memory_block *mem, int nid,*/ extern struct mutex text_mutex; +#ifdef CONFIG_ACPI_MRRM +int mrrm_max_mem_region (void ); +int memory_subsys_device_register (struct device *dev); +#else +static inline int mrrm_max_mem_region (void ) {return -EONENT ; } +static inline int memory_subsys_device_register (struct device *dev) {return -EINVAL ; } +#define memory_subsys_device_register memory_subsys_device_register +#endif +#endif /* _LINUX_MEMORY_H_ */
drivers/acpi/acpi_mrrm.c new file +159 -0
159 lines not on this page · raw message
drivers/base/memory.c +9 -0
@@ -699,6 +699,15 @@ static int __add_memory_block(struct memory_block *memory)return ret; } +#ifndef memory_subsys_device_register +int memory_subsys_device_register (struct device *dev) +{ + dev->bus = &memory_subsys; + +return device_register (dev); +} +#endif +static struct zone *early_node_zone_for_memory_block (struct memory_block *mem,int nid) {
arch/x86/Kconfig +1 -0
@@ -36,6 +36,7 @@ config X86_64 select ARCH_HAS_ELFCORE_COMPAT select ZONE_DMA32 select EXECMEM if DYNAMIC_FTRACE + select ACPI_MRRM if MEMORY_HOTPLUG config FORCE_DYNAMIC_FTRACE def_bool y
drivers/acpi/Kconfig +4 -0
@@ -576,6 +576,10 @@ config ACPI_FFH Enable this feature if you want to set up and install the FFH Address Space handler to handle FFH OpRegion in the firmware. +config ACPI_MRRM + bool + depends on MEMORY_HOTPLUG + source "drivers/acpi/pmic/Kconfig" config ACPI_VIOT
drivers/acpi/Makefile +1 -0
@@ -62,6 +62,7 @@ acpi-$(CONFIG_ACPI_WATCHDOG) += acpi_watchdog.o acpi-$(CONFIG_ACPI_PRMT) += prmt.o acpi-$(CONFIG_ACPI_PCC) += acpi_pcc.o acpi-$(CONFIG_ACPI_FFH) += acpi_ffh.o +acpi-$(CONFIG_ACPI_MRRM) += acpi_mrrm.o # Address translation acpi-$(CONFIG_ACPI_ADXL) += acpi_adxl.o
Signature · 1 line
2.48.1
kernel test robot
build botRe: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
kernel test robot noticed the following build errors:
Hi Tony,
kernel test robot noticed the following build errors:
[auto build test ERROR on a64dcfb451e254085a7daee5fe51bf22959d52d3]
url: https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427 base: a64dcfb451e254085a7daee5fe51bf22959d52d3 patch link: https://lore.kernel.org/r/20250210211223.6139-3-tony.luck%40intel.com patch subject: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI config: i386-buildonly-randconfig-006-20250211 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config) compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from mm/page_ext.c:6:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
| ^
1 error generated.
--
In file included from mm/slab_common.c:12:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
| ^
In file included from mm/slab_common.c:34:
In file included from mm/internal.h:13:
include/linux/mm_inline.h:47:41: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
47 | __mod_lruvec_state(lruvec, NR_LRU_BASE + lru, nr_pages);
| ~~~~~~~~~~~ ^ ~~~
include/linux/mm_inline.h:49:22: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
49 | NR_ZONE_LRU_BASE + lru, nr_pages);
| ~~~~~~~~~~~~~~~~ ^ ~~~
2 warnings and 1 error generated.
--
In file included from mm/mm_init.c:13:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:98:11: warning: array index 3 is past the end of the array (that has type 'unsigned long[2]') [-Warray-bounds]
98 | return (set->sig[3] | set->sig[2] |
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:98:25: warning: array index 2 is past the end of the array (that has type 'unsigned long[2]') [-Warray-bounds]
98 | return (set->sig[3] | set->sig[2] |
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:114:11: warning: array index 3 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
114 | return (set1->sig[3] == set2->sig[3]) &&
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:114:27: warning: array index 3 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
114 | return (set1->sig[3] == set2->sig[3]) &&
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:115:5: warning: array index 2 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
115 | (set1->sig[2] == set2->sig[2]) &&
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:115:21: warning: array index 2 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
115 | (set1->sig[2] == set2->sig[2]) &&
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:157:1: warning: array index 3 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
157 | _SIG_SET_BINOP(sigorsets, _sig_or)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/signal.h:138:8: note: expanded from macro '_SIG_SET_BINOP'
138 | a3 = a->sig[3]; a2 = a->sig[2]; \
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from mm/mm_init.c:16:
In file included from include/linux/mman.h:5:
In file included from include/linux/fs.h:33:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:157:1: warning: array index 2 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
157 | _SIG_SET_BINOP(sigorsets, _sig_or)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/signal.h:138:24: note: expanded from macro '_SIG_SET_BINOP'
138 | a3 = a->sig[3]; a2 = a->sig[2]; \
--
In file included from mm/migrate.c:22:
include/linux/mm_inline.h:47:41: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
47 | __mod_lruvec_state(lruvec, NR_LRU_BASE + lru, nr_pages);
| ~~~~~~~~~~~ ^ ~~~
include/linux/mm_inline.h:49:22: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
49 | NR_ZONE_LRU_BASE + lru, nr_pages);
| ~~~~~~~~~~~~~~~~ ^ ~~~
In file included from mm/migrate.c:43:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
| ^2 warnings and 1 error generated.
vim +/EONENT +196 include/linux/memory.h
191
192 #ifdef CONFIG_ACPI_MRRM
193 int mrrm_max_mem_region(void);
194 int memory_subsys_device_register(struct device *dev);
195 #else
> 196 static inline int mrrm_max_mem_region(void) { return -EONENT; } 197 static inline int memory_subsys_device_register(struct device *dev) { return -EINVAL; }
198 #define memory_subsys_device_register memory_subsys_device_register
199 #endif
200
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
RE: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
In reply to Tony Luckinclude/linux/memory.h void memory_block_add_nid(struct memory_block *mem, int nid, · line 196
Quoted 13 lines
include/linux/memory.h +5
-0@@ -189,4 +189,13 @@ void memory_block_add_nid(struct memory_block *mem, int nid,*/ extern struct mutex text_mutex; +#ifdef CONFIG_ACPI_MRRM +int mrrm_max_mem_region (void ); +int memory_subsys_device_register (struct device *dev); +#else +static inline int mrrm_max_mem_region (void ) {return -EONENT ; }
The lkp robot just pointed out my spelling error. Should be ENOENT.
include/linux/memory.h void memory_block_add_nid(struct memory_block *mem, int nid, · line 201
Quoted 5 lines
include/linux/memory.h +4
-0+static inline int memory_subsys_device_register (struct device *dev) {return -EINVAL ; } +#define memory_subsys_device_register memory_subsys_device_register +#endif +#endif /* _LINUX_MEMORY_H_ */
-Tony
kernel test robot
build botRe: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
In reply to Tony Luckkernel test robot noticed the following build errors:
Hi Tony,
kernel test robot noticed the following build errors:
[auto build test ERROR on a64dcfb451e254085a7daee5fe51bf22959d52d3]
url: https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427 base: a64dcfb451e254085a7daee5fe51bf22959d52d3 patch link: https://lore.kernel.org/r/20250210211223.6139-3-tony.luck%40intel.com patch subject: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI config: i386-buildonly-randconfig-004-20250211 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from kernel/vmcore_info.c:13:
include/linux/memory.h: In function 'mrrm_max_mem_region':
>> include/linux/memory.h:196:55: error: 'EONENT' undeclared (first use in this function); did you mean 'ENOENT'?
196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
| ^~~~~~
| ENOENT include/linux/memory.h:196:55: note: each undeclared identifier is reported only once for each function it appears in
vim +196 include/linux/memory.h
191
192 #ifdef CONFIG_ACPI_MRRM
193 int mrrm_max_mem_region(void);
194 int memory_subsys_device_register(struct device *dev);
195 #else
> 196 static inline int mrrm_max_mem_region(void) { return -EONENT; } 197 static inline int memory_subsys_device_register(struct device *dev) { return -EINVAL; }
198 #define memory_subsys_device_register memory_subsys_device_register
199 #endif
200
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
In reply to Tony Luckkernel test robot noticed the following build warnings:
Hi Tony,
kernel test robot noticed the following build warnings:
[auto build test WARNING on a64dcfb451e254085a7daee5fe51bf22959d52d3]
url: https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427 base: a64dcfb451e254085a7daee5fe51bf22959d52d3 patch link: https://lore.kernel.org/r/20250210211223.6139-3-tony.luck%40intel.com patch subject: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI config: x86_64-kismet-CONFIG_ACPI_MRRM-CONFIG_X86_64-0-0 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config) reproduce: (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
kismet warnings: (new ones prefixed by >>)
>> kismet: WARNING: unmet direct dependencies detected for ACPI_MRRM when selected by X86_64
WARNING: unmet direct dependencies detected for ACPI_MRRM
Depends on [n]: ACPI [=n] && MEMORY_HOTPLUG [=y] Selected by [y]:
- X86_64 [=y] && 64BIT [=y] && MEMORY_HOTPLUG [=y]
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
In reply to Tony LuckQuoted 6 lines · Tony Luck
Perf and resctrl users need an enumeration of which memory addresses are bound to which "region" tag.
Parse the ACPI MRRM table and add /sys entries for each memory range describing base address, length, and which region tags apply for same-socket and cross-socket access.
How does an example in /sys/devices/system/memory/ look like later?
From a quick glimpse, I am not sure if this really belongs into /sys/devices/system/memory/, but I am missing some information in cover letter / patch.
Signature · 3 lines
Cheers, David / dhildenb
PATCH 3/4 ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
In reply to Tony LuckUsers will likely want to know which node owns each memory range and which CPUs are local to the range.
Add a symlink to the node directory to provide both pieces of information.
--- drivers/acpi/acpi_mrrm.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
drivers/acpi/acpi_mrrm.c +29 -0
@@ -119,6 +119,31 @@ static struct attribute *memory_range_attrs[] = {ATTRIBUTE_GROUPS (memory_range); +static __init int add_node_link (struct mrrm_mem_range_entry *entry) +{ +struct node *node =NULL ; +int ret =0 ; +int nid; + +for_each_online_node (nid) { +for (int z =0 ; z <MAX_NR_ZONES ; z++) { +struct zone *zone =NODE_DATA (nid)->node_zones + z; + +if (!populated_zone (zone)) +continue ; +if (zone_intersects (zone,PHYS_PFN (entry->base),PHYS_PFN (entry->length))) { + node = node_devices[zone->node]; + goto found; + } + } + } +found: +if (node) + ret =sysfs_create_link (&entry->dev.kobj, &node->dev.kobj,"node" ); + +return ret; +} +static __init int add_boot_memory_ranges (void ) {char name[16 ];
@@ -140,6 +165,10 @@ static __init int add_boot_memory_ranges(void)put_device (&entry->dev);return ret; } + + ret =add_node_link (entry); +if (ret) +break ; }return ret;
Signature · 1 line
2.48.1
Greg Kroah-Hartman
maintainerRe: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
Quoted 29 lines · Tony Luck
Users will likely want to know which node owns each memory range and which CPUs are local to the range.
Add a symlink to the node directory to provide both pieces of information.
Signed-off-by Tony Luck <[email protected]>--- drivers/acpi/acpi_mrrm.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)drivers/acpi/acpi_mrrm.c +11
-0@@ -119,6 +119,31 @@ static struct attribute *memory_range_attrs[] = {ATTRIBUTE_GROUPS (memory_range); +static __init int add_node_link (struct mrrm_mem_range_entry *entry) +{ +struct node *node =NULL ; +int ret =0 ; +int nid; + +for_each_online_node (nid) { +for (int z =0 ; z <MAX_NR_ZONES ; z++) { +struct zone *zone =NODE_DATA (nid)->node_zones + z; + +if (!populated_zone (zone))
drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142
Quoted 10 lines
drivers/acpi/acpi_mrrm.c +10
-0+continue ; +if (zone_intersects (zone,PHYS_PFN (entry->base),PHYS_PFN (entry->length))) { + node = node_devices[zone->node]; + goto found; + } + } + } +found: +if (node) + ret =sysfs_create_link (&entry->dev.kobj, &node->dev.kobj,"node" );
What is going to remove this symlink if the memory goes away? Or do these never get removed?
symlinks in sysfs created like this always worry me. What is going to use it?
thanks,
greg k-h
Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
Quoted 44 lines · Greg Kroah-Hartman
Quoted 29 lines · Tony Luck
Users will likely want to know which node owns each memory range and which CPUs are local to the range.
Add a symlink to the node directory to provide both pieces of information.
Signed-off-by Tony Luck <[email protected]>--- drivers/acpi/acpi_mrrm.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)drivers/acpi/acpi_mrrm.c +11
-0@@ -119,6 +119,31 @@ static struct attribute *memory_range_attrs[] = {ATTRIBUTE_GROUPS (memory_range); +static __init int add_node_link (struct mrrm_mem_range_entry *entry) +{ +struct node *node =NULL ; +int ret =0 ; +int nid; + +for_each_online_node (nid) { +for (int z =0 ; z <MAX_NR_ZONES ; z++) { +struct zone *zone =NODE_DATA (nid)->node_zones + z; + +if (!populated_zone (zone))drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142
Quoted 10 lines
drivers/acpi/acpi_mrrm.c +10
-0+continue ; +if (zone_intersects (zone,PHYS_PFN (entry->base),PHYS_PFN (entry->length))) { + node = node_devices[zone->node]; + goto found; + } + } + } +found: +if (node) + ret =sysfs_create_link (&entry->dev.kobj, &node->dev.kobj,"node" );What is going to remove this symlink if the memory goes away? Or do these never get removed?
Quoted 2 lines
symlinks in sysfs created like this always worry me. What is going to use it?
On top of that, we seem to be building a separate hierarchy here.
/sys/devices/system/memory/ operates in memory block granularity.
/sys/devices/system/node/nodeX/ links to memory blocks that belong to it.
Why is the memory-block granularity insufficient, and why do we have to squeeze in another range API here?
Signature · 3 lines
Cheers, David / dhildenb
RE: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
Quoted 9 lines
Quoted 5 lines
What is going to remove this symlink if the memory goes away? Or do these never get removed?
symlinks in sysfs created like this always worry me. What is going to use it?
On top of that, we seem to be building a separate hierarchy here.
/sys/devices/system/memory/operates in memory block granularity.
What defines the memory blocks? I'd initially assumed some connection to the ACPI SRAT table. But on my test system there are only three entries in SRAT that define non-zero sized memory blocks (two on socket/node 0 and one on socket/node 1), yet there are:
memory0 .. memory32 directories
in /sys/devices/system/memory.
The phys_device and phys_index files aren't helping me figure out what each of them mean.
Quoted 4 lines
/sys/devices/system/node/nodeX/links to memory blocks that belong to it.Why is the memory-block granularity insufficient, and why do we have to squeeze in another range API here?
If an MRRM range consists of some set of memory blocks (making sure that no memory block spans across MRRM range boundaries, then I could add the {local,remote}_region_id files into the memory block directories.
This could work now while the region assignments are done by the BIOS. But in the future when OS gets the opportunity to change them it might be weird if an MRRM range consists of multiple memory block range, since the region_ids in each all update together.
/sys/devices/system/memory seemed like a logical place for memory ranges. But should I jump up a level and make a new /sys/devices/system/memory_regions directory to expose these ranges?
-Tony
Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
Quoted 8 lines · Tony
Quoted 8 lines
Quoted 5 lines
What is going to remove this symlink if the memory goes away? Or do these never get removed?
symlinks in sysfs created like this always worry me. What is going to use it?
On top of that, we seem to be building a separate hierarchy here.
Quoted 8 lines
Quoted 1 line
/sys/devices/system/memory/operates in memory block granularity.What defines the memory blocks? I'd initially assumed some connection to the ACPI SRAT table. But on my test system there are only three entries in SRAT that define non-zero sized memory blocks (two on socket/node 0 and one on socket/node 1), yet there are:
memory0 .. memory32 directories
in/sys/devices/system/memory.
Each memory block is the same size (e.g., 128 MiB .. 2 GiB on x86-64).
The default is memory section granularity (e.g., 128 MiB on x86-64), but some configs allow for increasing it: see arch/x86/mm/init_64.c:memory_block_size_bytes(), and in particular probe_memory_block_size().
They define in the granularity in which we can online/offline/add/remove physical memory managed by the buddy.
We create these block during boot/during hotplug, and link them to the relevant nodes.
They do not reflect the HW state, but the state Linux manages that memory (through the buddy).
Quoted 3 lines
The
phys_deviceandphys_indexfiles aren't helping me figure out what each of them mean.
Yes, see Documentation/admin-guide/mm/memory-hotplug.rst
phys_device is a legacy thing for s390x, and phys_index is just the memory block ID.
You can derive the address range corresponding to a memory block using the ID.
/sys/devices/system/memory/block_size_bytes tells you the size of each block.
Address range of block X:
[ X*block_size_bytes .. (X+1)*block_size_bytes )
Now, the whole interface her is designed for handling memory hotplug:
obj-$(CONFIG_MEMORY_HOTPLUG) += memory.o
It's worth noting that
1) Blocks might not be all-memory (e.g., memory holes). In that case, offlining/unplug is not supported.
2) Blocks might span multiple NUMA nodes (e.g., node ends / starts in the middle of a block). Similarly, in that case offlining/unplug is not supported.
I assume 1) is not a problem. I assume 2) could be a problem for your use case.
Quoted 11 lines
Quoted 5 lines
/sys/devices/system/node/nodeX/links to memory blocks that belong to it.Why is the memory-block granularity insufficient, and why do we have to squeeze in another range API here?
If an MRRM range consists of some set of memory blocks (making sure that no memory block spans across MRRM range boundaries, then I could add the {local,remote}_region_id files into the memory block directories.
Quoted 4 lines
This could work now while the region assignments are done by the BIOS. But in the future when OS gets the opportunity to change them it might be weird if an MRRM range consists of multiple memory block range, since the
region_idsin each all update together.
What about memory ranges not managed by the buddy (e.g., dax/pmem ranges not exposed to the buddy through dax/kmem driver, memory hidden from Linux using mem=X etc.)?
Quoted 5 lines
/sys/devices/system/memoryseemed like a logical place for memory ranges. But should I jump up a level and make a new/sys/devices/system/memory_regionsdirectory to expose these ranges?
Let's take one step back. We do have
1) /proc/iomem to list physical device ranges, without a notion of nodes / other information. Maybe we could extend it, but it might be hard. Depending on *what* information we need to expose and for which memory.
/proc/iomem also doesn't indicate "System RAM" for memory not managed by the buddy.
2) /sys/devices/system/memory/memoryX and /sys/devices/system/node/
Again, the memory part is more hotplugged focused, and we treat individual memory blocks as "memory block devices".
Reading:
"
The MRRM solution is to tag physical address ranges with "region IDs" so that platform firmware[1] can indicate the type of memory for each range (with separate tags available for local vs. remote access to each range).
The region IDs will be used to provide separate event counts for each region for "perf" and for the "resctrl" file system to monitor and control memory bandwidth in each region.
Users will need to know the address range(s) that are part of each region."
A couple of questions:
a) How volatile is that information at runtime? Can ranges / IDs change? I read above that user space might in the future be able to reconfigure the ranges.
b) How is hotplug/unplug handled?
c) How are memory ranges not managed by Linux handled?
It might make sense to expose what you need in a more specialized, acpi/MRRM/perf specific form, and not as generic as you currently envision it.
Signature · 3 lines
Cheers, David / dhildenb
RE: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
Quoted 5 lines
A couple of questions:
a) How volatile is that information at runtime? Can ranges / IDs change? I read above that user space might in the future be able to reconfigure the ranges.
Initial implementation has BIOS making all the choices. When there is a system that supports OS changes I envision making the "local_region_id" and "remote_region_id" files writeable for the sysadmin to make changes.
Note that the address ranges are fixed (and this isn't going to change).
Quoted 1 line
b) How is hotplug/unplug handled?
I'm looking for answers to this very good question. Plausibly systems might reserve address ranges for later hotplug. Those ranges could be enumerated in the MRRM table. But that is just a guess.
Quoted 1 line
c) How are memory ranges not managed by Linux handled?
It appears that all system memory is included in the range information. so access by BIOS to reserved memory ranges would be counted and controlled (unless SMI were to disable on entry). I'll ask this question.
Quoted 3 lines
It might make sense to expose what you need in a more specialized, acpi/MRRM/perf specific form, and not as generic as you currently envision it.
Agreed. The only thing going for /sys/devices/system/memory is the name. The actual semantics of everything below there don't match well with this usage.
Rafael: How do you feel about this (not implemented yet, just looking for a new spot to expose this)?
$ cd /sys/firmware/acpi
$ tree memory_ranges/
memory_ranges/
├── range0
│ ├── base
│ ├── length
│ ├── local_region_id
│ └── remote_region_id
├── range1
│ ├── base
│ ├── length
│ ├── local_region_id
│ └── remote_region_id
└── range2
├── base
├── length
├── local_region_id
└── remote_region_id4 directories, 12 files
-Tony
RE: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
In reply to Greg Kroah-HartmanQuoted 5 lines
drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142
Quoted 2 lines
drivers/acpi/acpi_mrrm.c +2
-0+if (node) + ret =sysfs_create_link (&entry->dev.kobj, &node->dev.kobj,"node" );What is going to remove this symlink if the memory goes away? Or do these never get removed?
There's currently no method for runtime changes to these memory ranges. They are described by a static ACPI table. I need to poke the folks that came up with this to ask how memory hotplug will be handled (since CXL seems to be making that fashionable again).
Quoted 2 lines
symlinks in sysfs created like this always worry me. What is going to use it?
<hand waves>User space tools that want to understand what the "per-region" monitoring and control features are actually operating on.</hand waves>
-Tony
Greg Kroah-Hartman
maintainerRe: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
Quoted 10 lines · Tony
Quoted 5 lines
drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142
Quoted 2 lines
drivers/acpi/acpi_mrrm.c +2
-0+if (node) + ret =sysfs_create_link (&entry->dev.kobj, &node->dev.kobj,"node" );What is going to remove this symlink if the memory goes away? Or do these never get removed?
There's currently no method for runtime changes to these memory ranges. They are described by a static ACPI table. I need to poke the folks that came up with this to ask how memory hotplug will be handled (since CXL seems to be making that fashionable again).
ACPI should be supporting memory hotplug today, at the very least "memory add", so surely you have some old boxes to test this with?
Quoted 5 lines
Quoted 2 lines
symlinks in sysfs created like this always worry me. What is going to use it?
<hand waves>User space tools that want to understand what the "per-region" monitoring and control features are actually operating on.</hand waves>
If you don't have a real user today, please don't include it now. Wait until it is actually needed.
thanks,
greg k-h
kernel test robot
build botRe: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
In reply to Tony Luckkernel test robot noticed the following build errors:
Hi Tony,
kernel test robot noticed the following build errors:
[auto build test ERROR on a64dcfb451e254085a7daee5fe51bf22959d52d3]
url: https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427 base: a64dcfb451e254085a7daee5fe51bf22959d52d3 patch link: https://lore.kernel.org/r/20250210211223.6139-4-tony.luck%40intel.com patch subject: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX config: x86_64-randconfig-003-20250211 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config) compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> drivers/acpi/acpi_mrrm.c:135:31: error: no member named 'node' in 'struct zone'
135 | node = node_devices[zone->node];
| ~~~~ ^
1 error generated.
vim +135 drivers/acpi/acpi_mrrm.c
121
122 static __init int add_node_link(struct mrrm_mem_range_entry *entry)
123 {
124 struct node *node = NULL;
125 int ret = 0;
126 int nid;
127
128 for_each_online_node(nid) {
129 for (int z = 0; z < MAX_NR_ZONES; z++) {
130 struct zone *zone = NODE_DATA(nid)->node_zones + z;
131
132 if (!populated_zone(zone))
133 continue;
134 if (zone_intersects(zone, PHYS_PFN(entry->base), PHYS_PFN(entry->length))) {
> 135 node = node_devices[zone->node];
136 goto found;
137 }
138 }
139 }
140 found:
141 if (node)
142 ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");
143
144 return ret;
145 }
146
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH 4/4 ACPI/MRRM: ABI documentation for /sys/devices/system/memory/rangeX
In reply to Tony LuckABI providing users with a mapping between physical address ranges and "region id" used by perf for uncore memory events and the resctrl file system for per-region memory monitoring and control.
--- .../ABI/testing/sysfs-devices-memory | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+)
Documentation/ABI/testing/sysfs-devices-memory +32 -0
@@ -118,3 +118,35 @@ Description: (RO) indicates whether or not the kernel updates relevant kexec segments on memory hot un/plug and/or on/offline events, avoiding the need to reload kdump kernel. + +What: /sys/devices/system/memory/rangeX/base +Date: January 2025 +Contact: Tony Luck <[email protected]> +Description: + On systems with the ACPI MRRM table reports the + base system physical address of memory range X. + +What: /sys/devices/system/memory/rangeX/length +Date: January 2025 +Contact: Tony Luck <[email protected]> +Description: + On systems with the ACPI MRRM table reports the + size of rangeX system physical memory. + +What: /sys/devices/system/memory/rangeX/local_region_id +Date: January 2025 +Contact: Tony Luck <[email protected]> +Description: + On systems with the ACPI MRRM table reports the + the region id associated with memory access by + agents local to this range of addresses. Reports + 0xff when no region id has been assigned. + +What: /sys/devices/system/memory/rangeX/remote_region_id +Date: January 2025 +Contact: Tony Luck <[email protected]> +Description: + On systems with the ACPI MRRM table reports the + the region id associated with memory access by + agents non-local to this range of addresses. Reports + 0xff when no region id has been assigned.
Signature · 1 line
2.48.1