Skip to contentSkip to conversation

Add interfaces for ACPI MRRM table

PATCH Superseded Tony Luck 5 participants 19 messages Feb 10, 2025 – Feb 13, 2025

Subsystems (6)
x86 ACPI MM core Driver core +2 more subsystems

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Tony Luck

author
#0 lore raw

cover Add interfaces for ACPI MRRM table

Memory used to be homogeneous. Then NUMA came along. Later different types of memory (persistent memory, on-package high bandwidth memory, CXL attached memory).

Each type of memory has its own performance characteristics, and users will need to monitor and control access by type.

The MRRM solution is to tag physical address ranges with "region IDs" so that platform firmware[1] can indicate the type of memory for each range (with separate tags available for local vs. remote access to each range).

The region IDs will be used to provide separate event counts for each region for "perf" and for the "resctrl" file system to monitor and control memory bandwidth in each region.

Users will need to know the address range(s) that are part of each region. This patch series adds /sys/devices/memory/rangeX directories to provide user space accessible enumeration.

-Tony

[1] MRRM definition allow for future expansion for the OS to assign these region IDs.

Fenghua Yu (1):
ACPICA: Define MRRM ACPI table

Tony Luck (3):
ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX ACPI/MRRM: ABI documentation for /sys/devices/system/memory/rangeX

 include/linux/memory.h                        |   9 +
 include/acpi/actbl3.h                         |  40 ++++
 drivers/acpi/acpi_mrrm.c                      | 188 ++++++++++++++++++
 drivers/base/memory.c                         |   9 +
 .../ABI/testing/sysfs-devices-memory          |  32 +++
 arch/x86/Kconfig                              |   1 +
 drivers/acpi/Kconfig                          |   4 +
 drivers/acpi/Makefile                         |   1 +
 8 files changed, 284 insertions(+)
 create mode 100644 drivers/acpi/acpi_mrrm.c

base-commit: a64dcfb451e254085a7daee5fe51bf22959d52d3

Signature · 1 line
2.48.1
author
#1 lore raw

PATCH 1/4 ACPICA: Define MRRM ACPI table

From: Fenghua Yu <[email protected]>

The MRRM table describes association between physical address ranges and "region numbers". This is used by:

1) The /sys/fs/resctrl filesystem to report memory traffic per-RMID for each region.
2) Perf subsystem to report memory related uncore events per region.

Structure defined in the Intel Resource Director Technology (RDT) Architecture specification downloadable from www.intel.com/sdm

---
 include/acpi/actbl3.h | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
include/acpi/actbl3.h +40 -0
index 5cd755143b7d..1b9a03ff73ba 100644
@@ -42,6 +42,7 @@
 #define ACPI_SIG_WSMT           "WSMT"	/* Windows SMM Security Mitigations Table */
 #define ACPI_SIG_XENV           "XENV"	/* Xen Environment table */
 #define ACPI_SIG_XXXX           "XXXX"	/* Intermediate AML header for ASL/ASL+ converter */
+#define ACPI_SIG_MRRM           "MRRM"	/* Memory Range and Region Mapping table */
 
 /*
  * All tables must be byte-packed to match the ACPI specification, since
@@ -793,6 +794,45 @@ struct acpi_table_xenv {
 	u8 event_flags;
 };
 
+/*******************************************************************************
+ *
+ * MRRM - Memory Range and Region Mapping (MRRM) table
+ *
+ ******************************************************************************/
+
+struct acpi_table_mrrm {
+	struct acpi_table_header header;
+	u8 max_mem_region;	/* Max Memory Regions supported */
+	u8 flags;		/* Region assignment type */
+	u8 reserved[26];
+	/* Memory range entry array */
+};
+#define ACPI_MRRM_FLAGS_REGION_ASSIGNMENT_OS	(1<<0)
+
+/*******************************************************************************
+ *
+ * Memory Range entry - Memory Range entry in MRRM table
+ *
+ ******************************************************************************/
+
+struct acpi_table_mrrm_mem_range_entry {
+	u16 type;		/* Type 0="MRRM" */
+	u16 length;		/* 32B + sizeof(Region-ID Programming Reg[]) */
+	u32 reserved;		/* Reserved */
+	u32 base_addr_low;	/* Low 32 bits of base addr of the mem range */
+	u32 base_addr_high;	/* High 32 bits of base addr of the mem range */
+	u32 len_low;		/* Low 32 bits of length of the mem range */
+	u32 len_high;		/* High 32 bits of length of the mem range */
+	u16 region_id_flags;	/* Valid local or remote Region-ID */
+	u8  local_region_id;	/* Platform-assigned static local Region-ID */
+	u8  remote_region_id;	/* Platform-assigned static remote Region-ID */
+	u32 reserved1;		/* Reserved */
+	/* Region-ID Programming Registers[] */
+};
+
+#define ACPI_MRRM_VALID_REGION_ID_FLAGS_LOCAL 	(1<<0)
+#define ACPI_MRRM_VALID_REGION_ID_FLAGS_REMOTE	(1<<1)
+
 /* Reset to default packing */
 
 #pragma pack()
Signature · 1 line
2.48.1
#2 lore raw

Re: [PATCH 1/4] ACPICA: Define MRRM ACPI table

Quoted 10 lines · Tony Luck

From: Fenghua Yu <[email protected]>

The MRRM table describes association between physical address ranges and "region numbers". This is used by:

1) The /sys/fs/resctrl filesystem to report memory traffic per-RMID for each region.
2) Perf subsystem to report memory related uncore events per region.

Quoted 5 lines

Structure defined in the Intel Resource Director Technology (RDT) Architecture specification downloadable from www.intel.com/sdm

So the process for ACPICA changes is that they first need to go into the upstream ACPICA project on GitHub.

Once merged there, you can submit a corresponding Linux patch pointing to the upstream commit, but submitting it is not mandatory because upstream material lands in the Linux kernel eventually automatically.

Quoted 66 lines
---
 include/acpi/actbl3.h | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
include/acpi/actbl3.h +40 -1
index 5cd755143b7d..1b9a03ff73ba 100644
@@ -42,6 +42,7 @@
 #define ACPI_SIG_WSMT           "WSMT" /* Windows SMM Security Mitigations Table */
 #define ACPI_SIG_XENV           "XENV" /* Xen Environment table */
 #define ACPI_SIG_XXXX           "XXXX" /* Intermediate AML header for ASL/ASL+ converter */
+#define ACPI_SIG_MRRM           "MRRM" /* Memory Range and Region Mapping table */
 
 /*
  * All tables must be byte-packed to match the ACPI specification, since
@@ -793,6 +794,45 @@ struct acpi_table_xenv {
        u8 event_flags;
 };
 
+/*******************************************************************************
+ *
+ * MRRM - Memory Range and Region Mapping (MRRM) table
+ *
+ ******************************************************************************/
+
+struct acpi_table_mrrm {
+       struct acpi_table_header header;
+       u8 max_mem_region;      /* Max Memory Regions supported */
+       u8 flags;               /* Region assignment type */
+       u8 reserved[26];
+       /* Memory range entry array */
+};
+#define ACPI_MRRM_FLAGS_REGION_ASSIGNMENT_OS   (1<<0)
+
+/*******************************************************************************
+ *
+ * Memory Range entry - Memory Range entry in MRRM table
+ *
+ ******************************************************************************/
+
+struct acpi_table_mrrm_mem_range_entry {
+       u16 type;               /* Type 0="MRRM" */
+       u16 length;             /* 32B + sizeof(Region-ID Programming Reg[]) */
+       u32 reserved;           /* Reserved */
+       u32 base_addr_low;      /* Low 32 bits of base addr of the mem range */
+       u32 base_addr_high;     /* High 32 bits of base addr of the mem range */
+       u32 len_low;            /* Low 32 bits of length of the mem range */
+       u32 len_high;           /* High 32 bits of length of the mem range */
+       u16 region_id_flags;    /* Valid local or remote Region-ID */
+       u8  local_region_id;    /* Platform-assigned static local Region-ID */
+       u8  remote_region_id;   /* Platform-assigned static remote Region-ID */
+       u32 reserved1;          /* Reserved */
+       /* Region-ID Programming Registers[] */
+};
+
+#define ACPI_MRRM_VALID_REGION_ID_FLAGS_LOCAL  (1<<0)
+#define ACPI_MRRM_VALID_REGION_ID_FLAGS_REMOTE (1<<1)
+
 /* Reset to default packing */
 
 #pragma pack()
--

2.48.1

author
#3 lore raw

PATCH 2/4 ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI

In reply to Tony Luck

Perf and resctrl users need an enumeration of which memory addresses are bound to which "region" tag.

Parse the ACPI MRRM table and add /sys entries for each memory range describing base address, length, and which region tags apply for same-socket and cross-socket access.

[Derived from code developed by Fenghua Yu <[email protected]>]

---
 include/linux/memory.h   |   9 +++
 drivers/acpi/acpi_mrrm.c | 159 +++++++++++++++++++++++++++++++++++++++
 drivers/base/memory.c    |   9 +++
 arch/x86/Kconfig         |   1 +
 drivers/acpi/Kconfig     |   4 +
 drivers/acpi/Makefile    |   1 +
 6 files changed, 183 insertions(+)
 create mode 100644 drivers/acpi/acpi_mrrm.c
include/linux/memory.h +9 -0
index c0afee5d126e..0a21943ce44d 100644
@@ -189,4 +189,13 @@ void memory_block_add_nid(struct memory_block *mem, int nid,
  */
 extern struct mutex text_mutex;
 
+#ifdef CONFIG_ACPI_MRRM
+int mrrm_max_mem_region(void);
+int memory_subsys_device_register(struct device *dev);
+#else
+static inline int mrrm_max_mem_region(void) { return -EONENT; }
+static inline int memory_subsys_device_register(struct device *dev) { return -EINVAL; }
+#define memory_subsys_device_register memory_subsys_device_register
+#endif
+
 #endif /* _LINUX_MEMORY_H_ */
drivers/acpi/acpi_mrrm.c new file +159 -0

159 lines not on this page · raw message

drivers/base/memory.c +9 -0
index 348c5dbbfa68..1f7853a4df5c 100644
@@ -699,6 +699,15 @@ static int __add_memory_block(struct memory_block *memory)
 	return ret;
 }
 
+#ifndef memory_subsys_device_register
+int memory_subsys_device_register(struct device *dev)
+{
+	dev->bus = &memory_subsys;
+
+	return device_register(dev);
+}
+#endif
+
 static struct zone *early_node_zone_for_memory_block(struct memory_block *mem,
 						     int nid)
 {
arch/x86/Kconfig +1 -0
index 87198d957e2f..96aa73e8fb13 100644
@@ -36,6 +36,7 @@ config X86_64
 	select ARCH_HAS_ELFCORE_COMPAT
 	select ZONE_DMA32
 	select EXECMEM if DYNAMIC_FTRACE
+	select ACPI_MRRM			if MEMORY_HOTPLUG
 
 config FORCE_DYNAMIC_FTRACE
 	def_bool y
drivers/acpi/Kconfig +4 -0
index d81b55f5068c..c3d1b0217e99 100644
@@ -576,6 +576,10 @@ config ACPI_FFH
 	  Enable this feature if you want to set up and install the FFH Address
 	  Space handler to handle FFH OpRegion in the firmware.
 
+config ACPI_MRRM
+	bool
+	depends on MEMORY_HOTPLUG
+
 source "drivers/acpi/pmic/Kconfig"
 
 config ACPI_VIOT
drivers/acpi/Makefile +1 -0
index 40208a0f5dfb..5092b518fc9b 100644
@@ -62,6 +62,7 @@ acpi-$(CONFIG_ACPI_WATCHDOG)	+= acpi_watchdog.o
 acpi-$(CONFIG_ACPI_PRMT)	+= prmt.o
 acpi-$(CONFIG_ACPI_PCC)		+= acpi_pcc.o
 acpi-$(CONFIG_ACPI_FFH)		+= acpi_ffh.o
+acpi-$(CONFIG_ACPI_MRRM)	+= acpi_mrrm.o
 
 # Address translation
 acpi-$(CONFIG_ACPI_ADXL)	+= acpi_adxl.o
Signature · 1 line
2.48.1
#4 lore raw

Re: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI

kernel test robot noticed the following build errors:

Hi Tony,

kernel test robot noticed the following build errors:

[auto build test ERROR on a64dcfb451e254085a7daee5fe51bf22959d52d3]

url:    https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427
base:   a64dcfb451e254085a7daee5fe51bf22959d52d3
patch link:    https://lore.kernel.org/r/20250210211223.6139-3-tony.luck%40intel.com
patch subject: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
config: i386-buildonly-randconfig-006-20250211 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config)
compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from mm/page_ext.c:6:

>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
     196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
         |                                                       ^
   1 error generated.
--
   In file included from mm/slab_common.c:12:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
     196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
         |                                                       ^
   In file included from mm/slab_common.c:34:
   In file included from mm/internal.h:13:
   include/linux/mm_inline.h:47:41: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
      47 |         __mod_lruvec_state(lruvec, NR_LRU_BASE + lru, nr_pages);
         |                                    ~~~~~~~~~~~ ^ ~~~
   include/linux/mm_inline.h:49:22: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
      49 |                                 NR_ZONE_LRU_BASE + lru, nr_pages);
         |                                 ~~~~~~~~~~~~~~~~ ^ ~~~
   2 warnings and 1 error generated.
--
   In file included from mm/mm_init.c:13:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
     196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
         |                                                       ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:98:11: warning: array index 3 is past the end of the array (that has type 'unsigned long[2]') [-Warray-bounds]
      98 |                 return (set->sig[3] | set->sig[2] |
         |                         ^        ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:98:25: warning: array index 2 is past the end of the array (that has type 'unsigned long[2]') [-Warray-bounds]
      98 |                 return (set->sig[3] | set->sig[2] |
         |                                       ^        ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:114:11: warning: array index 3 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
     114 |                 return  (set1->sig[3] == set2->sig[3]) &&
         |                          ^         ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:114:27: warning: array index 3 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
     114 |                 return  (set1->sig[3] == set2->sig[3]) &&
         |                                          ^         ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:115:5: warning: array index 2 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
     115 |                         (set1->sig[2] == set2->sig[2]) &&
         |                          ^         ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:115:21: warning: array index 2 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
     115 |                         (set1->sig[2] == set2->sig[2]) &&
         |                                          ^         ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:157:1: warning: array index 3 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
     157 | _SIG_SET_BINOP(sigorsets, _sig_or)
         | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/signal.h:138:8: note: expanded from macro '_SIG_SET_BINOP'
     138 |                 a3 = a->sig[3]; a2 = a->sig[2];                         \
         |                      ^      ~
   arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
      24 |         unsigned long sig[_NSIG_WORDS];
         |         ^
   In file included from mm/mm_init.c:16:
   In file included from include/linux/mman.h:5:
   In file included from include/linux/fs.h:33:
   In file included from include/linux/percpu-rwsem.h:7:
   In file included from include/linux/rcuwait.h:6:
   In file included from include/linux/sched/signal.h:6:
   include/linux/signal.h:157:1: warning: array index 2 is past the end of the array (that has type 'const unsigned long[2]') [-Warray-bounds]
     157 | _SIG_SET_BINOP(sigorsets, _sig_or)
         | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   include/linux/signal.h:138:24: note: expanded from macro '_SIG_SET_BINOP'
     138 |                 a3 = a->sig[3]; a2 = a->sig[2];                         \
--
   In file included from mm/migrate.c:22:
   include/linux/mm_inline.h:47:41: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
      47 |         __mod_lruvec_state(lruvec, NR_LRU_BASE + lru, nr_pages);
         |                                    ~~~~~~~~~~~ ^ ~~~
   include/linux/mm_inline.h:49:22: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
      49 |                                 NR_ZONE_LRU_BASE + lru, nr_pages);
         |                                 ~~~~~~~~~~~~~~~~ ^ ~~~
   In file included from mm/migrate.c:43:
>> include/linux/memory.h:196:55: error: use of undeclared identifier 'EONENT'
     196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
         |                                                       ^

2 warnings and 1 error generated.

vim +/EONENT +196 include/linux/memory.h

191
192 #ifdef CONFIG_ACPI_MRRM
193 int mrrm_max_mem_region(void);
194 int memory_subsys_device_register(struct device *dev);
195 #else

 > 196	static inline int mrrm_max_mem_region(void) { return -EONENT; }

197 static inline int memory_subsys_device_register(struct device *dev) { return -EINVAL; }
198 #define memory_subsys_device_register memory_subsys_device_register
199 #endif
200

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author
#5 lore raw

RE: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI

In reply to Tony Luck

include/linux/memory.h void memory_block_add_nid(struct memory_block *mem, int nid, · line 196

Quoted 13 lines
include/linux/memory.h +5 -0
index c0afee5d126e..0a21943ce44d 100644
@@ -189,4 +189,13 @@ void memory_block_add_nid(struct memory_block *mem, int nid,
  */
 extern struct mutex text_mutex;
 
+#ifdef CONFIG_ACPI_MRRM
+int mrrm_max_mem_region(void);
+int memory_subsys_device_register(struct device *dev);
+#else
+static inline int mrrm_max_mem_region(void) { return -EONENT; }

The lkp robot just pointed out my spelling error. Should be ENOENT.

include/linux/memory.h void memory_block_add_nid(struct memory_block *mem, int nid, · line 201

Quoted 5 lines
include/linux/memory.h +4 -0
+static inline int memory_subsys_device_register(struct device *dev) { return -EINVAL; }
+#define memory_subsys_device_register memory_subsys_device_register
+#endif
+
 #endif /* _LINUX_MEMORY_H_ */

-Tony

#6 lore raw

Re: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI

In reply to Tony Luck
kernel test robot noticed the following build errors:

Hi Tony,

kernel test robot noticed the following build errors:

[auto build test ERROR on a64dcfb451e254085a7daee5fe51bf22959d52d3]

url:    https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427
base:   a64dcfb451e254085a7daee5fe51bf22959d52d3
patch link:    https://lore.kernel.org/r/20250210211223.6139-3-tony.luck%40intel.com
patch subject: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
config: i386-buildonly-randconfig-004-20250211 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from kernel/vmcore_info.c:13:
include/linux/memory.h: In function 'mrrm_max_mem_region':

>> include/linux/memory.h:196:55: error: 'EONENT' undeclared (first use in this function); did you mean 'ENOENT'?
     196 | static inline int mrrm_max_mem_region(void) { return -EONENT; }
         |                                                       ^~~~~~
         |                                                       ENOENT

include/linux/memory.h:196:55: note: each undeclared identifier is reported only once for each function it appears in

vim +196 include/linux/memory.h

191
192 #ifdef CONFIG_ACPI_MRRM
193 int mrrm_max_mem_region(void);
194 int memory_subsys_device_register(struct device *dev);
195 #else

 > 196	static inline int mrrm_max_mem_region(void) { return -EONENT; }

197 static inline int memory_subsys_device_register(struct device *dev) { return -EINVAL; }
198 #define memory_subsys_device_register memory_subsys_device_register
199 #endif
200

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#7 lore raw

Re: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI

In reply to Tony Luck
kernel test robot noticed the following build warnings:

Hi Tony,

kernel test robot noticed the following build warnings:

[auto build test WARNING on a64dcfb451e254085a7daee5fe51bf22959d52d3]

url:    https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427
base:   a64dcfb451e254085a7daee5fe51bf22959d52d3
patch link:    https://lore.kernel.org/r/20250210211223.6139-3-tony.luck%40intel.com
patch subject: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI
config: x86_64-kismet-CONFIG_ACPI_MRRM-CONFIG_X86_64-0-0 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config)
reproduce: (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

kismet warnings: (new ones prefixed by >>)

>> kismet: WARNING: unmet direct dependencies detected for ACPI_MRRM when selected by X86_64

WARNING: unmet direct dependencies detected for ACPI_MRRM
Depends on [n]: ACPI [=n] && MEMORY_HOTPLUG [=y] Selected by [y]:
- X86_64 [=y] && 64BIT [=y] && MEMORY_HOTPLUG [=y]

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#8 lore raw

Re: [PATCH 2/4] ACPI/MRRM: Create /sys/devices/system/memory/rangeX ABI

In reply to Tony Luck
Quoted 6 lines · Tony Luck

Perf and resctrl users need an enumeration of which memory addresses are bound to which "region" tag.

Parse the ACPI MRRM table and add /sys entries for each memory range describing base address, length, and which region tags apply for same-socket and cross-socket access.

How does an example in /sys/devices/system/memory/ look like later?

From a quick glimpse, I am not sure if this really belongs into /sys/devices/system/memory/, but I am missing some information in cover letter / patch.

Signature · 3 lines
Cheers,

David / dhildenb
author
#9 lore raw

PATCH 3/4 ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

In reply to Tony Luck

Users will likely want to know which node owns each memory range and which CPUs are local to the range.

Add a symlink to the node directory to provide both pieces of information.

---
 drivers/acpi/acpi_mrrm.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
drivers/acpi/acpi_mrrm.c +29 -0
index 51ed9064e025..28b484943bbd 100644
@@ -119,6 +119,31 @@ static struct attribute *memory_range_attrs[] = {
 
 ATTRIBUTE_GROUPS(memory_range);
 
+static __init int add_node_link(struct mrrm_mem_range_entry *entry)
+{
+	struct node *node = NULL;
+	int ret = 0;
+	int nid;
+
+	for_each_online_node(nid) {
+		for (int z = 0; z < MAX_NR_ZONES; z++) {
+			struct zone *zone = NODE_DATA(nid)->node_zones + z;
+
+			if (!populated_zone(zone))
+				continue;
+			if (zone_intersects(zone, PHYS_PFN(entry->base), PHYS_PFN(entry->length))) {
+				node = node_devices[zone->node];
+				goto found;
+			}
+		}
+	}
+found:
+	if (node)
+		ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");
+
+	return ret;
+}
+
 static __init int add_boot_memory_ranges(void)
 {
 	char name[16];
@@ -140,6 +165,10 @@ static __init int add_boot_memory_ranges(void)
 			put_device(&entry->dev);
 			return ret;
 		}
+
+		ret = add_node_link(entry);
+		if (ret)
+			break;
 	}
 
 	return ret;
Signature · 1 line
2.48.1
#10 lore raw

Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

Quoted 29 lines · Tony Luck

Users will likely want to know which node owns each memory range and which CPUs are local to the range.

Add a symlink to the node directory to provide both pieces of information.

---
 drivers/acpi/acpi_mrrm.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
drivers/acpi/acpi_mrrm.c +11 -0
index 51ed9064e025..28b484943bbd 100644
@@ -119,6 +119,31 @@ static struct attribute *memory_range_attrs[] = {
 
 ATTRIBUTE_GROUPS(memory_range);
 
+static __init int add_node_link(struct mrrm_mem_range_entry *entry)
+{
+	struct node *node = NULL;
+	int ret = 0;
+	int nid;
+
+	for_each_online_node(nid) {
+		for (int z = 0; z < MAX_NR_ZONES; z++) {
+			struct zone *zone = NODE_DATA(nid)->node_zones + z;
+
+			if (!populated_zone(zone))

drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142

Quoted 10 lines
drivers/acpi/acpi_mrrm.c +10 -0
+				continue;
+			if (zone_intersects(zone, PHYS_PFN(entry->base), PHYS_PFN(entry->length))) {
+				node = node_devices[zone->node];
+				goto found;
+			}
+		}
+	}
+found:
+	if (node)
+		ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");

What is going to remove this symlink if the memory goes away? Or do these never get removed?

symlinks in sysfs created like this always worry me. What is going to use it?

thanks,

greg k-h

#11 lore raw

Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

Quoted 44 lines · Greg Kroah-Hartman
Quoted 29 lines · Tony Luck

Users will likely want to know which node owns each memory range and which CPUs are local to the range.

Add a symlink to the node directory to provide both pieces of information.

---
  drivers/acpi/acpi_mrrm.c | 29 +++++++++++++++++++++++++++++
  1 file changed, 29 insertions(+)
drivers/acpi/acpi_mrrm.c +11 -0
index 51ed9064e025..28b484943bbd 100644
@@ -119,6 +119,31 @@ static struct attribute *memory_range_attrs[] = {
 
  ATTRIBUTE_GROUPS(memory_range);
 
+static __init int add_node_link(struct mrrm_mem_range_entry *entry)
+{
+	struct node *node = NULL;
+	int ret = 0;
+	int nid;
+
+	for_each_online_node(nid) {
+		for (int z = 0; z < MAX_NR_ZONES; z++) {
+			struct zone *zone = NODE_DATA(nid)->node_zones + z;
+
+			if (!populated_zone(zone))

drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142

Quoted 10 lines
drivers/acpi/acpi_mrrm.c +10 -0
+				continue;
+			if (zone_intersects(zone, PHYS_PFN(entry->base), PHYS_PFN(entry->length))) {
+				node = node_devices[zone->node];
+				goto found;
+			}
+		}
+	}
+found:
+	if (node)
+		ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");

What is going to remove this symlink if the memory goes away? Or do these never get removed?

Quoted 2 lines

symlinks in sysfs created like this always worry me. What is going to use it?

On top of that, we seem to be building a separate hierarchy here.

/sys/devices/system/memory/ operates in memory block granularity.

/sys/devices/system/node/nodeX/ links to memory blocks that belong to it.

Why is the memory-block granularity insufficient, and why do we have to squeeze in another range API here?

Signature · 3 lines
Cheers,

David / dhildenb
author
#12 lore raw

RE: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

Quoted 9 lines
Quoted 5 lines

What is going to remove this symlink if the memory goes away? Or do these never get removed?

symlinks in sysfs created like this always worry me. What is going to use it?

On top of that, we seem to be building a separate hierarchy here.

/sys/devices/system/memory/ operates in memory block granularity.

What defines the memory blocks? I'd initially assumed some connection to the ACPI SRAT table. But on my test system there are only three entries in SRAT that define non-zero sized memory blocks (two on socket/node 0 and one on socket/node 1), yet there are:
memory0 .. memory32 directories
in /sys/devices/system/memory.

The phys_device and phys_index files aren't helping me figure out what each of them mean.

Quoted 4 lines

/sys/devices/system/node/nodeX/ links to memory blocks that belong to it.

Why is the memory-block granularity insufficient, and why do we have to squeeze in another range API here?

If an MRRM range consists of some set of memory blocks (making sure that no memory block spans across MRRM range boundaries, then I could add the {local,remote}_region_id files into the memory block directories.

This could work now while the region assignments are done by the BIOS. But in the future when OS gets the opportunity to change them it might be weird if an MRRM range consists of multiple memory block range, since the region_ids in each all update together.

/sys/devices/system/memory seemed like a logical place for memory ranges. But should I jump up a level and make a new /sys/devices/system/memory_regions directory to expose these ranges?

-Tony

#13 lore raw

Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

Quoted 8 lines · Tony
Quoted 8 lines
Quoted 5 lines

What is going to remove this symlink if the memory goes away? Or do these never get removed?

symlinks in sysfs created like this always worry me. What is going to use it?

On top of that, we seem to be building a separate hierarchy here.

Quoted 8 lines
Quoted 1 line

/sys/devices/system/memory/ operates in memory block granularity.

What defines the memory blocks? I'd initially assumed some connection to the ACPI SRAT table. But on my test system there are only three entries in SRAT that define non-zero sized memory blocks (two on socket/node 0 and one on socket/node 1), yet there are:
memory0 .. memory32 directories
in /sys/devices/system/memory.

Each memory block is the same size (e.g., 128 MiB .. 2 GiB on x86-64).

The default is memory section granularity (e.g., 128 MiB on x86-64), but some configs allow for increasing it: see arch/x86/mm/init_64.c:memory_block_size_bytes(), and in particular probe_memory_block_size().

They define in the granularity in which we can online/offline/add/remove physical memory managed by the buddy.

We create these block during boot/during hotplug, and link them to the relevant nodes.

They do not reflect the HW state, but the state Linux manages that memory (through the buddy).

Quoted 3 lines

The phys_device and phys_index files aren't helping me figure out what each of them mean.

Yes, see Documentation/admin-guide/mm/memory-hotplug.rst

phys_device is a legacy thing for s390x, and phys_index is just the memory block ID.

You can derive the address range corresponding to a memory block using the ID.

/sys/devices/system/memory/block_size_bytes tells you the size of each block.

Address range of block X:
[ X*block_size_bytes .. (X+1)*block_size_bytes )

Now, the whole interface her is designed for handling memory hotplug:

obj-$(CONFIG_MEMORY_HOTPLUG) += memory.o

It's worth noting that

1) Blocks might not be all-memory (e.g., memory holes). In that case, offlining/unplug is not supported.
2) Blocks might span multiple NUMA nodes (e.g., node ends / starts in the middle of a block). Similarly, in that case offlining/unplug is not supported.

I assume 1) is not a problem. I assume 2) could be a problem for your use case.

Quoted 11 lines
Quoted 5 lines

/sys/devices/system/node/nodeX/ links to memory blocks that belong to it.

Why is the memory-block granularity insufficient, and why do we have to squeeze in another range API here?

If an MRRM range consists of some set of memory blocks (making sure that no memory block spans across MRRM range boundaries, then I could add the {local,remote}_region_id files into the memory block directories.

Quoted 4 lines

This could work now while the region assignments are done by the BIOS. But in the future when OS gets the opportunity to change them it might be weird if an MRRM range consists of multiple memory block range, since the region_ids in each all update together.

What about memory ranges not managed by the buddy (e.g., dax/pmem ranges not exposed to the buddy through dax/kmem driver, memory hidden from Linux using mem=X etc.)?

Quoted 5 lines

/sys/devices/system/memory seemed like a logical place for memory ranges. But should I jump up a level and make a new /sys/devices/system/memory_regions directory to expose these ranges?

Let's take one step back. We do have

1) /proc/iomem to list physical device ranges, without a notion of nodes / other information. Maybe we could extend it, but it might be hard. Depending on *what* information we need to expose and for which memory.

/proc/iomem also doesn't indicate "System RAM" for memory not managed by the buddy.

2) /sys/devices/system/memory/memoryX and /sys/devices/system/node/

Again, the memory part is more hotplugged focused, and we treat individual memory blocks as "memory block devices".

Reading:

"
The MRRM solution is to tag physical address ranges with "region IDs" so that platform firmware[1] can indicate the type of memory for each range (with separate tags available for local vs. remote access to each range).

The region IDs will be used to provide separate event counts for each region for "perf" and for the "resctrl" file system to monitor and control memory bandwidth in each region.

Users will need to know the address range(s) that are part of each region."

A couple of questions:

a) How volatile is that information at runtime? Can ranges / IDs change? I read above that user space might in the future be able to reconfigure the ranges.

b) How is hotplug/unplug handled?

c) How are memory ranges not managed by Linux handled?

It might make sense to expose what you need in a more specialized, acpi/MRRM/perf specific form, and not as generic as you currently envision it.

Signature · 3 lines
Cheers,

David / dhildenb
author
#14 lore raw

RE: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

Quoted 5 lines

A couple of questions:

a) How volatile is that information at runtime? Can ranges / IDs change? I read above that user space might in the future be able to reconfigure the ranges.

Initial implementation has BIOS making all the choices. When there is a system that supports OS changes I envision making the "local_region_id" and "remote_region_id" files writeable for the sysadmin to make changes.

Note that the address ranges are fixed (and this isn't going to change).

Quoted 1 line

b) How is hotplug/unplug handled?

I'm looking for answers to this very good question. Plausibly systems might reserve address ranges for later hotplug. Those ranges could be enumerated in the MRRM table. But that is just a guess.

Quoted 1 line

c) How are memory ranges not managed by Linux handled?

It appears that all system memory is included in the range information. so access by BIOS to reserved memory ranges would be counted and controlled (unless SMI were to disable on entry). I'll ask this question.

Quoted 3 lines

It might make sense to expose what you need in a more specialized, acpi/MRRM/perf specific form, and not as generic as you currently envision it.

Agreed. The only thing going for /sys/devices/system/memory is the name. The actual semantics of everything below there don't match well with this usage.

Rafael: How do you feel about this (not implemented yet, just looking for a new spot to expose this)?

$ cd /sys/firmware/acpi
$ tree memory_ranges/
memory_ranges/
├── range0
│   ├── base
│   ├── length
│   ├── local_region_id
│   └── remote_region_id
├── range1
│   ├── base
│   ├── length
│   ├── local_region_id
│   └── remote_region_id
└── range2
    ├── base
    ├── length
    ├── local_region_id
    └── remote_region_id

4 directories, 12 files

-Tony

author
#15 lore raw

RE: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

In reply to Greg Kroah-Hartman
Quoted 5 lines

drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142

Quoted 2 lines
drivers/acpi/acpi_mrrm.c +2 -0
+	if (node)
+		ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");

What is going to remove this symlink if the memory goes away? Or do these never get removed?

There's currently no method for runtime changes to these memory ranges. They are described by a static ACPI table. I need to poke the folks that came up with this to ask how memory hotplug will be handled (since CXL seems to be making that fashionable again).

Quoted 2 lines

symlinks in sysfs created like this always worry me. What is going to use it?

<hand waves>User space tools that want to understand what the "per-region" monitoring and control features are actually operating on.</hand waves>

-Tony

#16 lore raw

Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

Quoted 10 lines · Tony
Quoted 5 lines

drivers/acpi/acpi_mrrm.c static struct attribute *memory_range_attrs[] = { · line 142

Quoted 2 lines
drivers/acpi/acpi_mrrm.c +2 -0
+	if (node)
+		ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");

What is going to remove this symlink if the memory goes away? Or do these never get removed?

There's currently no method for runtime changes to these memory ranges. They are described by a static ACPI table. I need to poke the folks that came up with this to ask how memory hotplug will be handled (since CXL seems to be making that fashionable again).

ACPI should be supporting memory hotplug today, at the very least "memory add", so surely you have some old boxes to test this with?

Quoted 5 lines
Quoted 2 lines

symlinks in sysfs created like this always worry me. What is going to use it?

<hand waves>User space tools that want to understand what the "per-region" monitoring and control features are actually operating on.</hand waves>

If you don't have a real user today, please don't include it now. Wait until it is actually needed.

thanks,

greg k-h

#17 lore raw

Re: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX

In reply to Tony Luck
kernel test robot noticed the following build errors:

Hi Tony,

kernel test robot noticed the following build errors:

[auto build test ERROR on a64dcfb451e254085a7daee5fe51bf22959d52d3]

url:    https://github.com/intel-lab-lkp/linux/commits/Tony-Luck/ACPICA-Define-MRRM-ACPI-table/20250211-051427
base:   a64dcfb451e254085a7daee5fe51bf22959d52d3
patch link:    https://lore.kernel.org/r/20250210211223.6139-4-tony.luck%40intel.com
patch subject: [PATCH 3/4] ACPI/MRRM: Add "node" symlink to /sys/devices/system/memory/rangeX
config: x86_64-randconfig-003-20250211 (https://download.01.org/0day-ci/archive/20250211/[email protected]/config)
compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250211/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/acpi/acpi_mrrm.c:135:31: error: no member named 'node' in 'struct zone'

135 | node = node_devices[zone->node];
| ~~~~ ^
1 error generated.

vim +135 drivers/acpi/acpi_mrrm.c

121
122 static __init int add_node_link(struct mrrm_mem_range_entry *entry)
123 {
124 struct node *node = NULL;
125 int ret = 0;
126 int nid;
127
128 for_each_online_node(nid) {
129 for (int z = 0; z < MAX_NR_ZONES; z++) {
130 struct zone *zone = NODE_DATA(nid)->node_zones + z;
131
132 if (!populated_zone(zone))
133 continue;
134 if (zone_intersects(zone, PHYS_PFN(entry->base), PHYS_PFN(entry->length))) {

 > 135					node = node_devices[zone->node];

136 goto found;
137 }
138 }
139 }
140 found:
141 if (node)
142 ret = sysfs_create_link(&entry->dev.kobj, &node->dev.kobj, "node");
143
144 return ret;
145 }
146

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author
#18 lore raw

PATCH 4/4 ACPI/MRRM: ABI documentation for /sys/devices/system/memory/rangeX

In reply to Tony Luck

ABI providing users with a mapping between physical address ranges and "region id" used by perf for uncore memory events and the resctrl file system for per-region memory monitoring and control.

---
 .../ABI/testing/sysfs-devices-memory          | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)
Documentation/ABI/testing/sysfs-devices-memory +32 -0
index cec65827e602..fb01739e016e 100644
@@ -118,3 +118,35 @@ Description:
 		(RO) indicates whether or not the kernel updates relevant kexec
 		segments on memory hot un/plug and/or on/offline events, avoiding the
 		need to reload kdump kernel.
+
+What:           /sys/devices/system/memory/rangeX/base
+Date:           January 2025
+Contact:	Tony Luck <[email protected]>
+Description:
+		On systems with the ACPI MRRM table reports the
+		base system physical address of memory range X.
+
+What:           /sys/devices/system/memory/rangeX/length
+Date:           January 2025
+Contact:	Tony Luck <[email protected]>
+Description:
+		On systems with the ACPI MRRM table reports the
+		size of rangeX system physical memory.
+
+What:           /sys/devices/system/memory/rangeX/local_region_id
+Date:           January 2025
+Contact:	Tony Luck <[email protected]>
+Description:
+		On systems with the ACPI MRRM table reports the
+		the region id associated with memory access by
+		agents local to this range of addresses. Reports
+		0xff when no region id has been assigned.
+
+What:           /sys/devices/system/memory/rangeX/remote_region_id
+Date:           January 2025
+Contact:	Tony Luck <[email protected]>
+Description:
+		On systems with the ACPI MRRM table reports the
+		the region id associated with memory access by
+		agents non-local to this range of addresses. Reports
+		0xff when no region id has been assigned.
Signature · 1 line
2.48.1