mm/mempolicy: add sysfs interface to override NUMA node bandwidth
Subsystems (3)
PATCH 1/1 mm/mempolicy: add sysfs interface to override NUMA node bandwidth
From: yeeli <[email protected]>
Automatic tuning for weighted interleaving [1] provides real benefits on systems with CXL support. However, platforms that lack HMAT or CDAT information cannot make use of this feature.
If the bandwidth reported by firmware or the device deviates from the actual measured bandwidth, administrators also lack a clear way to adjust the per-node weight values.
This patch introduces an optional Kconfig option, CONFIG_NUMA_BW_MANUAL_OVERRIDE (default n), which exposes node bandwidth R/W sysfs attributes under:
/sys/kernel/mm/mempolicy/weighted_interleave/bw_nodeN
The sysfs files are created and removed dynamically on node hotplug events, in sync with the existing weighted_interleave/nodeN attributes.
Userspace can write a single bandwidth value (in MB/s) to override both read_bandwidth and write_bandwidth for the corresponding NUMA node. The value is then propagated to the internal node_bw_table via mempolicy_set_node_perf().
This interface is intended for debugging and experimentation only.
[1] Link:
https://lkml.kernel.org/r/[email protected]
--- mm/Kconfig | 20 +++++++ mm/mempolicy.c | 148 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 168 insertions(+)
mm/Kconfig +20 -0
@@ -1441,6 +1441,26 @@ config NUMA_EMU into virtual nodes when booted with "numa=fake=N", where N is the number of nodes. This is only useful for debugging. +config NUMA_BW_MANUAL_OVERRIDE + bool "Allow manual override of per-NUMA-node bandwidth for weighted interleave" + depends on NUMA && SYSFS + default n + help + This option exposes writable sysfs attributes under + /sys/kernel/mm/mempolicy/weighted_interleave/bw_nodeN, allowing + userspace to manually set read/write bandwidth values for each NUMA node. + + These values update the internal node_bw_table and can influence + weighted interleave auto-tuning (if enabled). + + WARNING: This is intended for debugging, development, or platforms + with incorrect HMAT/CDAT firmware data. Overriding hardware-reported + bandwidth can lead to suboptimal performance, instability, or + incorrect resource allocation decisions. + + Say N unless you are actively developing or debugging bandwidth-aware + memory policies. + config ARCH_HAS_USER_SHADOW_STACK bool help
mm/mempolicy.c +148 -0
181 lines not on this page · raw message
Signature · 1 line
2.34.1
Huang, Ying
reviewerQuoted 19 lines · YeeLi
From: yeeli <[email protected]>
Automatic tuning for weighted interleaving [1] provides real benefits on systems with CXL support. However, platforms that lack HMAT or CDAT information cannot make use of this feature.
If the bandwidth reported by firmware or the device deviates from the actual measured bandwidth, administrators also lack a clear way to adjust the per-node weight values.
This patch introduces an optional
Kconfigoption,CONFIG_NUMA_BW_MANUAL_OVERRIDE(default n), which exposes node bandwidth R/W sysfs attributes under:
/sys/kernel/mm/mempolicy/weighted_interleave/bw_nodeNThe sysfs files are created and removed dynamically on node hotplug events, in sync with the existing weighted_interleave/nodeN attributes.
Quoted 6 lines
Userspace can write a single bandwidth value (in MB/s) to override both
read_bandwidthandwrite_bandwidthfor the corresponding NUMA node. The value is then propagated to the internalnode_bw_tableviamempolicy_set_node_perf().This interface is intended for debugging and experimentation only.
How to use this new interface? Can you provide an example?
---
Best Regards,
Huang, Ying
Quoted 3 lines
[snip]
Quoted 1 line
How to use this new interface? Can you provide an example?
Hi Ying,
A quick example:
1. Check current (original) bandwidth
cd /sys/kernel/mm/mempolicy/weighted_interleave
$ cat bw_node* 20000(MB/s) 10000(MB/s) 0
2. Check current interleave weights
$ cat node* 21 10 1
3. Override a node's bandwidth
$ echo 10000 > bw_node2
4. After override, check results:
$ cat bw_node* 20000(MB/s) 10000(MB/s) 10000(MB/s)
$ cat node* 2 1 1
Quoted 24 lines · On Thu, 12 Mar 2026 17:12:07 +0800 YeeLi <[email protected]> wrote:
From: yeeli <[email protected]>
Automatic tuning for weighted interleaving [1] provides real benefits on systems with CXL support. However, platforms that lack HMAT or CDAT information cannot make use of this feature.
If the bandwidth reported by firmware or the device deviates from the actual measured bandwidth, administrators also lack a clear way to adjust the per-node weight values.
This patch introduces an optional
Kconfigoption,CONFIG_NUMA_BW_MANUAL_OVERRIDE(default n), which exposes node bandwidth R/W sysfs attributes under:
/sys/kernel/mm/mempolicy/weighted_interleave/bw_nodeNThe sysfs files are created and removed dynamically on node hotplug events, in sync with the existing weighted_interleave/nodeN attributes.
Userspace can write a single bandwidth value (in MB/s) to override both
read_bandwidthandwrite_bandwidthfor the corresponding NUMA node. The value is then propagated to the internalnode_bw_tableviamempolicy_set_node_perf().
Quoted 6 lines
This interface is intended for debugging and experimentation only.
[1] Link:
https://lkml.kernel.org/r/[email protected]Signed-off-by yeeli <[email protected]>
As I note below, I'm not convinced this is the best layer to be injecting data at if we want a debug interface that allows us to reflect real hardware. Might be the sort of patch that is useful outside the tree though so thanks for sharing it!
Quoted 10 lines
--- mm/Kconfig | 20 +++++++ mm/mempolicy.c | 148 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 168 insertions(+)mm/Kconfig +0
-0
mm/Kconfig config NUMA_EMU · line 1447
Quoted 7 lines
mm/Kconfig +4
-0@@ -1441,6 +1441,26 @@ config NUMA_EMU into virtual nodes when booted with "numa=fake=N", where N is the number of nodes. This is only useful for debugging. +config NUMA_BW_MANUAL_OVERRIDE + bool "Allow manual override of per-NUMA-node bandwidth for weighted interleave" + depends on NUMA && SYSFS + default n
Drop this. Everything is default n (by default!)
mm/Kconfig config NUMA_EMU · line 1459
Quoted 12 lines
mm/Kconfig +12
-0+ help + This option exposes writable sysfs attributes under + /sys/kernel/mm/mempolicy/weighted_interleave/bw_nodeN, allowing + userspace to manually set read/write bandwidth values for each NUMA node. + + These values update the internal node_bw_table and can influence + weighted interleave auto-tuning (if enabled). + + WARNING: This is intended for debugging, development, or platforms + with incorrect HMAT/CDAT firmware data. Overriding hardware-reported + bandwidth can lead to suboptimal performance, instability, or + incorrect resource allocation decisions.
If it was an SRAT (for GPs) / HMAT issue I'd suggest table injection is an adequate way to do testing and debug.
CDAT is trickier, but maybe we should be thinking about a debug path to inject that as well - similar to what we do for ACPI tables.
From my point of view I'd rather see debug / testing interfaces that test the whole flow rather than just this top layer control.
However, I can see this is a useful patch for developers. But if you are developing debugging bandwidth-aware memory policies, I'm going to assume you don't mind building a kernel and adding this patch to make your life easier!
So to me, useful tool but I'm not 'yet' convinced it should be upstream.
Quoted 12 lines
mm/Kconfig +4
-0+ + Say N unless you are actively developing or debugging bandwidth-aware + memory policies. + config ARCH_HAS_USER_SHADOW_STACK bool helpmm/mempolicy.c +0
-0
mm/mempolicy.c int mempolicy_set_node_perf(unsigned int node, struct access_coordinate *coords) · line 229
Quoted 4 lines
mm/mempolicy.c +1
-0@@ -226,6 +226,7 @@ int mempolicy_set_node_perf(unsigned int node, struct access_coordinate *coords) bw_val =min (coords->read_bandwidth, coords->write_bandwidth); new_bw =kcalloc (nr_node_ids,sizeof (unsigned int ),GFP_KERNEL ); +
Stray change. Always check for these bnefore posting as they slow down patch sets for no good reason.
mm/mempolicy.c struct iw_node_attr { · line 3619
Quoted 9 lines
mm/mempolicy.c +2
-0if (!new_bw)return -ENOMEM ;@@ -3614,6 +3615,9 @@ struct iw_node_attr {struct sysfs_wi_group {struct kobject wi_kobj;struct mutex kobj_lock; +#ifdef CONFIG_NUMA_BW_MANUAL_OVERRIDE +struct iw_node_attr *bw_attrs[MAX_NUMNODES ];
Ok. It's debug so I guess this is ok rather than what is done for nattrs with nr_node_ids.
Alternative would be a pointer and a separate allocation.
mm/mempolicy.c static int sysfs_wi_node_add(int nid) · line 3865
Quoted 12 lines
mm/mempolicy.c +5
-0+#endif struct iw_node_attr *nattrs[]; };@@ -3855,6 +3859,128 @@ static int sysfs_wi_node_add(int nid)return ret; } +#ifdef CONFIG_NUMA_BW_MANUAL_OVERRIDE +static ssize_t bw_node_show (struct kobject *kobj, +struct kobj_attribute *attr, +char *buf)
Can fit on fewer lines without going past 80 chars.
mm/mempolicy.c static int sysfs_wi_node_add(int nid) · line 3871
Quoted 6 lines
mm/mempolicy.c +6
-0+{ +struct iw_node_attr *node_attr; + + node_attr = container_of(attr, struct iw_node_attr, kobj_attr); + +/*A Node without CDAT or HMAT*/
/* A node without either CDAT or HMAT */
Quoted 12 lines
mm/mempolicy.c +12
-0+if (!node_bw_table) +return sprintf (buf,"N/A\n" ); + +if (!node_bw_table[node_attr->nid]) +return sprintf (buf,"0\n" ); + +return sprintf (buf,"%u(MB/s)\n" , node_bw_table[node_attr->nid]); +} + +static ssize_t bw_node_store (struct kobject *kobj, +struct kobj_attribute *attr, +const char *buf,size_t count)
mm/mempolicy.c static int sysfs_wi_node_add(int nid) · line 3893
Quoted 10 lines
mm/mempolicy.c +10
-0+{ +struct iw_node_attr *node_attr; +unsigned long val =0 ; +int ret; +struct access_coordinate coords = { + .read_bandwidth =0 , + .write_bandwidth =0 , + }; + + node_attr = container_of(attr, struct iw_node_attr, kobj_attr);
Can do that at declaration of node_Attr above.
mm/mempolicy.c static int sysfs_wi_node_add(int nid) · line 3895
Quoted 2 lines
mm/mempolicy.c +2
-0+ + ret =kstrtoul (buf,0 , &val);
Check ret before filling in values.
Quoted 37 lines
mm/mempolicy.c +37
-0+ + coords.read_bandwidth = val; + coords.write_bandwidth = val; + +if (ret) +return ret; + +if (val >UINT_MAX ) +return -EINVAL ; + + ret =mempolicy_set_node_perf (node_attr->nid, &coords); +if (ret) +return ret; + +return count; +} + +static int sysfs_bw_node_add (int nid) +{ +int ret; +char *name; +struct iw_node_attr *new_attr; + +if (nid <0 || nid >= nr_node_ids) { +pr_err ("invalid node id: %d\n" , nid); +return -EINVAL ; + } + + new_attr =kzalloc (sizeof (*new_attr),GFP_KERNEL ); +if (!new_attr) +return -ENOMEM ; + + name =kasprintf (GFP_KERNEL ,"bw_node%d" , nid); +if (!name) { +kfree (new_attr); +return -ENOMEM ; + }
mm/mempolicy.c static int sysfs_wi_node_add(int nid) · line 3942
Quoted 10 lines
mm/mempolicy.c +10
-0+ +sysfs_attr_init (&new_attr->kobj_attr.attr); + new_attr->kobj_attr.attr.name = name; + new_attr->kobj_attr.attr.mode =0644 ; + new_attr->kobj_attr.show = bw_node_show; + new_attr->kobj_attr.store = bw_node_store; + new_attr->nid = nid; + +mutex_lock (&wi_group->kobj_lock); +if (wi_group->bw_attrs[nid]) {
Add another label and put the mutex unlock in the error path.
mm/mempolicy.c static int sysfs_wi_node_add(int nid) · line 3949
Quoted 7 lines
mm/mempolicy.c +7
-0+mutex_unlock (&wi_group->kobj_lock); + ret = -EEXIST ; + goto out; + } + + ret =sysfs_create_file (&wi_group->wi_kobj, &new_attr->kobj_attr.attr); +
No blank line here.
Quoted 44 lines
mm/mempolicy.c +37
-0+if (ret) { +mutex_unlock (&wi_group->kobj_lock); + goto out; + } + wi_group->bw_attrs[nid] = new_attr; +mutex_unlock (&wi_group->kobj_lock); +return 0 ; + +out: +kfree (new_attr->kobj_attr.attr.name); +kfree (new_attr); + return ret; +} + +static void sysfs_bw_node_delete (int nid) +{ +struct iw_node_attr *attr; + +if (nid <0 || nid >= nr_node_ids) +return ; + +mutex_lock (&wi_group->kobj_lock); + attr = wi_group->bw_attrs[nid]; + +if (attr) { +sysfs_remove_file (&wi_group->wi_kobj, &attr->kobj_attr.attr); +kfree (attr->kobj_attr.attr.name); +kfree (attr); + wi_group->nattrs[nid] =NULL ; + } +mutex_unlock (&wi_group->kobj_lock); +} +#endif +static int wi_node_notifier (struct notifier_block *nb,unsigned long action,void *data) {@@ -3868,9 +3994,22 @@ static int wi_node_notifier(struct notifier_block *nb,if (err)pr_err ("failed to add sysfs for node%d during hotplug: %d\n" , nid, err); + +#ifdef CONFIG_NUMA_BW_MANUAL_OVERRIDE + err =sysfs_bw_node_add (nid);
mm/mempolicy.c static int wi_node_notifier(struct notifier_block *nb, · line 4009
Quoted 10 lines
mm/mempolicy.c +7
-0+if (err) +pr_err ("failed to add sysfs bw_node%d: %d\n" , + nid, err); +#endif break ; +case NODE_REMOVED_LAST_MEMORY :sysfs_wi_node_delete (nid); + +#ifdef CONFIG_NUMA_BW_MANUAL_OVERRIDE
Probably better to stub the function than to scatter ifdefs throughout.
Quoted 22 lines
mm/mempolicy.c +12
-0+sysfs_bw_node_delete (nid); +#endif +break ; }@@ -3906,6 +4045,15 @@ static int __init add_weighted_interleave_group(struct kobject *mempolicy_kobj) nid, err); goto err_cleanup_kobj; } + +#ifdef CONFIG_NUMA_BW_MANUAL_OVERRIDE + err =sysfs_bw_node_add (nid); +if (err) { +pr_err ("failed to add sysfs bw_node%d during init: %d\n" , nid, err); + goto err_cleanup_kobj; + } +#endif + }hotplug_node_notifier (wi_node_notifier,DEFAULT_CALLBACK_PRI );
Hi Jonathan,
Thanks for the review.
Quoted 4 lines
As I note below, I'm not convinced this is the best layer to be injecting data at if we want a debug interface that allows us to reflect real hardware. Might be the sort of patch that is useful outside the tree though so thanks for sharing it!
Perhaps a small tool to compute weights from the desired bandwidth would be a better approach.
Quoted 9 lines
If it was an SRAT (for GPs) / HMAT issue I'd suggest table injection is an adequate way to do testing and debug.
CDAT is trickier, but maybe we should be thinking about a debug path to inject that as well - similar to what we do for ACPI tables.
From my point of view I'd rather see debug / testing interfaces that test the
whole flow rather than just this top layer control.
Quoted 6 lines
However, I can see this is a useful patch for developers. But if you are developing debugging bandwidth-aware memory policies, I'm going to assume you don't mind building a kernel and adding this patch to make your life easier!
So to me, useful tool but I'm not 'yet' convinced it should be upstream.
You are right, table injection is likely the better approach. During early hardware development the platform is often unstable, and firmware may not be able to report it accurately. Letting the OS handle debugging and evaluate the results can make experimentation easier.
And, thank you for checking the code.
Joshua Hahn
reviewerOn Thu, 12 Mar 2026 17:12:07 +0800 YeeLi <[email protected]> wrote:
Hello YeeLi,
I hope you are doing well! Thank you for the patch, I think it is an interesting and helpful debugging tool.
Quoted 3 lines
Automatic tuning for weighted interleaving [1] provides real benefits on systems with CXL support. However, platforms that lack HMAT or CDAT information cannot make use of this feature.
Thank you for the kind words : -) I'm also CCing Gregory Price, who is the original author of the weighted interleave mempolicy.
Quoted 3 lines
If the bandwidth reported by firmware or the device deviates from the actual measured bandwidth, administrators also lack a clear way to adjust the per-node weight values.
I just wanted to touch on this. Is this true? Weighted interleave already exposes sysfs that lets the user set the weights they want. On one hand I think it would be helpful for there to be a tool to see what the effects of setting bandwidth information on a system could be, if it had other side effects. I'm not sure if there is a strong case to do this purely for weighted interleave auto-tuning, which already has a manual weight override and pretty simple calculations : -)
Quoted 9 lines
This patch introduces an optional
Kconfigoption,CONFIG_NUMA_BW_MANUAL_OVERRIDE(default n), which exposes node bandwidth R/W sysfs attributes under:
/sys/kernel/mm/mempolicy/weighted_interleave/bw_nodeNThe sysfs files are created and removed dynamically on node hotplug events, in sync with the existing weighted_interleave/nodeN attributes.
Quoted 6 lines
Userspace can write a single bandwidth value (in MB/s) to override both
read_bandwidthandwrite_bandwidthfor the corresponding NUMA node. The value is then propagated to the internalnode_bw_tableviamempolicy_set_node_perf().This interface is intended for debugging and experimentation only.
Thank you for the helpful clarification. Just to be sure, are there other intended things that the user can test other than what weights weighted interleave would resolve to?
Thank you again for the patch, I hope you have a great day! Joshua
Quoted 8 lines
[1] Link:
https://lkml.kernel.org/r/[email protected]Signed-off-by yeeli <[email protected]>--- mm/Kconfig | 20 +++++++ mm/mempolicy.c | 148 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 168 insertions(+)
Joshua Hahn
reviewerQuoted 1 line · On Thu, 12 Mar 2026 08:00:21 -0700 Joshua Hahn <[email protected]> wrote:
On Thu, 12 Mar 2026 17:12:07 +0800 YeeLi <[email protected]> wrote:
Cc-ing Gregory's REAL address, this time... sorry for the noise!
Hi Joshua,
Thanks for your feedback and kind wishes.
Quoted 7 lines
I just wanted to touch on this. Is this true? Weighted interleave already exposes sysfs that lets the user set the weights they want. On one hand I think it would be helpful for there to be a tool to see what the effects of setting bandwidth information on a system could be, if it had other side effects. I'm not sure if there is a strong case to do this purely for weighted interleave auto-tuning, which already has a manual weight override and pretty simple calculations : -)
I agree — writing a small userspace tool to compute the weights based on actual measured bandwidths (from mlc or stream) is a really good approach.
Quoted 3 lines
Thank you for the helpful clarification. Just to be sure, are there other intended things that the user can test other than what weights weighted interleave would resolve to?
We can continue DRAM + CXL weighted interleave performance testing later, following Gregory Price's experiments [1].
[1] Link:
https://lore.kernel.org/linux-mm/[email protected]/
Gregory Price
maintainerQuoted 6 lines · YeeLi
From: yeeli <[email protected]>
If the bandwidth reported by firmware or the device deviates from the actual measured bandwidth, administrators also lack a clear way to adjust the per-node weight values.
If userland is setting the raw bandwidth stat, why can't they do a tiny bit of math to figure out the preferred weighting and just write the weights?
Quoted 8 lines
Userspace can write a single bandwidth value (in MB/s) to override both
read_bandwidthandwrite_bandwidthfor the corresponding NUMA node. The value is then propagated to the internalnode_bw_tableviamempolicy_set_node_perf().This interface is intended for debugging and experimentation only.
Should be posted as [RFC] then, I don't think we want this upstream.
~Gregory
Hi Gregory,
Quoted 3 lines
If userland is setting the raw bandwidth stat, why can't they do a tiny bit of math to figure out the preferred weighting and just write the weights?
We're not setting the raw bandwidth stat directly, but overriding the top-level bandwidth to observe the final effect.
Quoted 1 line
Should be posted as [RFC] then, I don't think we want this upstream.
Thanks for the suggestion.