Skip to contentSkip to conversation

Enhance sysfs handling for memory hotplug in weighted interleave

PATCH Superseded Rakie Kim 5 participants 15 messages Apr 4, 2025 – Apr 7, 2025

Subsystems (2)

v6 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Rakie Kim

author reviewer
#0 lore raw

v6 cover Enhance sysfs handling for memory hotplug in weighted interleave

The following patch series enhances the weighted interleave policy in the memory management subsystem by improving sysfs handling, fixing memory leaks, and introducing dynamic sysfs updates for memory hotplug support.

### Background
The weighted interleave policy distributes memory allocations across multiple NUMA nodes based on their performance weight, thereby optimizing memory bandwidth utilization. The weight values are configured through sysfs.

Previously, sysfs entries for weighted interleave were managed statically at initialization. This led to several issues:
- Memory Leaks: Improper kobject teardown caused memory leaks when initialization failed or when nodes were removed.
- Lack of Dynamic Updates: Sysfs attributes were created only during initialization, preventing nodes added at runtime from being recognized.
- Handling of Unusable Nodes: Sysfs entries were generated for all possible nodes (N_POSSIBLE), including memoryless or unavailable nodes, leading to sysfs entries for unusable nodes and potential misconfigurations.

### Patch Overview
1. [PATCH 1/3] mm/mempolicy: Fix memory leaks in weighted interleave sysfs
- Ensures proper cleanup of kobject allocations.
- Adds kobject_del() before kobject_put() to clean up sysfs state correctly.
- Prevents memory/resource leaks and improves teardown behavior.

2. [PATCH 2/3] mm/mempolicy: Prepare weighted interleave sysfs for memory hotplug
- Refactors static sysfs layout into a new sysfs_wi_group structure.
- Makes per-node sysfs attributes accessible to external modules.
- Lays groundwork for future hotplug support by enabling runtime modification.

3. [PATCH 3/3] mm/mempolicy: Support memory hotplug in weighted interleave
- Dynamically adds/removes sysfs entries when nodes are online/offline.
- Limits sysfs creation to nodes with memory, avoiding unusable node entries.
- Hooks into memory hotplug notifier for runtime updates.

These patches have been tested under CXL-based memory configurations, including hotplug scenarios, to ensure proper behavior and stability.

 mm/mempolicy.c | 194 ++++++++++++++++++++++++++++++++-----------------
 1 file changed, 126 insertions(+), 68 deletions(-)

base-commit: 38fec10eb60d687e30c8c6b5420d86e8149f7557

Signature · 1 line
2.34.1
author reviewer
#1 lore raw

PATCH v6 1/3 mm/mempolicy: Fix memory leaks in weighted interleave sysfs

Memory leaks occurred when removing sysfs attributes for weighted interleave. Improper kobject deallocation led to unreleased memory when initialization failed or when nodes were removed.

This patch resolves the issue by replacing unnecessary kfree() calls with proper kobject_del() and kobject_put() sequences, ensuring correct teardown and preventing memory leaks.

By explicitly calling kobject_del() before kobject_put(), the release function is now invoked safely, and internal sysfs state is correctly cleaned up. This guarantees that the memory associated with the kobject is fully released and avoids resource leaks, thereby improving system stability.

Fixes dce41f5ae253 ("mm/mempolicy: implement the sysfs-based weighted_interleave interface")
---
 mm/mempolicy.c | 64 +++++++++++++++++++++++++++-----------------------
 1 file changed, 34 insertions(+), 30 deletions(-)
mm/mempolicy.c +34 -30
index bbaadbeeb291..af3753925573 100644
@@ -3448,7 +3448,9 @@ static void sysfs_wi_release(struct kobject *wi_kobj)
 
 	for (i = 0; i < nr_node_ids; i++)
 		sysfs_wi_node_release(node_attrs[i], wi_kobj);
-	kobject_put(wi_kobj);
+
+	kfree(node_attrs);
+	kfree(wi_kobj);
 }
 
 static const struct kobj_type wi_ktype = {
@@ -3494,15 +3496,22 @@ static int add_weighted_interleave_group(struct kobject *root_kobj)
 	struct kobject *wi_kobj;
 	int nid, err;
 
-	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
-	if (!wi_kobj)
+	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
+			     GFP_KERNEL);
+	if (!node_attrs)
 		return -ENOMEM;
 
+	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
+	if (!wi_kobj) {
+		err = -ENOMEM;
+		goto node_out;
+	}
+
 	err = kobject_init_and_add(wi_kobj, &wi_ktype, root_kobj,
 				   "weighted_interleave");
 	if (err) {
-		kfree(wi_kobj);
-		return err;
+		kobject_put(wi_kobj);
+		goto err_out;
 	}
 
 	for_each_node_state(nid, N_POSSIBLE) {
@@ -3512,9 +3521,18 @@ static int add_weighted_interleave_group(struct kobject *root_kobj)
 			break;
 		}
 	}
-	if (err)
+	if (err) {
+		kobject_del(wi_kobj);
 		kobject_put(wi_kobj);
+		goto err_out;
+	}
+
 	return 0;
+
+node_out:
+	kfree(node_attrs);
+err_out:
+	return err;
 }
 
 static void mempolicy_kobj_release(struct kobject *kobj)
@@ -3528,7 +3546,6 @@ static void mempolicy_kobj_release(struct kobject *kobj)
 	mutex_unlock(&iw_table_lock);
 	synchronize_rcu();
 	kfree(old);
-	kfree(node_attrs);
 	kfree(kobj);
 }
 
@@ -3542,37 +3559,24 @@ static int __init mempolicy_sysfs_init(void)
 	static struct kobject *mempolicy_kobj;
 
 	mempolicy_kobj = kzalloc(sizeof(*mempolicy_kobj), GFP_KERNEL);
-	if (!mempolicy_kobj) {
-		err = -ENOMEM;
-		goto err_out;
-	}
-
-	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
-			     GFP_KERNEL);
-	if (!node_attrs) {
-		err = -ENOMEM;
-		goto mempol_out;
-	}
+	if (!mempolicy_kobj)
+		return -ENOMEM;
 
 	err = kobject_init_and_add(mempolicy_kobj, &mempolicy_ktype, mm_kobj,
 				   "mempolicy");
 	if (err)
-		goto node_out;
+		goto err_out;
 
 	err = add_weighted_interleave_group(mempolicy_kobj);
-	if (err) {
-		pr_err("mempolicy sysfs structure failed to initialize\n");
-		kobject_put(mempolicy_kobj);
-		return err;
-	}
+	if (err)
+		goto err_del;
 
-	return err;
-node_out:
-	kfree(node_attrs);
-mempol_out:
-	kfree(mempolicy_kobj);
+	return 0;
+
+err_del:
+	kobject_del(mempolicy_kobj);
 err_out:
-	pr_err("failed to add mempolicy kobject to the system\n");
+	kobject_put(mempolicy_kobj);
 	return err;
 }
 
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v6 1/3] mm/mempolicy: Fix memory leaks in weighted interleave sysfs

Quoted 14 lines · On Fri, 4 Apr 2025 16:46:19 +0900 Rakie Kim <[email protected]> wrote:

Memory leaks occurred when removing sysfs attributes for weighted interleave. Improper kobject deallocation led to unreleased memory when initialization failed or when nodes were removed.

This patch resolves the issue by replacing unnecessary kfree() calls with proper kobject_del() and kobject_put() sequences, ensuring correct teardown and preventing memory leaks.

By explicitly calling kobject_del() before kobject_put(), the release function is now invoked safely, and internal sysfs state is correctly cleaned up. This guarantees that the memory associated with the kobject is fully released and avoids resource leaks, thereby improving system stability.

Quoted 5 lines
Fixes dce41f5ae253 ("mm/mempolicy: implement the sysfs-based weighted_interleave interface")

I've pretty much forgotten earlier discussions so apologies if I revisit old ground!

The fix is fine I think. But the resulting code structure could be improved, without (I think) complicating what is here by much.

Quoted 24 lines
---
 mm/mempolicy.c | 64 +++++++++++++++++++++++++++-----------------------
 1 file changed, 34 insertions(+), 30 deletions(-)
mm/mempolicy.c +3 -2
index bbaadbeeb291..af3753925573 100644
@@ -3448,7 +3448,9 @@ static void sysfs_wi_release(struct kobject *wi_kobj)
 
 	for (i = 0; i < nr_node_ids; i++)
 		sysfs_wi_node_release(node_attrs[i], wi_kobj);
-	kobject_put(wi_kobj);
+
+	kfree(node_attrs);
+	kfree(wi_kobj);
 }
 
 static const struct kobj_type wi_ktype = {
@@ -3494,15 +3496,22 @@ static int add_weighted_interleave_group(struct kobject *root_kobj)
 	struct kobject *wi_kobj;
 	int nid, err;
 
-	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);

mm/mempolicy.c static int add_weighted_interleave_group(struct kobject *root_kobj) · line 3507

Quoted 10 lines
mm/mempolicy.c +7 -1
-	if (!wi_kobj)
+	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
+			     GFP_KERNEL);
+	if (!node_attrs)
 		return -ENOMEM;
 
+	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
+	if (!wi_kobj) {
+		err = -ENOMEM;
+		goto node_out;

As this is only place where we do kfree(node_attrs) why not just do that here and return directly.

Quoted 13 lines
mm/mempolicy.c +4 -2
+	}
+
 	err = kobject_init_and_add(wi_kobj, &wi_ktype, root_kobj,
 				   "weighted_interleave");
 	if (err) {
-		kfree(wi_kobj);
-		return err;
+		kobject_put(wi_kobj);
+		goto err_out;
 	}
 
 	for_each_node_state(nid, N_POSSIBLE) {

mm/mempolicy.c static int add_weighted_interleave_group(struct kobject *root_kobj) · line 3526

Quoted 7 lines
mm/mempolicy.c +2 -1
@@ -3512,9 +3521,18 @@ static int add_weighted_interleave_group(struct kobject *root_kobj)
 			break;
 		}
 	}
-	if (err)
+	if (err) {
+		kobject_del(wi_kobj);
 		kobject_put(wi_kobj);

For this and the one above, a unified exit kind of makes sense as can do two labels and have the put only once.

If not, why not move this up into the loop and return directly? If you move to an error handling block

err_del_obj:

	kobject_del(wi_kobj);
err_put_obj:
	kobject_put(wi_kobj);

then you could also do the goto from within that loop.

Quoted 33 lines
mm/mempolicy.c +8 -10
+		goto err_out;
+	}
+
 	return 0;
+
+node_out:
+	kfree(node_attrs);
+err_out:
+	return err;
 }
 
 static void mempolicy_kobj_release(struct kobject *kobj)
@@ -3528,7 +3546,6 @@ static void mempolicy_kobj_release(struct kobject *kobj)
 	mutex_unlock(&iw_table_lock);
 	synchronize_rcu();
 	kfree(old);
-	kfree(node_attrs);
 	kfree(kobj);
 }
 
@@ -3542,37 +3559,24 @@ static int __init mempolicy_sysfs_init(void)
 	static struct kobject *mempolicy_kobj;
 
 	mempolicy_kobj = kzalloc(sizeof(*mempolicy_kobj), GFP_KERNEL);
-	if (!mempolicy_kobj) {
-		err = -ENOMEM;
-		goto err_out;
-	}
-
-	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
-			     GFP_KERNEL);
-	if (!node_attrs) {
-		err = -ENOMEM;

mm/mempolicy.c static int __init mempolicy_sysfs_init(void) · line 3568

Quoted 10 lines
mm/mempolicy.c +3 -3
-		goto mempol_out;
-	}
+	if (!mempolicy_kobj)
+		return -ENOMEM;
 
 	err = kobject_init_and_add(mempolicy_kobj, &mempolicy_ktype, mm_kobj,
 				   "mempolicy");
 	if (err)
-		goto node_out;
+		goto err_out;
		goto err_put_kobj;

or something like that.

Quoted 12 lines
mm/mempolicy.c +2 -7
 	err = add_weighted_interleave_group(mempolicy_kobj);
-	if (err) {
-		pr_err("mempolicy sysfs structure failed to initialize\n");
-		kobject_put(mempolicy_kobj);
-		return err;
-	}
+	if (err)
+		goto err_del;
 
-	return err;
-node_out:

mm/mempolicy.c static int __init mempolicy_sysfs_init(void) · line 3579

Quoted 10 lines
mm/mempolicy.c +5 -4
-	kfree(node_attrs);
-mempol_out:
-	kfree(mempolicy_kobj);
+	return 0;
+
+err_del:
+	kobject_del(mempolicy_kobj);
 err_out:
-	pr_err("failed to add mempolicy kobject to the system\n");
+	kobject_put(mempolicy_kobj);

If we keep an err_out, usual expectation is all it does is return
+ maybe print a message. We'd not expect a put.

Quoted 3 lines
mm/mempolicy.c +0 -0
 	return err;
 }
 
author reviewer
#3 lore raw

Re: [PATCH v6 1/3] mm/mempolicy: Fix memory leaks in weighted interleave sysfs

Quoted 22 lines · On Fri, 4 Apr 2025 13:59:06 +0100 Jonathan Cameron…
Quoted 14 lines · On Fri, 4 Apr 2025 16:46:19 +0900 Rakie Kim <[email protected]> wrote:

Memory leaks occurred when removing sysfs attributes for weighted interleave. Improper kobject deallocation led to unreleased memory when initialization failed or when nodes were removed.

This patch resolves the issue by replacing unnecessary kfree() calls with proper kobject_del() and kobject_put() sequences, ensuring correct teardown and preventing memory leaks.

By explicitly calling kobject_del() before kobject_put(), the release function is now invoked safely, and internal sysfs state is correctly cleaned up. This guarantees that the memory associated with the kobject is fully released and avoids resource leaks, thereby improving system stability.

Quoted 5 lines
Fixes dce41f5ae253 ("mm/mempolicy: implement the sysfs-based weighted_interleave interface")
Quoted 6 lines

I've pretty much forgotten earlier discussions so apologies if I revisit old ground!

The fix is fine I think. But the resulting code structure could be improved, without (I think) complicating what is here by much.

Thank you for your response regarding this patch.
I appreciate your willingness to revisit this code and share your thoughts. Please feel free to provide suggestions anytime.

Quoted 30 lines
Quoted 30 lines
---
 mm/mempolicy.c | 64 +++++++++++++++++++++++++++-----------------------
 1 file changed, 34 insertions(+), 30 deletions(-)
mm/mempolicy.c +6 -3
index bbaadbeeb291..af3753925573 100644
@@ -3448,7 +3448,9 @@ static void sysfs_wi_release(struct kobject *wi_kobj)
 
 	for (i = 0; i < nr_node_ids; i++)
 		sysfs_wi_node_release(node_attrs[i], wi_kobj);
-	kobject_put(wi_kobj);
+
+	kfree(node_attrs);
+	kfree(wi_kobj);
 }
 
 static const struct kobj_type wi_ktype = {
@@ -3494,15 +3496,22 @@ static int add_weighted_interleave_group(struct kobject *root_kobj)
 	struct kobject *wi_kobj;
 	int nid, err;
 
-	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
-	if (!wi_kobj)
+	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
+			     GFP_KERNEL);
+	if (!node_attrs)
 		return -ENOMEM;
 
Quoted 6 lines

mm/mempolicy.c static int add_weighted_interleave_group(struct kobject *root_kobj) · line 3507

Quoted 4 lines
mm/mempolicy.c +4 -0
+	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
+	if (!wi_kobj) {
+		err = -ENOMEM;
+		goto node_out;

As this is only place where we do kfree(node_attrs) why not just do that here and return directly.

Regarding your suggestion:
Is the following change what you had in mind?

    wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
    if (!wi_kobj) {
        kfree(node_attrs);
        return -ENOMEM;
    }

I will apply this change accordingly.

Quoted 29 lines
Quoted 15 lines
mm/mempolicy.c +4 -2
+	}
+
 	err = kobject_init_and_add(wi_kobj, &wi_ktype, root_kobj,
 				   "weighted_interleave");
 	if (err) {
-		kfree(wi_kobj);
-		return err;
+		kobject_put(wi_kobj);
+		goto err_out;
 	}
 
 	for_each_node_state(nid, N_POSSIBLE) {

mm/mempolicy.c static int add_weighted_interleave_group(struct kobject *root_kobj) · line 3526

Quoted 7 lines
mm/mempolicy.c +2 -1
@@ -3512,9 +3521,18 @@ static int add_weighted_interleave_group(struct kobject *root_kobj)
 			break;
 		}
 	}
-	if (err)
+	if (err) {
+		kobject_del(wi_kobj);
 		kobject_put(wi_kobj);

For this and the one above, a unified exit kind of makes sense as can do two labels and have the put only once.

If not, why not move this up into the loop and return directly? If you move to an error handling block

Quoted 7 lines

err_del_obj:

	kobject_del(wi_kobj);
err_put_obj:
	kobject_put(wi_kobj);

then you could also do the goto from within that loop.

As for your second suggestion about restructuring the error handling, you are right that having unified labels helps make the flow cleaner. I will update the code to use:

err = kobject_init_and_add(wi_kobj, &wi_ktype, root_kobj,

                           "weighted_interleave");
if (err)
    goto err_put_kobj;
for_each_node_state(nid, N_POSSIBLE) {
    err = add_weight_node(nid, wi_kobj);
    if (err) {
        pr_err("failed to add sysfs [node%d]\n", nid);
        goto err_del_kobj;
    }
}

err_del_kobj:

    kobject_del(wi_kobj);
err_put_kobj:
    kobject_put(wi_kobj);
    return err;

This structure keeps error handling more consistent and avoids redundancy.

Quoted 57 lines
Quoted 34 lines
mm/mempolicy.c +8 -10
+		goto err_out;
+	}
+
 	return 0;
+
+node_out:
+	kfree(node_attrs);
+err_out:
+	return err;
 }
 
 static void mempolicy_kobj_release(struct kobject *kobj)
@@ -3528,7 +3546,6 @@ static void mempolicy_kobj_release(struct kobject *kobj)
 	mutex_unlock(&iw_table_lock);
 	synchronize_rcu();
 	kfree(old);
-	kfree(node_attrs);
 	kfree(kobj);
 }
 
@@ -3542,37 +3559,24 @@ static int __init mempolicy_sysfs_init(void)
 	static struct kobject *mempolicy_kobj;
 
 	mempolicy_kobj = kzalloc(sizeof(*mempolicy_kobj), GFP_KERNEL);
-	if (!mempolicy_kobj) {
-		err = -ENOMEM;
-		goto err_out;
-	}
-
-	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
-			     GFP_KERNEL);
-	if (!node_attrs) {
-		err = -ENOMEM;

mm/mempolicy.c static int __init mempolicy_sysfs_init(void) · line 3568

Quoted 10 lines
mm/mempolicy.c +3 -3
-		goto mempol_out;
-	}
+	if (!mempolicy_kobj)
+		return -ENOMEM;
 
 	err = kobject_init_and_add(mempolicy_kobj, &mempolicy_ktype, mm_kobj,
 				   "mempolicy");
 	if (err)
-		goto node_out;
+		goto err_out;
		goto err_put_kobj;

or something like that.

Quoted 10 lines
mm/mempolicy.c +2 -5
 	err = add_weighted_interleave_group(mempolicy_kobj);
-	if (err) {
-		pr_err("mempolicy sysfs structure failed to initialize\n");
-		kobject_put(mempolicy_kobj);
-		return err;
-	}
+	if (err)
+		goto err_del;
 
Quoted 14 lines

mm/mempolicy.c static int __init mempolicy_sysfs_init(void) · line 3579

Quoted 12 lines
mm/mempolicy.c +5 -6
-	return err;
-node_out:
-	kfree(node_attrs);
-mempol_out:
-	kfree(mempolicy_kobj);
+	return 0;
+
+err_del:
+	kobject_del(mempolicy_kobj);
 err_out:
-	pr_err("failed to add mempolicy kobject to the system\n");
+	kobject_put(mempolicy_kobj);

If we keep an err_out, usual expectation is all it does is return
+ maybe print a message. We'd not expect a put.

Lastly, I agree with your point about err_out.
I will revise it to use err_del_kobj and err_put_kobj as needed.

Thanks again for the detailed review.

Rakie

Quoted 5 lines
Quoted 4 lines
mm/mempolicy.c +0 -0
 	return err;
 }
 
author reviewer
#4 lore raw

PATCH v6 2/3 mm/mempolicy: Prepare weighted interleave sysfs for memory hotplug

In reply to Rakie Kim

Previously, the weighted interleave sysfs structure was statically managed during initialization. This prevented new nodes from being recognized when memory hotplug events occurred, limiting the ability to update or extend sysfs entries dynamically at runtime.

To address this, this patch refactors the sysfs infrastructure and encapsulates it within a new structure, sysfs_wi_group, which holds both the kobject and an array of node attribute pointers.

By allocating this group structure globally, the per-node sysfs attributes can be managed beyond initialization time, enabling external modules to insert or remove node entries in response to events such as memory hotplug or node online/offline transitions.

Instead of allocating all per-node sysfs attributes at once, the initialization path now uses the existing sysfs_wi_node_add() and sysfs_wi_node_delete() helpers. This refactoring makes it possible to modularly manage per-node sysfs entries and ensures the infrastructure is ready for runtime extension.

---
 mm/mempolicy.c | 73 ++++++++++++++++++++++----------------------------
 1 file changed, 32 insertions(+), 41 deletions(-)
mm/mempolicy.c +32 -41
index af3753925573..73a9405ff352 100644
@@ -3388,6 +3388,13 @@ struct iw_node_attr {
 	int nid;
 };
 
+struct sysfs_wi_group {
+	struct kobject wi_kobj;
+	struct iw_node_attr *nattrs[];
+};
+
+static struct sysfs_wi_group *wi_group;
+
 static ssize_t node_show(struct kobject *kobj, struct kobj_attribute *attr,
 			 char *buf)
 {
@@ -3430,27 +3437,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 	return count;
 }
 
-static struct iw_node_attr **node_attrs;
-
-static void sysfs_wi_node_release(struct iw_node_attr *node_attr,
-				  struct kobject *parent)
+static void sysfs_wi_node_delete(int nid)
 {
-	if (!node_attr)
+	if (!wi_group->nattrs[nid])
 		return;
-	sysfs_remove_file(parent, &node_attr->kobj_attr.attr);
-	kfree(node_attr->kobj_attr.attr.name);
-	kfree(node_attr);
+
+	sysfs_remove_file(&wi_group->wi_kobj,
+			  &wi_group->nattrs[nid]->kobj_attr.attr);
+	kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
+	kfree(wi_group->nattrs[nid]);
 }
 
 static void sysfs_wi_release(struct kobject *wi_kobj)
 {
-	int i;
-
-	for (i = 0; i < nr_node_ids; i++)
-		sysfs_wi_node_release(node_attrs[i], wi_kobj);
+	int nid;
 
-	kfree(node_attrs);
-	kfree(wi_kobj);
+	for (nid = 0; nid < nr_node_ids; nid++)
+		sysfs_wi_node_delete(nid);
+	kfree(wi_group);
 }
 
 static const struct kobj_type wi_ktype = {
@@ -3458,7 +3462,7 @@ static const struct kobj_type wi_ktype = {
 	.release = sysfs_wi_release,
 };
 
-static int add_weight_node(int nid, struct kobject *wi_kobj)
+static int sysfs_wi_node_add(int nid)
 {
 	struct iw_node_attr *node_attr;
 	char *name;
@@ -3480,58 +3484,45 @@ static int add_weight_node(int nid, struct kobject *wi_kobj)
 	node_attr->kobj_attr.store = node_store;
 	node_attr->nid = nid;
 
-	if (sysfs_create_file(wi_kobj, &node_attr->kobj_attr.attr)) {
+	if (sysfs_create_file(&wi_group->wi_kobj, &node_attr->kobj_attr.attr)) {
 		kfree(node_attr->kobj_attr.attr.name);
 		kfree(node_attr);
 		pr_err("failed to add attribute to weighted_interleave\n");
 		return -ENOMEM;
 	}
 
-	node_attrs[nid] = node_attr;
+	wi_group->nattrs[nid] = node_attr;
 	return 0;
 }
 
-static int add_weighted_interleave_group(struct kobject *root_kobj)
+static int add_weighted_interleave_group(struct kobject *mempolicy_kobj)
 {
-	struct kobject *wi_kobj;
 	int nid, err;
 
-	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
-			     GFP_KERNEL);
-	if (!node_attrs)
+	wi_group = kzalloc(struct_size(wi_group, nattrs, nr_node_ids),
+			GFP_KERNEL);
+	if (!wi_group)
 		return -ENOMEM;
 
-	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
-	if (!wi_kobj) {
-		err = -ENOMEM;
-		goto node_out;
-	}
-
-	err = kobject_init_and_add(wi_kobj, &wi_ktype, root_kobj,
+	err = kobject_init_and_add(&wi_group->wi_kobj, &wi_ktype, mempolicy_kobj,
 				   "weighted_interleave");
-	if (err) {
-		kobject_put(wi_kobj);
+	if (err)
 		goto err_out;
-	}
 
 	for_each_node_state(nid, N_POSSIBLE) {
-		err = add_weight_node(nid, wi_kobj);
+		err = sysfs_wi_node_add(nid);
 		if (err) {
 			pr_err("failed to add sysfs [node%d]\n", nid);
-			break;
+			goto err_del;
 		}
 	}
-	if (err) {
-		kobject_del(wi_kobj);
-		kobject_put(wi_kobj);
-		goto err_out;
-	}
 
 	return 0;
 
-node_out:
-	kfree(node_attrs);
+err_del:
+	kobject_del(&wi_group->wi_kobj);
 err_out:
+	kobject_put(&wi_group->wi_kobj);
 	return err;
 }
 
Signature · 1 line
2.34.1
#5 lore raw

Re: [PATCH v6 2/3] mm/mempolicy: Prepare weighted interleave sysfs for memory hotplug

Quoted 20 lines · On Fri, 4 Apr 2025 16:46:20 +0900 Rakie Kim <[email protected]> wrote:

Previously, the weighted interleave sysfs structure was statically managed during initialization. This prevented new nodes from being recognized when memory hotplug events occurred, limiting the ability to update or extend sysfs entries dynamically at runtime.

To address this, this patch refactors the sysfs infrastructure and encapsulates it within a new structure, sysfs_wi_group, which holds both the kobject and an array of node attribute pointers.

By allocating this group structure globally, the per-node sysfs attributes can be managed beyond initialization time, enabling external modules to insert or remove node entries in response to events such as memory hotplug or node online/offline transitions.

Instead of allocating all per-node sysfs attributes at once, the initialization path now uses the existing sysfs_wi_node_add() and sysfs_wi_node_delete() helpers. This refactoring makes it possible to modularly manage per-node sysfs entries and ensures the infrastructure is ready for runtime extension.

Quoted 4 lines

Hi Rakie,

Some things I was requesting in patch 1 are done here. Mostly I think what is wanted is moving some of that refactoring back to that patch rather than here.

Some of the label and function naming needs another look.

Jonathan

Quoted 23 lines
---
 mm/mempolicy.c | 73 ++++++++++++++++++++++----------------------------
 1 file changed, 32 insertions(+), 41 deletions(-)
mm/mempolicy.c +7 -0
index af3753925573..73a9405ff352 100644
@@ -3388,6 +3388,13 @@ struct iw_node_attr {
 	int nid;
 };
 
+struct sysfs_wi_group {
+	struct kobject wi_kobj;
+	struct iw_node_attr *nattrs[];
+};
+
+static struct sysfs_wi_group *wi_group;
+
 static ssize_t node_show(struct kobject *kobj, struct kobj_attribute *attr,
 			 char *buf)
 {

mm/mempolicy.c static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr, · line 3440

Quoted 8 lines
mm/mempolicy.c +1 -4
@@ -3430,27 +3437,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 	return count;
 }
 
-static struct iw_node_attr **node_attrs;
-
-static void sysfs_wi_node_release(struct iw_node_attr *node_attr,
-				  struct kobject *parent)
+static void sysfs_wi_node_delete(int nid)

Maybe stick to release naming to match the sysfs_wi_release() below? I don't really care about this.

Quoted 32 lines
mm/mempolicy.c +11 -11
 {
-	if (!node_attr)
+	if (!wi_group->nattrs[nid])
 		return;
-	sysfs_remove_file(parent, &node_attr->kobj_attr.attr);
-	kfree(node_attr->kobj_attr.attr.name);
-	kfree(node_attr);
+
+	sysfs_remove_file(&wi_group->wi_kobj,
+			  &wi_group->nattrs[nid]->kobj_attr.attr);
+	kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
+	kfree(wi_group->nattrs[nid]);
 }
 
 static void sysfs_wi_release(struct kobject *wi_kobj)
 {
-	int i;
-
-	for (i = 0; i < nr_node_ids; i++)
-		sysfs_wi_node_release(node_attrs[i], wi_kobj);
+	int nid;
 
-	kfree(node_attrs);
-	kfree(wi_kobj);
+	for (nid = 0; nid < nr_node_ids; nid++)
+		sysfs_wi_node_delete(nid);
+	kfree(wi_group);
 }
 
-static int add_weighted_interleave_group(struct kobject *root_kobj)
+static int add_weighted_interleave_group(struct kobject *mempolicy_kobj)

{

mm/mempolicy.c static int add_weight_node(int nid, struct kobject *wi_kobj) · line 3503

Quoted 8 lines
mm/mempolicy.c +2 -4
-	struct kobject *wi_kobj;
 	int nid, err;
 
-	node_attrs = kcalloc(nr_node_ids, sizeof(struct iw_node_attr *),
-			     GFP_KERNEL);
-	if (!node_attrs)
+	wi_group = kzalloc(struct_size(wi_group, nattrs, nr_node_ids),
+			GFP_KERNEL);

Align to after the (
I think it's a couple of spaces short of that.

Quoted 15 lines
mm/mempolicy.c +3 -9
+	if (!wi_group)
 		return -ENOMEM;
 
-	wi_kobj = kzalloc(sizeof(struct kobject), GFP_KERNEL);
-	if (!wi_kobj) {
-		err = -ENOMEM;
-		goto node_out;
-	}
-
-	err = kobject_init_and_add(wi_kobj, &wi_ktype, root_kobj,
+	err = kobject_init_and_add(&wi_group->wi_kobj, &wi_ktype, mempolicy_kobj,
 				   "weighted_interleave");
-	if (err) {
-		kobject_put(wi_kobj);
+	if (err)

mm/mempolicy.c static int add_weight_node(int nid, struct kobject *wi_kobj) · line 3516

Quoted 10 lines
mm/mempolicy.c +2 -3
 		goto err_out;
-	}
 
 	for_each_node_state(nid, N_POSSIBLE) {
-		err = add_weight_node(nid, wi_kobj);
+		err = sysfs_wi_node_add(nid);
 		if (err) {
 			pr_err("failed to add sysfs [node%d]\n", nid);
-			break;
+			goto err_del;

Ah! This is what I was looking for in patch 1, but it's down here. Move it back to there.

Quoted 6 lines
mm/mempolicy.c +0 -4
 		}
 	}
-	if (err) {
-		kobject_del(wi_kobj);
-		kobject_put(wi_kobj);
-		goto err_out;

mm/mempolicy.c static int add_weight_node(int nid, struct kobject *wi_kobj) · line 3525

Quoted 10 lines
mm/mempolicy.c +3 -3
-	}
 
 	return 0;
 
-node_out:
-	kfree(node_attrs);
+err_del:
+	kobject_del(&wi_group->wi_kobj);
 err_out:
+	kobject_put(&wi_group->wi_kobj);

Same issue as previous patch on naming of the label. Moving to this single error block is fine but belongs in patch 1.

Quoted 3 lines

return err;
}

#6 lore raw

Re: [PATCH v6 2/3] mm/mempolicy: Prepare weighted interleave sysfs for memory hotplug

Quoted 36 lines · Jonathan Cameron
Quoted 20 lines · On Fri, 4 Apr 2025 16:46:20 +0900 Rakie Kim <[email protected]> wrote:

Previously, the weighted interleave sysfs structure was statically managed during initialization. This prevented new nodes from being recognized when memory hotplug events occurred, limiting the ability to update or extend sysfs entries dynamically at runtime.

To address this, this patch refactors the sysfs infrastructure and encapsulates it within a new structure, sysfs_wi_group, which holds both the kobject and an array of node attribute pointers.

By allocating this group structure globally, the per-node sysfs attributes can be managed beyond initialization time, enabling external modules to insert or remove node entries in response to events such as memory hotplug or node online/offline transitions.

Instead of allocating all per-node sysfs attributes at once, the initialization path now uses the existing sysfs_wi_node_add() and sysfs_wi_node_delete() helpers. This refactoring makes it possible to modularly manage per-node sysfs entries and ensures the infrastructure is ready for runtime extension.

Quoted 4 lines

Hi Rakie,

Some things I was requesting in patch 1 are done here. Mostly I think what is wanted is moving some of that refactoring back to that patch rather than here.

Some of the label and function naming needs another look.

Jonathan

[..]

Quoted 4 lines
Quoted 4 lines
mm/mempolicy.c +0 -0
@@ -3430,27 +3437,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 	return count;
 }
 
Quoted 8 lines

mm/mempolicy.c static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr, · line 3440

Quoted 5 lines
mm/mempolicy.c +1 -4
-static struct iw_node_attr **node_attrs;
-
-static void sysfs_wi_node_release(struct iw_node_attr *node_attr,
-				  struct kobject *parent)
+static void sysfs_wi_node_delete(int nid)

Maybe stick to release naming to match the sysfs_wi_release() below? I don't really care about this.

I had asked for "delete" to pair with "add" and to not get confused with a final kobject_put() callback.

author reviewer
#7 lore raw

Re: [PATCH v6 2/3] mm/mempolicy: Prepare weighted interleave sysfs for memory hotplug

Quoted 51 lines · On Fri, 4 Apr 2025 10:23:10 -0700 Dan Williams <[email protected]> wrote:
Quoted 36 lines · Jonathan Cameron
Quoted 20 lines · On Fri, 4 Apr 2025 16:46:20 +0900 Rakie Kim <[email protected]> wrote:

Previously, the weighted interleave sysfs structure was statically managed during initialization. This prevented new nodes from being recognized when memory hotplug events occurred, limiting the ability to update or extend sysfs entries dynamically at runtime.

To address this, this patch refactors the sysfs infrastructure and encapsulates it within a new structure, sysfs_wi_group, which holds both the kobject and an array of node attribute pointers.

By allocating this group structure globally, the per-node sysfs attributes can be managed beyond initialization time, enabling external modules to insert or remove node entries in response to events such as memory hotplug or node online/offline transitions.

Instead of allocating all per-node sysfs attributes at once, the initialization path now uses the existing sysfs_wi_node_add() and sysfs_wi_node_delete() helpers. This refactoring makes it possible to modularly manage per-node sysfs entries and ensures the infrastructure is ready for runtime extension.

Quoted 4 lines

Hi Rakie,

Some things I was requesting in patch 1 are done here. Mostly I think what is wanted is moving some of that refactoring back to that patch rather than here.

Some of the label and function naming needs another look.

Jonathan

[..]

Quoted 4 lines
Quoted 4 lines
mm/mempolicy.c +0 -0
@@ -3430,27 +3437,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 	return count;
 }
 
Quoted 8 lines

mm/mempolicy.c static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr, · line 3440

Quoted 5 lines
mm/mempolicy.c +1 -4
-static struct iw_node_attr **node_attrs;
-
-static void sysfs_wi_node_release(struct iw_node_attr *node_attr,
-				  struct kobject *parent)
+static void sysfs_wi_node_delete(int nid)

Maybe stick to release naming to match the sysfs_wi_release() below? I don't really care about this.

Quoted 2 lines

I had asked for "delete" to pair with "add" and to not get confused with a final kobject_put() callback.

As Dan mentioned earlier, I also believe that "sysfs_wi_node_delete" is a more appropriate name than "sysfs_wi_node_release" because it pairs naturally with "sysfs_wi_node_add" and avoids confusion with a final kobject_put() callback.

Rakie

#8 lore raw

Re: [PATCH v6 2/3] mm/mempolicy: Prepare weighted interleave sysfs for memory hotplug

In reply to Dan Williams
Quoted 51 lines · On Fri, 4 Apr 2025 10:23:10 -0700 Dan Williams <[email protected]> wrote:
Quoted 36 lines · Jonathan Cameron
Quoted 20 lines · On Fri, 4 Apr 2025 16:46:20 +0900 Rakie Kim <[email protected]> wrote:

Previously, the weighted interleave sysfs structure was statically managed during initialization. This prevented new nodes from being recognized when memory hotplug events occurred, limiting the ability to update or extend sysfs entries dynamically at runtime.

To address this, this patch refactors the sysfs infrastructure and encapsulates it within a new structure, sysfs_wi_group, which holds both the kobject and an array of node attribute pointers.

By allocating this group structure globally, the per-node sysfs attributes can be managed beyond initialization time, enabling external modules to insert or remove node entries in response to events such as memory hotplug or node online/offline transitions.

Instead of allocating all per-node sysfs attributes at once, the initialization path now uses the existing sysfs_wi_node_add() and sysfs_wi_node_delete() helpers. This refactoring makes it possible to modularly manage per-node sysfs entries and ensures the infrastructure is ready for runtime extension.

Quoted 4 lines

Hi Rakie,

Some things I was requesting in patch 1 are done here. Mostly I think what is wanted is moving some of that refactoring back to that patch rather than here.

Some of the label and function naming needs another look.

Jonathan

[..]

Quoted 4 lines
Quoted 4 lines
mm/mempolicy.c +0 -0
@@ -3430,27 +3437,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 	return count;
 }
 
Quoted 8 lines

mm/mempolicy.c static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr, · line 3440

Quoted 5 lines
mm/mempolicy.c +1 -4
-static struct iw_node_attr **node_attrs;
-
-static void sysfs_wi_node_release(struct iw_node_attr *node_attr,
-				  struct kobject *parent)
+static void sysfs_wi_node_delete(int nid)  

Maybe stick to release naming to match the sysfs_wi_release() below? I don't really care about this.

Quoted 3 lines

I had asked for "delete" to pair with "add" and to not get confused with a final kobject_put() callback.

Fair enough.

author reviewer
#9 lore raw

PATCH v6 3/3 mm/mempolicy: Support memory hotplug in weighted interleave

In reply to Rakie Kim

The weighted interleave policy distributes page allocations across multiple NUMA nodes based on their performance weight, thereby improving memory bandwidth utilization. The weight values for each node are configured through sysfs.

Previously, sysfs entries for configuring weighted interleave were created for all possible nodes (N_POSSIBLE) at initialization, including nodes that might not have memory. However, not all nodes in N_POSSIBLE are usable at runtime, as some may remain memoryless or offline.
This led to sysfs entries being created for unusable nodes, causing potential misconfiguration issues.

To address this issue, this patch modifies the sysfs creation logic to:
1) Limit sysfs entries to nodes that are online and have memory, avoiding the creation of sysfs entries for nodes that cannot be used.
2) Support memory hotplug by dynamically adding and removing sysfs entries based on whether a node transitions into or out of the N_MEMORY state.

Additionally, the patch ensures that sysfs attributes are properly managed when nodes go offline, preventing stale or redundant entries from persisting in the system.

By making these changes, the weighted interleave policy now manages its sysfs entries more efficiently, ensuring that only relevant nodes are considered for interleaving, and dynamically adapting to memory hotplug events.

---
 mm/mempolicy.c | 109 ++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 86 insertions(+), 23 deletions(-)
mm/mempolicy.c +86 -23
index 73a9405ff352..f25c2c7f8fcf 100644
@@ -113,6 +113,7 @@
 #include <asm/tlbflush.h>
 #include <asm/tlb.h>
 #include <linux/uaccess.h>
+#include <linux/memory.h>
 
 #include "internal.h"
 
@@ -3390,6 +3391,7 @@ struct iw_node_attr {
 
 struct sysfs_wi_group {
 	struct kobject wi_kobj;
+	struct mutex kobj_lock;
 	struct iw_node_attr *nattrs[];
 };
 
@@ -3439,13 +3441,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 
 static void sysfs_wi_node_delete(int nid)
 {
-	if (!wi_group->nattrs[nid])
+	struct iw_node_attr *attr;
+
+	if (nid < 0 || nid >= nr_node_ids)
+		return;
+
+	mutex_lock(&wi_group->kobj_lock);
+	attr = wi_group->nattrs[nid];
+	if (!attr) {
+		mutex_unlock(&wi_group->kobj_lock);
 		return;
+	}
+
+	wi_group->nattrs[nid] = NULL;
+	mutex_unlock(&wi_group->kobj_lock);
 
-	sysfs_remove_file(&wi_group->wi_kobj,
-			  &wi_group->nattrs[nid]->kobj_attr.attr);
-	kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
-	kfree(wi_group->nattrs[nid]);
+	sysfs_remove_file(&wi_group->wi_kobj, &attr->kobj_attr.attr);
+	kfree(attr->kobj_attr.attr.name);
+	kfree(attr);
 }
 
 static void sysfs_wi_release(struct kobject *wi_kobj)
@@ -3464,35 +3477,80 @@ static const struct kobj_type wi_ktype = {
 
 static int sysfs_wi_node_add(int nid)
 {
-	struct iw_node_attr *node_attr;
+	int ret = 0;
 	char *name;
+	struct iw_node_attr *new_attr = NULL;
 
-	node_attr = kzalloc(sizeof(*node_attr), GFP_KERNEL);
-	if (!node_attr)
+	if (nid < 0 || nid >= nr_node_ids) {
+		pr_err("Invalid node id: %d\n", nid);
+		return -EINVAL;
+	}
+
+	new_attr = kzalloc(sizeof(struct iw_node_attr), GFP_KERNEL);
+	if (!new_attr)
 		return -ENOMEM;
 
 	name = kasprintf(GFP_KERNEL, "node%d", nid);
 	if (!name) {
-		kfree(node_attr);
+		kfree(new_attr);
 		return -ENOMEM;
 	}
 
-	sysfs_attr_init(&node_attr->kobj_attr.attr);
-	node_attr->kobj_attr.attr.name = name;
-	node_attr->kobj_attr.attr.mode = 0644;
-	node_attr->kobj_attr.show = node_show;
-	node_attr->kobj_attr.store = node_store;
-	node_attr->nid = nid;
+	mutex_lock(&wi_group->kobj_lock);
+	if (wi_group->nattrs[nid]) {
+		mutex_unlock(&wi_group->kobj_lock);
+		pr_info("Node [%d] already exists\n", nid);
+		kfree(new_attr);
+		kfree(name);
+		return 0;
+	}
+	wi_group->nattrs[nid] = new_attr;
 
-	if (sysfs_create_file(&wi_group->wi_kobj, &node_attr->kobj_attr.attr)) {
-		kfree(node_attr->kobj_attr.attr.name);
-		kfree(node_attr);
-		pr_err("failed to add attribute to weighted_interleave\n");
-		return -ENOMEM;
+	sysfs_attr_init(&wi_group->nattrs[nid]->kobj_attr.attr);
+	wi_group->nattrs[nid]->kobj_attr.attr.name = name;
+	wi_group->nattrs[nid]->kobj_attr.attr.mode = 0644;
+	wi_group->nattrs[nid]->kobj_attr.show = node_show;
+	wi_group->nattrs[nid]->kobj_attr.store = node_store;
+	wi_group->nattrs[nid]->nid = nid;
+
+	ret = sysfs_create_file(&wi_group->wi_kobj,
+				&wi_group->nattrs[nid]->kobj_attr.attr);
+	if (ret) {
+		kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
+		kfree(wi_group->nattrs[nid]);
+		wi_group->nattrs[nid] = NULL;
+		pr_err("Failed to add attribute to weighted_interleave: %d\n", ret);
 	}
+	mutex_unlock(&wi_group->kobj_lock);
 
-	wi_group->nattrs[nid] = node_attr;
-	return 0;
+	return ret;
+}
+
+static int wi_node_notifier(struct notifier_block *nb,
+			       unsigned long action, void *data)
+{
+	int err;
+	struct memory_notify *arg = data;
+	int nid = arg->status_change_nid;
+
+	if (nid < 0)
+		goto notifier_end;
+
+	switch(action) {
+	case MEM_ONLINE:
+		err = sysfs_wi_node_add(nid);
+		if (err) {
+			pr_err("failed to add sysfs [node%d]\n", nid);
+			return NOTIFY_BAD;
+		}
+		break;
+	case MEM_OFFLINE:
+		sysfs_wi_node_delete(nid);
+		break;
+	}
+
+notifier_end:
+	return NOTIFY_OK;
 }
 
 static int add_weighted_interleave_group(struct kobject *mempolicy_kobj)
@@ -3503,13 +3561,17 @@ static int add_weighted_interleave_group(struct kobject *mempolicy_kobj)
 			GFP_KERNEL);
 	if (!wi_group)
 		return -ENOMEM;
+	mutex_init(&wi_group->kobj_lock);
 
 	err = kobject_init_and_add(&wi_group->wi_kobj, &wi_ktype, mempolicy_kobj,
 				   "weighted_interleave");
 	if (err)
 		goto err_out;
 
-	for_each_node_state(nid, N_POSSIBLE) {
+	for_each_online_node(nid) {
+		if (!node_state(nid, N_MEMORY))
+			continue;
+
 		err = sysfs_wi_node_add(nid);
 		if (err) {
 			pr_err("failed to add sysfs [node%d]\n", nid);
@@ -3517,6 +3579,7 @@ static int add_weighted_interleave_group(struct kobject *mempolicy_kobj)
 		}
 	}
 
+	hotplug_memory_notifier(wi_node_notifier, DEFAULT_CALLBACK_PRI);
 	return 0;
 
 err_del:
Signature · 1 line
2.34.1
#10 lore raw

Re: [PATCH v6 3/3] mm/mempolicy: Support memory hotplug in weighted interleave

Quoted 22 lines · Rakie Kim

The weighted interleave policy distributes page allocations across multiple NUMA nodes based on their performance weight, thereby improving memory bandwidth utilization. The weight values for each node are configured through sysfs.

Previously, sysfs entries for configuring weighted interleave were created for all possible nodes (N_POSSIBLE) at initialization, including nodes that might not have memory. However, not all nodes in N_POSSIBLE are usable at runtime, as some may remain memoryless or offline.
This led to sysfs entries being created for unusable nodes, causing potential misconfiguration issues.

To address this issue, this patch modifies the sysfs creation logic to:
1) Limit sysfs entries to nodes that are online and have memory, avoiding the creation of sysfs entries for nodes that cannot be used.
2) Support memory hotplug by dynamically adding and removing sysfs entries based on whether a node transitions into or out of the N_MEMORY state.

Additionally, the patch ensures that sysfs attributes are properly managed when nodes go offline, preventing stale or redundant entries from persisting in the system.

Quoted 8 lines

By making these changes, the weighted interleave policy now manages its sysfs entries more efficiently, ensuring that only relevant nodes are considered for interleaving, and dynamically adapting to memory hotplug events.

For the memory-hotplug bits: Reviewed-by: Oscar Salvador <[email protected]<

Just one thing that caught my eye:

Cannot add_weighted_interleave_group be __init? AFAICS, it only gets called at boot time?

Signature · 2 lines
Oscar Salvador
SUSE Labs
author reviewer
#11 lore raw

Re: [PATCH v6 3/3] mm/mempolicy: Support memory hotplug in weighted interleave

Quoted 32 lines · On Fri, 4 Apr 2025 10:43:18 +0200 Oscar Salvador <[email protected]> wrote:
Quoted 22 lines · Rakie Kim

The weighted interleave policy distributes page allocations across multiple NUMA nodes based on their performance weight, thereby improving memory bandwidth utilization. The weight values for each node are configured through sysfs.

Previously, sysfs entries for configuring weighted interleave were created for all possible nodes (N_POSSIBLE) at initialization, including nodes that might not have memory. However, not all nodes in N_POSSIBLE are usable at runtime, as some may remain memoryless or offline.
This led to sysfs entries being created for unusable nodes, causing potential misconfiguration issues.

To address this issue, this patch modifies the sysfs creation logic to:
1) Limit sysfs entries to nodes that are online and have memory, avoiding the creation of sysfs entries for nodes that cannot be used.
2) Support memory hotplug by dynamically adding and removing sysfs entries based on whether a node transitions into or out of the N_MEMORY state.

Additionally, the patch ensures that sysfs attributes are properly managed when nodes go offline, preventing stale or redundant entries from persisting in the system.

Quoted 8 lines

By making these changes, the weighted interleave policy now manages its sysfs entries more efficiently, ensuring that only relevant nodes are considered for interleaving, and dynamically adapting to memory hotplug events.

Quoted 12 lines

For the memory-hotplug bits: Reviewed-by: Oscar Salvador <[email protected]<

Just one thing that caught my eye:

Cannot add_weighted_interleave_group be __init? AFAICS, it only gets called at boot time?

Signature · 2 lines
Oscar Salvador
SUSE Labs

Thank you for your response regarding this patch.
I agree that add_weighted_interleave_group can be marked with __init, as it is only called during system boot.
I will make this change in the next revision.

Rakie

#12 lore raw

Re: [PATCH v6 3/3] mm/mempolicy: Support memory hotplug in weighted interleave

In reply to Rakie Kim
Quoted 156 lines · Rakie Kim

The weighted interleave policy distributes page allocations across multiple NUMA nodes based on their performance weight, thereby improving memory bandwidth utilization. The weight values for each node are configured through sysfs.

Previously, sysfs entries for configuring weighted interleave were created for all possible nodes (N_POSSIBLE) at initialization, including nodes that might not have memory. However, not all nodes in N_POSSIBLE are usable at runtime, as some may remain memoryless or offline.
This led to sysfs entries being created for unusable nodes, causing potential misconfiguration issues.

To address this issue, this patch modifies the sysfs creation logic to:
1) Limit sysfs entries to nodes that are online and have memory, avoiding the creation of sysfs entries for nodes that cannot be used.
2) Support memory hotplug by dynamically adding and removing sysfs entries based on whether a node transitions into or out of the N_MEMORY state.

Additionally, the patch ensures that sysfs attributes are properly managed when nodes go offline, preventing stale or redundant entries from persisting in the system.

By making these changes, the weighted interleave policy now manages its sysfs entries more efficiently, ensuring that only relevant nodes are considered for interleaving, and dynamically adapting to memory hotplug events.

---
  mm/mempolicy.c | 109 ++++++++++++++++++++++++++++++++++++++-----------
  1 file changed, 86 insertions(+), 23 deletions(-)
mm/mempolicy.c +57 -22
index 73a9405ff352..f25c2c7f8fcf 100644
@@ -113,6 +113,7 @@
  #include <asm/tlbflush.h>
  #include <asm/tlb.h>
  #include <linux/uaccess.h>
+#include <linux/memory.h>
 
  #include "internal.h"
 
@@ -3390,6 +3391,7 @@ struct iw_node_attr {
 
  struct sysfs_wi_group {
  	struct kobject wi_kobj;
+	struct mutex kobj_lock;
  	struct iw_node_attr *nattrs[];
  };
 
@@ -3439,13 +3441,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 
  static void sysfs_wi_node_delete(int nid)
  {
-	if (!wi_group->nattrs[nid])
+	struct iw_node_attr *attr;
+
+	if (nid < 0 || nid >= nr_node_ids)
+		return;
+
+	mutex_lock(&wi_group->kobj_lock);
+	attr = wi_group->nattrs[nid];
+	if (!attr) {
+		mutex_unlock(&wi_group->kobj_lock);
  		return;
+	}
+
+	wi_group->nattrs[nid] = NULL;
+	mutex_unlock(&wi_group->kobj_lock);
 
-	sysfs_remove_file(&wi_group->wi_kobj,
-			  &wi_group->nattrs[nid]->kobj_attr.attr);
-	kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
-	kfree(wi_group->nattrs[nid]);
+	sysfs_remove_file(&wi_group->wi_kobj, &attr->kobj_attr.attr);
+	kfree(attr->kobj_attr.attr.name);
+	kfree(attr);
  }
 
  static void sysfs_wi_release(struct kobject *wi_kobj)
@@ -3464,35 +3477,80 @@ static const struct kobj_type wi_ktype = {
 
  static int sysfs_wi_node_add(int nid)
  {
-	struct iw_node_attr *node_attr;
+	int ret = 0;
  	char *name;
+	struct iw_node_attr *new_attr = NULL;
 
-	node_attr = kzalloc(sizeof(*node_attr), GFP_KERNEL);
-	if (!node_attr)
+	if (nid < 0 || nid >= nr_node_ids) {
+		pr_err("Invalid node id: %d\n", nid);
+		return -EINVAL;
+	}
+
+	new_attr = kzalloc(sizeof(struct iw_node_attr), GFP_KERNEL);
+	if (!new_attr)
  		return -ENOMEM;
 
  	name = kasprintf(GFP_KERNEL, "node%d", nid);
  	if (!name) {
-		kfree(node_attr);
+		kfree(new_attr);
  		return -ENOMEM;
  	}
 
-	sysfs_attr_init(&node_attr->kobj_attr.attr);
-	node_attr->kobj_attr.attr.name = name;
-	node_attr->kobj_attr.attr.mode = 0644;
-	node_attr->kobj_attr.show = node_show;
-	node_attr->kobj_attr.store = node_store;
-	node_attr->nid = nid;
+	mutex_lock(&wi_group->kobj_lock);
+	if (wi_group->nattrs[nid]) {
+		mutex_unlock(&wi_group->kobj_lock);
+		pr_info("Node [%d] already exists\n", nid);
+		kfree(new_attr);
+		kfree(name);
+		return 0;
+	}
+	wi_group->nattrs[nid] = new_attr;
 
-	if (sysfs_create_file(&wi_group->wi_kobj, &node_attr->kobj_attr.attr)) {
-		kfree(node_attr->kobj_attr.attr.name);
-		kfree(node_attr);
-		pr_err("failed to add attribute to weighted_interleave\n");
-		return -ENOMEM;
+	sysfs_attr_init(&wi_group->nattrs[nid]->kobj_attr.attr);
+	wi_group->nattrs[nid]->kobj_attr.attr.name = name;
+	wi_group->nattrs[nid]->kobj_attr.attr.mode = 0644;
+	wi_group->nattrs[nid]->kobj_attr.show = node_show;
+	wi_group->nattrs[nid]->kobj_attr.store = node_store;
+	wi_group->nattrs[nid]->nid = nid;
+
+	ret = sysfs_create_file(&wi_group->wi_kobj,
+				&wi_group->nattrs[nid]->kobj_attr.attr);
+	if (ret) {
+		kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
+		kfree(wi_group->nattrs[nid]);
+		wi_group->nattrs[nid] = NULL;
+		pr_err("Failed to add attribute to weighted_interleave: %d\n", ret);
  	}
+	mutex_unlock(&wi_group->kobj_lock);
 
-	wi_group->nattrs[nid] = node_attr;
-	return 0;
+	return ret;
+}
+
+static int wi_node_notifier(struct notifier_block *nb,
+			       unsigned long action, void *data)

mm/mempolicy.c static const struct kobj_type wi_ktype = { · line 3540

Quoted 10 lines
mm/mempolicy.c +10 -0
+{
+	int err;
+	struct memory_notify *arg = data;
+	int nid = arg->status_change_nid;
+
+	if (nid < 0)
+		goto notifier_end;
+
+	switch(action) {
+	case MEM_ONLINE:

MEM_ONLINE is too late, we cannot fail hotplug at that point.

Would MEM_GOING_ONLINE / MEM_CANCEL_ONLINE be better?

mm/mempolicy.c static const struct kobj_type wi_ktype = { · line 3544

Quoted 4 lines
mm/mempolicy.c +4 -0
+		err = sysfs_wi_node_add(nid);
+		if (err) {
+			pr_err("failed to add sysfs [node%d]\n", nid);
+			return NOTIFY_BAD;

Note that NOTIFY_BAD includes NOTIFY_STOP_MASK. So you wouldn't call other notifiers, but the overall memory onlining would succeed, which is bad.

If we don't care about the error (not prevent hotplug) we could only pr_warn() and continue. Maybe this (unlikely) case is not a good reason to stop memory from getting onlined. OTOH, it will barely ever trigger in practice ...

Signature · 3 lines
Cheers,

David / dhildenb
author reviewer
#13 lore raw

Re: [PATCH v6 3/3] mm/mempolicy: Support memory hotplug in weighted interleave

Quoted 170 lines · On Fri, 4 Apr 2025 22:45:00 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 156 lines · Rakie Kim

The weighted interleave policy distributes page allocations across multiple NUMA nodes based on their performance weight, thereby improving memory bandwidth utilization. The weight values for each node are configured through sysfs.

Previously, sysfs entries for configuring weighted interleave were created for all possible nodes (N_POSSIBLE) at initialization, including nodes that might not have memory. However, not all nodes in N_POSSIBLE are usable at runtime, as some may remain memoryless or offline.
This led to sysfs entries being created for unusable nodes, causing potential misconfiguration issues.

To address this issue, this patch modifies the sysfs creation logic to:
1) Limit sysfs entries to nodes that are online and have memory, avoiding the creation of sysfs entries for nodes that cannot be used.
2) Support memory hotplug by dynamically adding and removing sysfs entries based on whether a node transitions into or out of the N_MEMORY state.

Additionally, the patch ensures that sysfs attributes are properly managed when nodes go offline, preventing stale or redundant entries from persisting in the system.

By making these changes, the weighted interleave policy now manages its sysfs entries more efficiently, ensuring that only relevant nodes are considered for interleaving, and dynamically adapting to memory hotplug events.

---
  mm/mempolicy.c | 109 ++++++++++++++++++++++++++++++++++++++-----------
  1 file changed, 86 insertions(+), 23 deletions(-)
mm/mempolicy.c +57 -22
index 73a9405ff352..f25c2c7f8fcf 100644
@@ -113,6 +113,7 @@
  #include <asm/tlbflush.h>
  #include <asm/tlb.h>
  #include <linux/uaccess.h>
+#include <linux/memory.h>
 
  #include "internal.h"
 
@@ -3390,6 +3391,7 @@ struct iw_node_attr {
 
  struct sysfs_wi_group {
  	struct kobject wi_kobj;
+	struct mutex kobj_lock;
  	struct iw_node_attr *nattrs[];
  };
 
@@ -3439,13 +3441,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 
  static void sysfs_wi_node_delete(int nid)
  {
-	if (!wi_group->nattrs[nid])
+	struct iw_node_attr *attr;
+
+	if (nid < 0 || nid >= nr_node_ids)
+		return;
+
+	mutex_lock(&wi_group->kobj_lock);
+	attr = wi_group->nattrs[nid];
+	if (!attr) {
+		mutex_unlock(&wi_group->kobj_lock);
  		return;
+	}
+
+	wi_group->nattrs[nid] = NULL;
+	mutex_unlock(&wi_group->kobj_lock);
 
-	sysfs_remove_file(&wi_group->wi_kobj,
-			  &wi_group->nattrs[nid]->kobj_attr.attr);
-	kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
-	kfree(wi_group->nattrs[nid]);
+	sysfs_remove_file(&wi_group->wi_kobj, &attr->kobj_attr.attr);
+	kfree(attr->kobj_attr.attr.name);
+	kfree(attr);
  }
 
  static void sysfs_wi_release(struct kobject *wi_kobj)
@@ -3464,35 +3477,80 @@ static const struct kobj_type wi_ktype = {
 
  static int sysfs_wi_node_add(int nid)
  {
-	struct iw_node_attr *node_attr;
+	int ret = 0;
  	char *name;
+	struct iw_node_attr *new_attr = NULL;
 
-	node_attr = kzalloc(sizeof(*node_attr), GFP_KERNEL);
-	if (!node_attr)
+	if (nid < 0 || nid >= nr_node_ids) {
+		pr_err("Invalid node id: %d\n", nid);
+		return -EINVAL;
+	}
+
+	new_attr = kzalloc(sizeof(struct iw_node_attr), GFP_KERNEL);
+	if (!new_attr)
  		return -ENOMEM;
 
  	name = kasprintf(GFP_KERNEL, "node%d", nid);
  	if (!name) {
-		kfree(node_attr);
+		kfree(new_attr);
  		return -ENOMEM;
  	}
 
-	sysfs_attr_init(&node_attr->kobj_attr.attr);
-	node_attr->kobj_attr.attr.name = name;
-	node_attr->kobj_attr.attr.mode = 0644;
-	node_attr->kobj_attr.show = node_show;
-	node_attr->kobj_attr.store = node_store;
-	node_attr->nid = nid;
+	mutex_lock(&wi_group->kobj_lock);
+	if (wi_group->nattrs[nid]) {
+		mutex_unlock(&wi_group->kobj_lock);
+		pr_info("Node [%d] already exists\n", nid);
+		kfree(new_attr);
+		kfree(name);
+		return 0;
+	}
+	wi_group->nattrs[nid] = new_attr;
 
-	if (sysfs_create_file(&wi_group->wi_kobj, &node_attr->kobj_attr.attr)) {
-		kfree(node_attr->kobj_attr.attr.name);
-		kfree(node_attr);
-		pr_err("failed to add attribute to weighted_interleave\n");
-		return -ENOMEM;
+	sysfs_attr_init(&wi_group->nattrs[nid]->kobj_attr.attr);
+	wi_group->nattrs[nid]->kobj_attr.attr.name = name;
+	wi_group->nattrs[nid]->kobj_attr.attr.mode = 0644;
+	wi_group->nattrs[nid]->kobj_attr.show = node_show;
+	wi_group->nattrs[nid]->kobj_attr.store = node_store;
+	wi_group->nattrs[nid]->nid = nid;
+
+	ret = sysfs_create_file(&wi_group->wi_kobj,
+				&wi_group->nattrs[nid]->kobj_attr.attr);
+	if (ret) {
+		kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
+		kfree(wi_group->nattrs[nid]);
+		wi_group->nattrs[nid] = NULL;
+		pr_err("Failed to add attribute to weighted_interleave: %d\n", ret);
  	}
+	mutex_unlock(&wi_group->kobj_lock);
 
-	wi_group->nattrs[nid] = node_attr;
-	return 0;
+	return ret;
+}
+
+static int wi_node_notifier(struct notifier_block *nb,
+			       unsigned long action, void *data)

mm/mempolicy.c static const struct kobj_type wi_ktype = { · line 3540

Quoted 10 lines
mm/mempolicy.c +10 -0
+{
+	int err;
+	struct memory_notify *arg = data;
+	int nid = arg->status_change_nid;
+
+	if (nid < 0)
+		goto notifier_end;
+
+	switch(action) {
+	case MEM_ONLINE:

MEM_ONLINE is too late, we cannot fail hotplug at that point.

Quoted 1 line

Would MEM_GOING_ONLINE / MEM_CANCEL_ONLINE be better?

Hi David,

Thank you for raising these points. I would appreciate your clarification on the following:

Issue1: I want to invoke sysfs_wi_node_add() after a node with memory has been fully transitioned to the online state. Does replacing MEM_ONLINE with MEM_GOING_ONLINE or MEM_CANCEL_ONLINE still ensure that the node is considered online and usable by that point?

Quoted 11 lines

mm/mempolicy.c static const struct kobj_type wi_ktype = { · line 3544

Quoted 6 lines
mm/mempolicy.c +4 -0
+		err = sysfs_wi_node_add(nid);
+		if (err) {
+			pr_err("failed to add sysfs [node%d]\n", nid);
+			return NOTIFY_BAD;

Note that NOTIFY_BAD includes NOTIFY_STOP_MASK. So you wouldn't call other notifiers, but the overall memory onlining would succeed, which is bad.

Quoted 5 lines

If we don't care about the error (not prevent hotplug) we could only pr_warn() and continue. Maybe this (unlikely) case is not a good reason to stop memory from getting onlined. OTOH, it will barely ever trigger in practice ...

Issue2: Regarding your note about NOTIFY_BAD ? are you saying that if sysfs_wi_node_add() returns NOTIFY_BAD, it will trigger NOTIFY_STOP_MASK, preventing other notifiers from running, while still allowing the memory hotplug operation to complete?

If so, then I'm thinking of resolving both issues as follows:
- For Issue1: I keep using MEM_ONLINE, assuming it is safe and sufficient to ensure the node is fully online.
- For Issue2: I avoid returning NOTIFY_BAD from the notifier. Instead, I log the error using pr_err() and continue the operation.

This would result in the following code:

	if (nid < 0)
		return NOTIFY_OK;

	switch (action) {
	case MEM_ONLINE: // Issue1: keeping this unchanged
		err = sysfs_wi_node_add(nid);
		if (err) {
			pr_err("failed to add sysfs [node%d]\n", nid);
			// Issue2: Do not return NOTIFY_BAD
		}
		break;
	case MEM_OFFLINE:
		sysfs_wi_node_delete(nid);
		break;
	}

	// Always return NOTIFY_OK
	return NOTIFY_OK;

Please let me know if this approach is acceptable.

Rakie

Quoted 5 lines
Signature · 3 lines
Cheers,

David / dhildenb
#14 lore raw

Re: [PATCH v6 3/3] mm/mempolicy: Support memory hotplug in weighted interleave

Quoted 171 lines · Rakie Kim
Quoted 170 lines · On Fri, 4 Apr 2025 22:45:00 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 156 lines · Rakie Kim

The weighted interleave policy distributes page allocations across multiple NUMA nodes based on their performance weight, thereby improving memory bandwidth utilization. The weight values for each node are configured through sysfs.

Previously, sysfs entries for configuring weighted interleave were created for all possible nodes (N_POSSIBLE) at initialization, including nodes that might not have memory. However, not all nodes in N_POSSIBLE are usable at runtime, as some may remain memoryless or offline.
This led to sysfs entries being created for unusable nodes, causing potential misconfiguration issues.

To address this issue, this patch modifies the sysfs creation logic to:
1) Limit sysfs entries to nodes that are online and have memory, avoiding the creation of sysfs entries for nodes that cannot be used.
2) Support memory hotplug by dynamically adding and removing sysfs entries based on whether a node transitions into or out of the N_MEMORY state.

Additionally, the patch ensures that sysfs attributes are properly managed when nodes go offline, preventing stale or redundant entries from persisting in the system.

By making these changes, the weighted interleave policy now manages its sysfs entries more efficiently, ensuring that only relevant nodes are considered for interleaving, and dynamically adapting to memory hotplug events.

---
   mm/mempolicy.c | 109 ++++++++++++++++++++++++++++++++++++++-----------
   1 file changed, 86 insertions(+), 23 deletions(-)
mm/mempolicy.c +57 -22
index 73a9405ff352..f25c2c7f8fcf 100644
@@ -113,6 +113,7 @@
   #include <asm/tlbflush.h>
   #include <asm/tlb.h>
   #include <linux/uaccess.h>
+#include <linux/memory.h>
 
   #include "internal.h"
 
@@ -3390,6 +3391,7 @@ struct iw_node_attr {
 
   struct sysfs_wi_group {
   	struct kobject wi_kobj;
+	struct mutex kobj_lock;
   	struct iw_node_attr *nattrs[];
   };
 
@@ -3439,13 +3441,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 
   static void sysfs_wi_node_delete(int nid)
   {
-	if (!wi_group->nattrs[nid])
+	struct iw_node_attr *attr;
+
+	if (nid < 0 || nid >= nr_node_ids)
+		return;
+
+	mutex_lock(&wi_group->kobj_lock);
+	attr = wi_group->nattrs[nid];
+	if (!attr) {
+		mutex_unlock(&wi_group->kobj_lock);
   		return;
+	}
+
+	wi_group->nattrs[nid] = NULL;
+	mutex_unlock(&wi_group->kobj_lock);
 
-	sysfs_remove_file(&wi_group->wi_kobj,
-			  &wi_group->nattrs[nid]->kobj_attr.attr);
-	kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
-	kfree(wi_group->nattrs[nid]);
+	sysfs_remove_file(&wi_group->wi_kobj, &attr->kobj_attr.attr);
+	kfree(attr->kobj_attr.attr.name);
+	kfree(attr);
   }
 
   static void sysfs_wi_release(struct kobject *wi_kobj)
@@ -3464,35 +3477,80 @@ static const struct kobj_type wi_ktype = {
 
   static int sysfs_wi_node_add(int nid)
   {
-	struct iw_node_attr *node_attr;
+	int ret = 0;
   	char *name;
+	struct iw_node_attr *new_attr = NULL;
 
-	node_attr = kzalloc(sizeof(*node_attr), GFP_KERNEL);
-	if (!node_attr)
+	if (nid < 0 || nid >= nr_node_ids) {
+		pr_err("Invalid node id: %d\n", nid);
+		return -EINVAL;
+	}
+
+	new_attr = kzalloc(sizeof(struct iw_node_attr), GFP_KERNEL);
+	if (!new_attr)
   		return -ENOMEM;
 
   	name = kasprintf(GFP_KERNEL, "node%d", nid);
   	if (!name) {
-		kfree(node_attr);
+		kfree(new_attr);
   		return -ENOMEM;
   	}
 
-	sysfs_attr_init(&node_attr->kobj_attr.attr);
-	node_attr->kobj_attr.attr.name = name;
-	node_attr->kobj_attr.attr.mode = 0644;
-	node_attr->kobj_attr.show = node_show;
-	node_attr->kobj_attr.store = node_store;
-	node_attr->nid = nid;
+	mutex_lock(&wi_group->kobj_lock);
+	if (wi_group->nattrs[nid]) {
+		mutex_unlock(&wi_group->kobj_lock);
+		pr_info("Node [%d] already exists\n", nid);
+		kfree(new_attr);
+		kfree(name);
+		return 0;
+	}
+	wi_group->nattrs[nid] = new_attr;
 
-	if (sysfs_create_file(&wi_group->wi_kobj, &node_attr->kobj_attr.attr)) {
-		kfree(node_attr->kobj_attr.attr.name);
-		kfree(node_attr);
-		pr_err("failed to add attribute to weighted_interleave\n");
-		return -ENOMEM;
+	sysfs_attr_init(&wi_group->nattrs[nid]->kobj_attr.attr);
+	wi_group->nattrs[nid]->kobj_attr.attr.name = name;
+	wi_group->nattrs[nid]->kobj_attr.attr.mode = 0644;
+	wi_group->nattrs[nid]->kobj_attr.show = node_show;
+	wi_group->nattrs[nid]->kobj_attr.store = node_store;
+	wi_group->nattrs[nid]->nid = nid;
+
+	ret = sysfs_create_file(&wi_group->wi_kobj,
+				&wi_group->nattrs[nid]->kobj_attr.attr);
+	if (ret) {
+		kfree(wi_group->nattrs[nid]->kobj_attr.attr.name);
+		kfree(wi_group->nattrs[nid]);
+		wi_group->nattrs[nid] = NULL;
+		pr_err("Failed to add attribute to weighted_interleave: %d\n", ret);
   	}
+	mutex_unlock(&wi_group->kobj_lock);
 
-	wi_group->nattrs[nid] = node_attr;
-	return 0;
+	return ret;
+}
+
+static int wi_node_notifier(struct notifier_block *nb,
+			       unsigned long action, void *data)

mm/mempolicy.c static const struct kobj_type wi_ktype = { · line 3540

Quoted 10 lines
mm/mempolicy.c +10 -0
+{
+	int err;
+	struct memory_notify *arg = data;
+	int nid = arg->status_change_nid;
+
+	if (nid < 0)
+		goto notifier_end;
+
+	switch(action) {
+	case MEM_ONLINE:

MEM_ONLINE is too late, we cannot fail hotplug at that point.

Quoted 3 lines
Quoted 1 line

Would MEM_GOING_ONLINE / MEM_CANCEL_ONLINE be better?

Hi David,

Hi,

Quoted 8 lines

Thank you for raising these points. I would appreciate your clarification on the following:

Issue1: I want to invoke sysfs_wi_node_add() after a node with memory has been fully transitioned to the online state. Does replacing MEM_ONLINE with MEM_GOING_ONLINE or MEM_CANCEL_ONLINE still ensure that the node is considered online and usable by that point?

After MEM_GOING_ONLINE nothing can go wrong except that some other notifier fails MEM_GOING_ONLINE.

That happens rarely -- only if some other allocation, like for kasan, fails.

In MEM_CANCEL_ONLINE you have to undo what you did (remove the node again).

Quoted 18 lines
Quoted 12 lines

mm/mempolicy.c static const struct kobj_type wi_ktype = { · line 3544

Quoted 7 lines
mm/mempolicy.c +4 -0
+		err = sysfs_wi_node_add(nid);
+		if (err) {
+			pr_err("failed to add sysfs [node%d]\n", nid);
+			return NOTIFY_BAD;

Note that NOTIFY_BAD includes NOTIFY_STOP_MASK. So you wouldn't call other notifiers, but the overall memory onlining would succeed, which is bad.

Quoted 5 lines

If we don't care about the error (not prevent hotplug) we could only pr_warn() and continue. Maybe this (unlikely) case is not a good reason to stop memory from getting onlined. OTOH, it will barely ever trigger in practice ...

Quoted 4 lines

Issue2: Regarding your note about NOTIFY_BAD ? are you saying that if sysfs_wi_node_add() returns NOTIFY_BAD, it will trigger NOTIFY_STOP_MASK, preventing other notifiers from running, while still allowing the memory hotplug operation to complete?

Yes.

Quoted 28 lines

If so, then I'm thinking of resolving both issues as follows:
- For Issue1: I keep using MEM_ONLINE, assuming it is safe and sufficient to ensure the node is fully online.
- For Issue2: I avoid returning NOTIFY_BAD from the notifier. Instead, I log the error using pr_err() and continue the operation.

This would result in the following code:

	if (nid < 0)
		return NOTIFY_OK;

	switch (action) {
	case MEM_ONLINE: // Issue1: keeping this unchanged
		err = sysfs_wi_node_add(nid);
		if (err) {
			pr_err("failed to add sysfs [node%d]\n", nid);
			// Issue2: Do not return NOTIFY_BAD
		}
		break;
	case MEM_OFFLINE:
		sysfs_wi_node_delete(nid);
		break;
	}

	// Always return NOTIFY_OK
	return NOTIFY_OK;
Quoted 1 line

Please let me know if this approach is acceptable.

That would work. The alternative is failing during MEM_GOING_ONLINE if the allocation failed, and deleting the node during MEM_CANCEL_ONLINE.

Signature · 3 lines
Cheers,

David / dhildenb