mm/memory_hotplug: maintain N_NORMAL_MEMORY during hotplug

PATCH Applied Hao Li 6 participants 10 messages Mar 30 – Mar 31

Subsystems (2)

Memory management Memory hot(un)plug

lore

Share & tools

lore

Hao Li

author

2026-03-30 04:00 UTC #0 lore raw

PATCH v2 1/1 mm/memory_hotplug: maintain N_NORMAL_MEMORY during hotplug

N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.

Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.

This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.

Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.

Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")

Signed-off-by Hao Li <[email protected]>

---
Changes: simplify the code. (Thanks Joshua and David)

---
 mm/memory_hotplug.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

mm/memory_hotplug.c +20 -0

index bc805029da51..05a47953ef21 100644
@@ -1209,6 +1209,13 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 
 	if (node_arg.nid >= 0)
 		node_set_state(nid, N_MEMORY);
+	/*
+	 * Check whether we are adding normal memory to the node for the first
+	 * time.
+	 */
+	if (!node_state(nid, N_NORMAL_MEMORY) && zone_idx(zone) <= ZONE_NORMAL)
+		node_set_state(nid, N_NORMAL_MEMORY);
+
 	if (need_zonelists_rebuild)
 		build_all_zonelists(NULL);

@@ -1908,6 +1915,8 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	unsigned long flags;
 	char *reason;
 	int ret;
+	unsigned long normal_pages = 0;
+	enum zone_type zt;
 
 	/*
 	 * {on,off}lining is constrained to full memory sections (or more

@@ -2055,6 +2064,17 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	/* reinitialise watermarks and update pcp limits */
 	init_per_zone_wmark_min();
 
+	/*
+	 * Check whether this operation removes the last normal memory from
+	 * the node. We do this before clearing N_MEMORY to avoid the possible
+	 * transient "!N_MEMORY && N_NORMAL_MEMORY" state.
+	 */
+	if (zone_idx(zone) <= ZONE_NORMAL) {
+		for (zt = 0; zt <= ZONE_NORMAL; zt++)
+			normal_pages += pgdat->node_zones[zt].present_pages;
+		if (!normal_pages)
+			node_clear_state(node, N_NORMAL_MEMORY);
+	}
 	/*
 	 * Make sure to mark the node as memory-less before rebuilding the zone
 	 * list. Otherwise this node would still appear in the fallback lists.

Signature · 1 line

2.50.1

Harry Yoo (Oracle)

Reviewed-by on v2 1/1

2026-03-30 08:28 UTC #1 lore raw

Quoted 11 lines · Hao Li

N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.
Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.

Quoted 8 lines

This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.
Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")

The Fixes: tag indeed looks correct to me.

By the way, MM needs an explicit Cc: [email protected] for backporting.

Since this commit was introduced in v6.17 and we have v6.18 as LTS, this should to be backported to v6.18.

Quoted 2 lines

Signed-off-by Hao Li <[email protected]>
---

Otherwise looks good to me,

Reviewed-by Harry Yoo (Oracle) <[email protected]>

Signature · 2 lines

Cheers,
Harry / Hyeonggon

Hao Li

author

2026-03-30 09:34 UTC #2 lore raw

Quoted 19 lines · Harry Yoo (Oracle)

Quoted 18 lines · Hao Li
N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.
Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.
This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.
Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.

Quoted 8 lines

Quoted 1 line
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")
The Fixes: tag indeed looks correct to me.
By the way, MM needs an explicit Cc: [email protected] for backporting.
Since this commit was introduced in v6.17 and we have v6.18 as LTS, this should to be backported to v6.18.

I had gone back and forth on whether to add the Cc tag, but I'm happy to include it now!

Quoted 6 lines

Quoted 3 lines
Signed-off-by Hao Li <[email protected]>
---
Otherwise looks good to me,
Reviewed-by Harry Yoo (Oracle) <[email protected]>

Thanks for the review!

Signature · 2 lines

Thanks,
Hao

Andrew Morton

maintainer

2026-03-30 21:46 UTC #3 lore raw

Quoted 7 lines · On Mon, 30 Mar 2026 17:34:21 +0800 Hao Li <[email protected]> wrote:

Quoted 7 lines
Quoted 4 lines
Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")
The Fixes: tag indeed looks correct to me.

Quoted 7 lines

Quoted 4 lines
By the way, MM needs an explicit Cc: [email protected] for backporting.
Since this commit was introduced in v6.17 and we have v6.18 as LTS, this should to be backported to v6.18.
I had gone back and forth on whether to add the Cc tag, but I'm happy to include it now!

That's OK, I'm very cc:stable-vigilant. Thanks, I'll get this upstreamed during this -rc cycle.

I do like our cc:stable patches to include a clear description of the userspace-visible impacts of the bug. Two reasons:

- To explain to -stable maintainers why we're requesting the backport. If they care. Heck, and to explain to me why I'm doing this!

- To help out someone who is using an older kernel (there are many) who is dealing with a bug report. They're looking at this changelog thinking "hm, I wonder if this patch will fix that bug". Let's add a few words in there to help out with their pattern matching.

So, and not for the first time ;), what are the userspace-visible runtime effects of this bug?

Hao Li

author

2026-03-31 04:25 UTC #4 lore raw

Quoted 11 lines · Andrew Morton

Quoted 9 lines · On Mon, 30 Mar 2026 17:34:21 +0800 Hao Li <[email protected]> wrote:
Quoted 9 lines
Quoted 4 lines
Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")
The Fixes: tag indeed looks correct to me.
By the way, MM needs an explicit Cc: [email protected] for backporting.

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
Since this commit was introduced in v6.17 and we have v6.18 as LTS, this should to be backported to v6.18.
I had gone back and forth on whether to add the Cc tag, but I'm happy to include it now!
That's OK, I'm very cc:stable-vigilant. Thanks, I'll get this upstreamed during this -rc cycle.

Thanks!

Quoted 4 lines

I do like our cc:stable patches to include a clear description of the userspace-visible impacts of the bug. Two reasons:

Quoted 8 lines

- To explain to -stable maintainers why we're requesting the backport. If they care. Heck, and to explain to me why I'm doing this!
- To help out someone who is using an older kernel (there are many) who is dealing with a bug report. They're looking at this changelog thinking "hm, I wonder if this patch will fix that bug". Let's add a few words in there to help out with their pattern matching.

Agreed, and thanks for spelling out the rationale!

Quoted 4 lines

So, and not for the first time ;), what are the userspace-visible runtime effects of this bug?

The most visible effect is that /sys/devices/system/node/has_normal_memory does not report a node even after that node has gained normal memory via hotplug.

Also, list_lru-based shrinkers can undercount objects on such a node and may skip reclaim on that node entirely, which can lead to a higher memory footprint than expected.

Feel free to fold the above into the changelog if useful :)

Signature · 2 lines

Thanks,
Hao

Andrew Morton

maintainer

2026-03-31 05:15 UTC #5 lore raw

Quoted 9 lines · On Tue, 31 Mar 2026 12:24:46 +0800 Hao Li <[email protected]> wrote:

The most visible effect is that /sys/devices/system/node/has_normal_memory does not report a node even after that node has gained normal memory via hotplug.
Also, list_lru-based shrinkers can undercount objects on such a node and may skip reclaim on that node entirely, which can lead to a higher memory footprint than expected.
Feel free to fold the above into the changelog if useful :)

Very useful. Added, thanks!

Vlastimil Babka (SUSE)

Acked-by on v2 1/1

2026-03-30 12:07 UTC #6 lore raw

In reply to Hao Li

Quoted 15 lines · Hao Li

N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.
Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.
This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.

Quoted 5 lines

Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")
Signed-off-by Hao Li <[email protected]>

Acked-by Vlastimil Babka (SUSE) <[email protected]>

Thanks!

Quoted 51 lines

---
Changes: simplify the code. (Thanks Joshua and David)

---
 mm/memory_hotplug.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

mm/memory_hotplug.c +20 -0

index bc805029da51..05a47953ef21 100644
@@ -1209,6 +1209,13 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 
 	if (node_arg.nid >= 0)
 		node_set_state(nid, N_MEMORY);
+	/*
+	 * Check whether we are adding normal memory to the node for the first
+	 * time.
+	 */
+	if (!node_state(nid, N_NORMAL_MEMORY) && zone_idx(zone) <= ZONE_NORMAL)
+		node_set_state(nid, N_NORMAL_MEMORY);
+
 	if (need_zonelists_rebuild)
 		build_all_zonelists(NULL);

@@ -1908,6 +1915,8 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	unsigned long flags;
 	char *reason;
 	int ret;
+	unsigned long normal_pages = 0;
+	enum zone_type zt;
 
 	/*
 	 * {on,off}lining is constrained to full memory sections (or more

@@ -2055,6 +2064,17 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	/* reinitialise watermarks and update pcp limits */
 	init_per_zone_wmark_min();
 
+	/*
+	 * Check whether this operation removes the last normal memory from
+	 * the node. We do this before clearing N_MEMORY to avoid the possible
+	 * transient "!N_MEMORY && N_NORMAL_MEMORY" state.
+	 */
+	if (zone_idx(zone) <= ZONE_NORMAL) {
+		for (zt = 0; zt <= ZONE_NORMAL; zt++)
+			normal_pages += pgdat->node_zones[zt].present_pages;
+		if (!normal_pages)
+			node_clear_state(node, N_NORMAL_MEMORY);
+	}
 	/*
 	 * Make sure to mark the node as memory-less before rebuilding the zone
 	 * list. Otherwise this node would still appear in the fallback lists.

Joshua Hahn

Reviewed-by on v2 1/1

2026-03-30 13:53 UTC #7 lore raw

In reply to Hao Li

Quoted 15 lines · On Mon, 30 Mar 2026 11:57:49 +0800 Hao Li <[email protected]> wrote:

N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.
Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.
This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.

Quoted 5 lines

Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")
Signed-off-by Hao Li <[email protected]>

LGTM, thanks for the new version Hao! I hope you have a great day : -)

Reviewed-by Joshua Hahn <[email protected]>

Quoted 10 lines

---
Changes: simplify the code. (Thanks Joshua and David)
---
 mm/memory_hotplug.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
mm/memory_hotplug.c +0 -0
index bc805029da51..05a47953ef21 100644

David Hildenbrand (Arm)

maintainer Acked-by on v2 1/1

2026-03-30 14:20 UTC #8 lore raw

In reply to Hao Li

Quoted 11 lines · Hao Li

N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.
Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.

Quoted 8 lines

This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.
Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")

Agreed on CCing stable.

Quoted 39 lines

Signed-off-by Hao Li <[email protected]>

---
Changes: simplify the code. (Thanks Joshua and David)

---
 mm/memory_hotplug.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

mm/memory_hotplug.c +10 -0

index bc805029da51..05a47953ef21 100644
@@ -1209,6 +1209,13 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 
 	if (node_arg.nid >= 0)
 		node_set_state(nid, N_MEMORY);
+	/*
+	 * Check whether we are adding normal memory to the node for the first
+	 * time.
+	 */
+	if (!node_state(nid, N_NORMAL_MEMORY) && zone_idx(zone) <= ZONE_NORMAL)
+		node_set_state(nid, N_NORMAL_MEMORY);
+
 	if (need_zonelists_rebuild)
 		build_all_zonelists(NULL);

@@ -1908,6 +1915,8 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	unsigned long flags;
 	char *reason;
 	int ret;
+	unsigned long normal_pages = 0;
+	enum zone_type zt;
 
 	/*
 	 * {on,off}lining is constrained to full memory sections (or more

@@ -2055,6 +2064,17 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	/* reinitialise watermarks and update pcp limits */
 	init_per_zone_wmark_min();
 
+	/*

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, unsigned long nr_pages, · line 2077

Quoted 10 lines

mm/memory_hotplug.c +10 -0

+	 * Check whether this operation removes the last normal memory from
+	 * the node. We do this before clearing N_MEMORY to avoid the possible
+	 * transient "!N_MEMORY && N_NORMAL_MEMORY" state.
+	 */
+	if (zone_idx(zone) <= ZONE_NORMAL) {
+		for (zt = 0; zt <= ZONE_NORMAL; zt++)
+			normal_pages += pgdat->node_zones[zt].present_pages;
+		if (!normal_pages)
+			node_clear_state(node, N_NORMAL_MEMORY);
+	}

Was wondering whether we could just do an early exit, like:

for (zt = 0; zt <= ZONE_NORMAL; zt++) {
	if (pgdat->node_zones[zt].present_pages)
		break;
}
if (zt > ZONE_NORMAL)
	node_clear_state(node, N_NORMAL_MEMORY);

But ends up being harder to digest (incl. all variants I tried :) ), so

Acked-by David Hildenbrand (Arm) <[email protected]>

Signature · 3 lines

Cheers,

David

Hao Li

author

2026-03-31 04:14 UTC #9 lore raw

Quoted 76 lines · David Hildenbrand (Arm)

Quoted 11 lines · Hao Li
N_NORMAL_MEMORY is initialized from zone population at boot, but memory hotplug currently only updates N_MEMORY. As a result, a node that gains normal memory via hotplug can remain invisible to users iterating over N_NORMAL_MEMORY, while a node that loses its last normal memory can stay incorrectly marked as such.
Restore N_NORMAL_MEMORY maintenance directly in online_pages() and offline_pages(). Set the bit when a node that currently lacks normal memory onlines pages into a zone <= ZONE_NORMAL, and clear it when offlining removes the last present pages from zones <= ZONE_NORMAL.
Quoted 8 lines
This restores the intended semantics without bringing back the old status_change_nid_normal notifier plumbing which was removed in 8d2882a8edb8.
Current users that benefit include list_lru, zswap, nfsd filecache, hugetlb_cgroup, and has_normal_memory sysfs reporting.
Fixes 8d2882a8edb8 ("mm,memory_hotplug: remove status_change_nid_normal and update documentation")
Agreed on CCing stable.
Quoted 39 lines
Signed-off-by Hao Li <[email protected]>
---
Changes: simplify the code. (Thanks Joshua and David)
---
 mm/memory_hotplug.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
mm/memory_hotplug.c +10 -0
index bc805029da51..05a47953ef21 100644
@@ -1209,6 +1209,13 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 
 	if (node_arg.nid >= 0)
 		node_set_state(nid, N_MEMORY);
+	/*
+	 * Check whether we are adding normal memory to the node for the first
+	 * time.
+	 */
+	if (!node_state(nid, N_NORMAL_MEMORY) && zone_idx(zone) <= ZONE_NORMAL)
+		node_set_state(nid, N_NORMAL_MEMORY);
+
 	if (need_zonelists_rebuild)
 		build_all_zonelists(NULL);
 
@@ -1908,6 +1915,8 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	unsigned long flags;
 	char *reason;
 	int ret;
+	unsigned long normal_pages = 0;
+	enum zone_type zt;
 
 	/*
 	 * {on,off}lining is constrained to full memory sections (or more
@@ -2055,6 +2064,17 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	/* reinitialise watermarks and update pcp limits */
 	init_per_zone_wmark_min();
 
+	/*
mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, unsigned long nr_pages, · line 2077
Quoted 10 lines
mm/memory_hotplug.c +10 -0
+	 * Check whether this operation removes the last normal memory from
+	 * the node. We do this before clearing N_MEMORY to avoid the possible
+	 * transient "!N_MEMORY && N_NORMAL_MEMORY" state.
+	 */
+	if (zone_idx(zone) <= ZONE_NORMAL) {
+		for (zt = 0; zt <= ZONE_NORMAL; zt++)
+			normal_pages += pgdat->node_zones[zt].present_pages;
+		if (!normal_pages)
+			node_clear_state(node, N_NORMAL_MEMORY);
+	}
Was wondering whether we could just do an early exit, like:

Quoted 8 lines

for (zt = 0; zt <= ZONE_NORMAL; zt++) {
	if (pgdat->node_zones[zt].present_pages)
		break;
}
if (zt > ZONE_NORMAL)
	node_clear_state(node, N_NORMAL_MEMORY);
But ends up being harder to digest (incl. all variants I tried :) ), so

Thanks for exploring this.
Yeah, this looks like one of those cases where we have to make a bit of a tradeoff for readability :)

Quoted 2 lines

Acked-by David Hildenbrand (Arm) <[email protected]>

Thanks for the Ack!

Signature · 2 lines

Thanks,
Hao