Skip to contentSkip to conversation

mm/hugetlb Try to respect mempolicy when calculating surplus huge pages

PATCH In review Charles Haithcock 2 participants 7 messages Jul 7 – Sep 8

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover mm/hugetlb Try to respect mempolicy when calculating surplus huge pages

Presently, when calculating how many huge pages are needed when reserving surplus huge pages, the global count of free huge pages is used. When reserving with a mempolicy, the global count of free huge pages is used even if some/all of those free huge pages are on numa nodes outside of the mempolicy.

Reserving surplus huge pages is ultimately best effort even without a mempolicy. Restrictions from cpusets and mempolicies further complicate calculating correct numbers of surplus huge pages to reserve and maintaining which nodes those reservations belong to (see the comment in hugetlb_acct_memory).

However, we can do a little better when reserving surplus huge pages with a mempolicy. This patch changes how to calculate the necessary amount of surplus huge pages to reserve by considering the max of either the amount of free huge pages on nodes in the mempolicy or the global amount of free huge pages. We may still attempt to reserve huge pages outside the mempolicy, however, we end up being more likely to reserve from nodes in the mempolicy.

- v1: Modified needed calculation to use allowed_mems_nr(h) in order to consider free hugetlb pages in our mempolicy.
- v2: Folded in Joshua Hahn's recommendation [1] to further modify needed calculation to take the max of either the available hugetlb pages in the mempolicy or the globally available hugetlb pages. Allows allocations to prioritize nodes in the mempolicy but can still fall back to offnode allocations. Also added selftests to check only for the edgecase which caused this to initially be reported and sanity checks.
- v3: Split the change and tests into separate patches and clean up code to better align with kernel style guidelines. Likewise, fold in Usama Arif's correction [2].

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/

Charles Haithcock (2):
mm/hugetlb: Prefer mempolicy nodes for calcuating surplus hugepage
reservation
selftests/mm: add surplus hugetlb mempolicy reservation tests

 mm/hugetlb.c                                  |  42 +-
 tools/testing/selftests/mm/Makefile           |   3 +
 .../selftests/mm/hugetlb_surplus_mempolicy.c  | 491 ++++++++++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   1 +
 4 files changed, 517 insertions(+), 20 deletions(-)
 create mode 100644 tools/testing/selftests/mm/hugetlb_surplus_mempolicy.c
Signature · 1 line
2.55.0
#1 lore raw

PATCH 1/2 mm/hugetlb: Prefer mempolicy nodes for calcuating surplus hugepage reservation

Currently, reserving surplus hugetlb pages in a mempolicy still use global counters for calculating how many hugetlb pages to reserve. Modify this to consider the nodes in the mempolicy first but fall back to all possible nodes.

---

- v1: Modified needed calculation to use allowed_mems_nr(h) in order to consider free hugetlb pages in our mempolicy.
- v2: Folded in Joshua Hahn's recommendation [1] to further modify needed calculation to take the max of either the available hugetlb pages in the mempolicy or the globally available hugetlb pages. Allows allocations to prioritize nodes in the mempolicy but can still fall back to offnode allocations. Also added selftests to check only for the edgecase which caused this to initially be reported and sanity checks.
- v3: Split the change and tests into separate patches and clean up code to better align with kernel style guidelines. Likewise, fold in Usama Arif's correction [2].

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/hugetlb.c | 42 ++++++++++++++++++++++--------------------
 1 file changed, 22 insertions(+), 20 deletions(-)
mm/hugetlb.c +22 -20
index f24bf49be0..f5e678e997 100644
@@ -2255,6 +2255,23 @@ static nodemask_t *policy_mbind_nodemask(gfp_t gfp)
 	return NULL;
 }
 
+static unsigned int allowed_mems_nr(struct hstate *h)
+{
+	int node;
+	unsigned int nr = 0;
+	nodemask_t *mbind_nodemask;
+	unsigned int *array = h->free_huge_pages_node;
+	gfp_t gfp_mask = htlb_alloc_mask(h);
+
+	mbind_nodemask = policy_mbind_nodemask(gfp_mask);
+	for_each_node_mask(node, cpuset_current_mems_allowed) {
+		if (!mbind_nodemask || node_isset(node, *mbind_nodemask))
+			nr += array[node];
+	}
+
+	return nr;
+}
+
 /*
  * Increase the hugetlb pool such that it can accommodate a reservation
  * of size 'delta'.
@@ -2277,7 +2294,8 @@ static int gather_surplus_pages(struct hstate *h, long delta)
 		alloc_nodemask = cpuset_current_mems_allowed;
 
 	lockdep_assert_held(&hugetlb_lock);
-	needed = (h->resv_huge_pages + delta) - h->free_huge_pages;
+	needed = max_t(long, (delta - allowed_mems_nr(h)),
+			((h->resv_huge_pages + delta) - h->free_huge_pages));
 	if (needed <= 0) {
 		h->resv_huge_pages += delta;
 		return 0;
@@ -2311,8 +2329,9 @@ static int gather_surplus_pages(struct hstate *h, long delta)
 	 * because either resv_huge_pages or free_huge_pages may have changed.
 	 */
 	spin_lock_irq(&hugetlb_lock);
-	needed = (h->resv_huge_pages + delta) -
-			(h->free_huge_pages + allocated);
+	needed = max_t(long, ((delta - allowed_mems_nr(h)) - allocated),
+			((h->resv_huge_pages + delta) -
+				(h->free_huge_pages + allocated)));
 	if (needed > 0) {
 		if (alloc_ok)
 			goto retry;
@@ -4513,23 +4532,6 @@ static int __init hugepage_alloc_threads_setup(char *s)
 }
 __setup("hugepage_alloc_threads=", hugepage_alloc_threads_setup);
 
-static unsigned int allowed_mems_nr(struct hstate *h)
-{
-	int node;
-	unsigned int nr = 0;
-	nodemask_t *mbind_nodemask;
-	unsigned int *array = h->free_huge_pages_node;
-	gfp_t gfp_mask = htlb_alloc_mask(h);
-
-	mbind_nodemask = policy_mbind_nodemask(gfp_mask);
-	for_each_node_mask(node, cpuset_current_mems_allowed) {
-		if (!mbind_nodemask || node_isset(node, *mbind_nodemask))
-			nr += array[node];
-	}
-
-	return nr;
-}
-
 void hugetlb_report_meminfo(struct seq_file *m)
 {
 	struct hstate *h;
Signature · 1 line
2.55.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] mm/hugetlb: Prefer mempolicy nodes for calcuating surplus hugepage reservation

Quoted 4 lines · On Tue, 7 Jul 2026 12:28:55 -0600 Charles Haithcock <[email protected]>…

Currently, reserving surplus hugetlb pages in a mempolicy still use global counters for calculating how many hugetlb pages to reserve. Modify this to consider the nodes in the mempolicy first but fall back to all possible nodes.

Hi Charles, thank you for the patch, this looks good to me!

Personally I like the description of the commit in the cover letter because it provides a bit more context. Do other reviewers have thoughts on this? In any case, the code LGTM : -)

BTW this is a little strange but Sashiko seems to have failed to apply this patch [1] which is strange because when applying on the exact
same commit db6ed2a3d3106 "mm/swap, PM: hibernate: atomically replace hibernation pin" as Sashiko tried, it worked & compiled for me with no errors at all...

[1] https://sashiko.dev/#/patchset/20260707182900.483757-1-chaithco%40redhat.com
Sashiko's error log:

Trying baseline: mm/mm-new (db6ed2a3d31068ca5577466591b2521dee0508ca) Patch 41962/2 (ID: 108031) failed to apply.
Application failed.

Quoted 99 lines

---

- v1: Modified needed calculation to use allowed_mems_nr(h) in order to consider free hugetlb pages in our mempolicy.
- v2: Folded in Joshua Hahn's recommendation [1] to further modify needed calculation to take the max of either the available hugetlb pages in the mempolicy or the globally available hugetlb pages. Allows allocations to prioritize nodes in the mempolicy but can still fall back to offnode allocations. Also added selftests to check only for the edgecase which caused this to initially be reported and sanity checks.
- v3: Split the change and tests into separate patches and clean up code to better align with kernel style guidelines. Likewise, fold in Usama Arif's correction [2].

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/hugetlb.c | 42 ++++++++++++++++++++++--------------------
 1 file changed, 22 insertions(+), 20 deletions(-)
mm/hugetlb.c +22 -20
index f24bf49be0..f5e678e997 100644
@@ -2255,6 +2255,23 @@ static nodemask_t *policy_mbind_nodemask(gfp_t gfp)
 	return NULL;
 }
 
+static unsigned int allowed_mems_nr(struct hstate *h)
+{
+	int node;
+	unsigned int nr = 0;
+	nodemask_t *mbind_nodemask;
+	unsigned int *array = h->free_huge_pages_node;
+	gfp_t gfp_mask = htlb_alloc_mask(h);
+
+	mbind_nodemask = policy_mbind_nodemask(gfp_mask);
+	for_each_node_mask(node, cpuset_current_mems_allowed) {
+		if (!mbind_nodemask || node_isset(node, *mbind_nodemask))
+			nr += array[node];
+	}
+
+	return nr;
+}
+
 /*
  * Increase the hugetlb pool such that it can accommodate a reservation
  * of size 'delta'.
@@ -2277,7 +2294,8 @@ static int gather_surplus_pages(struct hstate *h, long delta)
 		alloc_nodemask = cpuset_current_mems_allowed;
 
 	lockdep_assert_held(&hugetlb_lock);
-	needed = (h->resv_huge_pages + delta) - h->free_huge_pages;
+	needed = max_t(long, (delta - allowed_mems_nr(h)),
+			((h->resv_huge_pages + delta) - h->free_huge_pages));
 	if (needed <= 0) {
 		h->resv_huge_pages += delta;
 		return 0;
@@ -2311,8 +2329,9 @@ static int gather_surplus_pages(struct hstate *h, long delta)
 	 * because either resv_huge_pages or free_huge_pages may have changed.
 	 */
 	spin_lock_irq(&hugetlb_lock);
-	needed = (h->resv_huge_pages + delta) -
-			(h->free_huge_pages + allocated);
+	needed = max_t(long, ((delta - allowed_mems_nr(h)) - allocated),
+			((h->resv_huge_pages + delta) -
+				(h->free_huge_pages + allocated)));
 	if (needed > 0) {
 		if (alloc_ok)
 			goto retry;
@@ -4513,23 +4532,6 @@ static int __init hugepage_alloc_threads_setup(char *s)
 }
 __setup("hugepage_alloc_threads=", hugepage_alloc_threads_setup);
 
-static unsigned int allowed_mems_nr(struct hstate *h)
-{
-	int node;
-	unsigned int nr = 0;
-	nodemask_t *mbind_nodemask;
-	unsigned int *array = h->free_huge_pages_node;
-	gfp_t gfp_mask = htlb_alloc_mask(h);
-
-	mbind_nodemask = policy_mbind_nodemask(gfp_mask);
-	for_each_node_mask(node, cpuset_current_mems_allowed) {
-		if (!mbind_nodemask || node_isset(node, *mbind_nodemask))
-			nr += array[node];
-	}
-
-	return nr;
-}
-
 void hugetlb_report_meminfo(struct seq_file *m)
 {
 	struct hstate *h;
Signature · 1 line
2.55.0
#3 lore raw

Re: [PATCH 1/2] mm/hugetlb: Prefer mempolicy nodes for calcuating surplus hugepage reservation

Quoted 7 lines · Joshua Hahn

On Tue, 7 Jul 2026 12:28:55 -0600 Charles Haithcock <[email protected]> wrote:

Hi Charles, thank you for the patch, this looks good to me!

Personally I like the description of the commit in the cover letter because it provides a bit more context. Do other reviewers have thoughts on this? In any case, the code LGTM : -)

Thank you! If people prefer, I can copy the cover letter contents to the main patch notes. Just let me know!

Quoted 4 lines

BTW this is a little strange but Sashiko seems to have failed to apply this patch [1] which is strange because when applying on the exact
same commit db6ed2a3d3106 "mm/swap, PM: hibernate: atomically replace hibernation pin" as Sashiko tried, it worked & compiled for me with no errors at all...

Interesting; is this something I need to act on in any way?

Quoted 1 line
#4 lore raw

Re: [PATCH 1/2] mm/hugetlb: Prefer mempolicy nodes for calcuating surplus hugepage reservation

Quoted 18 lines
Quoted 1 line · Joshua Hahn

On Tue, 7 Jul 2026 12:28:55 -0600 Charles Haithcock

Quoted 6 lines · <[email protected]> wrote:

Hi Charles, thank you for the patch, this looks good to me!

Personally I like the description of the commit in the cover letter because it provides a bit more context. Do other reviewers have thoughts on this? In any case, the code LGTM : -)

Thank you! If people prefer, I can copy the cover letter contents to the main
patch notes. Just let me know!

Quoted 1 line

BTW this is a little strange but Sashiko seems to have failed to

Quoted 8 lines

apply

Quoted 2 lines

this patch [1] which is strange because when applying on the exact same commit db6ed2a3d3106 "mm/swap, PM: hibernate: atomically

replace hibernation pin"

Quoted 1 line

as Sashiko tried, it worked & compiled for me with no errors at

all...

Interesting; is this something I need to act on in any way?

No not at all, I just thought it was worth pointing out since I had never seen before. Thanks again for the patch!

Joshua

#5 lore raw

PATCH 2/2 selftests/mm: add surplus hugetlb mempolicy reservation tests

In reply to Charles Haithcock

Verify that surplus hugetlb reservasations in the presence of mempolicies will prefer the nodes in the policy. A couple tests do basic sanity checking to validate if the rest of the tests should or should not work.

Note that this does not test the edge cases where reservations would fall back to other nodes; the intent of this patch series is to "do better" with reserving surplus pages within mempolicies. Later efforts may further improve this and result in testing those edge cases as false negatives.

---

- v1: Modified needed calculation to use allowed_mems_nr(h) in order to consider free hugetlb pages in our mempolicy.
- v2: Folded in Joshua Hahn's recommendation [1] to further modify needed calculation to take the max of either the available hugetlb pages in the mempolicy or the globally available hugetlb pages. Allows allocations to prioritize nodes in the mempolicy but can still fall back to offnode allocations. Also added selftests to check only for the edgecase which caused this to initially be reported and sanity checks.
- v3: Split the change and tests into separate patches and clean up code to better align with kernel style guidelines. Likewise, fold in Usama Arif's correction [2].

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 tools/testing/selftests/mm/Makefile           |   3 +
 .../selftests/mm/hugetlb_surplus_mempolicy.c  | 491 ++++++++++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   1 +
 3 files changed, 495 insertions(+)
 create mode 100644 tools/testing/selftests/mm/hugetlb_surplus_mempolicy.c
tools/testing/selftests/mm/Makefile +3 -0
index cd24596cdd..40de0938f3 100644
@@ -106,6 +106,7 @@ TEST_GEN_FILES += guard-regions
 TEST_GEN_FILES += merge
 TEST_GEN_FILES += rmap
 TEST_GEN_FILES += folio_split_race_test
+TEST_GEN_FILES += hugetlb_surplus_mempolicy
 
 ifneq ($(ARCH),arm64)
 TEST_GEN_FILES += soft-dirty
@@ -260,6 +261,8 @@ $(OUTPUT)/migration: LDLIBS += -lnuma
 
 $(OUTPUT)/rmap: LDLIBS += -lnuma
 
+$(OUTPUT)/hugetlb_surplus_mempolicy: LDLIBS += -lnuma
+
 local_config.mk local_config.h: check_config.sh
 	CC="$(CC)" CFLAGS="$(CFLAGS)" ./check_config.sh
 
tools/testing/selftests/mm/hugetlb_surplus_mempolicy.c new file +491 -0

491 lines not on this page · raw message

tools/testing/selftests/mm/run_vmtests.sh +1 -0
index c17b133a81..cd368ce590 100755
@@ -297,6 +297,7 @@ CATEGORY="hugetlb" run_test ./hugepage-mremap
 CATEGORY="hugetlb" run_test ./hugepage-vmemmap
 CATEGORY="hugetlb" run_test ./hugetlb-madvise
 CATEGORY="hugetlb" run_test ./hugetlb_dio
+CATEGORY="hugetlb" run_test ./hugetlb_surplus_mempolicy
 
 if [ "${HAVE_HUGEPAGES}" = "1" ]; then
 	nr_hugepages_tmp=$(cat /proc/sys/vm/nr_hugepages)
Signature · 1 line
2.55.0
#6 lore raw
In reply to Charles Haithcock
Quoted 53 lines · Charles Haithcock

Presently, when calculating how many huge pages are needed when reserving surplus huge pages, the global count of free huge pages is used. When reserving with a mempolicy, the global count of free huge
pages is used even if some/all of those free huge pages are on numa nodes outside of the mempolicy.

Reserving surplus huge pages is ultimately best effort even without a mempolicy. Restrictions from cpusets and mempolicies further complicate
calculating correct numbers of surplus huge pages to reserve and maintaining which nodes those reservations belong to (see the comment in
hugetlb_acct_memory).

However, we can do a little better when reserving surplus huge pages with a mempolicy. This patch changes how to calculate the necessary amount of surplus huge pages to reserve by considering the max of either
the amount of free huge pages on nodes in the mempolicy or the global amount of free huge pages. We may still attempt to reserve huge pages outside the mempolicy, however, we end up being more likely to reserve
from nodes in the mempolicy.

- v1: Modified needed calculation to use allowed_mems_nr(h) in order
  to consider free hugetlb pages in our mempolicy.
- v2: Folded in Joshua Hahn's recommendation [1] to further modify needed calculation to take the max of either the available hugetlb
  pages in the mempolicy or the globally available hugetlb pages. Allows
  allocations to prioritize nodes in the mempolicy but can still fall back to offnode allocations. Also added selftests to check only for the edgecase which caused this to initially be reported and sanity checks.
- v3: Split the change and tests into separate patches and clean up code
  to better align with kernel style guidelines. Likewise, fold in Usama
  Arif's correction [2].

[1]
https://lore.kernel.org/all/[email protected]/
[2]
https://lore.kernel.org/linux-mm/[email protected]/

Charles Haithcock (2):
  mm/hugetlb: Prefer mempolicy nodes for calcuating surplus hugepage reservation
  selftests/mm: add surplus hugetlb mempolicy reservation tests

Quoted 8 lines

 mm/hugetlb.c                                  |  42 +- tools/testing/selftests/mm/Makefile           |   3 + .../selftests/mm/hugetlb_surplus_mempolicy.c  | 491
++++++++++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   1 +

 4 files changed, 517 insertions(+), 20 deletions(-)

 create mode 100644 tools/testing/selftests/mm/hugetlb_surplus_mempolicy.c

Any followups on this? Please do let me know if I need to do anything else to help this patch along! :)