Skip to contentSkip to conversation

kho: make sure preservations do not span multiple NUMA nodes

PATCH Applied Pratyush Yadav 3 participants 8 messages Mar 9 – Mar 20

Subsystems (3)
lore
Share & tools
lore

Pratyush Yadav

author maintainer
#0 lore raw

PATCH 1/2 kho: make sure preservations do not span multiple NUMA nodes

From: "Pratyush Yadav (Google)" <[email protected]>

The KHO restoration machinery is not capable of dealing with preservations that span multiple NUMA nodes. kho_preserve_folio() guarantees the preservation will only span one NUMA node since folios can't span multiple nodes.

This leaves kho_preserve_pages(). While semantically kho_preserve_pages() only deals with 0-order pages, so all preservations should be single page only, in practice it combines preservations to higher orders for efficiency. This can result in a preservation spanning multiple nodes. Break up the preservations into a smaller order if that happens.

---

Notes:
Ref: https://lore.kernel.org/linux-mm/CA+CK2bDvaGmfkCPCMWM6gPcd4FfUyD6e5yWE+kNcma1vT3Jw3g@mail.gmail.com/

 kernel/liveupdate/kexec_handover.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +9 -1
index cc68a3692905..bc9bd18294ee 100644
@@ -869,9 +869,17 @@ int kho_preserve_pages(struct page *page, unsigned long nr_pages)
 	}
 
 	while (pfn < end_pfn) {
-		const unsigned int order =
+		unsigned int order =
 			min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn));
 
+		/*
+		 * Make sure all the pages in a single preservation are in the
+		 * same NUMA node. The restore machinery can not cope with a
+		 * preservation spanning multiple NUMA nodes.
+		 */
+		while (pfn_to_nid(pfn) != pfn_to_nid(pfn + (1UL << order) - 1))
+			order--;
+
 		err = __kho_preserve_order(track, pfn, order);
 		if (err) {
 			failed_pfn = pfn;
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
author maintainer
#1 lore raw

PATCH 2/2 kho: drop restriction on maximum page order

KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than MAX_PAGE_ORDER.

For one, it is possible to get a larger order when using kho_preserve_pages() if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.

For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.

There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.

---

Notes:
This patch was first sent with this RFC series [0]. I am sending it separately since it is an independent patch that is useful even without hugepage preservation. No changes since the RFC.

[0] https://lore.kernel.org/linux-mm/[email protected]/T/#u

 kernel/liveupdate/kexec_handover.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +1 -1
index bc9bd18294ee..1038e41ff9f9 100644
@@ -253,7 +253,7 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
 	 * check also implicitly makes sure phys is order-aligned since for
 	 * non-order-aligned phys addresses, magic will never be set.
 	 */
-	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC || info.order > MAX_PAGE_ORDER))
+	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
 		return NULL;
 	nr_pages = (1 << info.order);
 
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
maintainer Reviewed-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] kho: drop restriction on maximum page order

Quoted 13 lines · Pratyush Yadav

KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than MAX_PAGE_ORDER.

For one, it is possible to get a larger order when using kho_preserve_pages() if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.

For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.

Quoted 6 lines

There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.

One SOB should be enough ;-)

Quoted 28 lines

---

Notes:
This patch was first sent with this RFC series [0]. I am sending it separately since it is an independent patch that is useful even without hugepage preservation. No changes since the RFC.

[0] https://lore.kernel.org/linux-mm/[email protected]/T/#u

 kernel/liveupdate/kexec_handover.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +1 -1
index bc9bd18294ee..1038e41ff9f9 100644
@@ -253,7 +253,7 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
 	 * check also implicitly makes sure phys is order-aligned since for
 	 * non-order-aligned phys addresses, magic will never be set.
 	 */
-	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC || info.order > MAX_PAGE_ORDER))
+	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
 		return NULL;
 	nr_pages = (1 << info.order);
 
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#3 lore raw

Re: [PATCH 2/2] kho: drop restriction on maximum page order

Quoted 14 lines · Mike Rapoport
Quoted 13 lines · Pratyush Yadav

KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than MAX_PAGE_ORDER.

For one, it is possible to get a larger order when using kho_preserve_pages() if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.

For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.

Quoted 8 lines
Quoted 6 lines

There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.

One SOB should be enough ;-)

Hmm, I figured the unemployed me (who originally wrote the patch) and the employed-by-google me (who is doing this new version) would count as two separate entities and there should be a S-o-b for both.

Anyway, I am fine with dropping either one of the two.

Quoted 2 lines

Thanks!

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer
#4 lore raw

Re: [PATCH 2/2] kho: drop restriction on maximum page order

Quoted 20 lines · Pratyush Yadav
Quoted 18 lines · Mike Rapoport
Quoted 17 lines · Pratyush Yadav

KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than MAX_PAGE_ORDER.

For one, it is possible to get a larger order when using kho_preserve_pages() if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.

For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.

There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

One SOB should be enough ;-)

Hmm, I figured the unemployed me (who originally wrote the patch) and the employed-by-google me (who is doing this new version) would count as two separate entities and there should be a S-o-b for both.

Maybe than something like:

;-)

Quoted 12 lines

Anyway, I am fine with dropping either one of the two.

Quoted 2 lines

Thanks!

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#5 lore raw

Re: [PATCH 2/2] kho: drop restriction on maximum page order

Quoted 30 lines · Mike Rapoport
Quoted 20 lines · Pratyush Yadav
Quoted 18 lines · Mike Rapoport
Quoted 17 lines · Pratyush Yadav

KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than MAX_PAGE_ORDER.

For one, it is possible to get a larger order when using kho_preserve_pages() if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.

For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.

There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

One SOB should be enough ;-)

Hmm, I figured the unemployed me (who originally wrote the patch) and the employed-by-google me (who is doing this new version) would count as two separate entities and there should be a S-o-b for both.

Quoted 7 lines

Maybe than something like:

;-)

Yeah, that works too :-)

I'll use that when I redo some of the HugeTLB patches too

Signature · 2 lines
Regards,
Pratyush Yadav
Reviewed-by on v1 1/2
#6 lore raw
In reply to Pratyush Yadav
Quoted 51 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

The KHO restoration machinery is not capable of dealing with preservations that span multiple NUMA nodes. kho_preserve_folio() guarantees the preservation will only span one NUMA node since folios can't span multiple nodes.

This leaves kho_preserve_pages(). While semantically kho_preserve_pages() only deals with 0-order pages, so all preservations should be single page only, in practice it combines preservations to higher orders for efficiency. This can result in a preservation spanning multiple nodes. Break up the preservations into a smaller order if that happens.

---

Notes:
Ref: https://lore.kernel.org/linux-mm/CA+CK2bDvaGmfkCPCMWM6gPcd4FfUyD6e5yWE+kNcma1vT3Jw3g@mail.gmail.com/

kernel/liveupdate/kexec_handover.c | 10 +++++++++-

1 file changed, 9 insertions(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +9 -1
index cc68a3692905..bc9bd18294ee 100644
@@ -869,9 +869,17 @@ int kho_preserve_pages(struct page *page, unsigned long nr_pages)
 	}
 
 	while (pfn < end_pfn) {
-		const unsigned int order =
+		unsigned int order =
 			min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn));
 
+		/*
+		 * Make sure all the pages in a single preservation are in the
+		 * same NUMA node. The restore machinery can not cope with a
+		 * preservation spanning multiple NUMA nodes.
+		 */
+		while (pfn_to_nid(pfn) != pfn_to_nid(pfn + (1UL << order) - 1))
+			order--;
+
 		err = __kho_preserve_order(track, pfn, order);
 		if (err) {
 			failed_pfn = pfn;
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
maintainer Reviewed-by on v1 1/2
#7 lore raw
In reply to Pratyush Yadav
Quoted 14 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

The KHO restoration machinery is not capable of dealing with preservations that span multiple NUMA nodes. kho_preserve_folio() guarantees the preservation will only span one NUMA node since folios can't span multiple nodes.

This leaves kho_preserve_pages(). While semantically kho_preserve_pages() only deals with 0-order pages, so all preservations should be single page only, in practice it combines preservations to higher orders for efficiency. This can result in a preservation spanning multiple nodes. Break up the preservations into a smaller order if that happens.

Quoted 2 lines
Quoted 34 lines

---

Notes:
Ref: https://lore.kernel.org/linux-mm/CA+CK2bDvaGmfkCPCMWM6gPcd4FfUyD6e5yWE+kNcma1vT3Jw3g@mail.gmail.com/

 kernel/liveupdate/kexec_handover.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +9 -1
index cc68a3692905..bc9bd18294ee 100644
@@ -869,9 +869,17 @@ int kho_preserve_pages(struct page *page, unsigned long nr_pages)
 	}
 
 	while (pfn < end_pfn) {
-		const unsigned int order =
+		unsigned int order =
 			min(count_trailing_zeros(pfn), ilog2(end_pfn - pfn));
 
+		/*
+		 * Make sure all the pages in a single preservation are in the
+		 * same NUMA node. The restore machinery can not cope with a
+		 * preservation spanning multiple NUMA nodes.
+		 */
+		while (pfn_to_nid(pfn) != pfn_to_nid(pfn + (1UL << order) - 1))
+			order--;
+
 		err = __kho_preserve_order(track, pfn, order);
 		if (err) {
 			failed_pfn = pfn;
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Signature · 2 lines
Sincerely yours,
Mike.