kho: make sure preservations do not span multiple NUMA nodes
Subsystems (3)
Pratyush Yadav
maintainerPATCH 1/2 kho: make sure preservations do not span multiple NUMA nodes
From: "Pratyush Yadav (Google)" <[email protected]>
The KHO restoration machinery is not capable of dealing with preservations that span multiple NUMA nodes. kho_preserve_folio() guarantees the preservation will only span one NUMA node since folios can't span multiple nodes.
This leaves kho_preserve_pages(). While semantically kho_preserve_pages() only deals with 0-order pages, so all preservations should be single page only, in practice it combines preservations to higher orders for efficiency. This can result in a preservation spanning multiple nodes. Break up the preservations into a smaller order if that happens.
---
Notes:
Ref: https://lore.kernel.org/linux-mm/CA+CK2bDvaGmfkCPCMWM6gPcd4FfUyD6e5yWE+kNcma1vT3Jw3g@mail.gmail.com/
kernel/liveupdate/kexec_handover.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +9 -1
@@ -869,9 +869,17 @@ int kho_preserve_pages(struct page *page, unsigned long nr_pages) }while (pfn < end_pfn) { -const unsigned int order = +unsigned int order =min (count_trailing_zeros (pfn),ilog2 (end_pfn - pfn)); +/* +* Make sure all the pages in a single preservation are in the +* same NUMA node. The restore machinery can not cope with a +* preservation spanning multiple NUMA nodes. +*/ +while (pfn_to_nid (pfn) !=pfn_to_nid (pfn + (1UL << order) -1 )) + order--; + err =__kho_preserve_order (track, pfn, order);if (err) { failed_pfn = pfn;
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Pratyush Yadav
maintainerPATCH 2/2 kho: drop restriction on maximum page order
KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than MAX_PAGE_ORDER.
For one, it is possible to get a larger order when using kho_preserve_pages() if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.
For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.
There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.
---
Notes:
This patch was first sent with this RFC series [0]. I am sending it separately since it is an independent patch that is useful even without hugepage preservation. No changes since the RFC.
[0] https://lore.kernel.org/linux-mm/[email protected]/T/#u
kernel/liveupdate/kexec_handover.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +1 -1
@@ -253,7 +253,7 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)* check also implicitly makes sure phys is order-aligned since for * non-order-aligned phys addresses, magic will never be set. */ -if (WARN_ON_ONCE (info.magic !=KHO_PAGE_MAGIC || info.order >MAX_PAGE_ORDER )) +if (WARN_ON_ONCE (info.magic !=KHO_PAGE_MAGIC ))return NULL ; nr_pages = (1 << info.order);
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Re: [PATCH 2/2] kho: drop restriction on maximum page order
Quoted 13 lines · Pratyush Yadav
KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than
MAX_PAGE_ORDER.For one, it is possible to get a larger order when using
kho_preserve_pages()if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.
Quoted 6 lines
There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.
Signed-off-by Pratyush Yadav <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>
One SOB should be enough ;-)
Quoted 28 lines
---
Notes:
This patch was first sent with this RFC series [0]. I am sending it separately since it is an independent patch that is useful even without hugepage preservation. No changes since the RFC.[0] https://lore.kernel.org/linux-mm/[email protected]/T/#u
kernel/liveupdate/kexec_handover.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/liveupdate/kexec_handover.c +1
-1@@ -253,7 +253,7 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)* check also implicitly makes sure phys is order-aligned since for * non-order-aligned phys addresses, magic will never be set. */ -if (WARN_ON_ONCE (info.magic !=KHO_PAGE_MAGIC || info.order >MAX_PAGE_ORDER )) +if (WARN_ON_ONCE (info.magic !=KHO_PAGE_MAGIC ))return NULL ; nr_pages = (1 << info.order);Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerRe: [PATCH 2/2] kho: drop restriction on maximum page order
Quoted 14 lines · Mike Rapoport
Quoted 13 lines · Pratyush Yadav
KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than
MAX_PAGE_ORDER.For one, it is possible to get a larger order when using
kho_preserve_pages()if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.
Quoted 8 lines
Quoted 6 lines
There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.
Signed-off-by Pratyush Yadav <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>One SOB should be enough ;-)
Hmm, I figured the unemployed me (who originally wrote the patch) and the employed-by-google me (who is doing this new version) would count as two separate entities and there should be a S-o-b for both.
Anyway, I am fine with dropping either one of the two.
Quoted 2 lines
Reviewed-by Mike Rapoport (Microsoft) <[email protected]>
Thanks!
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Mike Rapoport
maintainerRe: [PATCH 2/2] kho: drop restriction on maximum page order
Quoted 20 lines · Pratyush Yadav
Quoted 18 lines · Mike Rapoport
Quoted 17 lines · Pratyush Yadav
KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than
MAX_PAGE_ORDER.For one, it is possible to get a larger order when using
kho_preserve_pages()if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.
There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Pratyush Yadav <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>One SOB should be enough ;-)
Hmm, I figured the unemployed me (who originally wrote the patch) and the employed-by-google me (who is doing this new version) would count as two separate entities and there should be a S-o-b for both.
Maybe than something like:
;-)
Quoted 12 lines
Anyway, I am fine with dropping either one of the two.
Quoted 2 lines
Reviewed-by Mike Rapoport (Microsoft) <[email protected]>Thanks!
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerRe: [PATCH 2/2] kho: drop restriction on maximum page order
Quoted 30 lines · Mike Rapoport
Quoted 20 lines · Pratyush Yadav
Quoted 18 lines · Mike Rapoport
Quoted 17 lines · Pratyush Yadav
KHO currently restricts the maximum order of a restored page to the maximum order supported by the buddy allocator. While this works fine for much of the data passed across kexec, it is possible to have pages larger than
MAX_PAGE_ORDER.For one, it is possible to get a larger order when using
kho_preserve_pages()if the number of pages is large enough, since it tries to combine multiple aligned 0-order preservations into one higher order preservation.For another, upcoming support for hugepages can have gigantic hugepages being preserved over KHO.
There is no real reason for this limit. The KHO preservation machinery can handle any page order. Remove this artificial restriction on max page order.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Pratyush Yadav <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>One SOB should be enough ;-)
Hmm, I figured the unemployed me (who originally wrote the patch) and the employed-by-google me (who is doing this new version) would count as two separate entities and there should be a S-o-b for both.
Quoted 7 lines
Maybe than something like:
Signed-off-by Pratyush Yadav (Hobbyist) <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>;-)
Yeah, that works too :-)
I'll use that when I redo some of the HugeTLB patches too
Signature · 2 lines
Regards, Pratyush Yadav
Quoted 51 lines · Pratyush Yadav
From: "Pratyush Yadav (Google)" <[email protected]>
The KHO restoration machinery is not capable of dealing with preservations that span multiple NUMA nodes.
kho_preserve_folio()guarantees the preservation will only span one NUMA node since folios can't span multiple nodes.This leaves
kho_preserve_pages(). While semanticallykho_preserve_pages()only deals with 0-order pages, so all preservations should be single page only, in practice it combines preservations to higher orders for efficiency. This can result in a preservation spanning multiple nodes. Break up the preservations into a smaller order if that happens.Suggested-by Pasha Tatashin <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>---
Notes:
Ref: https://lore.kernel.org/linux-mm/CA+CK2bDvaGmfkCPCMWM6gPcd4FfUyD6e5yWE+kNcma1vT3Jw3g@mail.gmail.com/
kernel/liveupdate/kexec_handover.c| 10 +++++++++-1 file changed, 9 insertions(+), 1 deletion(-)kernel/liveupdate/kexec_handover.c +9
-1@@ -869,9 +869,17 @@ int kho_preserve_pages(struct page *page, unsigned long nr_pages) }while (pfn < end_pfn) { -const unsigned int order = +unsigned int order =min (count_trailing_zeros (pfn),ilog2 (end_pfn - pfn)); +/* +* Make sure all the pages in a single preservation are in the +* same NUMA node. The restore machinery can not cope with a +* preservation spanning multiple NUMA nodes. +*/ +while (pfn_to_nid (pfn) !=pfn_to_nid (pfn + (1UL << order) -1 )) + order--; + err =__kho_preserve_order (track, pfn, order);if (err) { failed_pfn = pfn;Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Quoted 14 lines · Pratyush Yadav
From: "Pratyush Yadav (Google)" <[email protected]>
The KHO restoration machinery is not capable of dealing with preservations that span multiple NUMA nodes.
kho_preserve_folio()guarantees the preservation will only span one NUMA node since folios can't span multiple nodes.This leaves
kho_preserve_pages(). While semanticallykho_preserve_pages()only deals with 0-order pages, so all preservations should be single page only, in practice it combines preservations to higher orders for efficiency. This can result in a preservation spanning multiple nodes. Break up the preservations into a smaller order if that happens.
Quoted 2 lines
Suggested-by Pasha Tatashin <[email protected]>Signed-off-by Pratyush Yadav (Google) <[email protected]>
Quoted 34 lines
---
Notes:
Ref: https://lore.kernel.org/linux-mm/CA+CK2bDvaGmfkCPCMWM6gPcd4FfUyD6e5yWE+kNcma1vT3Jw3g@mail.gmail.com/kernel/liveupdate/kexec_handover.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)kernel/liveupdate/kexec_handover.c +9
-1@@ -869,9 +869,17 @@ int kho_preserve_pages(struct page *page, unsigned long nr_pages) }while (pfn < end_pfn) { -const unsigned int order = +unsigned int order =min (count_trailing_zeros (pfn),ilog2 (end_pfn - pfn)); +/* +* Make sure all the pages in a single preservation are in the +* same NUMA node. The restore machinery can not cope with a +* preservation spanning multiple NUMA nodes. +*/ +while (pfn_to_nid (pfn) !=pfn_to_nid (pfn + (1UL << order) -1 )) + order--; + err =__kho_preserve_order (track, pfn, order);if (err) { failed_pfn = pfn;Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Signature · 2 lines
Sincerely yours, Mike.