Partial revert "x86/xen: fix balloon target initialization for PVH dom0"
PATCH 1/1 Partial revert "x86/xen: fix balloon target initialization for PVH dom0"
This partially reverts commit 87af633689ce16ddb166c80f32b120e50b1295de so the current memory target for PV guests is still fetched from start_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.
Using get_num_physpages() is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.
Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
--- drivers/xen/balloon.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/xen/balloon.c +6 -2
@@ -724,6 +724,7 @@ static int __init balloon_add_regions(void)static int __init balloon_init (void ) {struct task_struct *task; +unsigned long current_pages;int rc;if (!xen_domain ())
@@ -731,12 +732,15 @@ static int __init balloon_init(void)pr_info ("Initialising balloon driver\n" ); -if (xen_released_pages >=get_num_physpages ()) { + current_pages =xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) + :get_num_physpages (); + +if (xen_released_pages >= current_pages) {WARN (1 ,"Released pages underflow current target" );return -ERANGE ; } - balloon_stats.current_pages =get_num_physpages () - xen_released_pages; + balloon_stats.current_pages = current_pages - xen_released_pages; balloon_stats.target_pages = balloon_stats.current_pages; balloon_stats.balloon_low =0 ; balloon_stats.balloon_high =0 ;
Signature · 1 line
2.51.0
Quoted 13 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.
Quoted 6 lines
Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
While kind of obvious, I guess this needs a:
So it doesn't fall through the cracks for backports.
Thanks, Roger.
Quoted 13 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.
Quoted 8 lines
Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
Reported-by James Dingwall <[email protected]>Signed-off-by Roger Pau Monné <[email protected]>
Thanks,
Jason
Quoted 20 lines · Jason Andryuk
Quoted 19 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
Quoted 4 lines
Quoted 2 lines
Reported-by James Dingwall <[email protected]>Signed-off-by Roger Pau Monné <[email protected]>Reviewed-by Jason Andryuk <[email protected]>
Thanks.
I've been considering what we discussed and as a separate follow up we might want to attempt to switch to using XENMEM_current_reservation for dom0? It might make the accounting in PVH dom0 better, as that's what the toolstack uses to set the xenstore target when initializing dom0 values.
Regards, Roger.
Quoted 26 lines · Roger Pau Monné
Quoted 20 lines · Jason Andryuk
Quoted 19 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
Quoted 4 lines
Quoted 2 lines
Reported-by James Dingwall <[email protected]>Signed-off-by Roger Pau Monné <[email protected]>Reviewed-by Jason Andryuk <[email protected]>
Quoted 7 lines
Thanks.
I've been considering what we discussed and as a separate follow up we might want to attempt to switch to using
XENMEM_current_reservationfor dom0? It might make the accounting in PVH dom0 better, as that's what the toolstack uses to set the xenstore target when initializing dom0 values.
Yes, I thought that could be a follow on. I've attached what I have tested, but it is based on a branch pre-dating xen_released_pages. xenmem_current_reservation with PVH dom0 seemed good without the xen_released_pages adjustment, but I don't know what that would be for a PVH dom0.
Regards,
Jason
From 8b628ad0ebe52c30e31298e868f2f5187f2f52da Mon Sep 17 00:00:00 2001
From: Jason Andryuk <[email protected]>
Date: Fri, 7 Nov 2025 16:44:53 -0500
Subject: [PATCH] xen/balloon: Initialize dom0 with XENMEM_current_reservation
The balloon driver bases its action off the memory/target and memory/static-max xenstore keys. These are set by the toolstack and match the domain's hypervisor allocated pages - domain_tot_pages().
However, PVH and HVM domains query get_num_physpages() for the initial balloon driver current and target pages. get_num_physpages() is different from dom_totain_pages(), and has been observed way off in a PVH dom0. Here a PVH dom0 is assigned 917503 pages (3.5GB), but get_num_physpages() is 7312455:
xen:balloon: pages curr 7312455 target 7312455
xen:balloon: current_reservation 917503
While Xen truncated the PVH dom0 memory map's RAM, Linux undoes that operation and restores RAM regions in pvh_reserve_extra_memory().
Use XENMEM_current_reveration to initialize the balloon driver current and target pages. This is the hypervisor value, and matches what the toolstack writes. This avoids any ballooning from the inital allocation. While domUs are affected, the implications of the change are unclear - only make the change for dom0.
--- drivers/xen/balloon.c | 9 ++++++--- drivers/xen/mem-reservation.c | 8 ++++++++ include/xen/interface/memory.h | 5 +++++ include/xen/mem-reservation.h | 2 ++ 4 files changed, 21 insertions(+), 3 deletions(-)
drivers/xen/balloon.c +6 -3
@@ -713,10 +713,13 @@ static int __init balloon_init(void)#ifdef CONFIG_XEN_PV balloon_stats.current_pages =xen_pv_domain () - ?min (xen_start_info->nr_pages - xen_released_pages, max_pfn) - :get_num_physpages (); + ?min (xen_start_info->nr_pages - xen_released_pages, max_pfn) : +xen_initial_domain () ?xenmem_current_reservation () : +get_num_physpages ();#else - balloon_stats.current_pages =get_num_physpages (); + balloon_stats.current_pages = +xen_initial_domain () ?xenmem_current_reservation () : +get_num_physpages ();#endif balloon_stats.target_pages = balloon_stats.current_pages; balloon_stats.balloon_low =0 ;
drivers/xen/mem-reservation.c +8 -0
@@ -113,3 +113,11 @@ int xenmem_reservation_decrease(int count, xen_pfn_t *frames)return HYPERVISOR_memory_op (XENMEM_decrease_reservation, &reservation); }EXPORT_SYMBOL_GPL (xenmem_reservation_decrease); + +long xenmem_current_reservation (void ) +{ +struct xen_memory_domain domain = { .domid =DOMID_SELF }; + +return HYPERVISOR_memory_op (XENMEM_current_reservation, &domain); +} +EXPORT_SYMBOL_GPL (xenmem_current_reservation);
include/xen/interface/memory.h +5 -0
@@ -104,6 +104,11 @@ DEFINE_GUEST_HANDLE_STRUCT(xen_memory_exchange);*/ #define XENMEM_maximum_ram_page 2 +struct xen_memory_domain { +/* [IN] Domain information is being queried for. */ +domid_t domid; +}; +/* * Returns the current or maximum memory reservation, in pages, of the * specified domain (may be DOMID_SELF). Returns -ve errcode on failure.
include/xen/mem-reservation.h +2 -0
@@ -57,4 +57,6 @@ int xenmem_reservation_increase(int count, xen_pfn_t *frames);int xenmem_reservation_decrease (int count,xen_pfn_t *frames); +long xenmem_current_reservation (void ); +#endif
Signature · 1 line
2.52.0
Quoted 120 lines · Jason Andryuk
Quoted 26 lines · Roger Pau Monné
Quoted 20 lines · Jason Andryuk
Quoted 19 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
Quoted 4 lines
Quoted 2 lines
Reported-by James Dingwall <[email protected]>Signed-off-by Roger Pau Monné <[email protected]>Reviewed-by Jason Andryuk <[email protected]>Quoted 7 lines
Thanks.
I've been considering what we discussed and as a separate follow up we might want to attempt to switch to using
XENMEM_current_reservationfor dom0? It might make the accounting in PVH dom0 better, as that's what the toolstack uses to set the xenstore target when initializing dom0 values.Yes, I thought that could be a follow on. I've attached what I have tested, but it is based on a branch pre-dating
xen_released_pages.xenmem_current_reservationwith PVH dom0 seemed good without thexen_released_pagesadjustment, but I don't know what that would be for a PVH dom0.Regards,
JasonFrom
8b628ad0ebe52c30e31298e868f2f5187f2f52daMon Sep 17 00:00:00 2001
From: Jason Andryuk <[email protected]>
Date: Fri, 7 Nov 2025 16:44:53 -0500
Subject: [PATCH] xen/balloon: Initialize dom0 withXENMEM_current_reservationThe balloon driver bases its action off the memory/target and memory/static-max xenstore keys. These are set by the toolstack and match the domain's hypervisor allocated pages -
domain_tot_pages().However, PVH and HVM domains query
get_num_physpages()for the initial balloon driver current and target pages.get_num_physpages()is different fromdom_totain_pages(), and has been observed way off in a PVH dom0. Here a PVH dom0 is assigned 917503 pages (3.5GB), butget_num_physpages()is 7312455:
xen:balloon: pages curr 7312455 target 7312455
xen:balloon:current_reservation917503While Xen truncated the PVH dom0 memory map's RAM, Linux undoes that operation and restores RAM regions in
pvh_reserve_extra_memory().Use
XENMEM_current_reverationto initialize the balloon driver current and target pages. This is the hypervisor value, and matches what the toolstack writes. This avoids any ballooning from the inital allocation. While domUs are affected, the implications of the change are unclear - only make the change for dom0.Signed-off-by Jason Andryuk <[email protected]>--- drivers/xen/balloon.c | 9 ++++++--- drivers/xen/mem-reservation.c | 8 ++++++++ include/xen/interface/memory.h | 5 +++++ include/xen/mem-reservation.h | 2 ++ 4 files changed, 21 insertions(+), 3 deletions(-)drivers/xen/balloon.c +6
-3@@ -713,10 +713,13 @@ static int __init balloon_init(void)#ifdef CONFIG_XEN_PV balloon_stats.current_pages =xen_pv_domain () - ?min (xen_start_info->nr_pages - xen_released_pages, max_pfn) - :get_num_physpages (); + ?min (xen_start_info->nr_pages - xen_released_pages, max_pfn) : +xen_initial_domain () ?xenmem_current_reservation () : +get_num_physpages ();#else - balloon_stats.current_pages =get_num_physpages (); + balloon_stats.current_pages = +xen_initial_domain () ?xenmem_current_reservation () : +get_num_physpages ();#endif balloon_stats.target_pages = balloon_stats.current_pages; balloon_stats.balloon_low =0 ;drivers/xen/mem-reservation.c +8
-0@@ -113,3 +113,11 @@ int xenmem_reservation_decrease(int count, xen_pfn_t *frames)return HYPERVISOR_memory_op (XENMEM_decrease_reservation, &reservation); }EXPORT_SYMBOL_GPL (xenmem_reservation_decrease); + +long xenmem_current_reservation (void ) +{ +struct xen_memory_domain domain = { .domid =DOMID_SELF }; + +return HYPERVISOR_memory_op (XENMEM_current_reservation, &domain); +} +EXPORT_SYMBOL_GPL (xenmem_current_reservation);include/xen/interface/memory.h +0
-0
include/xen/interface/memory.h DEFINE_GUEST_HANDLE_STRUCT(xen_memory_exchange); · line 110
Quoted 7 lines
include/xen/interface/memory.h +4
-0@@ -104,6 +104,11 @@ DEFINE_GUEST_HANDLE_STRUCT(xen_memory_exchange);*/ #define XENMEM_maximum_ram_page 2 +struct xen_memory_domain { +/* [IN] Domain information is being queried for. */ +domid_t domid; +};
Other callers that would use xen_memory_domain just pass a pointer to a domid_t, I think you could simplify the patch to the diff below, which sits on top of the patch here:
I haven't tested it yet to see whether that's OK to do on PV, I would think PV and PVH would be the same here, since the setting of the xenstore target value is based in the return of XENMEM_current_reservation for both.
Thanks, Roger.
---
drivers/xen/balloon.c +9 -3
@@ -724,7 +724,8 @@ static int __init balloon_add_regions(void)static int __init balloon_init (void ) {struct task_struct *task; -unsigned long current_pages; +unsigned long current_pages =0 ; +domid_t domid =DOMID_SELF ;int rc;if (!xen_domain ())
@@ -732,8 +733,13 @@ static int __init balloon_init(void)pr_info ("Initialising balloon driver\n" ); - current_pages =xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) - :get_num_physpages (); +if (xen_initial_domain ()) + current_pages =HYPERVISOR_memory_op (XENMEM_current_reservation, + &domid); +if (current_pages <=0 ) + current_pages = +xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) + :get_num_physpages ();if (xen_released_pages >= current_pages) {WARN (1 ,"Released pages underflow current target" );
drivers/xen/balloon.c static int __init balloon_add_regions(void) · line 727
Quoted 8 lines · Roger Pau Monné
drivers/xen/balloon.c +1
-1@@ -724,7 +724,8 @@ static int __init balloon_add_regions(void)static int __init balloon_init (void ) {struct task_struct *task; -unsigned long current_pages; +unsigned long current_pages =0 ;
With this, ...
drivers/xen/balloon.c static int __init balloon_init(void) · line 739
Quoted 10 lines
drivers/xen/balloon.c +4
-2@@ -732,8 +733,13 @@ static int __init balloon_init(void)pr_info ("Initialising balloon driver\n" ); - current_pages =xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) - :get_num_physpages (); +if (xen_initial_domain ()) + current_pages =HYPERVISOR_memory_op (XENMEM_current_reservation, + &domid); +if (current_pages <=0 )
... why <= ? Gives the impression you may mean to cover HYPERVISOR_memory_op() returning an error, yet that'll require a signed long variable then.
Jan
Quoted 7 lines
drivers/xen/balloon.c +3
-0+ current_pages = +xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) + :get_num_physpages ();if (xen_released_pages >= current_pages) {WARN (1 ,"Released pages underflow current target" );
Quoted 16 lines · Jan Beulich
drivers/xen/balloon.c static int __init balloon_add_regions(void) · line 727
Quoted 8 lines · Roger Pau Monné
drivers/xen/balloon.c +1
-1@@ -724,7 +724,8 @@ static int __init balloon_add_regions(void)static int __init balloon_init (void ) {struct task_struct *task; -unsigned long current_pages; +unsigned long current_pages =0 ;With this, ...
Quoted 4 lines
drivers/xen/balloon.c +0
-0@@ -732,8 +733,13 @@ static int __init balloon_init(void)pr_info ("Initialising balloon driver\n" );
Quoted 9 lines
drivers/xen/balloon.c static int __init balloon_init(void) · line 739
Quoted 6 lines
drivers/xen/balloon.c +4
-2- current_pages =xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) - :get_num_physpages (); +if (xen_initial_domain ()) + current_pages =HYPERVISOR_memory_op (XENMEM_current_reservation, + &domid); +if (current_pages <=0 )... why <= ? Gives the impression you may mean to cover
HYPERVISOR_memory_op()returning an error, yet that'll require a signed long variable then.
Oh, indeed, current_pages should be signed long. This was an untested mash-up of a slightly different patch, where I didn't realize that current_pages was unsigned.
Thanks, Roger.
Quoted 124 lines · Roger Pau Monné
Quoted 123 lines · Jason Andryuk
Quoted 26 lines · Roger Pau Monné
Quoted 20 lines · Jason Andryuk
Quoted 19 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
Quoted 4 lines
Quoted 2 lines
Reported-by James Dingwall <[email protected]>Signed-off-by Roger Pau Monné <[email protected]>Reviewed-by Jason Andryuk <[email protected]>Quoted 7 lines
Thanks.
I've been considering what we discussed and as a separate follow up we might want to attempt to switch to using
XENMEM_current_reservationfor dom0? It might make the accounting in PVH dom0 better, as that's what the toolstack uses to set the xenstore target when initializing dom0 values.Yes, I thought that could be a follow on. I've attached what I have tested, but it is based on a branch pre-dating
xen_released_pages.xenmem_current_reservationwith PVH dom0 seemed good without thexen_released_pagesadjustment, but I don't know what that would be for a PVH dom0.Regards,
JasonFrom
8b628ad0ebe52c30e31298e868f2f5187f2f52daMon Sep 17 00:00:00 2001
From: Jason Andryuk <[email protected]>
Date: Fri, 7 Nov 2025 16:44:53 -0500
Subject: [PATCH] xen/balloon: Initialize dom0 withXENMEM_current_reservationThe balloon driver bases its action off the memory/target and memory/static-max xenstore keys. These are set by the toolstack and match the domain's hypervisor allocated pages -
domain_tot_pages().However, PVH and HVM domains query
get_num_physpages()for the initial balloon driver current and target pages.get_num_physpages()is different fromdom_totain_pages(), and has been observed way off in a PVH dom0. Here a PVH dom0 is assigned 917503 pages (3.5GB), butget_num_physpages()is 7312455:
xen:balloon: pages curr 7312455 target 7312455
xen:balloon:current_reservation917503While Xen truncated the PVH dom0 memory map's RAM, Linux undoes that operation and restores RAM regions in
pvh_reserve_extra_memory().Use
XENMEM_current_reverationto initialize the balloon driver current and target pages. This is the hypervisor value, and matches what the toolstack writes. This avoids any ballooning from the inital allocation. While domUs are affected, the implications of the change are unclear - only make the change for dom0.Signed-off-by Jason Andryuk <[email protected]>--- drivers/xen/balloon.c | 9 ++++++--- drivers/xen/mem-reservation.c | 8 ++++++++ include/xen/interface/memory.h | 5 +++++ include/xen/mem-reservation.h | 2 ++ 4 files changed, 21 insertions(+), 3 deletions(-)drivers/xen/balloon.c +6
-3@@ -713,10 +713,13 @@ static int __init balloon_init(void)#ifdef CONFIG_XEN_PV balloon_stats.current_pages =xen_pv_domain () - ?min (xen_start_info->nr_pages - xen_released_pages, max_pfn) - :get_num_physpages (); + ?min (xen_start_info->nr_pages - xen_released_pages, max_pfn) : +xen_initial_domain () ?xenmem_current_reservation () : +get_num_physpages ();#else - balloon_stats.current_pages =get_num_physpages (); + balloon_stats.current_pages = +xen_initial_domain () ?xenmem_current_reservation () : +get_num_physpages ();#endif balloon_stats.target_pages = balloon_stats.current_pages; balloon_stats.balloon_low =0 ;drivers/xen/mem-reservation.c +8
-0@@ -113,3 +113,11 @@ int xenmem_reservation_decrease(int count, xen_pfn_t *frames)return HYPERVISOR_memory_op (XENMEM_decrease_reservation, &reservation); }EXPORT_SYMBOL_GPL (xenmem_reservation_decrease); + +long xenmem_current_reservation (void ) +{ +struct xen_memory_domain domain = { .domid =DOMID_SELF }; + +return HYPERVISOR_memory_op (XENMEM_current_reservation, &domain); +} +EXPORT_SYMBOL_GPL (xenmem_current_reservation);include/xen/interface/memory.h +0
-0@@ -104,6 +104,11 @@ DEFINE_GUEST_HANDLE_STRUCT(xen_memory_exchange);*/ #define XENMEM_maximum_ram_page 2
Quoted 8 lines
include/xen/interface/memory.h DEFINE_GUEST_HANDLE_STRUCT(xen_memory_exchange); · line 110
Quoted 4 lines
include/xen/interface/memory.h +4
-0+struct xen_memory_domain { +/* [IN] Domain information is being queried for. */ +domid_t domid; +};Other callers that would use
xen_memory_domainjust pass a pointer to adomid_t, I think you could simplify the patch to the diff below, which sits on top of the patch here:
Ah, yes, xen_memory_domain is just wrapping a domid.
Quoted 5 lines
I haven't tested it yet to see whether that's OK to do on PV, I would think PV and PVH would be the same here, since the setting of the xenstore target value is based in the return of
XENMEM_current_reservationfor both.
On a system with 32GB and dom0=pvh dom0_mem=7G:
[ 0.295201] xen:balloon: current_pages: 1835007 get_num_physpages 8220126 xen_released_pages 6385120 [ 0.295201] ------------[ cut here ]------------ [ 0.295201] Released pages underflow current target
8220126 - 6385120 = 1835006
And also for PV:
[ 1.406923] xen:balloon: current_pages: 1835008 get_num_physpages 8220127 xen_released_pages 6385120 [ 1.406928] ------------[ cut here ]------------ [ 1.406931] Released pages underflow current target
So we don't want to subtract xen_released_pages for dom0. Is xen_released_pages expected to be non-zero for a domU?
IIRC, for a domU, xl writes the xenstore nodes as the ~build time memory value, which doesn't include video ram. Later QEMU populates the videoram, which increases current reservation. Then the two values don't match when the domU initializes the balloon values.
Regards,
Jason
Quoted 17 lines · Jason Andryuk
Quoted 4 lines · Roger Pau Monné
I haven't tested it yet to see whether that's OK to do on PV, I would think PV and PVH would be the same here, since the setting of the xenstore target value is based in the return of
XENMEM_current_reservationfor both.On a system with 32GB and dom0=pvh dom0_mem=7G:
[ 0.295201] xen:balloon: current_pages: 1835007 get_num_physpages 8220126 xen_released_pages 6385120 [ 0.295201] ------------[ cut here ]------------ [ 0.295201] Released pages underflow current target8220126 - 6385120 = 1835006
And also for PV:
Quoted 8 lines
[ 1.406923] xen:balloon: current_pages: 1835008 get_num_physpages 8220127 xen_released_pages 6385120 [ 1.406928] ------------[ cut here ]------------ [ 1.406931] Released pages underflow current targetSo we don't want to subtract
xen_released_pagesfor dom0. Isxen_released_pagesexpected to be non-zero for a domU?
Oh, yes. In fact I think the patch here is wrong for PV dom0, as it shouldn't subtract xen_released_pages from xen_start_info->nr_pages. I will need to send v2.
Quoted 4 lines
IIRC, for a domU, xl writes the xenstore nodes as the ~build time memory value, which doesn't include video ram. Later QEMU populates the videoram, which increases current reservation. Then the two values don't match when the domU initializes the balloon values.
Yeah, the modifications done to the physmap by QEMU skew the target, so what's in xenstore doesn't match what XENMEM_current_reservation returns. However is very hard to fix this. We could attempt to make the toolstack write the xenstore node based on the return of XENMEM_current_reservation once QEMU has started. Sadly a domU would have no way to know whether the xenstore value accounts for the QEMU consumed memory or not. We would need to introduce a new target xenstore node, which is equally messy.
Thanks for the testing.
Roger.
Quoted 24 lines · Roger Pau Monné
Quoted 23 lines · Jason Andryuk
Quoted 4 lines · Roger Pau Monné
I haven't tested it yet to see whether that's OK to do on PV, I would think PV and PVH would be the same here, since the setting of the xenstore target value is based in the return of
XENMEM_current_reservationfor both.On a system with 32GB and dom0=pvh dom0_mem=7G:
[ 0.295201] xen:balloon: current_pages: 1835007 get_num_physpages 8220126 xen_released_pages 6385120 [ 0.295201] ------------[ cut here ]------------ [ 0.295201] Released pages underflow current target8220126 - 6385120 = 1835006
And also for PV:
[ 1.406923] xen:balloon: current_pages: 1835008 get_num_physpages 8220127 xen_released_pages 6385120 [ 1.406928] ------------[ cut here ]------------ [ 1.406931] Released pages underflow current target
Quoted 6 lines
Quoted 2 lines
So we don't want to subtract
xen_released_pagesfor dom0. Isxen_released_pagesexpected to be non-zero for a domU?Oh, yes. In fact I think the patch here is wrong for PV dom0, as it shouldn't subtract
xen_released_pagesfromxen_start_info->nr_pages. I will need to send v2.
To be clear, the numbers and warning are from the follow on current_reservation patch.
Regards,
Jason
Quoted 28 lines · Jason Andryuk
Quoted 27 lines · Roger Pau Monné
Quoted 25 lines · Jason Andryuk
Quoted 4 lines · Roger Pau Monné
I haven't tested it yet to see whether that's OK to do on PV, I would think PV and PVH would be the same here, since the setting of the xenstore target value is based in the return of
XENMEM_current_reservationfor both.On a system with 32GB and dom0=pvh dom0_mem=7G:
[ 0.295201] xen:balloon: current_pages: 1835007 get_num_physpages 8220126 xen_released_pages 6385120 [ 0.295201] ------------[ cut here ]------------ [ 0.295201] Released pages underflow current target8220126 - 6385120 = 1835006
And also for PV:
[ 1.406923] xen:balloon: current_pages: 1835008 get_num_physpages 8220127 xen_released_pages 6385120 [ 1.406928] ------------[ cut here ]------------ [ 1.406931] Released pages underflow current targetSo we don't want to subtract
xen_released_pagesfor dom0. Isxen_released_pagesexpected to be non-zero for a domU?
Quoted 6 lines
Quoted 3 lines
Oh, yes. In fact I think the patch here is wrong for PV dom0, as it shouldn't subtract
xen_released_pagesfromxen_start_info->nr_pages. I will need to send v2.To be clear, the numbers and warning are from the follow on
current_reservationpatch.
Yes, but I think it's also bad to use xen_start_info->nr_pages - xen_released_pages (my current proposal). xen_released_pages had a different meaning on PV which I kind of stolen for the unpopulated alloc work. It was originally meant to signal pages that are freed during the boot process when it's not possible to relocate them; see xen_set_identity_and_release_chunk. Unpopulated alloc doesn't free the pages it uses, because they are already free to start with.
I think I need to introduce a new counter that accounts for pages consumed by unpopulated alloc strictly, and not re-use the xen_released_pages counter.
All this memory accounting is far more complex than it should be sadly.
Thanks, Roger.
Quoted 58 lines · Roger Pau Monne
This partially reverts commit
87af633689ce16ddb166c80f32b120e50b1295deso the current memory target for PV guests is still fetched fromstart_info->nr_pages, which matches exactly what the toolstack sets the initial memory target to.Using
get_num_physpages()is possible on PV also, but needs adjusting to take into account the ISA hole and the PFN at 0 not considered usable memory depite being populated, and hence would need extra adjustments. Instead of carrying those extra adjustments switch back to the previous code. That leaves Linux with a difference in how current memory target is obtained for HVM vs PV, but that's better than adding extra logic just for PV.Also, for HVM the target is not (and has never been) accurately calculated, as in that case part of what starts as guest memory is reused by hvmloader and possibly other firmware to store ACPI tables and similar firmware information, thus the memory is no longer being reported as RAM in the memory map.
Reported-by James Dingwall <[email protected]>Signed-off-by Roger Pau Monné <[email protected]>--- drivers/xen/balloon.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/xen/balloon.c +6
-2@@ -724,6 +724,7 @@ static int __init balloon_add_regions(void)static int __init balloon_init (void ) {struct task_struct *task; +unsigned long current_pages;int rc;if (!xen_domain ())@@ -731,12 +732,15 @@ static int __init balloon_init(void)pr_info ("Initialising balloon driver\n" ); -if (xen_released_pages >=get_num_physpages ()) { + current_pages =xen_pv_domain () ?min (xen_start_info->nr_pages, max_pfn) + :get_num_physpages (); + +if (xen_released_pages >= current_pages) {WARN (1 ,"Released pages underflow current target" );return -ERANGE ; } - balloon_stats.current_pages =get_num_physpages () - xen_released_pages; + balloon_stats.current_pages = current_pages - xen_released_pages; balloon_stats.target_pages = balloon_stats.current_pages; balloon_stats.balloon_low =0 ; balloon_stats.balloon_high =0 ;Signature · 1 line
2.51.0
Thank you Roger, I tested this patch on the system which originally showed the error and the pci passthrough now works as expected.
Regards,
James