libxl: memory management patches
v3 cover libxl: memory management patches
Hi,
Here are three small but important libxl/xl memory management patches:
- libxl: xl mem-max et consortes must update static-max in xenstore too,
- xl: Allow user to configure xl mem-set behavior,
- xl: Improve xl documentation in regards to guest memory management.
Daniel
PATCH v3 1/3 libxl: xl mem-max et consortes must update static-max in xenstore too
xl mem-max et consortes must update static-max in xenstore too. Without this patch there is no chance to increase memory reservation for given domain above memory limit defined in config file. It means that memory hotplug is practicaly unusable without this patch.
--- tools/libxl/libxl.c | 3 +++ 1 file changed, 3 insertions(+)
tools/libxl/libxl.c +3 -0
@@ -3484,6 +3484,9 @@ int libxl_domain_setmaxmem(libxl_ctx *ctx, uint32_t domid, uint32_t max_memkb) goto out; } +libxl__xs_write (gc,XBT_NULL ,libxl__sprintf (gc,"%s/memory/static-max" , + dompath),"%" PRIu32, max_memkb); + rc =0 ; out:GC_FREE ;
Signature · 1 line
1.7.10.4
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Daniel Kiper writes ("[Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 4 lines
xl mem-max et consortes must update static-max in xenstore too. Without this patch there is no chance to increase memory reservation for given domain above memory limit defined in config file. It means that memory hotplug is practicaly unusable without this patch.
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
I thought that the static-max xenstore key was used for the former.
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 6 lines · Ian Jackson
Daniel Kiper writes ("[Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 4 lines
xl mem-max et consortes must update static-max in xenstore too. Without this patch there is no chance to increase memory reservation for given domain above memory limit defined in config file. It means that memory hotplug is practicaly unusable without this patch.
Quoted 6 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 6 lines · Ian Jackson
Quoted 5 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
Quoted 7 lines
Quoted 1 line
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
So that's not "enforcing a limit on guests".
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 15 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 6 lines · Ian Jackson
Quoted 5 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
Quoted 7 lines
Quoted 1 line
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
Quoted 3 lines
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not. That is why I think that static-max should be unconditionally changed or the guest should write something in xenstore to inform that it supports memory hotplug and relevant check should be waived.
Quoted 1 line
So that's not "enforcing a limit on guests".
OK, it should be phrased that xl checks that itself does not exceeds limits.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 7 lines · Ian Jackson
Quoted 3 lines
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
Right.
Quoted 3 lines
That is why I think that static-max should be unconditionally changed or the guest should write something in xenstore to inform that it supports memory hotplug and relevant check should be waived.
Right. I don't see what any of that has to do with "xl mem-max" though. The purpose of xl mem-max is to update the guest's enforcement limit.
I think it would be great for the guest to write something to xenstore saying it supports memory hotplug. How does the protocol work ? The guest writes "I support memory hotplug", and the tools then ignore static-max and simply update the memory target with a new value ? Do the tools need to do anything else ?
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 11 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 7 lines · Ian Jackson
Quoted 3 lines
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
Right.
Quoted 7 lines
Quoted 3 lines
That is why I think that static-max should be unconditionally changed or the guest should write something in xenstore to inform that it supports memory hotplug and relevant check should be waived.
Right. I don't see what any of that has to do with "xl mem-max" though. The purpose of xl mem-max is to update the guest's enforcement limit.
...and static-max if you would like to use memory hotplug in real. If not, then xl mem-set does not allow you to allocate more memory to a guest than static-max.
Quoted 3 lines
I think it would be great for the guest to write something to xenstore saying it supports memory hotplug. How does the protocol work ? The guest writes "I support memory hotplug", and the tools then ignore
More or less...
Quoted 2 lines
static-max and simply update the memory target with a new value ? Do the tools need to do anything else ?
I think that tools should not ignore static-max in any casy. Memory target should not be never ever above static-max.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 6 lines · Ian Jackson
Quoted 5 lines
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 3 lines
That is why I think that static-max should be unconditionally changed or the guest should write something in xenstore to inform that it supports memory hotplug and relevant check should be waived.
Quoted 7 lines
Quoted 3 lines
Right. I don't see what any of that has to do with "xl mem-max" though. The purpose of xl mem-max is to update the guest's enforcement limit.
...and static-max if you would like to use memory hotplug in real. If not, then xl mem-set does not allow you to allocate more memory to a guest than static-max.
You seem to be losing sight of the purpose of the static-max key in xenstore. It is for the benefit of the toolstack - and, indirectly, the user - so that we know the maximum amount of memory we can ask the guest to us.
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 1 line
No, memory hotplug should not change "static maximum" in any case.
Why shouldn't it ?
Quoted 3 lines
"static maximum" should define memory limit for a given guest. If "static maximum" is above "build maximum" then memory hotplug in the guest should be involved (if it is available).
If a guest supports memory hotplug, does it simply hotplug more memory response to changes in target ? Is there not a limit on how much memory hotplug a guest can cope with ?
Perhaps the guest should advertise its actual maximum ?
Quoted 5 lines
Quoted 2 lines
No, because a guest which is being asked to shrink will experience memory allocation failures if it doesn't shrink monotonically.
IIRC if you shrink memory for a given guest and it does not have memory balloon driver installed nothing happens.
Yes.
Quoted 2 lines
I did not tested it but it would be crazy when host could steal
random page from the guest without its acceptance.
It can't steal memory from the guest. But if the guest does any operation which allocates memory, when it is above the limit, that operation can fail.
So for example consider a guest with a balloon driver (but without memory hotplug) which was booted with 1G and subsequently ballooned down to 768M.
If the user says "xl mem-set blah 512M" then the limit and target are both set to 512M. If the guest's balloon driver, instead of shrinking the guest as it is supposed to, tries to allieviate some memory pressure inside the guest by allocating _more_ memory from Xen, that allocation will fail.
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 4 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 1 line
No, memory hotplug should not change "static maximum" in any case.
Why shouldn't it ?
FWIW, at least XenServer uses "static max" to mean "cannot be changed except by rebooting". It *also* uses *that* for HVM guests to say "how much memory to advertize to the guest". That is, if you boot a VM with memory = X and static_max = Y, then it will put Y amount of memory in the virtual e820 map, allocate X memory for the PoD, and then rely on the balloon driver to balloon from Y down to X during boot.
This is two slightly different meanings -- "how much to put in the guest e820" and "how much maximum a guest is ever allowed to have". The plain meaning of "static max" should be the second (i.e., can't be changed dynamically at run time). If we want a "max in the e820 at build time" we should probably come up with a new term; and if "how much the guest is ever allowed to have without rebooting" is not a useful concept, we should probably just discard it.
-George
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 14 lines · George Dunlap
Quoted 4 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 1 line
No, memory hotplug should not change "static maximum" in any case.
Why shouldn't it ?
FWIW, at least XenServer uses "static max" to mean "cannot be changed except by rebooting". It *also* uses *that* for HVM guests to say "how much memory to advertize to the guest". That is, if you boot a VM with memory = X and
static_max= Y, then it will put Y amount of memory in the virtual e820 map, allocate X memory for the PoD, and then rely on the balloon driver to balloon from Y down to X during boot.
Quoted 7 lines
This is two slightly different meanings -- "how much to put in the guest e820" and "how much maximum a guest is ever allowed to have". The plain meaning of "static max" should be the second (i.e., can't be changed dynamically at run time). If we want a "max in the e820 at build time" we should probably come up with a new term; and if "how much the guest is ever allowed to have without rebooting" is not a useful concept, we should probably just discard it.
George, thanks for this clarification.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
In reply to Ian JacksonQuoted 11 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 10 lines · Ian Jackson
Quoted 8 lines
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 3 lines
That is why I think that static-max should be unconditionally changed or the guest should write something in xenstore to inform that it supports memory hotplug and relevant check should be waived.
Right. I don't see what any of that has to do with "xl mem-max" though. The purpose of xl mem-max is to update the guest's enforcement limit.
Quoted 8 lines
Quoted 3 lines
...and static-max if you would like to use memory hotplug in real. If not, then xl mem-set does not allow you to allocate more memory to a guest than static-max.
You seem to be losing sight of the purpose of the static-max key in xenstore. It is for the benefit of the toolstack - and, indirectly, the user - so that we know the maximum amount of memory we can ask the guest to us.
OK, I understand that. However, static-max equal to maxmem makes sens for a guest without memory hotplug. If you have memory hotplug and you could not change static-max then this is an obstacle to use memory hotplug in real. You could not raise target above static-max using xl mem-set and you could not change target from guest because of "xen maximum" too.
Now we have two options:
- we could allow user to change static-max for a given guest by calling xl mem-max (my current solution); this way we change a bit meaning of static-max from "maximum amount of memory allowed for the guest (usualy all guest OS structures were prepared for this amount of memory but they do not need to be filled at boot time)" to "maximum amount of memory allowed for the guest at a given moment",
- we could leave static-max as is and use "xen maximum" as "maximum amount of memory allowed for the guest at a given moment"; However, in this case comparison with static-max in libxl_set_memory_target() should be changed to comparison with "xen maximum".
[...]
Quoted 19 lines
Quoted 5 lines
Quoted 2 lines
No, because a guest which is being asked to shrink will experience memory allocation failures if it doesn't shrink monotonically.
IIRC if you shrink memory for a given guest and it does not have memory balloon driver installed nothing happens.
Yes.
Quoted 2 lines
I did not tested it but it would be crazy when host could steal
random page from the guest without its acceptance.It can't steal memory from the guest. But if the guest does any operation which allocates memory, when it is above the limit, that operation can fail.
So for example consider a guest with a balloon driver (but without memory hotplug) which was booted with 1G and subsequently ballooned down to 768M.
Quoted 5 lines
If the user says "xl mem-set blah 512M" then the limit and target are both set to 512M. If the guest's balloon driver, instead of shrinking the guest as it is supposed to, tries to allieviate some memory pressure inside the guest by allocating _more_ memory from Xen, that allocation will fail.
I suppose it could be phrased like that:
Guests which attempts to exceed or exceeds the target may experience memory allocation failures.
It takes in to account your case and my case when a guest attempts to raise its allocation above target.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 28 lines · Daniel Kiper
Quoted 11 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 10 lines · Ian Jackson
Quoted 8 lines
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 3 lines
That is why I think that static-max should be unconditionally changed or the guest should write something in xenstore to inform that it supports memory hotplug and relevant check should be waived.
Right. I don't see what any of that has to do with "xl mem-max" though. The purpose of xl mem-max is to update the guest's enforcement limit.
Quoted 8 lines
Quoted 3 lines
...and static-max if you would like to use memory hotplug in real. If not, then xl mem-set does not allow you to allocate more memory to a guest than static-max.
You seem to be losing sight of the purpose of the static-max key in xenstore. It is for the benefit of the toolstack - and, indirectly, the user - so that we know the maximum amount of memory we can ask the guest to us.
OK, I understand that. However, static-max equal to maxmem makes sens for a guest without memory hotplug. If you have memory hotplug and you could not change static-max then this is an obstacle to use memory hotplug in real. You could not raise target above static-max using xl mem-set and you could not change target from guest because of "xen maximum" too.
Quoted 11 lines
Now we have two options:
- we could allow user to change static-max for a given guest by calling xl mem-max (my current solution); this way we change a bit meaning of static-max from "maximum amount of memory allowed for the guest (usualy all guest OS structures were prepared for this amount of memory but they do not need to be filled at boot time)" to "maximum amount of memory allowed for the guest at a given moment",
- we could leave static-max as is and use "xen maximum" as "maximum amount of memory allowed for the guest at a given moment"; However, in this case comparison with static-max inlibxl_set_memory_target()should be changed to comparison with "xen maximum".
How does this stuff work for physical memory hotplug? Understanding that might help us decide what admins expect (and is also directly relevant to HVM memory hotplug).
In the e820 of a physical system memory which is actually present at boot is obviously represented as E820_MEMORY, but how are the holes in the memory map where DIMMs could subsequently be physically inserted represented? Are they just "reserved" or is there a special "unpopulated memory" type?
And on the PV kernel side how does this appear to the guest? If you boot a "massively ballooned" guest (e.g. 1G out of max 1TB) does it automatically switch to making the bulk of the difference a hotpluggable region rather than a balloon region? What does "pre-ballooned" even mean to a guest which supports memory hotplug?
Do the kernels support memory unplug?
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 1 line · Ian Campbell
On Thu, 2013-04-11 at 13:24 +0100, Daniel Kiper wrote:
[...]
Quoted 12 lines
Quoted 11 lines
Now we have two options:
- we could allow user to change static-max for a given guest by calling xl mem-max (my current solution); this way we change a bit meaning of static-max from "maximum amount of memory allowed for the guest (usualy all guest OS structures were prepared for this amount of memory but they do not need to be filled at boot time)" to "maximum amount of memory allowed for the guest at a given moment",
- we could leave static-max as is and use "xen maximum" as "maximum amount of memory allowed for the guest at a given moment"; However, in this case comparison with static-max inlibxl_set_memory_target()should be changed to comparison with "xen maximum".
Quoted 3 lines
How does this stuff work for physical memory hotplug? Understanding that might help us decide what admins expect (and is also directly relevant to HVM memory hotplug).
Memory hotplug works in the same way in PV and HVM guest.
Quoted 5 lines
In the e820 of a physical system memory which is actually present at boot is obviously represented as
E820_MEMORY, but how are the holes in the memory map where DIMMs could subsequently be physically inserted represented? Are they just "reserved" or is there a special "unpopulated memory" type?
Region for hotplugged memory is not reserved in any way. Only memory available at boot have e820 entries. After hotplugging memory hardware places it at relevant address and informs system about new memory and its config usualy via ACPI. System admin must online new memory via sysfs interface.
Current memory hotplug implementation in balloon driver does not use ACPI and establishes placement for new memory itself (simply above max_pfn; algorithm is not perfect and it fails in some cases; I am going to fix it). Other things work like in physical case.
Quoted 5 lines
And on the PV kernel side how does this appear to the guest? If you boot a "massively ballooned" guest (e.g. 1G out of max 1TB) does it automatically switch to making the bulk of the difference a hotpluggable region rather than a balloon region? What does "pre-ballooned" even mean to a guest which supports memory hotplug?
Guest is "massively ballooned". There is no automatic change from balloon to memory hotplug.
Quoted 1 line
Do the kernels support memory unplug?
Linux Kernel supports memory unplug (hot remove) on baremetal. In Xen guest case memory is ballooned down after memory hotplug.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 47 lines · Daniel Kiper
Quoted 1 line · Ian Campbell
On Thu, 2013-04-11 at 13:24 +0100, Daniel Kiper wrote:
[...]
Quoted 12 lines
Quoted 11 lines
Now we have two options:
- we could allow user to change static-max for a given guest by calling xl mem-max (my current solution); this way we change a bit meaning of static-max from "maximum amount of memory allowed for the guest (usualy all guest OS structures were prepared for this amount of memory but they do not need to be filled at boot time)" to "maximum amount of memory allowed for the guest at a given moment",
- we could leave static-max as is and use "xen maximum" as "maximum amount of memory allowed for the guest at a given moment"; However, in this case comparison with static-max inlibxl_set_memory_target()should be changed to comparison with "xen maximum".Quoted 3 lines
How does this stuff work for physical memory hotplug? Understanding that might help us decide what admins expect (and is also directly relevant to HVM memory hotplug).
Memory hotplug works in the same way in PV and HVM guest.
Quoted 5 lines
In the e820 of a physical system memory which is actually present at boot is obviously represented as
E820_MEMORY, but how are the holes in the memory map where DIMMs could subsequently be physically inserted represented? Are they just "reserved" or is there a special "unpopulated memory" type?Region for hotplugged memory is not reserved in any way. Only memory available at boot have e820 entries. After hotplugging memory hardware places it at relevant address and informs system about new memory and its config usualy via ACPI. System admin must online new memory via sysfs interface.
Current memory hotplug implementation in balloon driver does not use ACPI and establishes placement for new memory itself (simply above
max_pfn; algorithm is not perfect and it fails in some cases; I am going to fix it). Other things work like in physical case.Quoted 5 lines
And on the PV kernel side how does this appear to the guest? If you boot a "massively ballooned" guest (e.g. 1G out of max 1TB) does it automatically switch to making the bulk of the difference a hotpluggable region rather than a balloon region? What does "pre-ballooned" even mean to a guest which supports memory hotplug?
Quoted 7 lines
Guest is "massively ballooned". There is no automatic change from balloon to memory hotplug.
Quoted 1 line
Do the kernels support memory unplug?
Linux Kernel supports memory unplug (hot remove) on baremetal. In Xen guest case memory is ballooned down after memory hotplug.
Ian, any additional thoughts, comments, ...? I would like to prepare next version of patch ASAP. I hope it will be included in Xen 4.3.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 1 line · Daniel Kiper
Ian, any additional thoughts, comments, ...?
Do you mean me or IanJ?
I think I'm still of the opinion that operations which may involve memory hotplug on the guest side rather than just ballooning should be explicit, at least at the libxl API layer. TBH I'm not sure what this would look like since the current API doesn't really seem to suggest a natural way to extend it like this.
At the xl layer I'm not so sure what the answer is but xl is easier to change if we aren't happy with it.
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Quoted 4 lines · Ian Campbell
Quoted 1 line · Daniel Kiper
Ian, any additional thoughts, comments, ...?
Do you mean me or IanJ?
You, but if IanJ has comments I am happy to hear them too...
Quoted 8 lines
I think I'm still of the opinion that operations which may involve memory hotplug on the guest side rather than just ballooning should be explicit, at least at the libxl API layer. TBH I'm not sure what this would look like since the current API doesn't really seem to suggest a natural way to extend it like this.
At the xl layer I'm not so sure what the answer is but xl is easier to change if we aren't happy with it.
I think that we should not care (from a host point of view) which mechanism is used in a guest to increase its memory size. From the host point of view in balloon case and in memory hotplug case we just allocate pages for the guest. We do not care where they will be placed and when they will be used. That is it.
However, we should create a mechanism which allows the guest to allocate memory above limit set at boot and not eat all host memory in uncontrolled manner (e.g. using memory hotplug mechanism). Last is provided by xl now but former is not. Currently there is no way to increase memory limit above memory limit set at boot (maxmem). Now I think we should just decide which value should be used as a cap reference: static-max or "xen maximum". Both are not perfect and some recalculation is always needed to establish real cap internaly in the guest.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too [and 1 more messages]"):
Quoted 5 lines
I think that we should not care (from a host point of view) which mechanism is used in a guest to increase its memory size. From the host point of view in balloon case and in memory hotplug case we just allocate pages for the guest. We do not care where they will be placed and when they will be used. That is it.
Yes, exactly.
Quoted 4 lines
However, we should create a mechanism which allows the guest to allocate memory above limit set at boot and not eat all host memory in uncontrolled manner (e.g. using memory hotplug mechanism). Last is provided by xl now but former is not.
Yes. That limit mechanism is the Xen memory limit, which by default is set by xl mem-set and with your changes can be independently set with xl mem-max.
Quoted 2 lines
Currently there is no way to increase memory limit above memory limit set at boot (maxmem).
That's the wrong way to think about it. Currently xl will allow neither the target nor the xen maximum to be increased beyond the boot maximum, because they think that won't work.
Quoted 4 lines
Now I think we should just decide which value should be used as a cap reference: static-max or "xen maximum". Both are not perfect and some recalculation is always needed to establish real cap internaly in the guest.
I don't know what "cap reference" is.
It seems to me that if the guest could in principle do memory hotplug without assistance from the toolstack, all that's needed is:
* Guest advertises memory hotplug capability in xenstore (presumably it advertises a maximum figure, rather than just promising to be able to use any amount of memory, since most guests will be able to hotplug only a certain amount?)
* When memory hotplug limit is advertised, libxl ignores static-max in xenstore in favour of the guest's advertised capability. xl mem-set (and if different, xl mem-max) can be told to set the guest's memory target (in xenstore), and the xen memory limit, to more than the boot time memory. When this happens the guest's balloon/hotplug driver can hotplug memory appropriately to bring the guest's memory usage to the requested target.
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
In reply to Daniel KiperQuoted 16 lines · Daniel Kiper
Quoted 15 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 13 lines · Ian Jackson
Quoted 6 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
Quoted 6 lines
Quoted 3 lines
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
This is why I previously queried whether memory hotplug ought not to be a conscious admin decision.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 21 lines · Ian Campbell
Quoted 20 lines · Daniel Kiper
Quoted 18 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 6 lines · Ian Jackson
Quoted 5 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
Quoted 7 lines
Quoted 1 line
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
Quoted 5 lines
Quoted 2 lines
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
This is why I previously queried whether memory hotplug ought not to be a conscious admin decision.
Sure but I do not understand how this is related to above. Addtitionally, which admin do you mean? Guest or host?
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 22 lines · Daniel Kiper
Quoted 21 lines · Ian Campbell
Quoted 20 lines · Daniel Kiper
Quoted 18 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 6 lines · Ian Jackson
Quoted 5 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
Quoted 7 lines
Quoted 1 line
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
This is why I previously queried whether memory hotplug ought not to be a conscious admin decision.
Sure but I do not understand how this is related to above.
Because an admin might know whether hotplug is (potentially) an option for the domain.
Quoted 1 line
Addtitionally, which admin do you mean? Guest or host?
Host.
Ian.
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 29 lines · Ian Campbell
Quoted 28 lines · Daniel Kiper
Quoted 26 lines · Ian Campbell
Quoted 16 lines · Daniel Kiper
Quoted 15 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 13 lines · Ian Jackson
Quoted 6 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
Quoted 6 lines
Quoted 3 lines
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
This is why I previously queried whether memory hotplug ought not to be a conscious admin decision.
Quoted 8 lines
Quoted 1 line
Sure but I do not understand how this is related to above.
Because an admin might know whether hotplug is (potentially) an option for the domain.
Quoted 1 line
Addtitionally, which admin do you mean? Guest or host?
Host.
I thought once that balloon driver with memory hotplug support might store something in xenstore. Maybe it will be sufficient.
Daniel
Re: [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too
Quoted 35 lines · Daniel Kiper
Quoted 34 lines · Ian Campbell
Quoted 22 lines · Daniel Kiper
Quoted 21 lines · Ian Campbell
Quoted 20 lines · Daniel Kiper
Quoted 18 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 1/3] libxl: xl mem-max et consortes must update static-max in xenstore too"):
Quoted 6 lines · Ian Jackson
Quoted 5 lines
Doesn't this, together with your previous patch, conflate the "static maximum" (ie, boot-time memory size which in the absence of memory hotplug can never be exceeded), with the "xen maximum" (ie the enforced memory limit) ?
Quoted 7 lines
Quoted 1 line
I thought that the static-max xenstore key was used for the former.
To some extent. However, static-max has always a bit larger value than "xen maximum". xl uses static-max to enforce limits on guests but it is just an info for guest itself. "xen maximum" is a kind of hard limit which could not be exceeded and is enforced on guest by Xen hypervisor.
The reason for xl using static-max is that in the absence of memory hotplug, attempting to raise a guest above static-max will not work. And this check takes effect in xl.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
OK but now it is quiet difficult (or close to impossible) to know in advance that a given guest supports memory hotplug or not.
This is why I previously queried whether memory hotplug ought not to be a conscious admin decision.
Sure but I do not understand how this is related to above.
Because an admin might know whether hotplug is (potentially) an option for the domain.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Addtitionally, which admin do you mean? Guest or host?
Host.
I thought once that balloon driver with memory hotplug support might store something in xenstore. Maybe it will be sufficient.
Same as in previous email...
Daniel
PATCH v3 2/3 xl: Allow user to configure xl mem-set behavior
In reply to Daniel KiperAdd mem_set_enforce_limit option to xl.conf file and equivalent xl command line option. Those two things give a chance to align xl mem-set behavior to xm mem-set behavior. Default xl mem-set behavior is not changed.
v3 - suggestions/fixes:
- add xl command line option
(suggested by Ian Jackson).
--- tools/libxl/xl.c | 4 ++++ tools/libxl/xl.h | 1 + tools/libxl/xl_cmdimpl.c | 16 ++++++++++------ tools/libxl/xl_cmdtable.c | 3 ++- 4 files changed, 17 insertions(+), 7 deletions(-)
tools/libxl/xl.c +4 -0
@@ -38,6 +38,7 @@ xentoollog_logger_stdiostream *logger;int dryrun_only;int force_execution;int autoballoon =1 ; +int mem_set_enforce_limit =1 ;char *blkdev_start;int run_hotplug_scripts =1 ;char *lockfile;
@@ -72,6 +73,9 @@ static void parse_global_config(const char *configfile,if (!xlu_cfg_get_long (config,"autoballoon" , &l,0 )) autoballoon = l; +if (!xlu_cfg_get_long (config,"mem_set_enforce_limit" , &l,0 )) + mem_set_enforce_limit = l; +if (!xlu_cfg_get_long (config,"run_hotplug_scripts" , &l,0 )) run_hotplug_scripts = l;
tools/libxl/xl.h +1 -0
@@ -143,6 +143,7 @@ int xl_child_pid(xlchildnum); /* returns 0 if child struct is not in use *//* global options */ extern int autoballoon; +extern int mem_set_enforce_limit;extern int run_hotplug_scripts;extern int dryrun_only;extern char *lockfile;
tools/libxl/xl_cmdimpl.c +10 -6
@@ -2495,7 +2495,7 @@ int main_memmax(int argc, char **argv)return 0 ; } -static void set_memory_target (uint32_t domid,const char *mem) +static void set_memory_target (uint32_t domid,const char *mem,int enforce_limit) {long long int memorykb;
@@ -2505,23 +2505,27 @@ static void set_memory_target(uint32_t domid, const char *mem)exit (3 ); } -libxl_set_memory_target (ctx, domid, memorykb,0 ,/* enforce */ 1 ); +libxl_set_memory_target (ctx, domid, memorykb,0 , enforce_limit); }int main_memset (int argc,char **argv) {uint32_t domid; -int opt =0 ; +int enforce_limit, opt =0 ;const char *mem; - SWITCH_FOREACH_OPT(opt, "", NULL, "mem-set", 2) { -/* No options */ + enforce_limit = mem_set_enforce_limit; + + SWITCH_FOREACH_OPT(opt, "e:", NULL, "mem-set", 2) { +case 'e' : + enforce_limit =atoi (optarg) ?1 :0 ; +break ; } domid =find_domain (argv[optind]); mem = argv[optind +1 ]; -set_memory_target (domid, mem); +set_memory_target (domid, mem, enforce_limit);return 0 ; }
tools/libxl/xl_cmdtable.c +2 -1
@@ -195,7 +195,8 @@ struct cmd_spec cmd_table[] = { {"mem-set" , &main_memset,0 ,1 ,"Set the current memory usage for a domain" , -"<Domain> <MemMB['b'[bytes]|'k'[KB]|'m'[MB]|'g'[GB]|'t'[TB]]>" , +"[-e <0|1>] <Domain> <MemMB['b'[bytes]|'k'[KB]|'m'[MB]|'g'[GB]|'t'[TB]]>" , +"-e <0|1> (Not)Enforce maximum memory allocation for a domain" }, { "button-press", &main_button_press, 0, 1,
Signature · 1 line
1.7.10.4
PATCH v3 3/3 xl: Improve xl documentation in regards to guest memory management
In reply to Daniel KiperImprove xl documentation in regards to guest memory management.
Some improvements are suggested by Ian Jackson and Konrad Rzeszutek Wilk.
--- docs/man/xl.conf.pod.5 | 15 +++++++++++++++ docs/misc/libxl_memory.txt | 38 +++++++++++++++++++++++-------------- docs/misc/xenstore-paths.markdown | 15 +++++++-------- tools/examples/xl.conf | 4 ++++ 4 files changed, 50 insertions(+), 22 deletions(-)
docs/man/xl.conf.pod.5 +15 -0
@@ -55,6 +55,21 @@ reduce the amount of memory given to domain 0 by default. Default: C<1> +=item B<mem_set_enforce_limit=BOOLEAN> + +If enabled, then C<xl mem-set> by default will set a memory allocation +target and enforce the maximum memory allocation for a given domain. +If disabled, then C<xl mem-set> by default will set a memory allocation +target only for a given domain. + +The memory allocation target is a suggested amount of memory which +should be allocated by a guest. It is nice when the guest follows this +value but it may have its own idea about memory requirments. However, +it could not allocate more memory than a ceiling known as the maximum +memory allocation. + +Default: C<1> + =item B<run_hotplug_scripts=BOOLEAN> If disabled hotplug scripts will be called from udev, as it used to
docs/misc/libxl_memory.txt +24 -14
@@ -15,14 +15,19 @@ | | | | | | target | | | | guest | | | build | | | | | | | | start | | | - static | | | | | | | | - maximum | +----------+ | + + + + - | | | | - | | | | + | | | | | | | | + | +----------+ | + + + + + static | | | | + maximum | | | | | | balloon | | build | | | | maximum | | | | - + +----------+ + + | +----------+ + + | | | + | | memory | + | | hotplug | + | | | + + +----------+ extra internal = LIBXL_MAXMEM_CONSTANT
@@ -31,8 +36,8 @@ static maximum = libxl_domain_build_info.max_memkb video = libxl_domain_build_info.video_memkb build start = libxl_domain_build_info.target_memkb - libxl_domain_setmaxmem -> xen maximum - libxl_set_memory_target -> actual target + libxl_domain_setmaxmem -> xen maximum & static maximum + libxl_set_memory_target -> actual target & optionaly xen maximum === Domain memory breakdown: PV guests ==================================
@@ -49,22 +54,27 @@ | | | | | | actual | maximum | | | guest | | | build | target | | | | | | | start | | | - static | | | | | | | | - maximum | +----------+ | + + + + + | | | | | | | | + | +----------+ | + + + + | | | | - | | | | - | | balloon | | build + static | | | | + maximum | | balloon | | build | | | | maximum | | | | - + +----------+ + + | +----------+ + + | | | + | | memory | + | | hotplug | + | | | + + +----------+ extra internal = LIBXL_MAXMEM_CONSTANT extra external = LIBXL_PV_EXTRA_MEMORY static maximum = libxl_domain_build_info.max_memkb build start = libxl_domain_build_info.target_memkb - libxl_domain_setmaxmem -> xen maximum - libxl_set_memory_target -> actual target + libxl_domain_setmaxmem -> xen maximum & static maximum + libxl_set_memory_target -> actual target & optionaly xen maximum ========================================================================= */
docs/misc/xenstore-paths.markdown +7 -8
@@ -124,18 +124,17 @@ This protocol is not currently well documented. #### ~/memory/static-max = MEMKB [] -Specifies a static maximum amount memory which this domain should -expect to be given. In the absence of in-guest memory hotplug support -this set on domain boot and is usually the maximum amount of RAM which -a guest can make use of. See [docs/misc/libxl_memory.txt][LIBXLMEM] -for a description of how memory is accounted for in toolstacks using -the libxl library. +Specifies a static maximum amount of memory which this domain should +expect to be given. It is usually the maximum amount of RAM which a guest +can make use of. This value is set on the domain boot and can be changed by +xl mem-max. See [docs/misc/libxl_memory.txt][LIBXLMEM] for a description +of how the memory is accounted for in toolstacks using the libxl library. #### ~/memory/target = MEMKB [] The current balloon target for the domain. The balloon driver within -the guest is expected to make every effort to every effort use no more -than this amount of RAM. +the guest is expected to make every effort to use no more than this +amount of RAM. #### ~/memory/videoram = MEMKB [HVM,INTERNAL]
tools/examples/xl.conf +4 -0
@@ -4,6 +4,10 @@ # memory to create a domain #autoballoon=1 +# xl mem-set by default will set a memory allocation target and enforce +# the maximum memory allocation for a given domain +#mem_set_enforce_limit=1 + # full path of the lockfile used by xl during domain creation #lockfile="/var/lock/xl"
Signature · 1 line
1.7.10.4
Re: [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management
Daniel Kiper writes ("[Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 1 line
Improve xl documentation in regards to guest memory management.
Thanks.
docs/man/xl.conf.pod.5 reduce the amount of memory given to domain 0 by default. · line 63
Quoted 6 lines
+=item B<mem_set_enforce_limit=BOOLEAN>
docs/man/xl.conf.pod.5 +5
-0+ +If enabled, then C<xl mem-set> by default will set a memory allocation +target and enforce the maximum memory allocation for a given domain. +If disabled, then C<xl mem-set> by default will set a memory allocation +target only for a given domain.
I think you mean "for each domain" rather than "for a given domain", and "will set only a memory allocation target for each domain" ?
docs/man/xl.conf.pod.5 reduce the amount of memory given to domain 0 by default. · line 69
Quoted 5 lines
docs/man/xl.conf.pod.5 +5
-0+The memory allocation target is a suggested amount of memory which +should be allocated by a guest. It is nice when the guest follows this +value but it may have its own idea about memory requirments. However, +it could not allocate more memory than a ceiling known as the maximum +memory allocation.
I would write "However the guest is prevented by Xen from allocating more memory than a ceiling known as the "xen maximum" memory allocation" or some such.
Quoted 14 lines
docs/misc/libxl_memory.txt +4
-4@@ -15,14 +15,19 @@ | | | | | | target | | | | guest | | | build | | | | | | | | start | | | - static | | | | | | | | - maximum | +----------+ | + + + + - | | | | - | | | | + | | | | | | | | + | +----------+ | + + + + + static | | | | + maximum | | | |
docs/misc/libxl_memory.txt line 30
Quoted 10 lines
docs/misc/libxl_memory.txt +6
-1| | balloon | | build | | | | maximum | | | | - + +----------+ + + | +----------+ + + | | | + | | memory | + | | hotplug | + | | | + + +----------+
This change to the diagram implies that the target, and and the xen maximum, never include any hotplug memory. I don't think that can be true.
Presumably hotplugged memory can be ballooned too. So perhaps we need another column ?
docs/misc/libxl_memory.txt
Quoted 7 lines
docs/misc/libxl_memory.txt +2
-2#### ~/memory/target = MEMKB [] The current balloon target for the domain. The balloon driver within -the guest is expected to make every effort to every effort use no more -than this amount of RAM. +the guest is expected to make every effort to use no more than this +amount of RAM.
Good fix.
It might be worth adding:
"Guests whose actual allocation exceeds the target may experience
memory allocation failures."
Thanks,
Ian.
Re: [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management
Quoted 5 lines · Ian Jackson
Daniel Kiper writes ("[Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 1 line
Improve xl documentation in regards to guest memory management.
Thanks.
Quoted 9 lines
docs/man/xl.conf.pod.5 reduce the amount of memory given to domain 0 by default. · line 63
Quoted 6 lines
+=item B<mem_set_enforce_limit=BOOLEAN>
docs/man/xl.conf.pod.5 +5
-0+ +If enabled, then C<xl mem-set> by default will set a memory allocation +target and enforce the maximum memory allocation for a given domain. +If disabled, then C<xl mem-set> by default will set a memory allocation +target only for a given domain.I think you mean "for each domain" rather than "for a given domain", and "will set only a memory allocation target for each domain" ?
Yes, thanks.
Quoted 30 lines
docs/man/xl.conf.pod.5 reduce the amount of memory given to domain 0 by default. · line 69
Quoted 5 lines
docs/man/xl.conf.pod.5 +5
-0+The memory allocation target is a suggested amount of memory which +should be allocated by a guest. It is nice when the guest follows this +value but it may have its own idea about memory requirments. However, +it could not allocate more memory than a ceiling known as the maximum +memory allocation.I would write "However the guest is prevented by Xen from allocating more memory than a ceiling known as the "xen maximum" memory allocation" or some such.
Quoted 20 lines
docs/misc/libxl_memory.txt +6
-5@@ -15,14 +15,19 @@ | | | | | | target | | | | guest | | | build | | | | | | | | start | | | - static | | | | | | | | - maximum | +----------+ | + + + + - | | | | - | | | | + | | | | | | | | + | +----------+ | + + + + + static | | | | + maximum | | | | | | balloon | | build | | | | maximum | | | | - + +----------+ + + | +----------+ + + | | |
Quoted 8 lines
docs/misc/libxl_memory.txt line 30
Quoted 4 lines
docs/misc/libxl_memory.txt +4
-0+ | | memory | + | | hotplug | + | | | + + +----------+This change to the diagram implies that the target, and and the xen maximum, never include any hotplug memory. I don't think that can be true.
Right, "xen maximum" should include hotplug memory. However, I think that "actual target" should be split to "suggested target" and "actual target". "suggested target" could be up to "static maximum". "actual target" always include only video and guest memory (currently alloceted memory by domain).
Quoted 2 lines
Presumably hotplugged memory can be ballooned too. So perhaps we need another column ?
I think that it is better to add comment that after hotplugging memory could be ballooned down.
Quoted 10 lines
docs/misc/libxl_memory.txt
Quoted 7 lines
docs/misc/libxl_memory.txt +2
-2#### ~/memory/target = MEMKB [] The current balloon target for the domain. The balloon driver within -the guest is expected to make every effort to every effort use no more -than this amount of RAM. +the guest is expected to make every effort to use no more than this +amount of RAM.Good fix.
Quoted 4 lines
It might be worth adding:
"Guests whose actual allocation exceeds the target may experience
memory allocation failures."
... or "Guests which attempts to exceed the target may experience memory allocation failures."
Daniel
Re: [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 6 lines · Ian Jackson
Quoted 2 lines
Presumably hotplugged memory can be ballooned too. So perhaps we need another column ?
I think that it is better to add comment that after hotplugging memory could be ballooned down.
I'd prefer not to make the diagram wrong. Perhaps it would be better to annotate the "static maximum" to say that it might be increased by memory hotplug.
Quoted 7 lines
Quoted 4 lines
It might be worth adding:
"Guests whose actual allocation exceeds the target may experience
memory allocation failures."... or "Guests which attempts to exceed the target may experience memory allocation failures."
No, because a guest which is being asked to shrink will experience memory allocation failures if it doesn't shrink monotonically.
Ian.
Re: [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management
Quoted 11 lines · Ian Jackson
Daniel Kiper writes ("Re: [Xen-devel] [PATCH v3 3/3] xl: Improve xl documentation in regards to guest memory management"):
Quoted 6 lines · Ian Jackson
Quoted 2 lines
Presumably hotplugged memory can be ballooned too. So perhaps we need another column ?
I think that it is better to add comment that after hotplugging memory could be ballooned down.
I'd prefer not to make the diagram wrong. Perhaps it would be better to annotate the "static maximum" to say that it might be increased by memory hotplug.
No, memory hotplug should not change "static maximum" in any case. "static maximum" should define memory limit for a given guest. If "static maximum" is above "build maximum" then memory hotplug in the guest should be involved (if it is available).
Quoted 10 lines
Quoted 7 lines
Quoted 4 lines
It might be worth adding:
"Guests whose actual allocation exceeds the target may experience
memory allocation failures."... or "Guests which attempts to exceed the target may experience memory allocation failures."
No, because a guest which is being asked to shrink will experience memory allocation failures if it doesn't shrink monotonically.
IIRC if you shrink memory for a given guest and it does not have memory balloon driver installed nothing happens. I did not tested it but it would be crazy when host could steal random page from the guest without its acceptance. It would lead to crash almost immediately. That is why I wrote "attempts" because memory allocation failures may happen when you increase reservation only.
Daniel