Skip to contentSkip to conversation

xl: Update memory info in xenstore when use 'xl mem-set'

PATCH In review Yu Zhiguo 4 participants 10 messages May 7, 2010 – May 14, 2010

lore
Share & tools
lore

Yu Zhiguo

author
#0 lore raw

PATCH 1/1 xl: Update memory info in xenstore when use 'xl mem-set'

'xl mem-set' should update memory info in xenstore's vm info.

tools/libxl/libxl.c +11 -0
diff -r ccae861f52f7 -r 5847ce53f5dc tools/libxl/libxl.c
@@ -2290,11 +2290,22 @@
     uint32_t videoram;
     char *videoram_s = NULL;
     char *dompath = libxl_xs_get_dompath(ctx, domid);
+    xc_domaininfo_t info;
+    struct libxl_dominfo ptr;
+    char *uuid;
 
     videoram_s = libxl_xs_read(ctx, XBT_NULL, libxl_sprintf(ctx, "%s/memory/videoram", dompath));
     videoram = videoram_s ? atoi(videoram_s) : 0;
 
     libxl_xs_write(ctx, XBT_NULL, libxl_sprintf(ctx, "%s/memory/target", dompath), "%lu", target_memkb);
+
+    rc = xc_domain_getinfolist(ctx->xch, domid, 1, &info);
+    if (rc != 1 || info.domain != domid)
+        return rc;
+    xcinfo2xlinfo(&info, &ptr);
+    uuid = libxl_uuid2string(ctx, ptr.uuid);
+    libxl_xs_write(ctx, XBT_NULL, libxl_sprintf(ctx, "/vm/%s/memory", uuid), "%lu", target_memkb / 1024);
+
     rc = xc_domain_setmaxmem(ctx->xch, domid, target_memkb + LIBXL_MAXMEM_CONSTANT);
     if (rc != 0)
         return rc;
Acked-by on v1 1/1
#1 lore raw

Yu Zhiguo writes ("[Xen-devel] [PATCH] xl: Update memory info in xenstore when use 'xl mem-set'"):

Quoted 1 line

'xl mem-set' should update memory info in xenstore's vm info.

Your patch looks sane (although I haven't tested it - I guess you have). So:

However, looking at this, it's far from clear why libxl_set_memory_target calls xc_domain_setmaxmem at all. Can anyone explain ?

Ian.

author
#2 lore raw

Hi Ian,

Quoted 4 lines · Ian Jackson

However, looking at this, it's far from clear why libxl_set_memory_target calls xc_domain_setmaxmem at all. Can anyone explain ?

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max'
Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory,
it is missing now.

2. fix 'mem-set'
Delete xc_domain_setmaxmem.
Get max memory from /local/domain/$domid/memory/static-max, and then do value check.
It seems that we cannot get max memory use libxc routines.

Do you agree with me?

Add, I found another problem about mem-set.

I used 'xl mem-set' set the memory, but the size displayed in output of 'xm list' did not change.
It seems that 'xm' get the memory size from xend, xend maintains some info of VM. I'm not sure how to resolve this problem? Get info from xenstore? What's your opinion?

# xl mem-set 3 204800
setting domid 3 memory to : 204800
# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
...
pvguest1                                     3   200     2        r--      0.2
# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
...
pvguest1                                     3   256     2     -b----      0.2

Regards
Yu Zhiguo

author
#3 lore raw

cover xl: Add subcommand mem-max and fix mem-set

Quoted 12 lines · Yu Zhiguo

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max'
Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory,
it is missing now.

Quoted 5 lines

2. fix 'mem-set'
Delete xc_domain_setmaxmem.
Get max memory from /local/domain/$domid/memory/static-max, and then do value check.
It seems that we cannot get max memory use libxc routines.

I fixed it, please check.

[PATCH 1/2] xl: Add command 'xl mem-max' [PATCH 2/2] xl: Add check for command 'xl mem-set'

Regards
Yu Zhiguo

author
#4 lore raw

Re: [PATCH 0/2] xl: Add subcommand mem-max and fix mem-set

Quoted 18 lines · Yu Zhiguo
Quoted 11 lines · Yu Zhiguo

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max'
Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory,
it is missing now.

Quoted 5 lines

2. fix 'mem-set'
Delete xc_domain_setmaxmem.
Get max memory from /local/domain/$domid/memory/static-max, and then do value check.
It seems that we cannot get max memory use libxc routines.

Quoted 5 lines

I fixed it, please check.

[PATCH 1/2] xl: Add command 'xl mem-max' [PATCH 2/2] xl: Add check for command 'xl mem-set'

It seems than /local/domain/$domid/memory/target cannot be created when use 'xl create', only /local/domain/0/memory/static-max is existent.
Isn't it a bug? I'll check the code.

I think it should be created like static-max.
* memory/target can be created when use 'xm create'

Quoted 12 lines

Regards
Yu Zhiguo

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Signature · 14 lines
Regards
Yu Zhiguo
--------------------------------------------------
Yu Zhiguo
Development Dept.I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
8/F., Civil Defense Building, No.189 Guangzhou Road,
Nanjing, 210029, China
TEL: +86+25-86630566-853
COINS: 79955-853
FAX: +86+25-83317685
MAIL: [email protected]
--------------------------------------------------
This communication is for use by the intended recipient(s) only and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not an intended recipient of this communication, you are hereby notified that any dissemination, distribution or copying hereof is strictly prohibited.  If you have received this communication in error, please notify me by reply e-mail, permanently delete this communication from your system, and destroy any hard copies you may have printed.
#5 lore raw

Re: [PATCH 0/2] xl: Add subcommand mem-max and fix mem-set

Quoted 25 lines · Yu Zhiguo
Quoted 18 lines · Yu Zhiguo
Quoted 11 lines · Yu Zhiguo

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max'
Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory,
it is missing now.

Quoted 5 lines

2. fix 'mem-set'
Delete xc_domain_setmaxmem.
Get max memory from /local/domain/$domid/memory/static-max, and then do value check.
It seems that we cannot get max memory use libxc routines.

Quoted 5 lines

I fixed it, please check.

[PATCH 1/2] xl: Add command 'xl mem-max' [PATCH 2/2] xl: Add check for command 'xl mem-set'

Quoted 7 lines

It seems than /local/domain/$domid/memory/target cannot be created when use 'xl create', only /local/domain/0/memory/static-max is existent.
Isn't it a bug? I'll check the code.

I think it should be created like static-max.
* memory/target can be created when use 'xm create'

Actually sorry if I didn't reply in time before you sent your patches, but I asked the xapi memory management expert (CC'ed) an opinion on the memset/memmax commands, I have the impression that he has something interesting to say about them.

Keir, could you please wait before applying these patches?

#6 lore raw

Re: [PATCH 0/2] xl: Add subcommand mem-max and fix mem-set

Quoted 34 lines · Stefano Stabellini
Quoted 25 lines · Yu Zhiguo
Quoted 18 lines · Yu Zhiguo
Quoted 11 lines · Yu Zhiguo

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max'
Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory,
it is missing now.

Quoted 5 lines

2. fix 'mem-set'
Delete xc_domain_setmaxmem.
Get max memory from /local/domain/$domid/memory/static-max, and then do value check.
It seems that we cannot get max memory use libxc routines.

Quoted 5 lines

I fixed it, please check.

[PATCH 1/2] xl: Add command 'xl mem-max' [PATCH 2/2] xl: Add check for command 'xl mem-set'

Quoted 7 lines

It seems than /local/domain/$domid/memory/target cannot be created when use 'xl create', only /local/domain/0/memory/static-max is existent.
Isn't it a bug? I'll check the code.

I think it should be created like static-max.
* memory/target can be created when use 'xm create'

Quoted 7 lines

Actually sorry if I didn't reply in time before you sent your patches, but I asked the xapi memory management expert (CC'ed) an opinion on the memset/memmax commands, I have the impression that he has something interesting to say about them.

Keir, could you please wait before applying these patches?

Oops I see now is too late :)
Jonathan, what do you think about them anyway?

author
#7 lore raw

PATCH 1/1 xl: Fix missing memory target in xenstore

In reply to Yu Zhiguo
Quoted 7 lines · Yu Zhiguo

It seems than /local/domain/$domid/memory/target cannot be created when use 'xl create', only /local/domain/0/memory/static-max is existent.
Isn't it a bug? I'll check the code.

I think it should be created like static-max.
* memory/target can be created when use 'xm create'

/local/domain/$domid/memory/target should be
created in xenstore when domain is created.

tools/libxl/libxl_dom.c +11 -11
diff -r d77a88f938c6 -r 3b473b9417e9 tools/libxl/libxl_dom.c
@@ -100,22 +100,22 @@
     xc_cpuid_apply_policy(ctx->xch, domid);
 #endif
 
-    ents = libxl_calloc(ctx, (10 + info->max_vcpus) * 2, sizeof(char *));
+    ents = libxl_calloc(ctx, (12 + info->max_vcpus) * 2, sizeof(char *));
     ents[0] = "memory/static-max";
     ents[1] = libxl_sprintf(ctx, "%d", info->max_memkb);
     ents[2] = "memory/target";
     ents[3] = libxl_sprintf(ctx, "%d", info->target_memkb);
-    ents[2] = "memory/videoram";
-    ents[3] = libxl_sprintf(ctx, "%d", info->video_memkb);
-    ents[4] = "domid";
-    ents[5] = libxl_sprintf(ctx, "%d", domid);
-    ents[6] = "store/port";
-    ents[7] = libxl_sprintf(ctx, "%"PRIu32, state->store_port);
-    ents[8] = "store/ring-ref";
-    ents[9] = libxl_sprintf(ctx, "%lu", state->store_mfn);
+    ents[4] = "memory/videoram";
+    ents[5] = libxl_sprintf(ctx, "%d", info->video_memkb);
+    ents[6] = "domid";
+    ents[7] = libxl_sprintf(ctx, "%d", domid);
+    ents[8] = "store/port";
+    ents[9] = libxl_sprintf(ctx, "%"PRIu32, state->store_port);
+    ents[10] = "store/ring-ref";
+    ents[11] = libxl_sprintf(ctx, "%lu", state->store_mfn);
     for (i = 0; i < info->max_vcpus; i++) {
-        ents[10+(i*2)]   = libxl_sprintf(ctx, "cpu/%d/availability", i);
-        ents[10+(i*2)+1] = (i && info->cur_vcpus && (i >= info->cur_vcpus))
+        ents[12+(i*2)]   = libxl_sprintf(ctx, "cpu/%d/availability", i);
+        ents[12+(i*2)+1] = (i && info->cur_vcpus && (i >= info->cur_vcpus))
                             ? "offline" : "online";
     }
#8 lore raw

Re: [PATCH 0/2] xl: Add subcommand mem-max and fix mem-set

In reply to Yu Zhiguo

Hi all

As one of the maintainers of the Xen API tool-stack [1], I have a couple of comments about these patches.

Currently, the Xen API tool-stack does not use libxenlight, but we're interested in moving to it in the near future.

Quoted 10 lines · Yu Zhiguo

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max' Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory, it is missing now.

From the point of view of the Xen API tool-stack, these values have different purposes.

1. The XenStore value: /local/domain/$domid/memory/static-max

This value reflects the maximum amount of physical host memory that's addressable by the guest OS.

The Xen API tool-stack writes this value into XenStore at the time of domain construction, and leaves the value alone until the domain is destroyed.

I think it would make sense for libxenlight to do the same.

Many balloon drivers read this value, along with the "target" field, in order to determine how large they should make their balloons:

balloon size = (static-max - target)

In the absence of hotplug support, balloon drivers expect the value of "static-max" to remain constant for the lifetime of the domain.

2. The Xen value: maxmem

Xen uses this value as a sort of "ratchet", to prevent a domain that's currently ballooned down from growing too large.

Xen will reject any call to domain_memory_increase_reservation that would increase the reservation beyond the value of maxmex.

It's possible to change the value of "maxmex" while a domain is running. Indeed, whenever the Xen API tool-stack writes a new balloon target into XenStore, it also calls xc_domain_setmaxmem with the same value.

This mechanism allows the tool-stack to prevent a balloon driver from claiming back more memory if the guest is already using more than its target memory allocation.

So, I think it's fair to say that locking these two values together does not make sense from the point of view of the Xen API tool-stack.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

2. fix 'mem-set' Delete xc_domain_setmaxmem. Get max memory from /local/domain/$domid/memory/static-max, and then do value check. It seems that we cannot get max memory use libxc routines.

If the purpose of "mem-set" is to set a balloon target for a guest, then it might also make sense to call xc_domain_setmaxmem with the same value, whenever the target changes, for the reasons stated above (preventing a domain that's currently ballooned down from growing too large).

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I fixed it, please check.

[PATCH 1/2] xl: Add command 'xl mem-max' [PATCH 2/2]
xl: Add check for command 'xl mem-set'

Cheers,
Jonathan

[1] http://www.xen.org/products/cloud_source.html

Acked-by Jonathan Knowles
#9 lore raw

Re: [PATCH 0/2] xl: Add subcommand mem-max and fix mem-set

Quoted 49 lines · Jonathan Knowles
Quoted 10 lines · Yu Zhiguo

I'm trying to add subcommand 'mem-max', I think xc_domain_setmaxmem should be used in it but not here.

I'll move this code to 'mem-max', and in 'mem-set', a check should be added because setting memory larger than max memory is invalid.

1. Add 'mem-max' Add libxl_domain_setmaxmem, it calls xc_domain_setmaxmem. /local/domain/$domid/memory/static-max should be updated when set max memory, it is missing now.

From the point of view of the Xen API tool-stack, these values
have different purposes.

1. The XenStore value: /local/domain/$domid/memory/static-max

This value reflects the maximum amount of physical host memory that's addressable by the guest OS.

The Xen API tool-stack writes this value into XenStore at the time of domain construction, and leaves the value alone until the domain is destroyed.

I think it would make sense for libxenlight to do the same.

Many balloon drivers read this value, along with the "target" field, in order to determine how large they should make their balloons:

balloon size = (static-max - target)

In the absence of hotplug support, balloon drivers expect the value of "static-max" to remain constant for the lifetime of the domain.

2. The Xen value: maxmem

Xen uses this value as a sort of "ratchet", to prevent a domain that's currently ballooned down from growing too large.

Xen will reject any call to domain_memory_increase_reservation that would increase the reservation beyond the value of maxmex.

It's possible to change the value of "maxmex" while a domain is running. Indeed, whenever the Xen API tool-stack writes a new balloon target into XenStore, it also calls xc_domain_setmaxmem with the same value.

Quoted 7 lines

This mechanism allows the tool-stack to prevent a balloon driver from claiming back more memory if the guest is already using more than its target memory allocation.

So, I think it's fair to say that locking these two values together does not make sense from the point of view of the Xen API tool-stack.

I think xl memset should change the memory currently used by the guest and xl memmax should change the size of the guest's address space and not the population.
For this reason libxl_set_memory_target should provide a way to enforce the memory target, calling xc_domain_setmaxmem.
On the other hand xl memmax shouldn't call xc_domain_setmaxmem because that is the upper bound of the memory reservation, it should just change static-max, that at the moment wouldn't do much, but we can imagine that in the future could trigger something useful in the guest.

---

tools/libxl/libxl.c +8 -8
diff -r 9d53864d7be6 tools/libxl/libxl.c
@@ -2468,7 +2468,6 @@
     char *mem, *endptr;
     uint32_t memorykb;
     char *dompath = libxl_xs_get_dompath(ctx, domid);
-    int rc;
 
     mem = libxl_xs_read(ctx, XBT_NULL, libxl_sprintf(ctx, "%s/memory/target", dompath));
     if (!mem) {
@@ -2486,20 +2485,16 @@
         return 1;
     }
 
-    rc = xc_domain_setmaxmem(ctx->xch, domid, max_memkb);
-    if (rc != 0)
-        return rc;
-
     if (domid != 0)
         libxl_xs_write(ctx, XBT_NULL, libxl_sprintf(ctx, "%s/memory/static-max", dompath), "%lu", max_memkb);
 
-    return rc;
+    return 0;
 }
 
-int libxl_set_memory_target(struct libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb)
+int libxl_set_memory_target(struct libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb, int enforce)
 {
     int rc = 0;
-    uint32_t memorykb, videoram;
+    uint32_t memorykb = 0, videoram = 0;
     char *memmax, *endptr, *videoram_s = NULL;
     char *dompath = libxl_xs_get_dompath(ctx, domid);
     xc_domaininfo_t info;
@@ -2539,6 +2533,11 @@
     uuid = libxl_uuid2string(ctx, ptr.uuid);
     libxl_xs_write(ctx, XBT_NULL, libxl_sprintf(ctx, "/vm/%s/memory", uuid), "%lu", target_memkb / 1024);
 
+    if (enforce || !domid)
+        memorykb = target_memkb;
+    rc = xc_domain_setmaxmem(ctx->xch, domid, memorykb + LIBXL_MAXMEM_CONSTANT);
+    if (rc != 0)
+        return rc;
     rc = xc_domain_memory_set_pod_target(ctx->xch, domid, (target_memkb - videoram) / 4, NULL, NULL, NULL);
     return rc;
 }
tools/libxl/libxl.h +1 -1
diff -r 9d53864d7be6 tools/libxl/libxl.h
@@ -340,7 +340,7 @@
 int libxl_domain_unpause(struct libxl_ctx *ctx, uint32_t domid);
 
 int libxl_domain_setmaxmem(struct libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb);
-int libxl_set_memory_target(struct libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb);
+int libxl_set_memory_target(struct libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb, int enforce);
 
 int libxl_console_attach(struct libxl_ctx *ctx, uint32_t domid, int cons_num);
 
tools/libxl/xl_cmdimpl.c +1 -1
diff -r 9d53864d7be6 tools/libxl/xl_cmdimpl.c
@@ -1262,7 +1262,7 @@
         exit(3);
     }
     printf("setting domid %d memory to : %d\n", domid, memorykb);
-    libxl_set_memory_target(&ctx, domid, memorykb);
+    libxl_set_memory_target(&ctx, domid, memorykb, /* enforce */ 1);
 }
 
 int main_memset(int argc, char **argv)