Propagate sharing and paging counts to libx{c/l}

PATCH New Andres Lagar-Cavilla 3 participants 15 messages Mar 12, 2012 – Mar 13, 2012

lore

Share & tools

lore

Andres Lagar-Cavilla

author

2012-03-12 19:53 UTC #0 lore raw

PATCH 1/1 Propagate sharing and paging counts to libx{c/l}

Currently commented out, no reason to disable it.

The hypervisor <-> libxc interface changes, though.

Signed-off-by Andres Lagar-Cavilla <[email protected]>

 tools/libxc/xc_domain.c     |  2 +-
 xen/common/domctl.c         |  2 +-
 xen/include/public/domctl.h |  2 +-
 tools/libxl/libxl.c         |  3 ++-
 tools/libxl/libxl_types.idl |  1 +
 5 files changed, 6 insertions(+), 4 deletions(-)

Andres Lagar-Cavilla

author

2012-03-12 19:53 UTC #1 lore raw

PATCH 1/1 Propagate domain shared pages via dominfo

 tools/libxc/xc_domain.c     |  2 +-
 xen/common/domctl.c         |  2 +-
 xen/include/public/domctl.h |  2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

Currently commented out, no reason to disable it.

Signed-off-by Andres Lagar-Cavilla <[email protected]>

tools/libxc/xc_domain.c +1 -1

diff -r addb6616e282 -r 94ec84eeca5d tools/libxc/xc_domain.c
@@ -234,7 +234,7 @@ int xc_domain_getinfo(xc_interface *xch,
 
         info->ssidref  = domctl.u.getdomaininfo.ssidref;
         info->nr_pages = domctl.u.getdomaininfo.tot_pages;
-        /* info->nr_shared_pages = domctl.u.getdomaininfo.shr_pages; */
+        info->nr_shared_pages = domctl.u.getdomaininfo.shr_pages;
         info->nr_paged_pages = domctl.u.getdomaininfo.paged_pages;
         info->max_memkb = domctl.u.getdomaininfo.max_pages << (PAGE_SHIFT-10);
         info->shared_info_frame = domctl.u.getdomaininfo.shared_info_frame;

xen/common/domctl.c +1 -1

diff -r addb6616e282 -r 94ec84eeca5d xen/common/domctl.c
@@ -153,7 +153,7 @@ void getdomaininfo(struct domain *d, str
 
     info->tot_pages         = d->tot_pages;
     info->max_pages         = d->max_pages;
-    /* info->shr_pages         = atomic_read(&d->shr_pages); */
+    info->shr_pages         = atomic_read(&d->shr_pages);
     info->paged_pages       = atomic_read(&d->paged_pages);
     info->shared_info_frame = mfn_to_gmfn(d, __pa(d->shared_info)>>PAGE_SHIFT);
     BUG_ON(SHARED_M2P(info->shared_info_frame));

xen/include/public/domctl.h +1 -1

diff -r addb6616e282 -r 94ec84eeca5d xen/include/public/domctl.h
@@ -94,7 +94,7 @@ struct xen_domctl_getdomaininfo {
     uint32_t flags;              /* XEN_DOMINF_* */
     uint64_aligned_t tot_pages;
     uint64_aligned_t max_pages;
-    /* uint64_aligned_t shr_pages; */
+    uint64_aligned_t shr_pages;
     uint64_aligned_t paged_pages;
     uint64_aligned_t shared_info_frame; /* GMFN of shared_info struct */
     uint64_aligned_t cpu_time;

Ian Campbell

2012-03-13 10:41 UTC #2 lore raw

Re: [PATCH 1 of 2] Propagate domain shared pages via dominfo

Quoted 12 lines · Andres Lagar-Cavilla

tools/libxc/xc_domain.c | 2 +-
 xen/common/domctl.c         |  2 +-
 xen/include/public/domctl.h |  2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
Currently commented out, no reason to disable it.
Signed-off-by Andres Lagar-Cavilla <[email protected]>
diff -r addb6616e282 -r 94ec84eeca5d xen/include/public/domctl.h

xen/include/public/domctl.h struct xen_domctl_getdomaininfo { · line 100

Quoted 8 lines

xen/include/public/domctl.h +1 -1

@@ -94,7 +94,7 @@ struct xen_domctl_getdomaininfo {
     uint32_t flags;              /* XEN_DOMINF_* */
     uint64_aligned_t tot_pages;
     uint64_aligned_t max_pages;
-    /* uint64_aligned_t shr_pages; */
+    uint64_aligned_t shr_pages;
     uint64_aligned_t paged_pages;
     uint64_aligned_t shared_info_frame; /* GMFN of shared_info struct */
     uint64_aligned_t cpu_time;

This doesn't appear to be commented out here, xen-unstable 25007:5d20d2f6ffed:

$ grep -A 4 -B 4 shr_pages xen/include/public/*
xen/include/public/domctl.h-#define XEN_DOMINF_shutdownshift 16
xen/include/public/domctl.h-    uint32_t flags;              /* XEN_DOMINF_* */
xen/include/public/domctl.h-    uint64_aligned_t tot_pages;
xen/include/public/domctl.h-    uint64_aligned_t max_pages;
xen/include/public/domctl.h:    uint64_aligned_t shr_pages;
xen/include/public/domctl.h-    uint64_aligned_t paged_pages;
xen/include/public/domctl.h-    uint64_aligned_t shared_info_frame; /* GMFN of shared_info struct */
xen/include/public/domctl.h-    uint64_aligned_t cpu_time;
xen/include/public/domctl.h-    uint32_t nr_online_vcpus;    /* Number of VCPUs currently online. */

I didn't look at the other hunks after I noticed this. I can't see anywhere in history where it was commented out -- perhaps a local patch on your end?

Ian.

Andres Lagar-Cavilla

author

2012-03-12 19:53 UTC #3 lore raw

PATCH 1/1 libxl: publish number of shared and paged pages

In reply to Andres Lagar-Cavilla

 tools/libxl/libxl.c         |  3 ++-
 tools/libxl/libxl_types.idl |  1 +
 2 files changed, 3 insertions(+), 1 deletions(-)

This is information is currently not harvested from libxc.

Signed-off-by Andres Lagar-Cavilla <[email protected]>

tools/libxl/libxl.c +2 -1

diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
@@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
         xlinfo->shutdown_reason  = ~0;
 
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
-    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
+    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;

tools/libxl/libxl_types.idl +1 -0

diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl_types.idl
@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Ian Campbell

2012-03-13 10:42 UTC #4 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Quoted 13 lines · Andres Lagar-Cavilla

tools/libxl/libxl.c | 3 ++-
 tools/libxl/libxl_types.idl |  1 +
 2 files changed, 3 insertions(+), 1 deletions(-)
This is information is currently not harvested from libxc.
Signed-off-by Andres Lagar-Cavilla <[email protected]>
diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c

tools/libxl/libxl.c static void xcinfo2xlinfo(const xc_domai · line 462

Quoted 5 lines

tools/libxl/libxl.c +1 -1

@@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
         xlinfo->shutdown_reason  = ~0;
 
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
-    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
+    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);

Same comment about not being commented out in xen-unstable. The other bits of this patch appear to make sense though.

Quoted 15 lines

tools/libxl/libxl.c +1 -0

+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;

diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl_types.idl

tools/libxl/libxl_types.idl +1 -0

@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Andres Lagar-Cavilla

author

2012-03-13 14:14 UTC #5 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Quoted 16 lines · Andres Lagar-Cavilla

Quoted 15 lines
tools/libxl/libxl.c | 3 ++-
 tools/libxl/libxl_types.idl |  1 +
 2 files changed, 3 insertions(+), 1 deletions(-)
This is information is currently not harvested from libxc.
Signed-off-by Andres Lagar-Cavilla <[email protected]>
diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
tools/libxl/libxl.c +0 -0
@@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
         xlinfo->shutdown_reason  = ~0;
 

Quoted 6 lines

tools/libxl/libxl.c static void xcinfo2xlinfo(const xc_domai · line 462
Quoted 3 lines
tools/libxl/libxl.c +1 -1
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
-    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
+    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
Same comment about not being commented out in xen-unstable. The other bits of this patch appear to make sense though.

I think the most accurate description for these two patches is "brain fart"

Here is the libxl patch, rebased.
Thanks
Andres

# HG changeset patch
# User Andres Lagar-Cavilla <[email protected]> # Date 1331648207 14400
# Node ID 94b89a5f14e512c2ad46ef9ad1628fba19858463 # Parent 5d20d2f6ffed0a49f030f04a8870f1926babbcbf
libxl: publish number of paged pages.

This is information is currently not harvested from libxc.

Signed-off-by Andres Lagar-Cavilla <[email protected]>

tools/libxl/libxl.c +1 -0

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl.c
@@ -460,6 +460,7 @@ static void xcinfo2xlinfo(const xc_domai
 
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
     xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;

tools/libxl/libxl_types.idl +1 -0

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl_types.idl
@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Ian Campbell

Acked-by

2012-03-13 14:32 UTC #6 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Quoted 6 lines

Quoted 6 lines
Quoted 6 lines
diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
tools/libxl/libxl.c +0 -0
@@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
         xlinfo->shutdown_reason  = ~0;
 

Quoted 8 lines

Quoted 6 lines
tools/libxl/libxl.c static void xcinfo2xlinfo(const xc_domai · line 462
Quoted 3 lines
tools/libxl/libxl.c +1 -1
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
-    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
+    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
Same comment about not being commented out in xen-unstable. The other bits of this patch appear to make sense though.
I think the most accurate description for these two patches is "brain fart"

;-)

Quoted 5 lines

libxl: publish number of paged pages.
This is information is currently not harvested from libxc.
Signed-off-by Andres Lagar-Cavilla <[email protected]>

Acked-by Ian Campbell <[email protected]>

But did you not want to print it from xl command or other?

Quoted 25 lines

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl.c

tools/libxl/libxl.c +1 -0

@@ -460,6 +460,7 @@ static void xcinfo2xlinfo(const xc_domai
 
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
     xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl_types.idl

tools/libxl/libxl_types.idl +1 -0

@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Andres Lagar-Cavilla

author

2012-03-13 14:40 UTC #7 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Quoted 20 lines

Quoted 13 lines
Quoted 6 lines
Quoted 6 lines
diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
tools/libxl/libxl.c +0 -0
@@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
         xlinfo->shutdown_reason  = ~0;
 
Quoted 6 lines
tools/libxl/libxl.c static void xcinfo2xlinfo(const xc_domai · line 462
Quoted 3 lines
tools/libxl/libxl.c +1 -1
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
-    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
+    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
Same comment about not being commented out in xen-unstable. The other bits of this patch appear to make sense though.
Quoted 2 lines
I think the most accurate description for these two patches is "brain fart"
;-)
Quoted 2 lines
libxl: publish number of paged pages.

Quoted 7 lines

Quoted 3 lines
This is information is currently not harvested from libxc.
Signed-off-by Andres Lagar-Cavilla <[email protected]>
Acked-by Ian Campbell <[email protected]>
But did you not want to print it from xl command or other?

Not in xl list (format compat with xm list). Where? Andres

Quoted 29 lines

Quoted 26 lines

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl.c

tools/libxl/libxl.c +1 -0

@@ -460,6 +460,7 @@ static void xcinfo2xlinfo(const xc_domai
 
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
     xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl_types.idl

tools/libxl/libxl_types.idl +1 -0

@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Ian Jackson

2012-03-13 15:35 UTC #8 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):

Quoted 3 lines

Quoted 1 line
But did you not want to print it from xl command or other?
Not in xl list (format compat with xm list). Where?

Tricky. Can you make the format compatible if paging is disabled ?

Ultimately the format of list is pretty annoying and inextensible.

Ian.

Ian Campbell

2012-03-13 15:40 UTC #9 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Quoted 8 lines · Ian Jackson

Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
Quoted 3 lines
Quoted 1 line
But did you not want to print it from xl command or other?
Not in xl list (format compat with xm list). Where?
Tricky. Can you make the format compatible if paging is disabled ?
Ultimately the format of list is pretty annoying and inextensible.

I was wondering if we should just bit the bullet and agree that only the machine readable "xl list -l" output is backwards compatible? Supporting machine parsing of the human readable "xl list" output is pretty limiting in many ways.

The default for "xl list -l" is now json output (which is autogenerated and will therefore pickup new field automatically) but there is an option to use the xm sxp syntax too.

If changing the machine readable output is actually impossible I suppose we could add a "--old-style" flag to ease the transition?

Ian.

Ian Jackson

2012-03-13 15:43 UTC #10 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Ian Campbell writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):

Quoted 4 lines

I was wondering if we should just bit the bullet and agree that only the machine readable "xl list -l" output is backwards compatible? Supporting machine parsing of the human readable "xl list" output is pretty limiting in many ways.

Mmmm.

Quoted 6 lines

The default for "xl list -l" is now json output (which is autogenerated and will therefore pickup new field automatically) but there is an option to use the xm sxp syntax too.
If changing the machine readable output is actually impossible I suppose we could add a "--old-style" flag to ease the transition?

Can we make it an environment variable ?

Ian.

Ian Campbell

2012-03-13 15:52 UTC #11 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Quoted 8 lines · Ian Jackson

Ian Campbell writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
Quoted 4 lines
I was wondering if we should just bit the bullet and agree that only the machine readable "xl list -l" output is backwards compatible? Supporting machine parsing of the human readable "xl list" output is pretty limiting in many ways.
Mmmm.

Quoted 8 lines

Quoted 6 lines
The default for "xl list -l" is now json output (which is autogenerated and will therefore pickup new field automatically) but there is an option to use the xm sxp syntax too.
If changing the machine readable output is actually impossible I suppose we could add a "--old-style" flag to ease the transition?
Can we make it an environment variable ?

I don't see why not. Say, $XL_USE_XM_COMPATIBLE_OUTPUT=1 ?

I suppose your rationale is that this makes it easier to fix a script which might use either xm or xl?

Ian.

Ian Jackson

2012-03-13 16:08 UTC #12 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

Ian Campbell writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):

Quoted 2 lines

I suppose your rationale is that this makes it easier to fix a script which might use either xm or xl?

Yes. It means whole programs can be fixed more easily.

Ian.

Andres Lagar-Cavilla

author

2012-03-13 15:43 UTC #13 lore raw

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

In reply to Ian Campbell

Quoted 16 lines · Ian Jackson

Quoted 9 lines
Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl:
publish number of shared and paged pages"):
Quoted 3 lines
Quoted 1 line
But did you not want to print it from xl command or other?
Not in xl list (format compat with xm list). Where?
Tricky. Can you make the format compatible if paging is disabled ?
Ultimately the format of list is pretty annoying and inextensible.
I was wondering if we should just bit the bullet and agree that only the machine readable "xl list -l" output is backwards compatible? Supporting machine parsing of the human readable "xl list" output is pretty limiting in many ways.

Quoted 6 lines

The default for "xl list -l" is now json output (which is autogenerated and will therefore pickup new field automatically) but there is an option to use the xm sxp syntax too.
If changing the machine readable output is actually impossible I suppose we could add a "--old-style" flag to ease the transition?

Happy to add the paging and sharing stats to the extended list format, once a choice is made.
Andres

Quoted 4 lines