libxl: Extend CPU affinity specification and enable it in config file.
PATCH v2 1/1 libxl: Extend CPU affinity specification and enable it in config file.
Hello Everyone,
This series slightly extends the current support for specifying CPU affinity, basically adding the support for "^<cpuid>" kind of entries (i.e., "^6", meaning "not on CPU#6), and enables doing so in a VM's config file, like it (probably?) was possible with `xm'.
With respect to v1:
* Reworked (hopefully improved) the parsing of the cpu-pin string
* Removed forward declarations (as asked during review)
* Added some helper functions (as asked during review)
* Put a saner default for cpu-pinning for libxl (as asked during review)
Thanks and Regards,
Dario
--
generalize-vcpupin-parsig.patch
support-cpus-par-in-config-file.patch
Signature · 14 lines
tools/libxl/libxl.c | 14 +++++++++ tools/libxl/libxl.h | 2 + tools/libxl/libxl_create.c | 3 + tools/libxl/libxl_dom.c | 1 + tools/libxl/libxl_types.idl | 1 + tools/libxl/libxl_utils.h | 14 +++++++++ tools/libxl/xl_cmdimpl.c | 235 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------- 7 files changed, 190 insertions(+), 80 deletions(-) -- <<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
PATCH v2 1/1 libxl: extend pCPUs specification for vcpu-pin.
Allow for "^<cpuid>" syntax while specifying the pCPUs list during a vcpu-pin. This enables doing the following:
xl vcpu-pin 1 1 0-4,^2
and achieving:
xl vcpu-list
Name ID VCPU CPU State Time(s) CPU Affinity
...
Squeeze_pv 1 1 3 -b- 2.4 0-1,3-4
...
tools/libxl/libxl_utils.h +2 -0
@@ -71,6 +71,8 @@ int libxl_cpumap_test(libxl_cpumap *cpumvoid libxl_cpumap_set (libxl_cpumap *cpumap,int cpu);void libxl_cpumap_reset (libxl_cpumap *cpumap,int cpu);#define libxl_for_each_cpu (var, map) for (var = 0; var < (map).size * 8; var++) +#define libxl_for_each_set_cpu (v, m) for (v = 0; v < (m).size * 8; v++) \ + if (libxl_cpumap_test(&(m), v))int libxl_cpuarray_alloc (libxl_ctx *ctx,libxl_cpuarray *cpuarray);
tools/libxl/xl_cmdimpl.c +69 -28
114 lines not on this page · raw message
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
# HG changeset patch
# Parent 370924e204dc29e12cd807dd730974d6b2bc53d3
libxl: extend pCPUs specification for vcpu-pin.
Allow for "^<cpuid>" syntax while specifying the pCPUs list during a vcpu-pin. This enables doing the following:
xl vcpu-pin 1 1 0-4,^2
and achieving:
xl vcpu-list
Name ID VCPU CPU State Time(s) CPU Affinity
...
Squeeze_pv 1 1 3 -b- 2.4 0-1,3-4
...
tools/libxl/libxl_utils.h +2 -0
@@ -71,6 +71,8 @@ int libxl_cpumap_test(libxl_cpumap *cpumvoid libxl_cpumap_set (libxl_cpumap *cpumap,int cpu);void libxl_cpumap_reset (libxl_cpumap *cpumap,int cpu);#define libxl_for_each_cpu (var, map) for (var = 0; var < (map).size * 8; var++) +#define libxl_for_each_set_cpu (v, m) for (v = 0; v < (m).size * 8; v++) \ + if (libxl_cpumap_test(&(m), v))int libxl_cpuarray_alloc (libxl_ctx *ctx,libxl_cpuarray *cpuarray);
tools/libxl/xl_cmdimpl.c +69 -28
114 lines not on this page · raw message
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Re: [PATCHv2 1 of 2] libxl: extend pCPUs specification for vcpu-pin.
Quoted 7 lines · Dario Faggioli
Allow for "^<cpuid>" syntax while specifying the pCPUs list during a vcpu-pin. This enables doing the following:
xl vcpu-pin 1 1 0-4,^2
and achieving:
Quoted 7 lines
xl vcpu-list
Name ID VCPU CPU State Time(s) CPU Affinity
...
Squeeze_pv1 1 3 -b- 2.4 0-1,3-4
...Signed-off-by Dario Faggioli <[email protected]>
Is 0-4,^2-3 (== 0-1,4) a useful thing to want to write? I don't think it should block this patch going in though.
Ian.
Re: [PATCHv2 1 of 2] libxl: extend pCPUs specification for vcpu-pin.
Quoted 7 lines · Ian Campbell
Quoted 1 line
Signed-off-by Dario Faggioli <[email protected]>Acked-by Ian Campbell <[email protected]>Is 0-4,^2-3 (== 0-1,4) a useful thing to want to write? I don't think it should block this patch going in though.
As I have to respin, I don't think that wouldn't be so hard to add. I'll give it a try and see if I cat put it there like super-quick and repost both patches...
Thanks,
Dario
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Re: [PATCHv2 1 of 2] libxl: extend pCPUs specification for vcpu-pin.
Quoted 1 line · Dario Faggioli
As I have to respin, I don't think that wouldn't be so hard to add.
^I don't think that _would_ be so hard to add.
:-)
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
PATCH v2 1/1 libxl: allow for specifying the CPU affinity in the config file.
In reply to Dario FaggioliEnable CPU affinity specification in a VM's config file with the exact syntax `xl vcpu-pin' provides.
tools/libxl/libxl.c +14 -0
@@ -2663,6 +2663,20 @@ int libxl_set_vcpuaffinity(libxl_ctx *ctreturn 0 ; } +int libxl_set_vcpuaffinity_all (libxl_ctx *ctx,uint32_t domid, +unsigned int max_vcpus,libxl_cpumap *cpumap) +{ +int i, rc =0 ; + +for (i =0 ; i < max_vcpus; i++) { +if (libxl_set_vcpuaffinity (ctx, domid, i, cpumap)) { +LIBXL__LOG (ctx,LIBXL__LOG_WARNING ,"no affinity for cpu %d" , i); + rc =ERROR_FAIL ; + } + } +return rc; +} +int libxl_set_vcpuonline (libxl_ctx *ctx,uint32_t domid,libxl_cpumap *cpumap) {GC_INIT (ctx);
tools/libxl/libxl.h +2 -0
@@ -560,6 +560,8 @@ libxl_vcpuinfo *libxl_list_vcpu(libxl_ct int *nb_vcpu, int *nrcpus);int libxl_set_vcpuaffinity (libxl_ctx *ctx,uint32_t domid,uint32_t vcpuid,libxl_cpumap *cpumap); +int libxl_set_vcpuaffinity_all (libxl_ctx *ctx,uint32_t domid, +unsigned int max_vcpus,libxl_cpumap *cpumap);int libxl_set_vcpuonline (libxl_ctx *ctx,uint32_t domid,libxl_cpumap *cpumap);int libxl_get_sched_id (libxl_ctx *ctx);
tools/libxl/libxl_create.c +3 -0
@@ -71,6 +71,9 @@ int libxl_init_build_info(libxl_ctx *ctxmemset (b_info,'\0' ,sizeof (*b_info)); b_info->max_vcpus =1 ; b_info->cur_vcpus =1 ; +if (libxl_cpumap_alloc (ctx, &b_info->cpumap)) +return ERROR_NOMEM ; +libxl_cpumap_set_any (&b_info->cpumap); b_info->max_memkb =32 *1024 ; b_info->target_memkb = b_info->max_memkb; b_info->disable_migrate =0 ;
tools/libxl/libxl_dom.c +1 -0
@@ -65,6 +65,7 @@ int libxl__build_pre(libxl__gc *gc, uintlibxl_ctx *ctx =libxl__gc_owner (gc);int tsc_mode;xc_domain_max_vcpus (ctx->xch, domid, info->max_vcpus); +libxl_set_vcpuaffinity_all (ctx, domid, info->max_vcpus, &info->cpumap);xc_domain_setmaxmem (ctx->xch, domid, info->target_memkb +LIBXL_MAXMEM_CONSTANT );if (info->type ==LIBXL_DOMAIN_TYPE_PV ) xc_domain_set_memmap_limit(ctx->xch, domid,
tools/libxl/libxl_types.idl +1 -0
@@ -162,6 +162,7 @@ libxl_domain_create_info = Struct("domai libxl_domain_build_info = Struct("domain_build_info",[ ("max_vcpus", integer), ("cur_vcpus", integer), + ("cpumap", libxl_cpumap), ("tsc_mode", libxl_tsc_mode), ("max_memkb", uint32), ("target_memkb", uint32),
tools/libxl/libxl_utils.h +12 -0
@@ -70,6 +70,18 @@ int libxl_cpumap_alloc(libxl_ctx *ctx, lint libxl_cpumap_test (libxl_cpumap *cpumap,int cpu);void libxl_cpumap_set (libxl_cpumap *cpumap,int cpu);void libxl_cpumap_reset (libxl_cpumap *cpumap,int cpu); +static inline void libxl_cpumap_set_any (libxl_cpumap *cpumap) +{ +memset (cpumap->map,-1 , cpumap->size); +} +static inline void libxl_cpumap_set_none (libxl_cpumap *cpumap) +{ +memset (cpumap->map,0 , cpumap->size); +} +static inline int libxl_cpumap_cpu_valid (libxl_cpumap *cpumap,int cpu) +{ +return cpu >=0 && cpu < (cpumap->size *8 ); +}#define libxl_for_each_cpu (var, map) for (var = 0; var < (map).size * 8; var++)#define libxl_for_each_set_cpu (v, m) for (v = 0; v < (m).size * 8; v++) \ if (libxl_cpumap_test(&(m), v))
tools/libxl/xl_cmdimpl.c +87 -53
191 lines not on this page · raw message
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
# HG changeset patch
# Parent d76603510485bbd69b30905599f2883bc0fb9b67
libxl: allow for specifying the CPU affinity in the config file.
Enable CPU affinity specification in a VM's config file with the exact syntax `xl vcpu-pin' provides.
tools/libxl/libxl.c +14 -0
@@ -2663,6 +2663,20 @@ int libxl_set_vcpuaffinity(libxl_ctx *ctreturn 0 ; } +int libxl_set_vcpuaffinity_all (libxl_ctx *ctx,uint32_t domid, +unsigned int max_vcpus,libxl_cpumap *cpumap) +{ +int i, rc =0 ; + +for (i =0 ; i < max_vcpus; i++) { +if (libxl_set_vcpuaffinity (ctx, domid, i, cpumap)) { +LIBXL__LOG (ctx,LIBXL__LOG_WARNING ,"no affinity for cpu %d" , i); + rc =ERROR_FAIL ; + } + } +return rc; +} +int libxl_set_vcpuonline (libxl_ctx *ctx,uint32_t domid,libxl_cpumap *cpumap) {GC_INIT (ctx);
tools/libxl/libxl.h +2 -0
@@ -560,6 +560,8 @@ libxl_vcpuinfo *libxl_list_vcpu(libxl_ct int *nb_vcpu, int *nrcpus);int libxl_set_vcpuaffinity (libxl_ctx *ctx,uint32_t domid,uint32_t vcpuid,libxl_cpumap *cpumap); +int libxl_set_vcpuaffinity_all (libxl_ctx *ctx,uint32_t domid, +unsigned int max_vcpus,libxl_cpumap *cpumap);int libxl_set_vcpuonline (libxl_ctx *ctx,uint32_t domid,libxl_cpumap *cpumap);int libxl_get_sched_id (libxl_ctx *ctx);
tools/libxl/libxl_create.c +3 -0
@@ -71,6 +71,9 @@ int libxl_init_build_info(libxl_ctx *ctxmemset (b_info,'\0' ,sizeof (*b_info)); b_info->max_vcpus =1 ; b_info->cur_vcpus =1 ; +if (libxl_cpumap_alloc (ctx, &b_info->cpumap)) +return ERROR_NOMEM ; +libxl_cpumap_set_any (&b_info->cpumap); b_info->max_memkb =32 *1024 ; b_info->target_memkb = b_info->max_memkb; b_info->disable_migrate =0 ;
tools/libxl/libxl_dom.c +1 -0
@@ -65,6 +65,7 @@ int libxl__build_pre(libxl__gc *gc, uintlibxl_ctx *ctx =libxl__gc_owner (gc);int tsc_mode;xc_domain_max_vcpus (ctx->xch, domid, info->max_vcpus); +libxl_set_vcpuaffinity_all (ctx, domid, info->max_vcpus, &info->cpumap);xc_domain_setmaxmem (ctx->xch, domid, info->target_memkb +LIBXL_MAXMEM_CONSTANT );if (info->type ==LIBXL_DOMAIN_TYPE_PV ) xc_domain_set_memmap_limit(ctx->xch, domid,
tools/libxl/libxl_types.idl +1 -0
@@ -162,6 +162,7 @@ libxl_domain_create_info = Struct("domai libxl_domain_build_info = Struct("domain_build_info",[ ("max_vcpus", integer), ("cur_vcpus", integer), + ("cpumap", libxl_cpumap), ("tsc_mode", libxl_tsc_mode), ("max_memkb", uint32), ("target_memkb", uint32),
tools/libxl/libxl_utils.h +12 -0
@@ -70,6 +70,18 @@ int libxl_cpumap_alloc(libxl_ctx *ctx, lint libxl_cpumap_test (libxl_cpumap *cpumap,int cpu);void libxl_cpumap_set (libxl_cpumap *cpumap,int cpu);void libxl_cpumap_reset (libxl_cpumap *cpumap,int cpu); +static inline void libxl_cpumap_set_any (libxl_cpumap *cpumap) +{ +memset (cpumap->map,-1 , cpumap->size); +} +static inline void libxl_cpumap_set_none (libxl_cpumap *cpumap) +{ +memset (cpumap->map,0 , cpumap->size); +} +static inline int libxl_cpumap_cpu_valid (libxl_cpumap *cpumap,int cpu) +{ +return cpu >=0 && cpu < (cpumap->size *8 ); +}#define libxl_for_each_cpu (var, map) for (var = 0; var < (map).size * 8; var++)#define libxl_for_each_set_cpu (v, m) for (v = 0; v < (m).size * 8; v++) \ if (libxl_cpumap_test(&(m), v))
tools/libxl/xl_cmdimpl.c +87 -53
191 lines not on this page · raw message
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Re: [PATCHv2 2 of 2] libxl: allow for specifying the CPU affinity in the config file.
Quoted 10 lines · Dario Faggioli
Enable CPU affinity specification in a VM's config file with the exact syntax `xl vcpu-pin' provides.
Signed-off-by Dario Faggioli <[email protected]>diff -r
d76603510485tools/libxl/libxl.ctools/libxl/libxl.c +0
-0@@ -2663,6 +2663,20 @@ int libxl_set_vcpuaffinity(libxl_ctx *ctreturn 0 ;
tools/libxl/libxl.c int libxl_set_vcpuaffinity(libxl_ctx *ct · line 2673
Quoted 10 lines
tools/libxl/libxl.c +8
-0} +int libxl_set_vcpuaffinity_all (libxl_ctx *ctx,uint32_t domid, +unsigned int max_vcpus,libxl_cpumap *cpumap) +{ +int i, rc =0 ; + +for (i =0 ; i < max_vcpus; i++) +if (libxl_set_vcpuaffinity (ctx, domid, i, cpumap)) { +LIBXL__LOG (ctx,LIBXL__LOG_WARNING ,"no affinity for cpu %d" , i);
"failed to set affinity for ..." would better describe what had happened.
Quoted 9 lines
tools/libxl/libxl.c +6
-0+ rc =ERROR_FAIL ; + } + } +return rc; +} +int libxl_set_vcpuonline (libxl_ctx *ctx,uint32_t domid,libxl_cpumap *cpumap) {GC_INIT (ctx);
[...]
tools/libxl/xl_cmdimpl.c static void dolog(const char *file, int · line 292
Quoted 10 lines
if (libxl_cpumap_test(&(m), v))diff -r
d76603510485tools/libxl/xl_cmdimpl.ctools/libxl/xl_cmdimpl.c +2
-0@@ -288,16 +288,72 @@ static void dolog(const char *file, intlibxl_write_exactly (NULL , logfile, s, rc,NULL ,NULL ); } +static void print_bitmap (uint8_t *map,int maplen,FILE *stream) +{
[...]
I assumed this was pure code motion so I didn't read it.
Quoted 23 lines
tools/libxl/xl_cmdimpl.c +9
-1+} +static void printf_info (int domid,libxl_domain_config *d_config,libxl_device_model_info *dm_info) { -int i; +int i, nr_cpus =-1 ;libxl_dominfo info; +libxl_physinfo physinfo;libxl_domain_create_info *c_info = &d_config->c_info;libxl_domain_build_info *b_info = &d_config->b_info; +if (libxl_get_physinfo (ctx, &physinfo) ==0 ) + nr_cpus = physinfo.nr_cpus; +else +fprintf (stderr,"libxl_physinfo failed.\n" ); +printf ("(domain\n\t(domid %d)\n" , domid);printf ("\t(create_info)\n" );printf ("\t(hvm %d)\n" , c_info->type ==LIBXL_DOMAIN_TYPE_HVM );
tools/libxl/xl_cmdimpl.c static void printf_info(int domid, · line 388
Quoted 5 lines
tools/libxl/xl_cmdimpl.c +2
-0@@ -328,6 +384,9 @@ static void printf_info(int domid,printf ("\t(build_info)\n" );printf ("\t(max_vcpus %d)\n" , b_info->max_vcpus); +printf ("\t(CPU affinity " ); +print_bitmap (b_info->cpumap.map, nr_cpus, stdout);
If libxl_get_physinfo failed then nr_cpus would == -1 here.
However I don't think it is even necessary to extend this (legacy) sexp any further. I'm about to send a patch which will use the print the json representation (which is autogenerated).
If we were to extend it then it should match the xend/xm output since it's only purpose is for xm compatibility.
tools/libxl/xl_cmdimpl.c static void split_string_into_string_lis · line 631
Quoted 9 lines
tools/libxl/xl_cmdimpl.c +2
-0+printf (")\n" );printf ("\t(tsc_mode %s)\n" ,libxl_tsc_mode_to_string (b_info->tsc_mode));printf ("\t(max_memkb %d)\n" , b_info->max_memkb);printf ("\t(target_memkb %d)\n" , b_info->target_memkb);@@ -569,6 +628,8 @@ static void split_string_into_string_lisfree (s); } +static int vcpupin_parse (char *cpu,libxl_cpumap *cpumap);
There are no calls to vcpupin_parse added after this point by this patch so is this necessary? I'd prefer moving the function up in any case or at least declaring all the forward references near the top of the file.
Ian.
Re: [PATCHv2 2 of 2] libxl: allow for specifying the CPU affinity in the config file.
Quoted 11 lines · Ian Campbell
tools/libxl/libxl.c int libxl_set_vcpuaffinity(libxl_ctx *ct · line 2673
Quoted 8 lines
tools/libxl/libxl.c +8
-0+int libxl_set_vcpuaffinity_all (libxl_ctx *ctx,uint32_t domid, +unsigned int max_vcpus,libxl_cpumap *cpumap) +{ +int i, rc =0 ; + +for (i =0 ; i < max_vcpus; i++) +if (libxl_set_vcpuaffinity (ctx, domid, i, cpumap)) { +LIBXL__LOG (ctx,LIBXL__LOG_WARNING ,"no affinity for cpu %d" , i);"failed to set affinity for ..." would better describe what had happened.
Ok.
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+static void print_bitmap(uint8_t *map, int maplen, FILE *stream) +{[...]
I assumed this was pure code motion so I didn't read it.
It is.
Quoted 9 lines
tools/libxl/xl_cmdimpl.c static void printf_info(int domid, · line 388
Quoted 6 lines
tools/libxl/xl_cmdimpl.c +2
-0@@ -328,6 +384,9 @@ static void printf_info(int domid,printf ("\t(build_info)\n" );printf ("\t(max_vcpus %d)\n" , b_info->max_vcpus); +printf ("\t(CPU affinity " ); +print_bitmap (b_info->cpumap.map, nr_cpus, stdout);If
libxl_get_physinfofailed thennr_cpuswould == -1 here.
Quoted 4 lines
However I don't think it is even necessary to extend this (legacy) sexp any further. I'm about to send a patch which will use the print the json representation (which is autogenerated).
Ok, then I'll drop this.
Quoted 6 lines
Review static void printf_info(int domid, · line 389
Quoted 1 line
(quoted diff) +1
-0+static int vcpupin_parse(char *cpu, libxl_cpumap *cpumap);There are no calls to
vcpupin_parseadded after this point by this patch so is this necessary? I'd prefer moving the function up in any case or at least declaring all the forward references near the top of the file.
Oops. I thought I removed all these fwd-decls. Sorry, seems this one escaped... Will nuke it. :-)
Thanks,
Dario
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ------------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel