Support CPUID/MSR data in migration streams
v2 cover Support CPUID/MSR data in migration streams
Here is v2 of the work to plumb CPUID/MSR data into the migration stream.
Patches 1 and 2 are cleanup. 3-8 are concerned with introducing the STATIC_DATA_END record. 9-11 are some libxl work to reposition CPUID construction during domain create. 12-14 are the introduction of the X86_{CPUID,MSR}_DATA records, and 15-17 are the final adjustments in libxc and libxl to use them.
This series does have a net change in behaviour for CPUID handing in migrated domains. See patch 16 for details.
Some acks are carried forwards from the v1 review. Others are not. The majority change has been to shuffle the order of actions to hopefully make the logic much more clear to follow.
~Andrew
Andrew Cooper (17):
tools/libxl: Remove libxl_cpuid_{set,apply_policy}() from the API tools/libxl: Simplify callback handling in libxl-save-helper tools/migration: Drop IHDR_VERSION constant from libxc and python docs/migration Specify migration v3 and STATIC_DATA_END python/migration: Update validation logic to understand a v3 stream libxc/restore: Support v3 streams and handle STATIC_DATA_END libxc/restore: STATIC_DATA_END inference for v2 compatibility libxc/save: Write a v3 stream
tools/libxl: Provide a static_data_done callback for domain restore tools/libxl: Plumb a restore boolean down into libxl__build_pre() tools/libxl: Re-position CPUID handling during domain construction docs/migration: Specify X86_{CPUID,MSR}_POLICY records libxc/restore: Handle X86_{CPUID,MSR}_DATA records libxc/save: Write X86_{CPUID,MSR}_DATA records tools/libx[cl]: Plumb 'missing' through static_data_done() up into libxl tools/libxc: Restore CPUID/MSR data found in the migration stream docs/xl.cfg: Rewrite cpuid= section
docs/man/xl.cfg.5.pod.in | 74 +++++++++++++----- docs/specs/libxc-migration-stream.pandoc | 81 ++++++++++++++++++- tools/libxc/include/xenguest.h | 11 +++ tools/libxc/xc_cpuid_x86.c | 8 +- tools/libxc/xc_sr_common.c | 3 + tools/libxc/xc_sr_common.h | 35 ++++++++- tools/libxc/xc_sr_common_x86.c | 120 +++++++++++++++++++++++++++++ tools/libxc/xc_sr_common_x86.h | 25 ++++++ tools/libxc/xc_sr_restore.c | 61 ++++++++++++++- tools/libxc/xc_sr_restore_x86_hvm.c | 10 +++ tools/libxc/xc_sr_restore_x86_pv.c | 27 +++++++ tools/libxc/xc_sr_save.c | 20 ++++- tools/libxc/xc_sr_save_x86_hvm.c | 6 ++ tools/libxc/xc_sr_save_x86_pv.c | 14 +++- tools/libxc/xc_sr_stream_format.h | 4 +- tools/libxl/libxl.h | 26 ++++++- tools/libxl/libxl_cpuid.c | 6 +- tools/libxl/libxl_create.c | 37 ++++++++- tools/libxl/libxl_dm.c | 2 +- tools/libxl/libxl_dom.c | 20 +++-- tools/libxl/libxl_internal.h | 10 ++- tools/libxl/libxl_nocpuid.c | 6 +- tools/libxl/libxl_save_helper.c | 16 ++-- tools/libxl/libxl_save_msgs_gen.pl | 3 +- tools/python/scripts/convert-legacy-stream | 13 +++- tools/python/scripts/verify-stream-v2 | 2 +- tools/python/xen/migration/libxc.py | 74 ++++++++++++++++-- 27 files changed, 635 insertions(+), 79 deletions(-)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 1/17 tools/libxl: Remove libxl_cpuid_{set, apply_policy}() from the API
These functions should never have been exposed. They don't have external users, and can't usefully be used for several reasons.
Move libxl_cpuid_{set,apply_policy}() to being internal functions, and leave an equivalent of the nop stubs in the API for caller compatibility.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
RFC for obvious reasons. An alternative would be to #if 0 them, which would result in a compile failure rather than silent stubbing. I'm not sure which is least bad, but I don't think either are going to cause a problem in practice.
--- tools/libxl/libxl.h | 26 ++++++++++++++++++++++---- tools/libxl/libxl_cpuid.c | 6 +++--- tools/libxl/libxl_dom.c | 4 ++-- tools/libxl/libxl_internal.h | 4 ++++ tools/libxl/libxl_nocpuid.c | 6 +++--- 5 files changed, 34 insertions(+), 12 deletions(-)
tools/libxl/libxl.h +22 -4
@@ -665,7 +665,7 @@ typedef struct libxl__ctx libxl_ctx;#if LIBXL_API_VERSION !=0x040200 &&LIBXL_API_VERSION !=0x040300 && \LIBXL_API_VERSION !=0x040400 &&LIBXL_API_VERSION !=0x040500 && \LIBXL_API_VERSION !=0x040700 &&LIBXL_API_VERSION !=0x040800 && \ -LIBXL_API_VERSION !=0x041300 +LIBXL_API_VERSION !=0x041300 &&LIBXL_API_VERSION !=0x041400 #error Unknown LIBXL_API_VERSION#endif #endif
@@ -2323,9 +2323,27 @@ libxl_device_pci *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num);int libxl_cpuid_parse_config (libxl_cpuid_policy_list *cpuid,const char * str);int libxl_cpuid_parse_config_xend (libxl_cpuid_policy_list *cpuid,const char * str); -void libxl_cpuid_apply_policy (libxl_ctx *ctx,uint32_t domid); -void libxl_cpuid_set (libxl_ctx *ctx,uint32_t domid, -libxl_cpuid_policy_list cpuid); +#if LIBXL_API_VERSION <0x041400 +/* +* Dropped from the API in Xen 4.14. At the time of writing, these functions +* don't appear to ever have had external callers. +* +* These have always been used internally during domain construction, and +* can't easily be used externally because of their implicit parameters in +* other pieces of global state. +* +* Furthermore, an API user can't usefully determine whether they get +* libxl_cpuid (the real implementation) or libxl_nocpuid (no-op stubs). +* +* The internal behaviour of these functions also needs to change. Therefore +* for simplicitly, provide the no-op stubs. Yes technically this is an API +* change in some cases for existing software, but there is 0 of that in +* practice. +*/ +static inline void libxl_cpuid_apply_policy (libxl_ctx *ctx,uint32_t domid) {} +static inline void libxl_cpuid_set (libxl_ctx *ctx,uint32_t domid, +libxl_cpuid_policy_list cpuid) {} +#endif /* * Functions for allowing users of libxl to store private data
tools/libxl/libxl_cpuid.c +3 -3
@@ -410,13 +410,13 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,return 0 ; } -void libxl_cpuid_apply_policy (libxl_ctx *ctx,uint32_t domid) +void libxl__cpuid_apply_policy (libxl_ctx *ctx,uint32_t domid) {xc_cpuid_apply_policy (ctx->xch, domid,NULL ,0 ); } -void libxl_cpuid_set (libxl_ctx *ctx,uint32_t domid, -libxl_cpuid_policy_list cpuid) +void libxl__cpuid_set (libxl_ctx *ctx,uint32_t domid, +libxl_cpuid_policy_list cpuid) {int i;char *cpuid_res[4 ];
tools/libxl/libxl_dom.c +2 -2
@@ -454,9 +454,9 @@ int libxl__build_post(libxl__gc *gc, uint32_t domid,if (rc)return rc; -libxl_cpuid_apply_policy (ctx, domid); +libxl__cpuid_apply_policy (ctx, domid);if (info->cpuid !=NULL ) -libxl_cpuid_set (ctx, domid, info->cpuid); +libxl__cpuid_set (ctx, domid, info->cpuid);if (info->type ==LIBXL_DOMAIN_TYPE_HVM && !libxl_ms_vm_genid_is_zero (&info->u.hvm.ms_vm_genid)) {
tools/libxl/libxl_internal.h +4 -0
@@ -2042,6 +2042,10 @@ struct libxl__cpuid_policy {char *policy[4 ]; }; +_hidden void libxl__cpuid_apply_policy(libxl_ctx *ctx, uint32_t domid); +_hidden void libxl__cpuid_set(libxl_ctx *ctx, uint32_t domid, +libxl_cpuid_policy_list cpuid); +/* Calls poll() again - useful to check whether a signaled condition * is still true. Cannot fail. Returns currently-true revents. */ _hidden short libxl__fd_poll_recheck (libxl__egc *egc,int fd,short events);
tools/libxl/libxl_nocpuid.c +3 -3
@@ -34,12 +34,12 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,return 0 ; } -void libxl_cpuid_apply_policy (libxl_ctx *ctx,uint32_t domid) +void libxl__cpuid_apply_policy (libxl_ctx *ctx,uint32_t domid) { } -void libxl_cpuid_set (libxl_ctx *ctx,uint32_t domid, -libxl_cpuid_policy_list cpuid) +void libxl__cpuid_set (libxl_ctx *ctx,uint32_t domid, +libxl_cpuid_policy_list cpuid) { }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 01/17] tools/libxl: Remove libxl_cpuid_{set, apply_policy}() from the API
Quoted 7 lines · Andrew Cooper
These functions should never have been exposed. They don't have external users, and can't usefully be used for several reasons.
Move libxl_cpuid_{set,apply_policy}() to being internal functions, and leave an equivalent of the nop stubs in the API for caller compatibility.
Signed-off-by Andrew Cooper <[email protected]>
This has already been committed (c/s dacb80f9757c011161cec6609f39837c9ea8caa8) as part of a smaller series developed subsequent to posting this series, which made sense to go in ahead.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 2/17 tools/libxl: Simplify callback handling in libxl-save-helper
In reply to Andrew CooperThe {save,restore}_callback helpers can have their scope reduced vastly, and helper_setcallbacks_{save,restore}() doesn't need to use a ternary operator to write 0 (meaning NULL) into an already zeroed structure.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
--- tools/libxl/libxl_save_helper.c | 16 ++++++---------- tools/libxl/libxl_save_msgs_gen.pl | 2 +- 2 files changed, 7 insertions(+), 11 deletions(-)
tools/libxl/libxl_save_helper.c +6 -10
@@ -217,8 +217,6 @@ int helper_getreply(void *user)/*----- other callbacks -----*/ -static struct save_callbacks helper_save_callbacks; -static void startup (const char *op) {xtl_log (&logger,XTL_DEBUG ,0 ,program,"starting %s" ,op);
@@ -234,8 +232,6 @@ static void complete(int retval) {exit (0 ); } -static struct restore_callbacks helper_restore_callbacks; -int main (int argc,char **argv) {int r;
@@ -247,6 +243,7 @@ int main(int argc, char **argv)assert (mode);if (!strcmp (mode,"--save-domain" )) { +static struct save_callbacks cb; io_fd =atoi (NEXTARG ); recv_fd =atoi (NEXTARG );
@@ -256,16 +253,16 @@ int main(int argc, char **argv)xc_stream_type_t stream_type =strtoul (NEXTARG ,0 ,10 );assert (!*++argv); -helper_setcallbacks_save (&helper_save_callbacks, cbflags); +helper_setcallbacks_save (&cb, cbflags);startup ("save" );setup_signals (save_signal_handler); - r =xc_domain_save (xch, io_fd, dom, flags, &helper_save_callbacks, - stream_type, recv_fd); + r =xc_domain_save (xch, io_fd, dom, flags, &cb, stream_type, recv_fd);complete (r); }else if (!strcmp (mode,"--restore-domain" )) { +static struct restore_callbacks cb; io_fd =atoi (NEXTARG ); send_back_fd =atoi (NEXTARG );
@@ -278,7 +275,7 @@ int main(int argc, char **argv)xc_stream_type_t stream_type =strtoul (NEXTARG ,0 ,10 );assert (!*++argv); -helper_setcallbacks_restore (&helper_restore_callbacks, cbflags); +helper_setcallbacks_restore (&cb, cbflags);unsigned long store_mfn =0 ;unsigned long console_mfn =0 ;
@@ -288,8 +285,7 @@ int main(int argc, char **argv) r =xc_domain_restore (xch, io_fd, dom, store_evtchn, &store_mfn, store_domid, console_evtchn, &console_mfn, - console_domid, stream_type, - &helper_restore_callbacks, send_back_fd); + console_domid, stream_type, &cb, send_back_fd);helper_stub_restore_results (store_mfn,console_mfn,0 );complete (r);
tools/libxl/libxl_save_msgs_gen.pl +1 -1
@@ -333,7 +333,7 @@ END_ALWAYS my $c_v = "(1u<<$msgnum)"; my $c_cb = "cbs->$name"; $f_more_sr->(" if ($c_cb) cbflags |= $c_v;\n", $enumcallbacks); - $f_more_sr->(" $c_cb = (cbflags & $c_v) ? ${encode}_${name} : 0;\n", + $f_more_sr->(" if (cbflags & $c_v) $c_cb = ${encode}_${name};\n", $setcallbacks); } $f_more_sr->(" return 1;\n }\n\n");
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 02/17] tools/libxl: Simplify callback handling in libxl-save-helper
Andrew Cooper writes ("[PATCH v2 02/17] tools/libxl: Simplify callback handling in libxl-save-helper"):
Quoted 5 lines
The {save,restore}_callback helpers can have their scope reduced vastly, and helper_setcallbacks_{save,restore}() doesn't need to use a ternary operator to write 0 (meaning NULL) into an already zeroed structure.
No functional change.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 3/17 tools/migration: Drop IHDR_VERSION constant from libxc and python
In reply to Andrew CooperMigration v3 is in the process of being introduced, meaning that the code has to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say "v2 (or later)". What matters is the distinction vs legacy streams.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
--- tools/libxc/xc_sr_restore.c | 6 +++--- tools/libxc/xc_sr_save.c | 2 +- tools/libxc/xc_sr_stream_format.h | 1 - tools/python/scripts/convert-legacy-stream | 6 +++--- tools/python/scripts/verify-stream-v2 | 2 +- tools/python/xen/migration/libxc.py | 9 ++++----- 6 files changed, 12 insertions(+), 14 deletions(-)
tools/libxc/xc_sr_restore.c +3 -3
@@ -35,10 +35,10 @@ static int read_headers(struct xc_sr_context *ctx)return -1 ; } -if ( ihdr.version !=IHDR_VERSION ) +if ( ihdr.version !=2 ) { -ERROR ("Invalid Version: Expected %d, Got %d" , -IHDR_VERSION , ihdr.version); +ERROR ("Invalid Version: Expected 2, Got %d" , + ihdr.version);return -1 ; }
tools/libxc/xc_sr_save.c +1 -1
@@ -13,7 +13,7 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)struct xc_sr_ihdr ihdr = { .marker =IHDR_MARKER , .id =htonl (IHDR_ID ), - .version =htonl (IHDR_VERSION ), + .version =htonl (2 ), .options =htons (IHDR_OPT_LITTLE_ENDIAN ), };struct xc_sr_dhdr dhdr = {
tools/libxc/xc_sr_stream_format.h +0 -1
@@ -23,7 +23,6 @@ struct xc_sr_ihdr#define IHDR_MARKER 0xffffffffffffffffULL#define IHDR_ID 0x58454E46U -#define IHDR_VERSION 2#define _IHDR_OPT_ENDIAN 0#define IHDR_OPT_LITTLE_ENDIAN (0 << _IHDR_OPT_ENDIAN)
tools/python/scripts/convert-legacy-stream +3 -3
@@ -79,7 +79,7 @@ def write_libxc_ihdr(): stream_write(pack(libxc.IHDR_FORMAT, libxc.IHDR_MARKER, # Marker libxc.IHDR_IDENT, # Ident - libxc.IHDR_VERSION, # Version + 2, # Version libxc.IHDR_OPT_LE, # Options 0, 0)) # Reserved
@@ -632,13 +632,13 @@ def main(): usage = ("%prog [options] -i INPUT -o OUTPUT" " -w WIDTH -g GUEST"), description = - "Convert a legacy stream to a v2 stream") + "Convert a legacy stream to a v2 (or later) stream") # Required options parser.add_option("-i", "--in", dest = "fin", metavar = "<FD or FILE>", help = "Legacy input to convert") parser.add_option("-o", "--out", dest = "fout", metavar = "<FD or FILE>", - help = "v2 format output") + help = "v2 (or later) format output") parser.add_option("-w", "--width", dest = "twidth", metavar = "<32/64>", choices = ["32", "64"], help = "Legacy toolstack bitness")
tools/python/scripts/verify-stream-v2 +1 -1
@@ -108,7 +108,7 @@ def main(): parser = OptionParser(usage = "%prog [options]", description = - "Verify a stream according to the v2 spec") + "Verify a stream according to the v2 (or later) spec") # Optional options parser.add_option("-i", "--in", dest = "fin", metavar = "<FD or FILE>",
tools/python/xen/migration/libxc.py +4 -5
@@ -19,7 +19,6 @@ IHDR_MARKER = 0xffffffffffffffff IHDR_IDENT = 0x58454E46 # "XENF" in ASCII -IHDR_VERSION = 2 IHDR_OPT_BIT_ENDIAN = 0 IHDR_OPT_LE = (0 << IHDR_OPT_BIT_ENDIAN)
@@ -113,7 +112,7 @@ HVM_PARAMS_FORMAT = "II" class VerifyLibxc(VerifyBase): - """ Verify a Libxc v2 stream """ + """ Verify a Libxc v2 (or later) stream """ def __init__(self, info, read): VerifyBase.__init__(self, info, read)
@@ -144,9 +143,9 @@ def verify_ihdr(self): raise StreamError("Bad image id: Expected 0x%x, got 0x%x" % (IHDR_IDENT, ident)) - if version != IHDR_VERSION: - raise StreamError("Unknown image version: Expected %d, got %d" % - (IHDR_VERSION, version)) + if version != 2: + raise StreamError("Unknown image version: Expected 2, got %d" % + (version, )) if options & IHDR_OPT_RESZ_MASK: raise StreamError("Reserved bits set in image options field: 0x%x" %
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python
Andrew Cooper writes ("[PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python"):
Quoted 6 lines
Migration v3 is in the process of being introduced, meaning that the code has to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say "v2 (or later)". What matters is the distinction vs legacy streams.
Can I request that you use a manifest constant for 2', rather than sprinkling literal 2's everywhere ? Something like IHDR_VERSION_2 ? This may seem pointless, but it will mean that it is possible to grep the code much more easily for things which are relevant to v2 or v3 or whatever.
(I don't it's necessary to go to any great lengths to substitute the value of IHDR_VERSION_2 into error messages; a literal 2 in the string is OK I think.)
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python
Quoted 13 lines · Ian Jackson
Andrew Cooper writes ("[PATCH v2 03/17]
tools/migration: DropIHDR_VERSIONconstant from libxc and python"):Quoted 6 lines
Migration v3 is in the process of being introduced, meaning that the code has to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say "v2 (or later)". What matters is the distinction vs legacy streams.
Can I request that you use a manifest constant for
2', rather than sprinkling literal2's everywhere ? Something likeIHDR_VERSION_2? This may seem pointless, but it will mean that it is possible to grep the code much more easily for things which are relevant to v2 or v3 or whatever.
Quoted 3 lines
(I don't it's necessary to go to any great lengths to substitute the value of
IHDR_VERSION_2into error messages; a literal 2 in the string is OK I think.)
As I reply previously... The value comes out of a pipe, and is checked exactly once.
You can already grep for everything, using format_version which is where this number is stashed for the duration of restore.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python
Andrew Cooper writes ("Re: [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python"):
Quoted 14 lines · Ian Jackson
Quoted 13 lines
Andrew Cooper writes ("[PATCH v2 03/17]
tools/migration: DropIHDR_VERSIONconstant from libxc and python"):Quoted 6 lines
Migration v3 is in the process of being introduced, meaning that the code has to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say "v2 (or later)". What matters is the distinction vs legacy streams.
Can I request that you use a manifest constant for
2', rather than sprinkling literal2's everywhere ? Something likeIHDR_VERSION_2? This may seem pointless, but it will mean that it is possible to grep the code much more easily for things which are relevant to v2 or v3 or whatever.
Quoted 6 lines
Quoted 3 lines
(I don't it's necessary to go to any great lengths to substitute the value of
IHDR_VERSION_2into error messages; a literal 2 in the string is OK I think.)As I reply previously... The value comes out of a pipe, and is checked exactly once.
I think we are talking at cross purposes.
I am suggesting that you replace every instance of a numeric literal `2' in the code with IHDR_VERSION_2 (which would be a #define or enum for 2).
I count 4 such literals.
Quoted 2 lines
You can already grep for everything, using
format_versionwhich is where this number is stashed for the duration of restore.
None of the things I am talking about have "format_version" nearby. They tend to have variants on "version" but that is a poor thing to grep for.
Am I making any more sense now ?
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python
Quoted 20 lines · Ian Jackson
Andrew Cooper writes ("Re: [PATCH v2 03/17]
tools/migration: DropIHDR_VERSIONconstant from libxc and python"):Quoted 14 lines · Ian Jackson
Quoted 13 lines
Andrew Cooper writes ("[PATCH v2 03/17]
tools/migration: DropIHDR_VERSIONconstant from libxc and python"):Quoted 6 lines
Migration v3 is in the process of being introduced, meaning that the code has to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say "v2 (or later)". What matters is the distinction vs legacy streams.
Can I request that you use a manifest constant for
2', rather than sprinkling literal2's everywhere ? Something likeIHDR_VERSION_2? This may seem pointless, but it will mean that it is possible to grep the code much more easily for things which are relevant to v2 or v3 or whatever.Quoted 5 lines
Quoted 3 lines
(I don't it's necessary to go to any great lengths to substitute the value of
IHDR_VERSION_2into error messages; a literal 2 in the string is OK I think.)As I reply previously... The value comes out of a pipe, and is checked exactly once.
Quoted 7 lines
I think we are talking at cross purposes.
I am suggesting that you replace every instance of a numeric literal `2' in the code with
IHDR_VERSION_2(which would be a #define or enum for 2).I count 4 such literals.
I presume you mean here 2x send IHDR and 2x receive IHDR, one C and one Python in each case.
I understand what you're suggesting. I completely disagree with it, as it obfuscates the logic and introduces a source of bugs for zero gain.
IHDR_VERSION_* isn't grepable. You've got to find the only instance of it in code to figure out what to search for.
Quoted 5 lines
Quoted 2 lines
You can already grep for everything, using
format_versionwhich is where this number is stashed for the duration of restore.None of the things I am talking about have "
format_version" nearby. They tend to have variants on "version" but that is a poor thing to grep for.
"ctx->restore.format_version = ihdr.version;" (the only instance in the codebase at this point in the series) is immediately after ihdr is sanity checked, which is sole time where idhr.version has its value checked (in the restore path. There is also exactly one place in the save side, and any more than this would be buggy code).
The only thing IHDR_VERSION_* usefully gets you is the ability to get the defines accidentally wrong, and introduce bugs that way.
~Andrew
Ian Jackson
Acked-byRe: [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python [and 1 more messages]
Andrew Cooper writes ("Re: [PATCH v2 03/17] tools/migration: Drop IHDR_VERSION constant from libxc and python"):
Quoted 5 lines
I presume you mean here 2x send IHDR and 2x receive IHDR, one C and one Python in each case.
I understand what you're suggesting. I completely disagree with it, as it obfuscates the logic and introduces a source of bugs for zero gain.
...
Quoted 2 lines
The only thing IHDR_VERSION_* usefully gets you is the ability to get the defines accidentally wrong, and introduce bugs that way.
Andrew Cooper writes ("Re: [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility"):
Quoted 6 lines · Ian Jackson
Quoted 3 lines
You could handle that with a small bit of code around one of the call sites to adjust the error handling. (Also, what a mess, but I guess we're here now...)
... which is the majority the code you're trying to abstract away.
...
Quoted 3 lines
tl;dr I could put an #ifdef x86'd static inline in the root common header (
xc_sr_common.h), but the overall complexity is greater than what is presented here.
I still don't agree with you I'm afraid. I went back through our messages, and looked at the code again, and I think we are probably still not communicating well. However, I thought about how best to deal with this disagreement. As the actual author of much of this code, and certainly the person putting a lot of effort in, you should get quite a bit of leeway.
I considered taking your branch and showing you what I meant in code terms. But ultimately I think this is a waste of our time and I don't want us to get into a pointless argument. I don't think these issues matter enough to be worth the debate. I don't think there are actual bugs here - we're talking about matters of style and taste.
So,
It would probably have been better for me to have got to this point earlier.
Sorry,
Ian.
PATCH v2 4/17 docs/migration Specify migration v3 and STATIC_DATA_END
In reply to Andrew CooperMigration data can be split into two parts - that which is invariant of guest execution, and that which is not. Separate these two with the STATIC_DATA_END record.
The short term, we want to move the x86 CPU Policy data into the stream. In the longer term, we want to provisionally send the static data only to the destination as a more robust compatibility check. In both cases, we will want a callback into the higher level toolstack.
Mandate the presence of the STATIC_DATA_END record, and declare this v3, along with instructions for how to compatibly interpret a v2 stream.
---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
--- docs/specs/libxc-migration-stream.pandoc | 39 +++++++++++++++++++++++++++++--- tools/libxc/xc_sr_common.c | 1 + tools/libxc/xc_sr_stream_format.h | 1 + tools/python/xen/migration/libxc.py | 2 ++ 4 files changed, 40 insertions(+), 3 deletions(-)
docs/specs/libxc-migration-stream.pandoc +36 -3
@@ -127,7 +127,7 @@ marker 0xFFFFFFFFFFFFFFFF. id 0x58454E46 ("XENF" in ASCII). -version 0x00000002. The version of this specification. +version 0x00000003. The version of this specification. options bit 0: Endianness. 0 = little-endian, 1 = big-endian.
@@ -620,6 +620,21 @@ The count of pfns is: record->length/sizeof(uint64_t). \clearpage +STATIC_DATA_END +--------------- + +A static data end record marks the end of the static state. I.e. state which +is invariant of guest execution. + + + 0 1 2 3 4 5 6 7 octet + +-------------------------------------------------+ + +The end record contains no fields; its body_length is 0. + +\clearpage + + Layout ======
@@ -639,7 +654,9 @@ A typical save record for an x86 PV guest image would look like: * Image header * Domain header -* X86_PV_INFO record +* Static data records: + * X86_PV_INFO record + * STATIC_DATA_END * X86_PV_P2M_FRAMES record * Many PAGE_DATA records * X86_TSC_INFO
@@ -667,6 +684,8 @@ A typical save record for an x86 HVM guest image would look like: * Image header * Domain header +* Static data records: + * STATIC_DATA_END * Many PAGE_DATA records * X86_TSC_INFO * HVM_PARAMS
@@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like: HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect the validity of architectural state in the context. +Compatibility with older versions +================================= + +v3 compat with v2 +----------------- + +A v3 stream is compatible with a v2 stream, but mandates the presense of a +STATIC_DATA_END record ahead of any memory/register content. This is to ease +the introduction of new static configuration records over time. + +A v3-compatible reciever interpreting a v2 stream should infer the position of +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END +had been sent. Legacy Images (x86 only) -======================== +------------------------ Restoring legacy images from older tools shall be handled by translating the legacy format image into this new format.
tools/libxc/xc_sr_common.c +1 -0
@@ -36,6 +36,7 @@ static const char *const mandatory_rec_types[] = [REC_TYPE_VERIFY] = "Verify", [REC_TYPE_CHECKPOINT] = "Checkpoint", [REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST] = "Checkpoint dirty pfn list", + [REC_TYPE_STATIC_DATA_END] = "Static data end", };const char *rec_type_to_str (uint32_t type)
tools/libxc/xc_sr_stream_format.h +1 -0
@@ -73,6 +73,7 @@ struct xc_sr_rhdr#define REC_TYPE_VERIFY 0x0000000dU#define REC_TYPE_CHECKPOINT 0x0000000eU#define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST 0x0000000fU +#define REC_TYPE_STATIC_DATA_END 0x00000010U#define REC_TYPE_OPTIONAL 0x80000000U
tools/python/xen/migration/libxc.py +2 -0
@@ -56,6 +56,7 @@ REC_TYPE_verify = 0x0000000d REC_TYPE_checkpoint = 0x0000000e REC_TYPE_checkpoint_dirty_pfn_list = 0x0000000f +REC_TYPE_static_data_end = 0x00000010 rec_type_to_str = { REC_TYPE_end : "End",
@@ -74,6 +75,7 @@ REC_TYPE_verify : "Verify", REC_TYPE_checkpoint : "Checkpoint", REC_TYPE_checkpoint_dirty_pfn_list : "Checkpoint dirty pfn list", + REC_TYPE_static_data_end : "Static data end", } # page_data
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 5/17 python/migration: Update validation logic to understand a v3 stream
In reply to Andrew Cooper---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
--- tools/python/xen/migration/libxc.py | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)
tools/python/xen/migration/libxc.py +22 -4
@@ -119,6 +119,7 @@ class VerifyLibxc(VerifyBase): def __init__(self, info, read): VerifyBase.__init__(self, info, read) + self.version = 0 self.squashed_pagedata_records = 0
@@ -145,9 +146,12 @@ def verify_ihdr(self): raise StreamError("Bad image id: Expected 0x%x, got 0x%x" % (IHDR_IDENT, ident)) - if version != 2: - raise StreamError("Unknown image version: Expected 2, got %d" % - (version, )) + if not (2 <= version <= 3): + raise StreamError( + "Unknown image version: Expected 2 <= ver <= 3, got %d" % + (version, )) + + self.version = version if options & IHDR_OPT_RESZ_MASK: raise StreamError("Reserved bits set in image options field: 0x%x" %
@@ -164,7 +168,8 @@ def verify_ihdr(self): "Stream is not native endianess - unable to validate") endian = ["little", "big"][options & IHDR_OPT_LE] - self.info("Libxc Image Header: %s endian" % (endian, )) + self.info("Libxc Image Header: Version %d, %s endian" % + (version, endian)) def verify_dhdr(self):
@@ -424,6 +429,16 @@ def verify_record_checkpoint_dirty_pfn_list(self, content): raise RecordError("Found checkpoint dirty pfn list record in stream") + def verify_record_static_data_end(self, content): + """ static data end record """ + + if len(content) != 0: + raise RecordError("End record with non-zero length") + + if self.version < 3: + raise RecordError("Static data end record found in v2 stream") + + record_verifiers = { REC_TYPE_end: VerifyLibxc.verify_record_end,
@@ -465,4 +480,7 @@ def verify_record_checkpoint_dirty_pfn_list(self, content): VerifyLibxc.verify_record_checkpoint, REC_TYPE_checkpoint_dirty_pfn_list: VerifyLibxc.verify_record_checkpoint_dirty_pfn_list, + + REC_TYPE_static_data_end: + VerifyLibxc.verify_record_static_data_end, }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 05/17] python/migration: Update validation logic to understand a v3 stream
Andrew Cooper writes ("[PATCH v2 05/17] python/migration: Update validation logic to understand a v3 stream"):
Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>
I'm not sure if I comment about manifest constants for `2' apply here. This is a validation program, not a production tool, AIUI, so it's less critical to be able to get it right. What do you think ?
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 6/17 libxc/restore: Support v3 streams and handle STATIC_DATA_END
In reply to Andrew CooperHigher level toolstacks may wish to know when the static data is complete, so introduce a restore_callback for the purpose.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
v2:
* Split/rearranged from v1
--- tools/libxc/include/xenguest.h | 3 +++ tools/libxc/xc_sr_common.h | 3 +++ tools/libxc/xc_sr_restore.c | 29 +++++++++++++++++++++++++++-- 3 files changed, 33 insertions(+), 2 deletions(-)
tools/libxc/include/xenguest.h +3 -0
@@ -139,6 +139,9 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,/* callbacks provided by xc_domain_restore */ struct restore_callbacks { +/* Called once the STATIC_DATA_END record has been received. */ +int (*static_data_done)(void *data); +/* Called after a new checkpoint to suspend the guest. */ int (*suspend)(void *data);
tools/libxc/xc_sr_common.h +3 -0
@@ -253,6 +253,9 @@ struct xc_sr_context/* Currently buffering records between a checkpoint */ bool buffer_all_records; +/* Whether a STATIC_DATA_END record has been seen. */ +bool seen_static_data_end; +/* * With Remus/COLO, we buffer the records sent by the primary at checkpoint, * in case the primary will fail, we can recover from the last
tools/libxc/xc_sr_restore.c +27 -2
@@ -35,9 +35,9 @@ static int read_headers(struct xc_sr_context *ctx)return -1 ; } -if ( ihdr.version !=2 ) +if ( ihdr.version <2 || ihdr.version >3 ) { -ERROR ("Invalid Version: Expected 2, Got %d" , +ERROR ("Invalid Version: Expected 2 <= ver <= 3, Got %d" , ihdr.version);return -1 ; }
@@ -631,6 +631,27 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)return 0 ; } +static int handle_static_data_end (struct xc_sr_context *ctx) +{ +xc_interface *xch = ctx->xch; +int rc =0 ; + +if ( ctx->restore.seen_static_data_end ) + { +ERROR ("Multiple STATIC_DATA_END records found" ); +return -1 ; + } + + ctx->restore.seen_static_data_end = true; + +if ( ctx->restore.callbacks->static_data_done && + (rc = ctx->restore.callbacks->static_data_done ( + ctx->restore.callbacks->data) !=0 ) ) +ERROR ("static_data_done() callback failed: %d\n" , rc); + +return rc; +} +static int process_record (struct xc_sr_context *ctx,struct xc_sr_record *rec) {xc_interface *xch = ctx->xch;
@@ -654,6 +675,10 @@ static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) rc =handle_checkpoint (ctx);break ; +case REC_TYPE_STATIC_DATA_END : + rc =handle_static_data_end (ctx); +break ; +default : rc = ctx->restore.ops.process_record (ctx, rec);break ;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 6/17 libxc/restore: Support v3 streams and handle STATIC_DATA_END
Andrew Cooper writes ("[PATCH v2 06/17] libxc/restore: Support v3 streams and handle STATIC_DATA_END"):
Quoted 2 lines
Higher level toolstacks may wish to know when the static data is complete, so introduce a
restore_callbackfor the purpose.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 7/17 libxc/restore: STATIC_DATA_END inference for v2 compatibility
In reply to Andrew CooperA v3 stream can compatibly read a v2 stream by inferring the position of the STATIC_DATA_END record.
v2 compatibility is only needed for x86. No other architectures exist yet, but they will have a minimum of v3 when introduced.
The x86 HVM compatibility point being in handle_page_data() (which is common code) is a bit awkward. However, as the two compatibility points are subtly different, and it is (intentionally) not possible to call into arch specific code from common code (except via the ops hooks), use some #ifdef-ary and opencode the check, rather than make handle_page_data() a per-arch helper.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
v2:
* Split/rearranged from v1
* Rewrite the commit message to explain why compatibility is done this way.
--- tools/libxc/include/xenguest.h | 2 +- tools/libxc/xc_sr_common.h | 5 ++++- tools/libxc/xc_sr_restore.c | 27 ++++++++++++++++++++++++++- tools/libxc/xc_sr_restore_x86_pv.c | 17 +++++++++++++++++ 4 files changed, 48 insertions(+), 3 deletions(-)
tools/libxc/include/xenguest.h +1 -1
@@ -139,7 +139,7 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,/* callbacks provided by xc_domain_restore */ struct restore_callbacks { -/* Called once the STATIC_DATA_END record has been received. */ +/* Called once the STATIC_DATA_END record has been received/inferred. */ int (*static_data_done)(void *data);/* Called after a new checkpoint to suspend the guest. */
tools/libxc/xc_sr_common.h +4 -1
@@ -253,7 +253,7 @@ struct xc_sr_context/* Currently buffering records between a checkpoint */ bool buffer_all_records; -/* Whether a STATIC_DATA_END record has been seen. */ +/* Whether a STATIC_DATA_END record has been seen/inferred. */ bool seen_static_data_end;/*
@@ -428,6 +428,9 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec);int populate_pfns (struct xc_sr_context *ctx,unsigned int count,const xen_pfn_t *original_pfns,const uint32_t *types); +/* Handle a STATIC_DATA_END record. */ +int handle_static_data_end (struct xc_sr_context *ctx); +#endif /* * Local variables:
tools/libxc/xc_sr_restore.c +26 -1
@@ -342,6 +342,31 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec)xen_pfn_t *pfns =NULL , pfn;uint32_t *types =NULL , type; +/* +* v2 compatibility only exists for x86 streams. This is a bit of a +* bodge, but it is less bad than duplicating handle_page_data() between +* different architectures. +*/ +#if defined(__i386__) || defined(__x86_64__) +/* v2 compat. Infer the position of STATIC_DATA_END. */ +if ( ctx->restore.format_version <3 && !ctx->restore.seen_static_data_end ) + { + rc =handle_static_data_end (ctx); +if ( rc ) + { +ERROR ("Inferred STATIC_DATA_END record failed" ); + goto err; + } + rc =-1 ; + } + +if ( !ctx->restore.seen_static_data_end ) + { +ERROR ("No STATIC_DATA_END seen" ); + goto err; + } +#endif +if ( rec->length <sizeof (*pages) ) { ERROR("PAGE_DATA record truncated: length %u, min %zu",
@@ -631,7 +656,7 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)return 0 ; } -static int handle_static_data_end (struct xc_sr_context *ctx) +int handle_static_data_end (struct xc_sr_context *ctx) {xc_interface *xch = ctx->xch;int rc =0 ;
tools/libxc/xc_sr_restore_x86_pv.c +17 -0
@@ -679,6 +679,23 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx,unsigned int start, end, x, fpp =PAGE_SIZE / ctx->x86.pv.width;int rc; +/* v2 compat. Infer the position of STATIC_DATA_END. */ +if ( ctx->restore.format_version <3 && !ctx->restore.seen_static_data_end ) + { + rc =handle_static_data_end (ctx); +if ( rc ) + { +ERROR ("Inferred STATIC_DATA_END record failed" ); +return rc; + } + } + +if ( !ctx->restore.seen_static_data_end ) + { +ERROR ("No STATIC_DATA_END seen" ); +return -1 ; + } +if ( !ctx->x86.pv.restore.seen_pv_info ) {ERROR ("Not yet received X86_PV_INFO record" );
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 7/17 libxc/restore: STATIC_DATA_END inference for v2 compatibility
Andrew Cooper writes ("[PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility"):
Quoted 11 lines
A v3 stream can compatibly read a v2 stream by inferring the position of the
STATIC_DATA_ENDrecord.v2 compatibility is only needed for x86. No other architectures exist yet, but they will have a minimum of v3 when introduced.
The x86 HVM compatibility point being in
handle_page_data()(which is common code) is a bit awkward. However, as the two compatibility points are subtly different, and it is (intentionally) not possible to call into arch specific code from common code (except via the ops hooks), use some #ifdef-ary and opencode the check, rather than makehandle_page_data()a per-arch helper.
...
tools/libxc/xc_sr_restore.c static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) · line 355
Quoted 6 lines
tools/libxc/xc_sr_restore.c +6
-0+#if defined(__i386__) || defined(__x86_64__) +/* v2 compat. Infer the position of STATIC_DATA_END. */ +if ( ctx->restore.format_version <3 && !ctx->restore.seen_static_data_end ) + { + rc =handle_static_data_end (ctx); +if ( rc )
These 17 lines appears twice, in basically identical form. Could it be refactored ?
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility
Quoted 13 lines · Ian Jackson
Andrew Cooper writes ("[PATCH v2 07/17] libxc/restore:
STATIC_DATA_ENDinference for v2 compatibility"):Quoted 11 lines
A v3 stream can compatibly read a v2 stream by inferring the position of the
STATIC_DATA_ENDrecord.v2 compatibility is only needed for x86. No other architectures exist yet, but they will have a minimum of v3 when introduced.
The x86 HVM compatibility point being in
handle_page_data()(which is common code) is a bit awkward. However, as the two compatibility points are subtly different, and it is (intentionally) not possible to call into arch specific code from common code (except via the ops hooks), use some #ifdef-ary and opencode the check, rather than makehandle_page_data()a per-arch helper....
Quoted 8 lines
tools/libxc/xc_sr_restore.c static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) · line 355
Quoted 6 lines
tools/libxc/xc_sr_restore.c +6
-0+#if defined(__i386__) || defined(__x86_64__) +/* v2 compat. Infer the position of STATIC_DATA_END. */ +if ( ctx->restore.format_version <3 && !ctx->restore.seen_static_data_end ) + { + rc =handle_static_data_end (ctx); +if ( rc )These 17 lines appears twice, in basically identical form. Could it be refactored ?
Not really, no.
The error handling (i.e. half of those 17 lines) is different, making it somewhat awkward to fit into a static inline.
More importantly however, by design, common code can't call arch-specific code without a restore_ops hook. Deduping these would require breaking the restriction which is currently doing a decent job of keeping x86-isms out of common code.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility
Andrew Cooper writes ("Re: [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility"):
Quoted 8 lines · Ian Jackson
Quoted 2 lines
These 17 lines appears twice, in basically identical form. Could it be refactored ?
Not really, no.
The error handling (i.e. half of those 17 lines) is different, making it somewhat awkward to fit into a static inline.
You could handle that with a small bit of code around one of the call sites to adjust the error handling. (Also, what a mess, but I guess we're here now...)
Quoted 4 lines
More importantly however, by design, common code can't call arch-specific code without a
restore_opshook. Deduping these would require breaking the restriction which is currently doing a decent job of keeping x86-isms out of common code.
I'm afraid you're going to have to explain that to me at a bit greater length. The biggest thing that is confusing me about your statement here is that your patch is indeed adding x86-specific code to a common file. But also I don't understand the comment about restore_ops hooks - do you mean something in restore_callbacks ?
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility
Quoted 11 lines · Ian Jackson
Andrew Cooper writes ("Re: [PATCH v2 07/17] libxc/restore:
STATIC_DATA_ENDinference for v2 compatibility"):Quoted 7 lines · Ian Jackson
Quoted 2 lines
These 17 lines appears twice, in basically identical form. Could it be refactored ?
Not really, no.
The error handling (i.e. half of those 17 lines) is different, making it somewhat awkward to fit into a static inline.
You could handle that with a small bit of code around one of the call sites to adjust the error handling. (Also, what a mess, but I guess we're here now...)
... which is the majority the code you're trying to abstract away.
Quoted 10 lines
Quoted 5 lines
More importantly however, by design, common code can't call arch-specific code without a
restore_opshook. Deduping these would require breaking the restriction which is currently doing a decent job of keeping x86-isms out of common code.I'm afraid you're going to have to explain that to me at a bit greater length. The biggest thing that is confusing me about your statement here is that your patch is indeed adding x86-specific code to a common file. But also I don't understand the comment about
restore_opshooks - do you mean something inrestore_callbacks?
No. restore_callbacks are plumbing between libxl-save-helper and libxc.
restore_ops are internal to the restore code, and come in x86_pv and x86_hvm flavours, with ARM existing in some theoretical future. The design of the common save/restore code had deliberate measures put in place to make it hard to get arch-specific details slip into common code, so porting to different architectures didn't have to start by doing a bunch of cleanup.
tl;dr I could put an #ifdef x86'd static inline in the root common header (xc_sr_common.h), but the overall complexity is greater than what is presented here.
~Andrew
Ian Jackson
Acked-byRe: [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility
Andrew Cooper writes ("Re: [PATCH v2 07/17] libxc/restore: STATIC_DATA_END inference for v2 compatibility"):
Quoted 21 lines · Ian Jackson
Quoted 10 lines
Andrew Cooper writes ("Re: [PATCH v2 07/17] libxc/restore:
STATIC_DATA_ENDinference for v2 compatibility"):Quoted 4 lines
More importantly however, by design, common code can't call arch-specific code without a
restore_opshook. Deduping these would require breaking the restriction which is currently doing a decent job of keeping x86-isms out of common code.I'm afraid you're going to have to explain that to me at a bit greater length. The biggest thing that is confusing me about your statement here is that your patch is indeed adding x86-specific code to a common file. But also I don't understand the comment about
restore_opshooks - do you mean something inrestore_callbacks?No.
restore_callbacksare plumbing between libxl-save-helper and libxc.
restore_opsare internal to the restore code, and come inx86_pvandx86_hvmflavours, with ARM existing in some theoretical future. The design of the common save/restore code had deliberate measures put in place to make it hard to get arch-specific details slip into common code, so porting to different architectures didn't have to start by doing a bunch of cleanup.
Quoted 3 lines
tl;dr I could put an #ifdef x86'd static inline in the root common header (
xc_sr_common.h), but the overall complexity is greater than what is presented here.
Well, I still don't quite follow but as you point out on irc I haven't replied for too long. I don't think I should withhold my ack at this stage.
Thanks,
Ian.
PATCH v2 8/17 libxc/save: Write a v3 stream
In reply to Andrew CooperIntroduce a new static_data() hook which is responsible for writing out any static data records. The HVM side continues to be a no-op, while the PV side moves write_x86_pv_info() into this earlier hook. The the common code writes out a STATIC_DATA_END record, and the stream version is bumped to 3.
Update convert-legacy-stream to write a v3 stream, because this will bypass the compatibly logic in libxc.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
--- tools/libxc/xc_sr_common.h | 10 ++++++++-- tools/libxc/xc_sr_save.c | 20 +++++++++++++++++++- tools/libxc/xc_sr_save_x86_hvm.c | 6 ++++++ tools/libxc/xc_sr_save_x86_pv.c | 10 ++++++---- tools/python/scripts/convert-legacy-stream | 9 ++++++++- 5 files changed, 47 insertions(+), 8 deletions(-)
tools/libxc/xc_sr_common.h +8 -2
@@ -63,8 +63,14 @@ struct xc_sr_save_opsint (*setup)(struct xc_sr_context *ctx);/** -* Send records which need to be at the start of the stream. This is -* called once, after the Image and Domain headers are written. +* Send static records at the head of the stream. This is called once, +* after the Image and Domain headers are written. +*/ +int (*static_data)(struct xc_sr_context *ctx); + +/** +* Send dynamic records which need to be at the start of the stream. This +* is called after the STATIC_DATA_END record is written. */ int (*start_of_stream)(struct xc_sr_context *ctx);
tools/libxc/xc_sr_save.c +19 -1
@@ -13,7 +13,7 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)struct xc_sr_ihdr ihdr = { .marker =IHDR_MARKER , .id =htonl (IHDR_ID ), - .version =htonl (2 ), + .version =htonl (3 ), .options =htons (IHDR_OPT_LITTLE_ENDIAN ), };struct xc_sr_dhdr dhdr = {
@@ -55,6 +55,16 @@ static int write_end_record(struct xc_sr_context *ctx) }/* +* Writes a STATIC_DATA_END record into the stream. +*/ +static int write_static_data_end_record (struct xc_sr_context *ctx) +{ +struct xc_sr_record end = { .type =REC_TYPE_STATIC_DATA_END }; + +return write_record (ctx, &end); +} + +/* * Writes a CHECKPOINT record into the stream. */ static int write_checkpoint_record (struct xc_sr_context *ctx)
@@ -849,6 +859,14 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)if ( rc ) goto err; + rc = ctx->save.ops.static_data (ctx); +if ( rc ) + goto err; + + rc =write_static_data_end_record (ctx); +if ( rc ) + goto err; + rc = ctx->save.ops.start_of_stream (ctx);if ( rc ) goto err;
tools/libxc/xc_sr_save_x86_hvm.c +6 -0
@@ -170,6 +170,11 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)return 0 ; } +static int x86_hvm_static_data (struct xc_sr_context *ctx) +{ +return 0 ; +} +static int x86_hvm_start_of_stream (struct xc_sr_context *ctx) {return 0 ;
@@ -228,6 +233,7 @@ struct xc_sr_save_ops save_ops_x86_hvm = .pfn_to_gfn = x86_hvm_pfn_to_gfn, .normalise_page = x86_hvm_normalise_page, .setup = x86_hvm_setup, + .static_data = x86_hvm_static_data, .start_of_stream = x86_hvm_start_of_stream, .start_of_checkpoint = x86_hvm_start_of_checkpoint, .end_of_checkpoint = x86_hvm_end_of_checkpoint,
tools/libxc/xc_sr_save_x86_pv.c +6 -4
@@ -1052,14 +1052,15 @@ static int x86_pv_setup(struct xc_sr_context *ctx)return 0 ; } +static int x86_pv_static_data (struct xc_sr_context *ctx) +{ +return write_x86_pv_info (ctx); +} +static int x86_pv_start_of_stream (struct xc_sr_context *ctx) {int rc; - rc =write_x86_pv_info (ctx); -if ( rc ) -return rc; -/* * Ideally should be able to change during migration. Currently * corruption will occur if the contents or location of the P2M changes
@@ -1126,6 +1127,7 @@ struct xc_sr_save_ops save_ops_x86_pv = .pfn_to_gfn = x86_pv_pfn_to_gfn, .normalise_page = x86_pv_normalise_page, .setup = x86_pv_setup, + .static_data = x86_pv_static_data, .start_of_stream = x86_pv_start_of_stream, .start_of_checkpoint = x86_pv_start_of_checkpoint, .end_of_checkpoint = x86_pv_end_of_checkpoint,
tools/python/scripts/convert-legacy-stream +8 -1
@@ -79,7 +79,7 @@ def write_libxc_ihdr(): stream_write(pack(libxc.IHDR_FORMAT, libxc.IHDR_MARKER, # Marker libxc.IHDR_IDENT, # Ident - 2, # Version + 3, # Version libxc.IHDR_OPT_LE, # Options 0, 0)) # Reserved
@@ -166,6 +166,9 @@ def write_libxc_hvm_params(params): pack(libxc.HVM_PARAMS_FORMAT, len(params) / 2, 0), pack("Q" * len(params), *params)) +def write_libxc_static_data_end(): + write_record(libxc.REC_TYPE_static_data_end) + def write_libxl_end(): write_record(libxl.REC_TYPE_end)
@@ -590,6 +593,10 @@ def read_legacy_stream(vm): if pv: read_pv_extended_info(vm) + + write_libxc_static_data_end() + + if pv: read_pv_p2m_frames(vm) read_chunks(vm)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 9/17 tools/libxl: Provide a static_data_done callback for domain restore
In reply to Andrew CooperThis will be needed shortly to provide backwards compatiblity for migration streams which do not have CPUID information contained within them.
No functional change yet.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2:
* Split/rearranged from v1
--- tools/libxl/libxl_create.c | 12 ++++++++++++ tools/libxl/libxl_save_msgs_gen.pl | 1 + 2 files changed, 13 insertions(+)
tools/libxl/libxl_create.c +12 -0
@@ -1227,6 +1227,7 @@ static void domcreate_bootloader_done(libxl__egc *egc, dcs->srs.dcs = dcs;/* Restore */ + callbacks->static_data_done = libxl__srm_callout_callback_static_data_done; callbacks->restore_results = libxl__srm_callout_callback_restore_results;/* COLO only supports HVM now because it does not work very
@@ -1296,6 +1297,17 @@ static void libxl__colo_restore_setup_done(libxl__egc *egc,libxl__stream_read_start (egc, &dcs->srs); } +int libxl__srm_callout_callback_static_data_done (void *user) +{ +libxl__save_helper_state *shs = user; +libxl__domain_create_state *dcs = shs->caller_state; +STATE_AO_GC (dcs->ao); + +/* Nothing to do (yet). */ + +return 0 ; +} +void libxl__srm_callout_callback_restore_results (xen_pfn_t store_mfn,xen_pfn_t console_mfn,void *user) {
tools/libxl/libxl_save_msgs_gen.pl +1 -0
@@ -29,6 +29,7 @@ our @msgs = ( [ 'srcxA', "wait_checkpoint", [] ], [ 'scxA', "switch_qemu_logdirty", [qw(uint32_t domid unsigned enable)] ], + [ 'rcxW', "static_data_done", [] ], [ 'rcx', "restore_results", ['xen_pfn_t', 'store_gfn', 'xen_pfn_t', 'console_gfn'] ], [ 'srW', "complete", [qw(int retval
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 10/17 tools/libxl: Plumb a restore boolean down into libxl__build_pre()
In reply to Andrew CooperTo fix CPUID handling, libxl__build_pre() is going to have to distinguish between a brand new VM vs one which is being migrated-in/resumed.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2:
* New. This is c/s aacc1430064 "tools/libxl: Plumb domain_create_state down into libxl__build_pre()" take-2, without any collateral damage to stubdoms.
--- tools/libxl/libxl_create.c | 9 +++++---- tools/libxl/libxl_dm.c | 2 +- tools/libxl/libxl_dom.c | 4 +++- tools/libxl/libxl_internal.h | 6 ++++-- 4 files changed, 13 insertions(+), 8 deletions(-)
tools/libxl/libxl_create.c +5 -4
@@ -434,14 +434,15 @@ static void init_console_info(libxl__gc *gc,int libxl__domain_build (libxl__gc *gc,libxl_domain_config *d_config,uint32_t domid, -libxl__domain_build_state *state) +libxl__domain_build_state *state, +bool restore) {libxl_domain_build_info *const info = &d_config->b_info;char **vments =NULL , **localents =NULL ;struct timeval start_time;int i, ret; - ret =libxl__build_pre (gc, domid, d_config, state); + ret =libxl__build_pre (gc, domid, d_config, state, restore);if (ret) goto out;
@@ -1218,7 +1219,7 @@ static void domcreate_bootloader_done(libxl__egc *egc, dcs->sdss.callback = domcreate_devmodel_started;if (restore_fd <0 && dcs->domid_soft_reset ==INVALID_DOMID ) { - rc =libxl__domain_build (gc, d_config, domid, state); + rc =libxl__domain_build (gc, d_config, domid, state, false);domcreate_rebuild_done (egc, dcs, rc);return ; }
@@ -1245,7 +1246,7 @@ static void domcreate_bootloader_done(libxl__egc *egc, goto out; } - rc =libxl__build_pre (gc, domid, d_config, state); + rc =libxl__build_pre (gc, domid, d_config, state, restore_fd >=0 );if (rc) goto out;
tools/libxl/libxl_dm.c +1 -1
@@ -2197,7 +2197,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)if (ret) goto out;uint32_t dm_domid = sdss->pvqemu.guest_domid; - ret =libxl__domain_build (gc, dm_config, dm_domid, stubdom_state); + ret =libxl__domain_build (gc, dm_config, dm_domid, stubdom_state, false);if (ret) goto out;
tools/libxl/libxl_dom.c +3 -1
@@ -244,7 +244,9 @@ static int numa_place_domain(libxl__gc *gc, uint32_t domid, }int libxl__build_pre (libxl__gc *gc,uint32_t domid, -libxl_domain_config *d_config,libxl__domain_build_state *state) +libxl_domain_config *d_config, +libxl__domain_build_state *state, +bool restore) {libxl_domain_build_info *const info = &d_config->b_info;libxl_ctx *ctx =libxl__gc_owner (gc);
tools/libxl/libxl_internal.h +4 -2
@@ -1390,7 +1390,8 @@ _hidden void libxl__domain_build_state_dispose(libxl__domain_build_state *s); _hidden int libxl__build_pre(libxl__gc *gc, uint32_t domid,libxl_domain_config *const d_config, -libxl__domain_build_state *state); +libxl__domain_build_state *state, +bool restore); _hidden int libxl__build_post(libxl__gc *gc, uint32_t domid,libxl_domain_build_info *info,libxl__domain_build_state *state,char **vms_ents,char **local_ents);
@@ -1963,7 +1964,8 @@ _hidden int libxl__domain_make(libxl__gc *gc, _hidden int libxl__domain_build(libxl__gc *gc,libxl_domain_config *d_config,uint32_t domid, -libxl__domain_build_state *state); +libxl__domain_build_state *state, +bool restore);/* for device model creation */ _hidden const char *libxl__domain_device_model(libxl__gc *gc,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 10/17] tools/libxl: Plumb a restore boolean down into libxl__build_pre()
Andrew Cooper writes ("[PATCH v2 10/17] tools/libxl: Plumb a restore boolean down into libxl__build_pre()"):
Quoted 2 lines
To fix CPUID handling,
libxl__build_pre()is going to have to distinguish between a brand new VM vs one which is being migrated-in/resumed.
The distinction between libxl__build_pre and these other functions is not particularly principled. It is a legacy from an old API (prior to the existince of *create) where libxl callers were expected to "build" a domain first and then do other things to it.
Maybe it would be better to pass this via libxl__domain_build_state rather than as an additional parameter ?
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 10/17] tools/libxl: Plumb a restore boolean down into libxl__build_pre()
Quoted 10 lines · Ian Jackson
Andrew Cooper writes ("[PATCH v2 10/17]
tools/libxl: Plumb a restore boolean down intolibxl__build_pre()"):Quoted 2 lines
To fix CPUID handling,
libxl__build_pre()is going to have to distinguish between a brand new VM vs one which is being migrated-in/resumed.The distinction between
libxl__build_preand these other functions is not particularly principled. It is a legacy from an old API (prior to the existince of *create) where libxl callers were expected to "build" a domain first and then do other things to it.Maybe it would be better to pass this via
libxl__domain_build_staterather than as an additional parameter ?
Well - I tried a similar approach the first time around, and it broke stubdoms so badly it needed reverting.
(Untrim the commit details)
Quoted 3 lines
v2:
* New. This is c/saacc1430064"tools/libxl: Plumbdomain_create_statedown intolibxl__build_pre()" take-2, without any collateral damage to stubdoms.
The actual information we want is in libxl__domain_create_state (specifically, restore_fd >= -1). I first tried plumbing dcs down, to avoid stashing the same information in two different structures at different times.
Sadly, plumbing dcs didn't work because it is common between the real domain and the stubdom (and this lead to the stubdom getting no settings at all). What we want to do is only influence the CPUID construction of the main domain (which may be migrating in), whereas the stubdom always wants fresh settings.
I could duplicate it into dbs, and at a guess that would probably work, but isn't it taking a bad problem and making it worse?
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 10/17] tools/libxl: Plumb a restore boolean down into libxl__build_pre()
Andrew Cooper writes ("Re: [PATCH v2 10/17] tools/libxl: Plumb a restore boolean down into libxl__build_pre()"):
Quoted 18 lines · Ian Jackson
Quoted 2 lines
Maybe it would be better to pass this via
libxl__domain_build_staterather than as an additional parameter ?Well - I tried a similar approach the first time around, and it broke stubdoms so badly it needed reverting.
(Untrim the commit details)
Quoted 3 lines
v2:
* New. This is c/saacc1430064"tools/libxl: Plumbdomain_create_statedown intolibxl__build_pre()" take-2, without any collateral damage to stubdoms.The actual information we want is in
libxl__domain_create_state(specifically,restore_fd>= -1). I first tried plumbing dcs down, to avoid stashing the same information in two different structures at different times.
Quoted 5 lines
Sadly, plumbing dcs didn't work because it is common between the real domain and the stubdom (and this lead to the stubdom getting no settings at all). What we want to do is only influence the CPUID construction of the main domain (which may be migrating in), whereas the stubdom always wants fresh settings.
Right. Thanks for the explanation, which make sense to me.
Quoted 2 lines
I could duplicate it into dbs, and at a guess that would probably work, but isn't it taking a bad problem and making it worse?
I think that is fine.
Conceptually I think it's like this:
These structs take place of huge lists of parameters. When we create a service domain, we need to pass a new list of parameters (_build_state), we also pass some of the original ones (_create_state). If we are talking about parameters that need to be different for a service domain, they should be in _build_state; even if the main domain's version is effectively a copy of something in _create_state.
So I think adding a "restoring" or "restore" boolean to dbs is probably right.
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v3 10/17 tools/libxl: Plumb a restore boolean into libxl__domain_build_state
In reply to Andrew CooperTo fix CPUID handling, libxl__build_pre() is going to have to distinguish between a brand new VM vs one which is being migrated-in/resumed.
Transcribe dcs->restore_fd into dbs->restore in initiate_domain_create() only (specifically avoiding the stubdom state in libxl__spawn_stub_dm()).
While tweaking initiate_domain_create(), make a new dbs alias and simplify later code, and drop the local restore_fd alias as the new dbs->restore is more intuitive in context.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2:
* New. This is c/s aacc1430064 "tools/libxl: Plumb domain_create_state down into libxl__build_pre()" take-2, without any collateral damage to stubdoms.
v3:
* Extend libxl__domain_build_state instead of adding a new parameter to libxl__build_pre().
--- tools/libxl/libxl_create.c | 14 +++++++------- tools/libxl/libxl_internal.h | 4 ++++ 2 files changed, 11 insertions(+), 7 deletions(-)
tools/libxl/libxl_create.c +7 -7
@@ -1181,18 +1181,18 @@ static void initiate_domain_create(libxl__egc *egc,/* convenience aliases */ libxl_domain_config *const d_config = dcs->guest_config; -const int restore_fd = dcs->restore_fd; +libxl__domain_build_state *dbs = &dcs->build_state;libxl__xswait_init (&dcs->console_xswait); domid = dcs->domid; -libxl__domain_build_state_init (&dcs->build_state); +libxl__domain_build_state_init (dbs); + dbs->restore = dcs->restore_fd >=0 ; ret =libxl__domain_config_setdefault (gc,d_config,domid);if (ret) goto error_out; - ret =libxl__domain_make (gc, d_config, &dcs->build_state, &domid, - dcs->soft_reset); + ret =libxl__domain_make (gc, d_config, dbs, &domid, dcs->soft_reset);if (ret) {LOGD (ERROR , domid,"cannot make domain: %d" , ret); dcs->guest_domid = domid;
@@ -1236,7 +1236,7 @@ static void initiate_domain_create(libxl__egc *egc,if (ret) goto error_out; -if (restore_fd >=0 || dcs->soft_reset) { +if (dbs->restore || dcs->soft_reset) {LOGD (DEBUG , domid,"restoring, not running bootloader" );domcreate_bootloader_done (egc, &dcs->bl,0 ); }else {
@@ -1247,8 +1247,8 @@ static void initiate_domain_create(libxl__egc *egc, dcs->bl.disk = bootdisk; dcs->bl.domid = dcs->guest_domid; - dcs->bl.kernel = &dcs->build_state.pv_kernel; - dcs->bl.ramdisk = &dcs->build_state.pv_ramdisk; + dcs->bl.kernel = &dbs->pv_kernel; + dcs->bl.ramdisk = &dbs->pv_ramdisk;libxl__bootloader_run (egc, &dcs->bl); }
tools/libxl/libxl_internal.h +4 -0
@@ -1397,6 +1397,10 @@ typedef struct {/* ARM only to deal with broken firmware */ uint32_t clock_frequency; + +/* Whether this domain is being migrated/restored, or booting fresh. Only +* applicable to the primary domain, not support domains (e.g. stub QEMU). */ +bool restore; } libxl__domain_build_state; _hidden void libxl__domain_build_state_init(libxl__domain_build_state *s);
Signature · 1 line
2.11.0
Re: [PATCH v3 10/17] tools/libxl: Plumb a restore boolean into libxl__domain_build_state
Andrew Cooper writes ("[PATCH v3 10/17] tools/libxl: Plumb a restore boolean into libxl__domain_build_state"):
Quoted 11 lines
To fix CPUID handling,
libxl__build_pre()is going to have to distinguish between a brand new VM vs one which is being migrated-in/resumed.Transcribe
dcs->restore_fdintodbs->restoreininitiate_domain_create()only (specifically avoiding the stubdom state inlibxl__spawn_stub_dm()).While tweaking
initiate_domain_create(), make a new dbs alias and simplify later code, and drop the localrestore_fdalias as the newdbs->restoreis more intuitive in context.No functional change.
PATCH v2 11/17 tools/libxl: Re-position CPUID handling during domain construction
In reply to Andrew CooperCPUID handling needs to be earlier in construction. Move it from its current position in libxl__build_post() to libxl__build_pre() for fresh builds, and libxl__srm_callout_callback_static_data_done() for the migration/resume case.
Later changes will make the migration/resume case conditional on whether CPUID data was present in the migration stream, and the libxc layer took care of restoring it.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
--- tools/libxl/libxl_create.c | 8 +++++++- tools/libxl/libxl_dom.c | 16 ++++++++++++---- 2 files changed, 19 insertions(+), 5 deletions(-)
tools/libxl/libxl_create.c +7 -1
@@ -1303,8 +1303,14 @@ int libxl__srm_callout_callback_static_data_done(void *user)libxl__save_helper_state *shs = user;libxl__domain_create_state *dcs = shs->caller_state;STATE_AO_GC (dcs->ao); +libxl_ctx *ctx =libxl__gc_owner (gc); + +const libxl_domain_config *d_config = dcs->guest_config; +const libxl_domain_build_info *info = &d_config->b_info; -/* Nothing to do (yet). */ +libxl__cpuid_apply_policy (ctx, dcs->guest_domid); +if (info->cpuid !=NULL ) +libxl__cpuid_set (ctx, dcs->guest_domid, info->cpuid);return 0 ; }
tools/libxl/libxl_dom.c +12 -4
@@ -389,6 +389,18 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, rc =libxl__arch_domain_create (gc, d_config, domid); +/* Construct a CPUID policy, but only for brand new domains. Domains +* being migrated-in/restored have CPUID handled during the +* static_data_done() callback. */ +if (!restore) { +/* For x86 at least, libxl_cpuid_apply_policy() takes an implicit +* parameter, HVM_PARAM_PAE_ENABLED, which is only set up in +* libxl__arch_domain_create(). */ +libxl_cpuid_apply_policy (ctx, domid); +if (info->cpuid !=NULL ) +libxl_cpuid_set (ctx, domid, info->cpuid); + } +return rc; }
@@ -456,10 +468,6 @@ int libxl__build_post(libxl__gc *gc, uint32_t domid,if (rc)return rc; -libxl__cpuid_apply_policy (ctx, domid); -if (info->cpuid !=NULL ) -libxl__cpuid_set (ctx, domid, info->cpuid); -if (info->type ==LIBXL_DOMAIN_TYPE_HVM && !libxl_ms_vm_genid_is_zero (&info->u.hvm.ms_vm_genid)) { rc = libxl__ms_vm_genid_set(gc, domid,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 11/17 tools/libxl: Re-position CPUID handling during domain construction
CPUID handling needs to be earlier in construction. Move it from its current position in libxl__build_post() to libxl__build_pre() for fresh builds, and libxl__srm_callout_callback_static_data_done() for the migration/resume case.
Later changes will make the migration/resume case conditional on whether CPUID data was present in the migration stream, and the libxc layer took care of restoring it.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2.1:
* Rebase over libxl__cpuid_legacy() changes
--- tools/libxl/libxl_create.c | 6 +++++- tools/libxl/libxl_dom.c | 8 ++++++-- 2 files changed, 11 insertions(+), 3 deletions(-)
tools/libxl/libxl_create.c +5 -1
@@ -1313,8 +1313,12 @@ int libxl__srm_callout_callback_static_data_done(void *user)libxl__save_helper_state *shs = user;libxl__domain_create_state *dcs = shs->caller_state;STATE_AO_GC (dcs->ao); +libxl_ctx *ctx =libxl__gc_owner (gc); + +libxl_domain_config *d_config = dcs->guest_config; +libxl_domain_build_info *info = &d_config->b_info; -/* Nothing to do (yet). */ +libxl__cpuid_legacy (ctx, dcs->guest_domid, info);return 0 ; }
tools/libxl/libxl_dom.c +6 -2
@@ -389,6 +389,12 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, rc =libxl__arch_domain_create (gc, d_config, domid); +/* Construct a CPUID policy, but only for brand new domains. Domains +* being migrated-in/restored have CPUID handled during the +* static_data_done() callback. */ +if (!restore) +libxl__cpuid_legacy (ctx, domid, info); +return rc; }
@@ -456,8 +462,6 @@ int libxl__build_post(libxl__gc *gc, uint32_t domid,if (rc)return rc; -libxl__cpuid_legacy (ctx, domid, info); -if (info->type ==LIBXL_DOMAIN_TYPE_HVM && !libxl_ms_vm_genid_is_zero (&info->u.hvm.ms_vm_genid)) { rc = libxl__ms_vm_genid_set(gc, domid,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 12/17 docs/migration: Specify X86_{CPUID, MSR}_POLICY records
In reply to Andrew CooperThese two records move blobs from the XEN_DOMCTL_{get,set}_cpu_policy hypercall.
---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Tim Deegan <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
--- docs/specs/libxc-migration-stream.pandoc | 42 +++++++++++++++++++++++++++++++ tools/libxc/xc_sr_common.c | 2 ++ tools/libxc/xc_sr_stream_format.h | 2 ++ tools/python/xen/migration/libxc.py | 43 ++++++++++++++++++++++++++++++++ 4 files changed, 89 insertions(+)
docs/specs/libxc-migration-stream.pandoc +42 -0
@@ -634,6 +634,46 @@ The end record contains no fields; its body_length is 0. \clearpage +X86_CPUID_POLICY +---------------- + +CPUID policy content, as accessed by the XEN_DOMCTL_{get,set}_cpu_policy +hypercall sub-ops. + + 0 1 2 3 4 5 6 7 octet + +-------------------------------------------------+ + | CPUID_policy | + ... + +-------------------------------------------------+ + +-------------------------------------------------------------------- +Field Description +------------ --------------------------------------------------- +CPUID_policy Array of xen_cpuid_leaf_t[]'s +-------------------------------------------------------------------- + +\clearpage + +X86_MSR_POLICY +-------------- + +MSR policy content, as accessed by the XEN_DOMCTL_{get,set}_cpu_policy +hypercall sub-ops. + + 0 1 2 3 4 5 6 7 octet + +-------------------------------------------------+ + | MSR_policy | + ... + +-------------------------------------------------+ + +-------------------------------------------------------------------- +Field Description +---------- --------------------------------------------------- +MSR_policy Array of xen_msr_entry_t[]'s +-------------------------------------------------------------------- + +\clearpage + Layout ======
@@ -656,6 +696,7 @@ A typical save record for an x86 PV guest image would look like: * Domain header * Static data records: * X86_PV_INFO record + * X86_{CPUID,MSR}_POLICY * STATIC_DATA_END * X86_PV_P2M_FRAMES record * Many PAGE_DATA records
@@ -685,6 +726,7 @@ A typical save record for an x86 HVM guest image would look like: * Image header * Domain header * Static data records: + * X86_{CPUID,MSR}_POLICY * STATIC_DATA_END * Many PAGE_DATA records * X86_TSC_INFO
tools/libxc/xc_sr_common.c +2 -0
@@ -37,6 +37,8 @@ static const char *const mandatory_rec_types[] = [REC_TYPE_CHECKPOINT] = "Checkpoint", [REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST] = "Checkpoint dirty pfn list", [REC_TYPE_STATIC_DATA_END] = "Static data end", + [REC_TYPE_X86_CPUID_POLICY] = "x86 CPUID policy", + [REC_TYPE_X86_MSR_POLICY] = "x86 MSR policy", };const char *rec_type_to_str (uint32_t type)
tools/libxc/xc_sr_stream_format.h +2 -0
@@ -74,6 +74,8 @@ struct xc_sr_rhdr#define REC_TYPE_CHECKPOINT 0x0000000eU#define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST 0x0000000fU#define REC_TYPE_STATIC_DATA_END 0x00000010U +#define REC_TYPE_X86_CPUID_POLICY 0x00000011U +#define REC_TYPE_X86_MSR_POLICY 0x00000012U#define REC_TYPE_OPTIONAL 0x80000000U
tools/python/xen/migration/libxc.py +43 -0
71 lines not on this page · raw message
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 12/17] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
Andrew Cooper writes ("[PATCH v2 12/17] docs/migration: Specify X86_{CPUID,MSR}_POLICY records"):
Quoted 2 lines
These two records move blobs from the XEN_DOMCTL_{get,set}_cpu_policy hypercall.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 13/17 libxc/restore: Handle X86_{CPUID, MSR}_DATA records
In reply to Andrew CooperFor now, the data are just stashed, and discarded at the end.
A future change will restore the data, once libxl has been adjusted to avoid clobbering the data.
No functional change.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
--- tools/libxc/xc_sr_common.h | 10 ++++++++++ tools/libxc/xc_sr_common_x86.c | 40 +++++++++++++++++++++++++++++++++++++ tools/libxc/xc_sr_common_x86.h | 14 +++++++++++++ tools/libxc/xc_sr_restore_x86_hvm.c | 9 +++++++++ tools/libxc/xc_sr_restore_x86_pv.c | 9 +++++++++ 5 files changed, 82 insertions(+)
tools/libxc/xc_sr_common.h +10 -0
@@ -296,6 +296,16 @@ struct xc_sr_context { struct /* x86 */ { + /* Common save/restore data. */ + union + { + struct + { + /* X86_{CPUID,MSR}_DATA blobs for CPU Policy. */ + struct xc_sr_blob cpuid, msr; + } restore; + }; + struct /* x86 PV guest. */ { /* 4 or 8; 32 or 64 bit domain */
tools/libxc/xc_sr_common_x86.c +40 -0
@@ -42,6 +42,46 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)return 0 ; } +int handle_x86_cpuid_policy (struct xc_sr_context *ctx,struct xc_sr_record *rec) +{ +xc_interface *xch = ctx->xch; +int rc; + +if ( rec->length ==0 || + rec->length %sizeof (xen_cpuid_leaf_t) !=0 ) + { +ERROR ("X86_CPUID_POLICY size %u should be multiple of %zu" , + rec->length,sizeof (xen_cpuid_leaf_t)); +return -1 ; + } + + rc =update_blob (&ctx->x86.restore.cpuid, rec->data, rec->length); +if ( rc ) +ERROR ("Unable to allocate %u bytes for X86_CPUID_POLICY" , rec->length); + +return rc; +} + +int handle_x86_msr_policy (struct xc_sr_context *ctx,struct xc_sr_record *rec) +{ +xc_interface *xch = ctx->xch; +int rc; + +if ( rec->length ==0 || + rec->length %sizeof (xen_msr_entry_t) !=0 ) + { +ERROR ("X86_MSR_POLICY size %u should be multiple of %zu" , + rec->length,sizeof (xen_cpuid_leaf_t)); +return -1 ; + } + + rc =update_blob (&ctx->x86.restore.msr, rec->data, rec->length); +if ( rc ) +ERROR ("Unable to allocate %u bytes for X86_MSR_POLICY" , rec->length); + +return rc; +} +/* * Local variables: * mode: C
tools/libxc/xc_sr_common_x86.h +14 -0
@@ -14,6 +14,20 @@ int write_x86_tsc_info(struct xc_sr_context *ctx);*/ int handle_x86_tsc_info (struct xc_sr_context *ctx,struct xc_sr_record *rec); +/* +* Parses an X86_CPUID_POLICY record and stashes the content for application +* when a STATIC_DATA_END record is encountered. +*/ +int handle_x86_cpuid_policy (struct xc_sr_context *ctx, +struct xc_sr_record *rec); + +/* +* Parses an X86_MSR_POLICY record and stashes the content for application +* when a STATIC_DATA_END record is encountered. +*/ +int handle_x86_msr_policy (struct xc_sr_context *ctx, +struct xc_sr_record *rec); +#endif /* * Local variables:
tools/libxc/xc_sr_restore_x86_hvm.c +9 -0
@@ -171,6 +171,12 @@ static int x86_hvm_process_record(struct xc_sr_context *ctx,case REC_TYPE_HVM_PARAMS :return handle_hvm_params (ctx, rec); +case REC_TYPE_X86_CPUID_POLICY : +return handle_x86_cpuid_policy (ctx, rec); + +case REC_TYPE_X86_MSR_POLICY : +return handle_x86_msr_policy (ctx, rec); +default :return RECORD_NOT_PROCESSED ; }
@@ -227,6 +233,9 @@ static int x86_hvm_cleanup(struct xc_sr_context *ctx) {free (ctx->x86.hvm.restore.context.ptr); +free (ctx->x86.restore.cpuid.ptr); +free (ctx->x86.restore.msr.ptr); +return 0 ; }
tools/libxc/xc_sr_restore_x86_pv.c +9 -0
@@ -1102,6 +1102,12 @@ static int x86_pv_process_record(struct xc_sr_context *ctx,case REC_TYPE_X86_TSC_INFO :return handle_x86_tsc_info (ctx, rec); +case REC_TYPE_X86_CPUID_POLICY : +return handle_x86_cpuid_policy (ctx, rec); + +case REC_TYPE_X86_MSR_POLICY : +return handle_x86_msr_policy (ctx, rec); +default :return RECORD_NOT_PROCESSED ; }
@@ -1173,6 +1179,9 @@ static int x86_pv_cleanup(struct xc_sr_context *ctx)if ( ctx->x86.pv.m2p )munmap (ctx->x86.pv.m2p, ctx->x86.pv.nr_m2p_frames *PAGE_SIZE ); +free (ctx->x86.restore.cpuid.ptr); +free (ctx->x86.restore.msr.ptr); +return 0 ; }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 14/17 libxc/save: Write X86_{CPUID, MSR}_DATA records
In reply to Andrew CooperWith all other plumbing in place, obtain the CPU Policy from Xen and write it into the migration stream.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
--- tools/libxc/xc_sr_common_x86.c | 50 ++++++++++++++++++++++++++++++++++++++++ tools/libxc/xc_sr_common_x86.h | 6 +++++ tools/libxc/xc_sr_save_x86_hvm.c | 2 +- tools/libxc/xc_sr_save_x86_pv.c | 12 +++++++++- 4 files changed, 68 insertions(+), 2 deletions(-)
tools/libxc/xc_sr_common_x86.c +50 -0
@@ -42,6 +42,56 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)return 0 ; } +int write_x86_cpu_policy_records (struct xc_sr_context *ctx) +{ +xc_interface *xch = ctx->xch; +struct xc_sr_record cpuid = { .type =REC_TYPE_X86_CPUID_POLICY , }; +struct xc_sr_record msrs = { .type =REC_TYPE_X86_MSR_POLICY , }; +uint32_t nr_leaves =0 , nr_msrs =0 ; +int rc; + +if (xc_get_cpu_policy_size (xch, &nr_leaves, &nr_msrs) <0 ) + { +PERROR ("Unable to get CPU Policy size" ); +return -1 ; + } + + cpuid.data =malloc (nr_leaves *sizeof (xen_cpuid_leaf_t)); + msrs.data =malloc (nr_msrs *sizeof (xen_msr_entry_t)); +if ( !cpuid.data || !msrs.data ) + { +ERROR ("Cannot allocate memory for CPU Policy" ); + rc =-1 ; + goto out; + } + +if (xc_get_domain_cpu_policy (xch, ctx->domid, &nr_leaves, cpuid.data, + &nr_msrs, msrs.data) ) + { +PERROR ("Unable to get d%d CPU Policy" , ctx->domid); + rc =-1 ; + goto out; + } + + cpuid.length = nr_leaves *sizeof (xen_cpuid_leaf_t); +if ( cpuid.length ) + { + rc =write_record (ctx, &cpuid); +if ( rc ) + goto out; + } + + msrs.length = nr_msrs *sizeof (xen_msr_entry_t); +if ( msrs.length ) + rc =write_record (ctx, &msrs); + + out: +free (cpuid.data); +free (msrs.data); + +return rc; +} +int handle_x86_cpuid_policy (struct xc_sr_context *ctx,struct xc_sr_record *rec) {xc_interface *xch = ctx->xch;
tools/libxc/xc_sr_common_x86.h +6 -0
@@ -15,6 +15,12 @@ int write_x86_tsc_info(struct xc_sr_context *ctx);int handle_x86_tsc_info (struct xc_sr_context *ctx,struct xc_sr_record *rec);/* +* Obtains a domains CPU Policy from Xen, and writes X86_{CPUID,MSR}_POLICY +* records into the stream. +*/ +int write_x86_cpu_policy_records (struct xc_sr_context *ctx); + +/* * Parses an X86_CPUID_POLICY record and stashes the content for application * when a STATIC_DATA_END record is encountered. */
tools/libxc/xc_sr_save_x86_hvm.c +1 -1
@@ -172,7 +172,7 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)static int x86_hvm_static_data (struct xc_sr_context *ctx) { -return 0 ; +return write_x86_cpu_policy_records (ctx); }static int x86_hvm_start_of_stream (struct xc_sr_context *ctx)
tools/libxc/xc_sr_save_x86_pv.c +11 -1
@@ -1054,7 +1054,17 @@ static int x86_pv_setup(struct xc_sr_context *ctx)static int x86_pv_static_data (struct xc_sr_context *ctx) { -return write_x86_pv_info (ctx); +int rc; + + rc =write_x86_pv_info (ctx); +if ( rc ) +return rc; + + rc =write_x86_cpu_policy_records (ctx); +if ( rc ) +return rc; + +return 0 ; }static int x86_pv_start_of_stream (struct xc_sr_context *ctx)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: Ping [PATCH v2 14/17] libxc/save: Write X86_{CPUID,MSR}_DATA records
Quoted 128 lines · Andrew Cooper
With all other plumbing in place, obtain the CPU Policy from Xen and write it into the migration stream.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>--- tools/libxc/xc_sr_common_x86.c | 50 ++++++++++++++++++++++++++++++++++++++++ tools/libxc/xc_sr_common_x86.h | 6 +++++ tools/libxc/xc_sr_save_x86_hvm.c | 2 +- tools/libxc/xc_sr_save_x86_pv.c | 12 +++++++++- 4 files changed, 68 insertions(+), 2 deletions(-)tools/libxc/xc_sr_common_x86.c +50
-0@@ -42,6 +42,56 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)return 0 ; } +int write_x86_cpu_policy_records (struct xc_sr_context *ctx) +{ +xc_interface *xch = ctx->xch; +struct xc_sr_record cpuid = { .type =REC_TYPE_X86_CPUID_POLICY , }; +struct xc_sr_record msrs = { .type =REC_TYPE_X86_MSR_POLICY , }; +uint32_t nr_leaves =0 , nr_msrs =0 ; +int rc; + +if (xc_get_cpu_policy_size (xch, &nr_leaves, &nr_msrs) <0 ) + { +PERROR ("Unable to get CPU Policy size" ); +return -1 ; + } + + cpuid.data =malloc (nr_leaves *sizeof (xen_cpuid_leaf_t)); + msrs.data =malloc (nr_msrs *sizeof (xen_msr_entry_t)); +if ( !cpuid.data || !msrs.data ) + { +ERROR ("Cannot allocate memory for CPU Policy" ); + rc =-1 ; + goto out; + } + +if (xc_get_domain_cpu_policy (xch, ctx->domid, &nr_leaves, cpuid.data, + &nr_msrs, msrs.data) ) + { +PERROR ("Unable to get d%d CPU Policy" , ctx->domid); + rc =-1 ; + goto out; + } + + cpuid.length = nr_leaves *sizeof (xen_cpuid_leaf_t); +if ( cpuid.length ) + { + rc =write_record (ctx, &cpuid); +if ( rc ) + goto out; + } + + msrs.length = nr_msrs *sizeof (xen_msr_entry_t); +if ( msrs.length ) + rc =write_record (ctx, &msrs); + + out: +free (cpuid.data); +free (msrs.data); + +return rc; +} +int handle_x86_cpuid_policy (struct xc_sr_context *ctx,struct xc_sr_record *rec) {xc_interface *xch = ctx->xch;tools/libxc/xc_sr_common_x86.h +6
-0@@ -15,6 +15,12 @@ int write_x86_tsc_info(struct xc_sr_context *ctx);int handle_x86_tsc_info (struct xc_sr_context *ctx,struct xc_sr_record *rec);/* +* Obtains a domains CPU Policy from Xen, and writes X86_{CPUID,MSR}_POLICY +* records into the stream. +*/ +int write_x86_cpu_policy_records (struct xc_sr_context *ctx); + +/* * Parses an X86_CPUID_POLICY record and stashes the content for application * when a STATIC_DATA_END record is encountered. */ tools/libxc/xc_sr_save_x86_hvm.c +1
-1@@ -172,7 +172,7 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)static int x86_hvm_static_data (struct xc_sr_context *ctx) { -return 0 ; +return write_x86_cpu_policy_records (ctx); }static int x86_hvm_start_of_stream (struct xc_sr_context *ctx)tools/libxc/xc_sr_save_x86_pv.c +11
-1@@ -1054,7 +1054,17 @@ static int x86_pv_setup(struct xc_sr_context *ctx)static int x86_pv_static_data (struct xc_sr_context *ctx) { -return write_x86_pv_info (ctx); +int rc; + + rc =write_x86_pv_info (ctx); +if ( rc ) +return rc; + + rc =write_x86_cpu_policy_records (ctx); +if ( rc ) +return rc; + +return 0 ; }static int x86_pv_start_of_stream (struct xc_sr_context *ctx)
Re: [PATCH v2 14/17] libxc/save: Write X86_{CPUID,MSR}_DATA records
In reply to Andrew CooperAndrew Cooper writes ("[PATCH v2 14/17] libxc/save: Write X86_{CPUID,MSR}_DATA records"):
Quoted 2 lines
With all other plumbing in place, obtain the CPU Policy from Xen and write it into the migration stream.
Thanks for your earlier explanation in the thread:
In all cases with migration development, the receive side logic (previous patch) has to come before the save side logic (this patch), or the result will break bisection with the receive side choking on an unknown record type.
From the "whole series" point of view, compatibility is also the destination side discarding the data because libxl still needs its order of CPUID handling shuffling to cope.
I still would have some comments about the compatibility implications *in the commit message*; maybe an abbreviated version the text above.
Nevertheless,
PATCH v2 15/17 tools/libx[cl]: Plumb 'missing' through static_data_done() up into libxl
In reply to Andrew CooperPre Xen-4.14 streams will not contain any CPUID/MSR information. There is nothing libxc can do about this, and will have to rely on the higher level toolstack to provide backwards compatibility.
To facilitate this, extend the static_data_done() callback, highlighting the missing information, and modify libxl to use it. At the libxc level, this requires an arch-specific hook which, for now, always reports CPUID and MSR as missing. This will be adjusted in a later change.
No overall functional change - this is just plumbing.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2:
* Split/rearrange from v1
* Don't re-evalute 'k' on migrate
--- tools/libxc/include/xenguest.h | 12 ++++++++++-- tools/libxc/xc_sr_common.h | 9 +++++++++ tools/libxc/xc_sr_common_x86.c | 8 ++++++++ tools/libxc/xc_sr_common_x86.h | 5 +++++ tools/libxc/xc_sr_restore.c | 7 ++++++- tools/libxc/xc_sr_restore_x86_hvm.c | 1 + tools/libxc/xc_sr_restore_x86_pv.c | 1 + tools/libxl/libxl_create.c | 18 ++++++++++++++---- tools/libxl/libxl_save_msgs_gen.pl | 2 +- 9 files changed, 55 insertions(+), 8 deletions(-)
tools/libxc/include/xenguest.h +10 -2
@@ -139,8 +139,16 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,/* callbacks provided by xc_domain_restore */ struct restore_callbacks { -/* Called once the STATIC_DATA_END record has been received/inferred. */ -int (*static_data_done)(void *data); +/* +* Called once the STATIC_DATA_END record has been received/inferred. +* +* For compatibility with older streams, provides a list of static data +* expected to be found in the stream, which was missing. A higher level +* toolstack is responsible for providing any necessary compatibiltiy. +*/ +#define XGR_SDD_MISSING_CPUID (1 << 0) +#define XGR_SDD_MISSING_MSR (1 << 1) +int (*static_data_done)(unsigned int missing,void *data);/* Called after a new checkpoint to suspend the guest. */ int (*suspend)(void *data);
tools/libxc/xc_sr_common.h +9 -0
@@ -159,6 +159,15 @@ struct xc_sr_restore_opsint (*process_record)(struct xc_sr_context *ctx,struct xc_sr_record *rec);/** +* Perform any actions required after the static data has arrived. Called +* when the STATIC_DATA_COMPLETE record has been recieved/inferred. +* 'missing' should be filled in for any data item the higher level +* toolstack needs to provide compatiblity for. +*/ +int (*static_data_complete)(struct xc_sr_context *ctx, +unsigned int *missing); + +/** * Perform any actions required after the stream has been finished. Called * after the END record has been received. */
tools/libxc/xc_sr_common_x86.c +8 -0
@@ -132,6 +132,14 @@ int handle_x86_msr_policy(struct xc_sr_context *ctx, struct xc_sr_record *rec)return rc; } +int x86_static_data_complete (struct xc_sr_context *ctx,unsigned int *missing) +{ +/* TODO: Become conditional on there being no data in the stream. */ + *missing =XGR_SDD_MISSING_MSR |XGR_SDD_MISSING_CPUID ; + +return 0 ; +} +/* * Local variables: * mode: C
tools/libxc/xc_sr_common_x86.h +5 -0
@@ -34,6 +34,11 @@ int handle_x86_cpuid_policy(struct xc_sr_context *ctx,int handle_x86_msr_policy (struct xc_sr_context *ctx,struct xc_sr_record *rec); +/* +* Perform common x86 actions required after the static data has arrived. +*/ +int x86_static_data_complete (struct xc_sr_context *ctx,unsigned int *missing); +#endif /* * Local variables:
tools/libxc/xc_sr_restore.c +6 -1
@@ -659,6 +659,7 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)int handle_static_data_end (struct xc_sr_context *ctx) {xc_interface *xch = ctx->xch; +unsigned int missing =0 ;int rc =0 ;if ( ctx->restore.seen_static_data_end )
@@ -669,9 +670,13 @@ int handle_static_data_end(struct xc_sr_context *ctx) ctx->restore.seen_static_data_end = true; + rc = ctx->restore.ops.static_data_complete (ctx, &missing); +if ( rc ) +return rc; +if ( ctx->restore.callbacks->static_data_done && (rc = ctx->restore.callbacks->static_data_done ( - ctx->restore.callbacks->data) !=0 ) ) + missing, ctx->restore.callbacks->data) !=0 ) )ERROR ("static_data_done() callback failed: %d\n" , rc);return rc;
tools/libxc/xc_sr_restore_x86_hvm.c +1 -0
@@ -248,6 +248,7 @@ struct xc_sr_restore_ops restore_ops_x86_hvm = .localise_page = x86_hvm_localise_page, .setup = x86_hvm_setup, .process_record = x86_hvm_process_record, + .static_data_complete = x86_static_data_complete, .stream_complete = x86_hvm_stream_complete, .cleanup = x86_hvm_cleanup, };
tools/libxc/xc_sr_restore_x86_pv.c +1 -0
@@ -1194,6 +1194,7 @@ struct xc_sr_restore_ops restore_ops_x86_pv = .localise_page = x86_pv_localise_page, .setup = x86_pv_setup, .process_record = x86_pv_process_record, + .static_data_complete = x86_static_data_complete, .stream_complete = x86_pv_stream_complete, .cleanup = x86_pv_cleanup, };
tools/libxl/libxl_create.c +14 -4
@@ -1298,7 +1298,8 @@ static void libxl__colo_restore_setup_done(libxl__egc *egc,libxl__stream_read_start (egc, &dcs->srs); } -int libxl__srm_callout_callback_static_data_done (void *user) +int libxl__srm_callout_callback_static_data_done (unsigned int missing, +void *user) {libxl__save_helper_state *shs = user;libxl__domain_create_state *dcs = shs->caller_state;
@@ -1308,9 +1309,18 @@ int libxl__srm_callout_callback_static_data_done(void *user)const libxl_domain_config *d_config = dcs->guest_config;const libxl_domain_build_info *info = &d_config->b_info; -libxl__cpuid_apply_policy (ctx, dcs->guest_domid); -if (info->cpuid !=NULL ) -libxl__cpuid_set (ctx, dcs->guest_domid, info->cpuid); +/* +* CPUID/MSR information is not present in pre Xen-4.14 streams. +* +* Libxl used to always regenerate the CPUID policy from first principles +* on migrate. Continue to do so for backwards compatibility when the +* stream doesn't contain any CPUID data. +*/ +if (missing &XGR_SDD_MISSING_CPUID ) { +libxl__cpuid_apply_policy (ctx, dcs->guest_domid); +if (info->cpuid !=NULL ) +libxl__cpuid_set (ctx, dcs->guest_domid, info->cpuid); + }return 0 ; }
tools/libxl/libxl_save_msgs_gen.pl +1 -1
@@ -29,7 +29,7 @@ our @msgs = ( [ 'srcxA', "wait_checkpoint", [] ], [ 'scxA', "switch_qemu_logdirty", [qw(uint32_t domid unsigned enable)] ], - [ 'rcxW', "static_data_done", [] ], + [ 'rcxW', "static_data_done", [qw(unsigned missing)] ], [ 'rcx', "restore_results", ['xen_pfn_t', 'store_gfn', 'xen_pfn_t', 'console_gfn'] ], [ 'srW', "complete", [qw(int retval
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 15/17 tools/libx[cl]: Plumb 'missing' through static_data_done() up into libxl
Pre Xen-4.14 streams will not contain any CPUID/MSR information. There is nothing libxc can do about this, and will have to rely on the higher level toolstack to provide backwards compatibility.
To facilitate this, extend the static_data_done() callback, highlighting the missing information, and modify libxl to use it. At the libxc level, this requires an arch-specific hook which, for now, always reports CPUID and MSR as missing. This will be adjusted in a later change.
No overall functional change - this is just plumbing.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2:
* Split/rearrange from v1
* Don't re-evalute 'k' on migrate
v2.1:
* Rebase over libxl__cpuid_legacy() changes
--- tools/libxc/include/xenguest.h | 12 ++++++++++-- tools/libxc/xc_sr_common.h | 9 +++++++++ tools/libxc/xc_sr_common_x86.c | 8 ++++++++ tools/libxc/xc_sr_common_x86.h | 5 +++++ tools/libxc/xc_sr_restore.c | 7 ++++++- tools/libxc/xc_sr_restore_x86_hvm.c | 1 + tools/libxc/xc_sr_restore_x86_pv.c | 1 + tools/libxl/libxl_create.c | 13 +++++++++++-- tools/libxl/libxl_save_msgs_gen.pl | 2 +- 9 files changed, 52 insertions(+), 6 deletions(-)
tools/libxc/include/xenguest.h +10 -2
@@ -139,8 +139,16 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,/* callbacks provided by xc_domain_restore */ struct restore_callbacks { -/* Called once the STATIC_DATA_END record has been received/inferred. */ -int (*static_data_done)(void *data); +/* +* Called once the STATIC_DATA_END record has been received/inferred. +* +* For compatibility with older streams, provides a list of static data +* expected to be found in the stream, which was missing. A higher level +* toolstack is responsible for providing any necessary compatibiltiy. +*/ +#define XGR_SDD_MISSING_CPUID (1 << 0) +#define XGR_SDD_MISSING_MSR (1 << 1) +int (*static_data_done)(unsigned int missing,void *data);/* Called after a new checkpoint to suspend the guest. */ int (*suspend)(void *data);
tools/libxc/xc_sr_common.h +9 -0
@@ -159,6 +159,15 @@ struct xc_sr_restore_opsint (*process_record)(struct xc_sr_context *ctx,struct xc_sr_record *rec);/** +* Perform any actions required after the static data has arrived. Called +* when the STATIC_DATA_COMPLETE record has been recieved/inferred. +* 'missing' should be filled in for any data item the higher level +* toolstack needs to provide compatiblity for. +*/ +int (*static_data_complete)(struct xc_sr_context *ctx, +unsigned int *missing); + +/** * Perform any actions required after the stream has been finished. Called * after the END record has been received. */
tools/libxc/xc_sr_common_x86.c +8 -0
@@ -132,6 +132,14 @@ int handle_x86_msr_policy(struct xc_sr_context *ctx, struct xc_sr_record *rec)return rc; } +int x86_static_data_complete (struct xc_sr_context *ctx,unsigned int *missing) +{ +/* TODO: Become conditional on there being no data in the stream. */ + *missing =XGR_SDD_MISSING_MSR |XGR_SDD_MISSING_CPUID ; + +return 0 ; +} +/* * Local variables: * mode: C
tools/libxc/xc_sr_common_x86.h +5 -0
@@ -34,6 +34,11 @@ int handle_x86_cpuid_policy(struct xc_sr_context *ctx,int handle_x86_msr_policy (struct xc_sr_context *ctx,struct xc_sr_record *rec); +/* +* Perform common x86 actions required after the static data has arrived. +*/ +int x86_static_data_complete (struct xc_sr_context *ctx,unsigned int *missing); +#endif /* * Local variables:
tools/libxc/xc_sr_restore.c +6 -1
@@ -659,6 +659,7 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)int handle_static_data_end (struct xc_sr_context *ctx) {xc_interface *xch = ctx->xch; +unsigned int missing =0 ;int rc =0 ;if ( ctx->restore.seen_static_data_end )
@@ -669,9 +670,13 @@ int handle_static_data_end(struct xc_sr_context *ctx) ctx->restore.seen_static_data_end = true; + rc = ctx->restore.ops.static_data_complete (ctx, &missing); +if ( rc ) +return rc; +if ( ctx->restore.callbacks->static_data_done && (rc = ctx->restore.callbacks->static_data_done ( - ctx->restore.callbacks->data) !=0 ) ) + missing, ctx->restore.callbacks->data) !=0 ) )ERROR ("static_data_done() callback failed: %d\n" , rc);return rc;
tools/libxc/xc_sr_restore_x86_hvm.c +1 -0
@@ -258,6 +258,7 @@ struct xc_sr_restore_ops restore_ops_x86_hvm = .localise_page = x86_hvm_localise_page, .setup = x86_hvm_setup, .process_record = x86_hvm_process_record, + .static_data_complete = x86_static_data_complete, .stream_complete = x86_hvm_stream_complete, .cleanup = x86_hvm_cleanup, };
tools/libxc/xc_sr_restore_x86_pv.c +1 -0
@@ -1194,6 +1194,7 @@ struct xc_sr_restore_ops restore_ops_x86_pv = .localise_page = x86_pv_localise_page, .setup = x86_pv_setup, .process_record = x86_pv_process_record, + .static_data_complete = x86_static_data_complete, .stream_complete = x86_pv_stream_complete, .cleanup = x86_pv_cleanup, };
tools/libxl/libxl_create.c +11 -2
@@ -1308,7 +1308,8 @@ static void libxl__colo_restore_setup_done(libxl__egc *egc,libxl__stream_read_start (egc, &dcs->srs); } -int libxl__srm_callout_callback_static_data_done (void *user) +int libxl__srm_callout_callback_static_data_done (unsigned int missing, +void *user) {libxl__save_helper_state *shs = user;libxl__domain_create_state *dcs = shs->caller_state;
@@ -1318,7 +1319,15 @@ int libxl__srm_callout_callback_static_data_done(void *user)libxl_domain_config *d_config = dcs->guest_config;libxl_domain_build_info *info = &d_config->b_info; -libxl__cpuid_legacy (ctx, dcs->guest_domid, info); +/* +* CPUID/MSR information is not present in pre Xen-4.14 streams. +* +* Libxl used to always regenerate the CPUID policy from first principles +* on migrate. Continue to do so for backwards compatibility when the +* stream doesn't contain any CPUID data. +*/ +if (missing &XGR_SDD_MISSING_CPUID ) +libxl__cpuid_legacy (ctx, dcs->guest_domid, info);return 0 ; }
tools/libxl/libxl_save_msgs_gen.pl +1 -1
@@ -29,7 +29,7 @@ our @msgs = ( [ 'srcxA', "wait_checkpoint", [] ], [ 'scxA', "switch_qemu_logdirty", [qw(uint32_t domid unsigned enable)] ], - [ 'rcxW', "static_data_done", [] ], + [ 'rcxW', "static_data_done", [qw(unsigned missing)] ], [ 'rcx', "restore_results", ['xen_pfn_t', 'store_gfn', 'xen_pfn_t', 'console_gfn'] ], [ 'srW', "complete", [qw(int retval
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2.1 15/17] tools/libx[cl]: Plumb 'missing' through static_data_done() up into libxl
Andrew Cooper writes ("[PATCH v2.1 15/17] tools/libx[cl]: Plumb 'missing' through static_data_done() up into libxl"):
Quoted 10 lines
Pre Xen-4.14 streams will not contain any CPUID/MSR information. There is nothing libxc can do about this, and will have to rely on the higher level toolstack to provide backwards compatibility.
To facilitate this, extend the
static_data_done()callback, highlighting the missing information, and modify libxl to use it. At the libxc level, this requires an arch-specific hook which, for now, always reports CPUID and MSR as missing. This will be adjusted in a later change.No overall functional change - this is just plumbing.
Thanks for the additional explanation and the explanation on IRC. I'm now convinced that this is the best way to do it.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 16/17 tools/libxc: Restore CPUID/MSR data found in the migration stream
In reply to Andrew CooperWith all other pieces in place, it is now safe to restore the CPUID and MSR data in the migration stream, rather than discarding them and using the higher level toolstacks compatibility logic.
While this is a small patch, it has large implications for migrated/resumed domains. Most obviously, the CPU family/model/stepping data, cache/tlb/etc. will no longer change behind the guests back.
Another change is the interpretation of the Xend cpuid strings. The 'k' option is not a sensible thing to have ever supported, and 's' is how how the stream will end up behaving.
---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
--- tools/libxc/xc_cpuid_x86.c | 8 ++++---- tools/libxc/xc_sr_common_x86.c | 26 ++++++++++++++++++++++++-- 2 files changed, 28 insertions(+), 6 deletions(-)
tools/libxc/xc_cpuid_x86.c +4 -4
@@ -291,10 +291,9 @@ int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid,* '0' -> force to 0 * 'x' -> we don't care (use default) * 'k' -> pass through host value -* 's' -> pass through the first time and then keep the same value -* across save/restore and migration. +* 's' -> legacy alias for 'k' * -* For 's' and 'x' the configuration is overwritten with the value applied. +* In all cases, the returned string consists of just '0' and '1'. */ int xc_cpuid_set( xc_interface *xch, uint32_t domid, const unsigned int *input,
@@ -420,7 +419,8 @@ int xc_cpuid_set(clear_feature (31 - j, regs[i]); config_transformed[i][j] = config[i][j]; -if ( config[i][j] =='s' ) +/* All non 0/1 values get overwritten. */ +if ( (config[i][j] & ~1 ) !='0' ) config_transformed[i][j] ='0' + val; } }
tools/libxc/xc_sr_common_x86.c +24 -2
@@ -134,8 +134,30 @@ int handle_x86_msr_policy(struct xc_sr_context *ctx, struct xc_sr_record *rec)int x86_static_data_complete (struct xc_sr_context *ctx,unsigned int *missing) { -/* TODO: Become conditional on there being no data in the stream. */ - *missing =XGR_SDD_MISSING_MSR |XGR_SDD_MISSING_CPUID ; +xc_interface *xch = ctx->xch; +uint32_t nr_leaves =0 , nr_msrs =0 ; +uint32_t err_l = ~0 , err_s = ~0 , err_m = ~0 ; + +if ( ctx->x86.restore.cpuid.ptr ) + nr_leaves = ctx->x86.restore.cpuid.size /sizeof (xen_cpuid_leaf_t); +else + *missing |=XGR_SDD_MISSING_CPUID ; + +if ( ctx->x86.restore.msr.ptr ) + nr_msrs = ctx->x86.restore.msr.size /sizeof (xen_msr_entry_t); +else + *missing |=XGR_SDD_MISSING_MSR ; + +if ( (nr_leaves || nr_msrs) && +xc_set_domain_cpu_policy (xch, ctx->domid, + nr_leaves, ctx->x86.restore.cpuid.ptr, + nr_msrs, ctx->x86.restore.msr.ptr, + &err_l, &err_s, &err_m) ) + { +PERROR ("Failed to set CPUID policy: leaf %08x, subleaf %08x, msr %08x" , + err_l, err_s, err_m); +return -1 ; + }return 0 ; }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 16/17] tools/libxc: Restore CPUID/MSR data found in the migration stream
Quoted 11 lines · Andrew Cooper
With all other pieces in place, it is now safe to restore the CPUID and MSR data in the migration stream, rather than discarding them and using the higher level toolstacks compatibility logic.
While this is a small patch, it has large implications for migrated/resumed domains. Most obviously, the CPU family/model/stepping data, cache/tlb/etc. will no longer change behind the guests back.
Another change is the interpretation of the Xend cpuid strings. The 'k' option is not a sensible thing to have ever supported, and 's' is how how the stream will end up behaving.
I'm a little confused I guess - I'd have expected such a description to mean that ...
tools/libxc/xc_cpuid_x86.c int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid, · line 294
Quoted 9 lines
tools/libxc/xc_cpuid_x86.c +1
-2@@ -291,10 +291,9 @@ int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid,* '0' -> force to 0 * 'x' -> we don't care (use default) * 'k' -> pass through host value -* 's' -> pass through the first time and then keep the same value -* across save/restore and migration. +* 's' -> legacy alias for 'k'
... 's' remains documented as is, and 'k' to become a legacy alias.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 16/17] tools/libxc: Restore CPUID/MSR data found in the migration stream
Quoted 15 lines · Jan Beulich
Quoted 11 lines · Andrew Cooper
With all other pieces in place, it is now safe to restore the CPUID and MSR data in the migration stream, rather than discarding them and using the higher level toolstacks compatibility logic.
While this is a small patch, it has large implications for migrated/resumed domains. Most obviously, the CPU family/model/stepping data, cache/tlb/etc. will no longer change behind the guests back.
Another change is the interpretation of the Xend cpuid strings. The 'k' option is not a sensible thing to have ever supported, and 's' is how how the stream will end up behaving.
I'm a little confused I guess - I'd have expected such a description to mean that ...
Quoted 10 lines
tools/libxc/xc_cpuid_x86.c int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid, · line 294
Quoted 9 lines
tools/libxc/xc_cpuid_x86.c +1
-2@@ -291,10 +291,9 @@ int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid,* '0' -> force to 0 * 'x' -> we don't care (use default) * 'k' -> pass through host value -* 's' -> pass through the first time and then keep the same value -* across save/restore and migration. +* 's' -> legacy alias for 'k' ... 's' remains documented as is, and 'k' to become a legacy alias.
Given that s has never worked in the past, k is the only plausible one used by people.
As they mean the same now, we could specify it either way around, but this way round gives users less work to do.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 17/17 docs/xl.cfg: Rewrite cpuid= section
In reply to Andrew CooperThis is partly to adjust the description of 'k' and 's' seeing as they have changed, but mostly restructuring the information for clarity.
In particular, use indentation to clearly separate the areas discussing libxl format from xend format. In addition, extend the xend format section to discuss subleaf notation.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>
v2:
* New
--- docs/man/xl.cfg.5.pod.in | 74 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 53 insertions(+), 21 deletions(-)
docs/man/xl.cfg.5.pod.in +53 -21
@@ -1964,26 +1964,42 @@ This option is disabled by default. Configure the value returned when a guest executes the CPUID instruction. Two versions of config syntax are recognized: libxl and xend. -The libxl syntax is a comma separated list of key=value pairs, preceded by the -word "host". A few keys take a numerical value, all others take a single -character which describes what to do with the feature bit. - -Possible values for a single feature bit: +Both formats use a common notation for specifying a single feature bit. +Possible values are: '1' -> force the corresponding bit to 1 '0' -> force to 0 'x' -> Get a safe value (pass through and mask with the default policy) - 'k' -> pass through the host bit value - 's' -> as 'k' but preserve across save/restore and migration (not implemented) + 'k' -> pass through the host bit value (at boot only - value preserved on migrate) + 's' -> legacy alias for 'k' -Note: when specifying B<cpuid> for hypervisor leaves (0x4000xxxx major group) -only the lowest 8 bits of leaf's 0x4000xx00 EAX register are processed, the -rest are ignored (these 8 bits signify maximum number of hypervisor leaves). +B<Libxl format>: + +=over 4 + +The libxl format is a single string, starting with the word "host", and +followed by a comma separated list of key=value pairs. A few keys take a +numerical value, all others take a single character which describes what to do +with the feature bit. e.g.: + +=over 4 + +cpuid="host,tm=0,sse3=0" + +=back List of keys taking a value: + +=over 4 + apicidsize brandid clflush family localapicid maxleaf maxhvleaf model nc proccount procpkg stepping +=back + List of keys taking a character: + +=over 4 + 3dnow 3dnowext 3dnowprefetch abm acpi adx aes altmovcr8 apic arat avx avx2 avx512-4fmaps avx512-4vnniw avx512bw avx512cd avx512dq avx512er avx512f avx512ifma avx512pf avx512vbmi avx512vl bmi1 bmi2 clflushopt clfsh clwb cmov
@@ -1997,21 +2013,37 @@ ssse3 svm svm_decode svm_lbrv svm_npt svm_nrips svm_pausefilt svm_tscrate svm_vmcbclean syscall sysenter tbm tm tm2 topoext tsc tsc-deadline tsc_adjust umip vme vmx wdt x2apic xop xsave xtpr +=back + +=back + +B<Xend format>: -The xend syntax is a list of values in the form of -'leafnum:register=bitstring,register=bitstring' - "leafnum" is the requested function, - "register" is the response register to modify - "bitstring" represents all bits in the register, its length must be 32 chars. - Each successive character represent a lesser-significant bit, possible values - are listed above in the libxl section. +=over 4 -Example to hide two features from the guest: 'tm', which is bit #29 in EDX, and -'pni' (SSE3), which is bit #0 in ECX: +Xend format consists of an array of one or more strings of the form +"leaf:reg=bitstring,...". e.g. (matching the libxl example above): -xend: [ "1:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0,edx=xx0xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" ] +=over 4 -libxl: "host,tm=0,sse3=0" +cpuid=["1:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0,edx=xx0xxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ...] + +=back + +"leaf" is an integer, either decimal or hex with a "0x" prefix. e.g. to +specify something in the AMD feature leaves, use "0x80000001:ecx=...". + +Some leaves have subleaves which can be specified as "leaf,subleaf". e.g. for +the Intel structured feature leaf, use "7,0:ebx=..." + +The bitstring represents all bits in the register, its length must be 32 +chars. Each successive character represent a lesser-significant bit. + +=back + +Note: when specifying B<cpuid> for hypervisor leaves (0x4000xxxx major group) +only the lowest 8 bits of leaf's 0x4000xx00 EAX register are processed, the +rest are ignored (these 8 bits signify maximum number of hypervisor leaves). More info about the CPUID instruction can be found in the processor manuals, and on Wikipedia: L<https://en.wikipedia.org/wiki/CPUID>
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: Ping [PATCH v2 17/17] docs/xl.cfg: Rewrite cpuid= section
Quoted 125 lines · Andrew Cooper
This is partly to adjust the description of 'k' and 's' seeing as they have changed, but mostly restructuring the information for clarity.
In particular, use indentation to clearly separate the areas discussing libxl format from xend format. In addition, extend the xend format section to discuss subleaf notation.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Anthony PERARD <[email protected]>v2:
* New--- docs/man/xl.cfg.5.pod.in | 74 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 53 insertions(+), 21 deletions(-)docs/man/xl.cfg.5.pod.in +53
-21@@ -1964,26 +1964,42 @@ This option is disabled by default. Configure the value returned when a guest executes the CPUID instruction. Two versions of config syntax are recognized: libxl and xend. -The libxl syntax is a comma separated list of key=value pairs, preceded by the -word "host". A few keys take a numerical value, all others take a single -character which describes what to do with the feature bit. - -Possible values for a single feature bit: +Both formats use a common notation for specifying a single feature bit. +Possible values are: '1' -> force the corresponding bit to 1 '0' -> force to 0 'x' -> Get a safe value (pass through and mask with the default policy) - 'k' -> pass through the host bit value - 's' -> as 'k' but preserve across save/restore and migration (not implemented) + 'k' -> pass through the host bit value (at boot only - value preserved on migrate) + 's' -> legacy alias for 'k' -Note: when specifying B<cpuid> for hypervisor leaves (0x4000xxxx major group) -only the lowest 8 bits of leaf's 0x4000xx00 EAX register are processed, the -rest are ignored (these 8 bits signify maximum number of hypervisor leaves). +B<Libxl format>: + +=over 4 + +The libxl format is a single string, starting with the word "host", and +followed by a comma separated list of key=value pairs. A few keys take a +numerical value, all others take a single character which describes what to do +with the feature bit. e.g.: + +=over 4 + +cpuid="host,tm=0,sse3=0" + +=back List of keys taking a value: + +=over 4 + apicidsize brandid clflush family localapicid maxleaf maxhvleaf model nc proccount procpkg stepping +=back + List of keys taking a character: + +=over 4 + 3dnow 3dnowext 3dnowprefetch abm acpi adx aes altmovcr8 apic arat avx avx2 avx512-4fmaps avx512-4vnniw avx512bw avx512cd avx512dq avx512er avx512f avx512ifma avx512pf avx512vbmi avx512vl bmi1 bmi2 clflushopt clfsh clwb cmov@@ -1997,21 +2013,37 @@ ssse3 svm svm_decode svm_lbrv svm_npt svm_nrips svm_pausefilt svm_tscrate svm_vmcbclean syscall sysenter tbm tm tm2 topoext tsc tsc-deadline tsc_adjust umip vme vmx wdt x2apic xop xsave xtpr +=back + +=back + +B<Xend format>: -The xend syntax is a list of values in the form of -'leafnum:register=bitstring,register=bitstring' - "leafnum" is the requested function, - "register" is the response register to modify - "bitstring" represents all bits in the register, its length must be 32 chars. - Each successive character represent a lesser-significant bit, possible values - are listed above in the libxl section. +=over 4 -Example to hide two features from the guest: 'tm', which is bit #29 in EDX, and -'pni' (SSE3), which is bit #0 in ECX: +Xend format consists of an array of one or more strings of the form +"leaf:reg=bitstring,...". e.g. (matching the libxl example above): -xend: [ "1:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0,edx=xx0xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" ] +=over 4 -libxl: "host,tm=0,sse3=0" +cpuid=["1:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0,edx=xx0xxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ...] + +=back + +"leaf" is an integer, either decimal or hex with a "0x" prefix. e.g. to +specify something in the AMD feature leaves, use "0x80000001:ecx=...". + +Some leaves have subleaves which can be specified as "leaf,subleaf". e.g. for +the Intel structured feature leaf, use "7,0:ebx=..." + +The bitstring represents all bits in the register, its length must be 32 +chars. Each successive character represent a lesser-significant bit. + +=back + +Note: when specifying B<cpuid> for hypervisor leaves (0x4000xxxx major group) +only the lowest 8 bits of leaf's 0x4000xx00 EAX register are processed, the +rest are ignored (these 8 bits signify maximum number of hypervisor leaves). More info about the CPUID instruction can be found in the processor manuals, and on Wikipedia: L<https://en.wikipedia.org/wiki/CPUID>
Re: [PATCH v2 17/17] docs/xl.cfg: Rewrite cpuid= section
In reply to Andrew CooperAndrew Cooper writes ("[PATCH v2 17/17] docs/xl.cfg: Rewrite cpuid= section"):
Quoted 6 lines
This is partly to adjust the description of 'k' and 's' seeing as they have changed, but mostly restructuring the information for clarity.
In particular, use indentation to clearly separate the areas discussing libxl format from xend format. In addition, extend the xend format section to discuss subleaf notation.