domain context infrastructure
v7 cover domain context infrastructure
From: Paul Durrant <[email protected]>
The first 3 patches of the series are largely the same as v6 but patches 4 thru 9 are basically new (patch 7 being inherited from v6 but heavily re-worked).
Patch 4 is cleanup work that I noticed needed doing whilst working patch 5.
Paul Durrant (9):
xen/common: introduce a new framework for save/restore of 'domain'
context
xen/common/domctl: introduce XEN_DOMCTL_get/setdomaincontext tools/misc: add xen-domctx to present domain context docs/specs: add missing definitions to libxc-migration-stream docs / tools: specific migration v4 to include DOMAIN_CONTEXT tools/libxc: split restore handler handle_shared_info() functionality common/domain: add a domain context record for shared_info... x86/time: add a domain context record for tsc_info... tools/libxc: add DOMAIN_CONTEXT records to the migration stream...
.gitignore | 1 + docs/specs/libxc-migration-stream.pandoc | 69 ++++- tools/flask/policy/modules/xen.if | 4 +- tools/libxc/include/xenctrl.h | 5 + tools/libxc/xc_domain.c | 56 ++++ tools/libxc/xc_sr_common.c | 1 + tools/libxc/xc_sr_common.h | 3 + tools/libxc/xc_sr_common_x86.c | 20 -- tools/libxc/xc_sr_common_x86.h | 6 - tools/libxc/xc_sr_restore.c | 45 +++- tools/libxc/xc_sr_restore_x86_pv.c | 31 ++- tools/libxc/xc_sr_save.c | 52 +++- tools/libxc/xc_sr_save_x86_hvm.c | 5 - tools/libxc/xc_sr_save_x86_pv.c | 22 -- tools/libxc/xc_sr_stream_format.h | 1 + tools/misc/Makefile | 4 + tools/misc/xen-domctx.c | 290 +++++++++++++++++++++ tools/python/xen/migration/libxc.py | 2 + xen/arch/x86/time.c | 34 ++- xen/common/Makefile | 1 + xen/common/domain.c | 114 ++++++++ xen/common/domctl.c | 173 +++++++++++++ xen/common/save.c | 315 +++++++++++++++++++++++ xen/include/asm-x86/time.h | 5 +- xen/include/public/arch-arm/hvm/save.h | 5 + xen/include/public/arch-x86/hvm/save.h | 5 + xen/include/public/domctl.h | 41 +++ xen/include/public/save.h | 111 ++++++++ xen/include/xen/save.h | 170 ++++++++++++ xen/xsm/flask/hooks.c | 6 + xen/xsm/flask/policy/access_vectors | 4 + 31 files changed, 1525 insertions(+), 76 deletions(-) create mode 100644 tools/misc/xen-domctx.c create mode 100644 xen/common/save.c create mode 100644 xen/include/public/save.h create mode 100644 xen/include/xen/save.h
---
Cc: Andrew Cooper <[email protected]>
Cc: Daniel De Graaf <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: "Marek Marczykowski-Górecki" <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Volodymyr Babchuk <[email protected]>
Cc: Wei Liu <[email protected]>
Signature · 1 line
2.20.1
PATCH v7 1/9 xen/common: introduce a new framework for save/restore of 'domain' context
To allow enlightened HVM guests (i.e. those that have PV drivers) to be migrated without their co-operation it will be necessary to transfer 'PV' state such as event channel state, grant entry state, etc.
Currently there is a framework (entered via the hvm_save/load() functions) that allows a domain's 'HVM' (architectural) state to be transferred but 'PV' state is also common with pure PV guests and so this framework is not really suitable.
This patch adds the new public header and low level implementation of a new common framework, entered via the domain_save/load() functions. Subsequent patches will introduce other parts of the framework, and code that will make use of it within the current version of the libxc migration stream.
This patch also marks the HVM-only framework as deprecated in favour of the new framework.
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Volodymyr Babchuk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v7:
- Add an option to domain_load_end() to ignore unconsumed data, which will be needed by a subsequent patch
- Kept acks since the modification is very small
v4:
- Addressed further comments from Jan
v3:
- Addressed comments from Julien and Jan
- Save handlers no longer need to state entry length up-front
- Save handlers expected to deal with multiple instances internally
- Entries are now auto-padded to 8 byte boundary
v2:
- Allow multi-stage save/load to avoid the need to double-buffer
- Get rid of the masks and add an 'ignore' flag instead
- Create copy function union to preserve const save buffer
- Deprecate HVM-only framework
--- xen/common/Makefile | 1 + xen/common/save.c | 315 +++++++++++++++++++++++++ xen/include/public/arch-arm/hvm/save.h | 5 + xen/include/public/arch-x86/hvm/save.h | 5 + xen/include/public/save.h | 89 +++++++ xen/include/xen/save.h | 170 +++++++++++++ 6 files changed, 585 insertions(+) create mode 100644 xen/common/save.c create mode 100644 xen/include/public/save.h create mode 100644 xen/include/xen/save.h
xen/common/Makefile +1 -0
@@ -39,6 +39,7 @@ obj-y += radix-tree.o obj-y += rbtree.o obj-y += rcupdate.o obj-y += rwlock.o +obj-y += save.o obj-y += shutdown.o obj-y += softirq.o obj-y += sort.o
xen/common/save.c new file +315 -0
315 lines not on this page · raw message
xen/include/public/arch-arm/hvm/save.h +5 -0
@@ -26,6 +26,11 @@#ifndef __XEN_PUBLIC_HVM_SAVE_ARM_H__#define __XEN_PUBLIC_HVM_SAVE_ARM_H__ +/* +* Further use of HVM state is deprecated. New state records should only +* be added to the domain state header: public/save.h +*/ +#endif /*
xen/include/public/arch-x86/hvm/save.h +5 -0
@@ -648,6 +648,11 @@ struct hvm_msr {*/ #define HVM_SAVE_CODE_MAX 20 +/* +* Further use of HVM state is deprecated. New state records should only +* be added to the domain state header: public/save.h +*/ +#endif /* __XEN_PUBLIC_HVM_SAVE_X86_H__ */ /*
xen/include/public/save.h new file +89 -0
89 lines not on this page · raw message
xen/include/xen/save.h new file +170 -0
170 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH v7 1/9] xen/common: introduce a new framework for save/restore of 'domain' context
Quoted 3 lines · Paul Durrant
v7:
- Add an option todomain_load_end()to ignore unconsumed data, which will be needed by a subsequent patch
May I suggest to name the parameter "ignore_tail" instead of "ignore_data", as typically you don't mean to ignore all of it?
Jan
Re: [PATCH v7 1/9] xen/common: introduce a new framework for save/restore of 'domain' context
Quoted 8 lines · Jan Beulich
Quoted 3 lines · Paul Durrant
v7:
- Add an option todomain_load_end()to ignore unconsumed data, which will be needed by a subsequent patchMay I suggest to name the parameter "
ignore_tail" instead of "ignore_data", as typically you don't mean to ignore all of it?
Hmm, looking at patch 7 it's indeed all of the body which gets ignored. Not sure what the longer term expectations here are.
Jan
RE: [PATCH v7 1/9] xen/common: introduce a new framework for save/restore of 'domain' context
Quoted 16 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 14:54
To: Paul Durrant <[email protected]>
Cc: [email protected]; Paul Durrant <[email protected]>; Julien Grall <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Stefano Stabellini <[email protected]>; Wei Liu <[email protected]>; Volodymyr Babchuk <[email protected]>; Roger Pau Monné <[email protected]>
Subject: Re: [PATCH v7 1/9] xen/common: introduce a new framework for save/restore of 'domain' contextQuoted 5 lines · Jan Beulich
Quoted 3 lines · Paul Durrant
v7:
- Add an option todomain_load_end()to ignore unconsumed data, which will be needed by a subsequent patch
Quoted 7 lines
Quoted 3 lines
May I suggest to name the parameter "
ignore_tail" instead of "ignore_data", as typically you don't mean to ignore all of it?Hmm, looking at patch 7 it's indeed all of the body which gets ignored. Not sure what the longer term expectations here are.
I think it is useful to be able to consume as much or as little of a record as is needed.
Paul
PATCH v7 2/9 xen/common/domctl: introduce XEN_DOMCTL_get/setdomaincontext
In reply to Paul DurrantThese domctls provide a mechanism to get and set domain context from the toolstack.
---
Cc: Daniel De Graaf <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Stefano Stabellini <[email protected]>
v4:
- Add missing zero pad checks
v3:
- Addressed comments from Julien and Jan
- Use vmalloc() rather than xmalloc_bytes()
v2:
- drop mask parameter
- const-ify some more buffers
--- tools/flask/policy/modules/xen.if | 4 +- tools/libxc/include/xenctrl.h | 5 + tools/libxc/xc_domain.c | 56 +++++++++ xen/common/domctl.c | 173 ++++++++++++++++++++++++++++ xen/include/public/domctl.h | 41 +++++++ xen/xsm/flask/hooks.c | 6 + xen/xsm/flask/policy/access_vectors | 4 + 7 files changed, 287 insertions(+), 2 deletions(-)
tools/flask/policy/modules/xen.if +2 -2
@@ -53,7 +53,7 @@ define(`create_domain_common', ` allow $1 $2:domain2 { set_cpu_policy settsc setscheduler setclaim set_vnumainfo get_vnumainfo cacheflush psr_cmt_op psr_alloc soft_reset - resource_map get_cpu_policy }; + resource_map get_cpu_policy setcontext }; allow $1 $2:security check_context; allow $1 $2:shadow enable; allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage mmuext_op updatemp };
@@ -97,7 +97,7 @@ define(`migrate_domain_out', ` allow $1 $2:hvm { gethvmc getparam }; allow $1 $2:mmu { stat pageinfo map_read }; allow $1 $2:domain { getaddrsize getvcpucontext pause destroy }; - allow $1 $2:domain2 gettsc; + allow $1 $2:domain2 { gettsc getcontext }; allow $1 $2:shadow { enable disable logdirty }; ')
tools/libxc/include/xenctrl.h +5 -0
@@ -867,6 +867,11 @@ int xc_domain_hvm_setcontext(xc_interface *xch,uint8_t *hvm_ctxt, uint32_t size); +int xc_domain_getcontext (xc_interface *xch,uint32_t domid, +void *ctxt_buf,size_t *size); +int xc_domain_setcontext (xc_interface *xch,uint32_t domid, +const void *ctxt_buf,size_t size); +/** * This function will return guest IO ABI protocol *
tools/libxc/xc_domain.c +56 -0
62 lines not on this page · raw message
xen/common/domctl.c +173 -0
191 lines not on this page · raw message
xen/include/public/domctl.h +41 -0
59 lines not on this page · raw message
xen/xsm/flask/hooks.c +6 -0
@@ -745,6 +745,12 @@ static int flask_domctl(struct domain *d, int cmd)case XEN_DOMCTL_get_cpu_policy:return current_has_perm (d,SECCLASS_DOMAIN2 ,DOMAIN2__GET_CPU_POLICY ); +case XEN_DOMCTL_setdomaincontext: +return current_has_perm (d,SECCLASS_DOMAIN2 ,DOMAIN2__SETCONTEXT ); + +case XEN_DOMCTL_getdomaincontext: +return current_has_perm (d,SECCLASS_DOMAIN2 ,DOMAIN2__GETCONTEXT ); +default :return avc_unknown_permission ("domctl" , cmd); }
xen/xsm/flask/policy/access_vectors +4 -0
@@ -245,6 +245,10 @@ class domain2 resource_map # XEN_DOMCTL_get_cpu_policy get_cpu_policy +# XEN_DOMCTL_setdomaincontext + setcontext +# XEN_DOMCTL_getdomaincontext + getcontext } # Similar to class domain, but primarily contains domctls related to HVM domains
Signature · 1 line
2.20.1
PATCH v7 3/9 tools/misc: add xen-domctx to present domain context
In reply to Paul DurrantThis tool is analogous to 'xen-hvmctx' which presents HVM context. Subsequent patches will add 'dump' functions when new records are introduced.
---
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
NOTE: Ian requested ack from Andrew
v3:
- Re-worked to avoid copying onto stack
- Added optional typecode and instance arguments
v2:
- Change name from 'xen-ctx' to 'xen-domctx'
--- .gitignore | 1 + tools/misc/Makefile | 4 + tools/misc/xen-domctx.c | 200 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 205 insertions(+) create mode 100644 tools/misc/xen-domctx.c
.gitignore +1 -0
@@ -211,6 +211,7 @@ tools/misc/xen_cpuperf tools/misc/xen-cpuid tools/misc/xen-detect tools/misc/xen-diag +tools/misc/xen-domctx tools/misc/xen-tmem-list-parse tools/misc/xen-livepatch tools/misc/xenperf
tools/misc/Makefile +4 -0
@@ -31,6 +31,7 @@ INSTALL_SBIN += xenpm INSTALL_SBIN += xenwatchdogd INSTALL_SBIN += xen-livepatch INSTALL_SBIN += xen-diag +INSTALL_SBIN += xen-domctx INSTALL_SBIN += $(INSTALL_SBIN-y) # Everything to be installed in a private bin/
@@ -114,6 +115,9 @@ xen-livepatch: xen-livepatch.o xen-diag: xen-diag.o $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS) +xen-domctx: xen-domctx.o + $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS) + xen-lowmemd: xen-lowmemd.o $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenevtchn) $(LDLIBS_libxenctrl) $(LDLIBS_libxenstore) $(APPEND_LDFLAGS)
tools/misc/xen-domctx.c new file +200 -0
200 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH v7 4/9 docs/specs: add missing definitions to libxc-migration-stream
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
The STATIC_DATA_END, X86_CPUID_POLICY and X86_MSR_POLICY record types have sections explaining what they are but their values are not defined. Indeed their values are defined as "Reserved for future mandatory records."
Also, the spec revision is adjusted to match the migration stream version and an END record is added to the description of a 'typical save record for and x86 HVM guest.'
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Wei Liu <[email protected]>
v7:
- New in v7
--- docs/specs/libxc-migration-stream.pandoc | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
docs/specs/libxc-migration-stream.pandoc +9 -2
@@ -3,7 +3,7 @@ Andrew Cooper <<[email protected]>> Wen Congyang <<[email protected]>> Yang Hongyang <<[email protected]>> -% Revision 2 +% Revision 3 Introduction ============
@@ -227,7 +227,13 @@ type 0x00000000: END 0x0000000F: CHECKPOINT_DIRTY_PFN_LIST (Secondary -> Primary) - 0x00000010 - 0x7FFFFFFF: Reserved for future _mandatory_ + 0x00000010: STATIC_DATA_END + + 0x00000011: X86_CPUID_POLICY + + 0x00000012: X86_MSR_POLICY + + 0x00000013 - 0x7FFFFFFF: Reserved for future _mandatory_ records. 0x80000000 - 0xFFFFFFFF: Reserved for future _optional_
@@ -732,6 +738,7 @@ A typical save record for an x86 HVM guest image would look like: * X86_TSC_INFO * HVM_PARAMS * HVM_CONTEXT +* END record HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect the validity of architectural state in the context.
Signature · 1 line
2.20.1
PATCH v7 5/9 docs / tools: specific migration v4 to include DOMAIN_CONTEXT
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
A new 'domain context' framework was recently introduced to facilitate transfer of state for both PV and HVM guests. Hence this patch mandates the presence of a new DOMAIN_CONTEXT record in version 4 of the libxc migration stream.
This record will incorprate the content of the domain's 'shared_info' page and the TSC informaiton so the SHARED_INFO and TSC_INFO records are deprecated. It is intended that, in future, this record will contain state currently present in the HVM_CONTEXT record. However, for compatibility with earlier migration streams, the version 4 stream format continues to specify an HVM_CONTEXT record and XEN_DOMCTL_sethvmcontext will continue to accept all content of that record that may be present in a version 3 stream.
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Marek Marczykowski-Górecki" <[email protected]>
v7:
- New in v7
--- docs/specs/libxc-migration-stream.pandoc | 62 ++++++++++++++++++------ tools/libxc/xc_sr_common.c | 1 + tools/libxc/xc_sr_stream_format.h | 1 + tools/python/xen/migration/libxc.py | 2 + 4 files changed, 52 insertions(+), 14 deletions(-)
docs/specs/libxc-migration-stream.pandoc +48 -14
@@ -3,7 +3,7 @@ Andrew Cooper <<[email protected]>> Wen Congyang <<[email protected]>> Yang Hongyang <<[email protected]>> -% Revision 3 +% Revision 4 Introduction ============
@@ -127,7 +127,7 @@ marker 0xFFFFFFFFFFFFFFFF. id 0x58454E46 ("XENF" in ASCII). -version 0x00000003. The version of this specification. +version 0x00000004. The version of this specification. options bit 0: Endianness. 0 = little-endian, 1 = big-endian.
@@ -209,9 +209,9 @@ type 0x00000000: END 0x00000006: X86_PV_VCPU_XSAVE - 0x00000007: SHARED_INFO + 0x00000007: SHARED_INFO (deprecated) - 0x00000008: X86_TSC_INFO + 0x00000008: X86_TSC_INFO (deprecated) 0x00000009: HVM_CONTEXT
@@ -233,7 +233,9 @@ type 0x00000000: END 0x00000012: X86_MSR_POLICY - 0x00000013 - 0x7FFFFFFF: Reserved for future _mandatory_ + 0x00000013: DOMAIN_CONTEXT + + 0x00000014 - 0x7FFFFFFF: Reserved for future _mandatory_ records. 0x80000000 - 0xFFFFFFFF: Reserved for future _optional_
@@ -442,10 +444,11 @@ X86_PV_VCPU_MSRS XEN_DOMCTL_{get,set}\_vcpu_msrs \clearpage -SHARED_INFO ------------ +SHARED_INFO (deprecated) +------------------------ -The content of the Shared Info page. +The content of the Shared Info page. This is incorporated into the +DOMAIN_CONTEXT record as of specification version 4. 0 1 2 3 4 5 6 7 octet +-------------------------------------------------+
@@ -462,11 +465,12 @@ shared_info Contents of the shared info page. This record \clearpage -X86_TSC_INFO ------------- +X86_TSC_INFO (deprecated) +------------------------- Domain TSC information, as accessed by the -XEN_DOMCTL_{get,set}tscinfo hypercall sub-ops. +XEN_DOMCTL_{get,set}tscinfo hypercall sub-ops. This is incorporated into the +DOMAIN_CONTEXT record as of specification version 4. 0 1 2 3 4 5 6 7 octet +------------------------+------------------------+
@@ -680,6 +684,25 @@ MSR_policy Array of xen_msr_entry_t[]'s \clearpage +DOMAIN_CONTEXT +-------------- + +Domain context, as accessed by the +XEN_DOMCTL_{get,set}domaincontext hypercall sub-ops. + + 0 1 2 3 4 5 6 7 octet + +-------------------------------------------------+ + | dom_ctx | + ... + +-------------------------------------------------+ + +-------------------------------------------------------------------- +Field Description +----------- --------------------------------------------------- +dom_ctx The Domain Context blob from Xen. +-------------------------------------------------------------------- + +\clearpage Layout ======
@@ -706,8 +729,7 @@ A typical save record for an x86 PV guest image would look like: * STATIC_DATA_END * X86_PV_P2M_FRAMES record * Many PAGE_DATA records -* X86_TSC_INFO -* SHARED_INFO record +* DOMAIN_CONTEXT * VCPU context records for each online VCPU * X86_PV_VCPU_BASIC record * X86_PV_VCPU_EXTENDED record
@@ -735,7 +757,7 @@ A typical save record for an x86 HVM guest image would look like: * X86_{CPUID,MSR}_POLICY * STATIC_DATA_END * Many PAGE_DATA records -* X86_TSC_INFO +* DOMAIN_CONTEXT * HVM_PARAMS * HVM_CONTEXT * END record
@@ -746,6 +768,18 @@ the validity of architectural state in the context. Compatibility with older versions ================================= +v4 compat with v3 +----------------- + +A v4 stream is compatible with a v3 stream, but mandates the presence of a +DOMAIN_CONTEXT record. This incorporates context such as the content of +the domain's Shared Info page and the TSC information, hence the SHARED_INFO +and TSC_INFO records are deprecated. +It also supercedes HVM_CONTEXT and, over time, data that is currently part of +the HVM_CONTEXT blob will move to the DOMAIN_CONTEXT blob. Xen, however, will +continue to accept all defined HVM_CONTEXT records so a v4-compatible +receiver can still accept an unmodified v3 stream. + v3 compat with v2 -----------------
tools/libxc/xc_sr_common.c +1 -0
@@ -39,6 +39,7 @@ static const char *const mandatory_rec_types[] = [REC_TYPE_STATIC_DATA_END] = "Static data end", [REC_TYPE_X86_CPUID_POLICY] = "x86 CPUID policy", [REC_TYPE_X86_MSR_POLICY] = "x86 MSR policy", + [REC_TYPE_DOMAIN_CONTEXT] = "Domain context", };const char *rec_type_to_str (uint32_t type)
tools/libxc/xc_sr_stream_format.h +1 -0
@@ -76,6 +76,7 @@ struct xc_sr_rhdr#define REC_TYPE_STATIC_DATA_END 0x00000010U#define REC_TYPE_X86_CPUID_POLICY 0x00000011U#define REC_TYPE_X86_MSR_POLICY 0x00000012U +#define REC_TYPE_DOMAIN_CONTEXT 0x00000013U#define REC_TYPE_OPTIONAL 0x80000000U
tools/python/xen/migration/libxc.py +2 -0
@@ -59,6 +59,7 @@ REC_TYPE_checkpoint_dirty_pfn_list = 0x0000000f REC_TYPE_static_data_end = 0x00000010 REC_TYPE_x86_cpuid_policy = 0x00000011 REC_TYPE_x86_msr_policy = 0x00000012 +REC_TYPE_domain_context = 0x00000013 rec_type_to_str = { REC_TYPE_end : "End",
@@ -80,6 +81,7 @@ rec_type_to_str = { REC_TYPE_static_data_end : "Static data end", REC_TYPE_x86_cpuid_policy : "x86 CPUID policy", REC_TYPE_x86_msr_policy : "x86 MSR policy", + REC_TYPE_domain_context : "Domain context", } # page_data
Signature · 1 line
2.20.1
PATCH v7 6/9 tools/libxc: split restore handler handle_shared_info() functionality
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
The code is invoked when a SHARED_INFO record is processed but actually performs two functions:
[1] Copy the 'vcpu_info' and 'arch' substructures from the record into the new domain's shared_info
[2] Clear out any pending event channel state, mask all channels and also clear 'arch.pfn_to_mfn_frame_list_list'
To facilitate deprecation of the SHARED_INFO record in version 4 of the migration stream, this patch splits the functionality, leaving [1] in handle_shared_info() and adding update_shared_info() (called from x86_pv_stream_complete()) to deal with [2].
---
Cc: Ian Jackson <[email protected]>
Cc: Wei Liu <[email protected]>
v7:
- New in v7
--- tools/libxc/xc_sr_restore_x86_pv.c | 31 +++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-)
tools/libxc/xc_sr_restore_x86_pv.c +30 -1
@@ -880,7 +880,6 @@ static int handle_shared_info(struct xc_sr_context *ctx, struct xc_sr_record *rec) {xc_interface *xch = ctx->xch; -unsigned int i;int rc =-1 ;shared_info_any_t *guest_shinfo =NULL ;const shared_info_any_t *old_shinfo = rec->data;
@@ -911,6 +910,32 @@ static int handle_shared_info(struct xc_sr_context *ctx,MEMCPY_FIELD (guest_shinfo, old_shinfo, vcpu_info, ctx->x86.pv.width);MEMCPY_FIELD (guest_shinfo, old_shinfo, arch, ctx->x86.pv.width); + rc =0 ; + + err: +if ( guest_shinfo ) +munmap (guest_shinfo,PAGE_SIZE ); + +return rc; +} + +static int update_shared_info (struct xc_sr_context *ctx) +{ +xc_interface *xch = ctx->xch; +unsigned int i; +int rc =-1 ; +shared_info_any_t *guest_shinfo =NULL ; + + guest_shinfo =xc_map_foreign_range ( + xch, ctx->domid,PAGE_SIZE ,PROT_READ |PROT_WRITE , + ctx->dominfo.shared_info_frame); +if ( !guest_shinfo ) + { +PERROR ("Failed to map Shared Info at mfn %#lx" , + ctx->dominfo.shared_info_frame); + goto err; + } +SET_FIELD (guest_shinfo, arch.pfn_to_mfn_frame_list_list,0 , ctx->x86.pv.width);
@@ -1122,6 +1147,10 @@ static int x86_pv_stream_complete(struct xc_sr_context *ctx)xc_interface *xch = ctx->xch;int rc; + rc =update_shared_info (ctx); +if ( rc ) +return rc; + rc =update_vcpu_context (ctx);if ( rc )return rc;
Signature · 1 line
2.20.1
PATCH v7 7/9 common/domain: add a domain context record for shared_info...
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
... and update xen-domctx to dump some information describing the record.
NOTE: Processing of the content during restore is currently limited to
PV domains, and matches processing of the PV-only SHARED_INFO record done by libxc. All content is, however, saved such that restore processing can be modified in future without requiring a new record format.
---
Cc: Ian Jackson <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Stefano Stabellini <[email protected]>
v7:
- Only restore vcpu_info and arch sub-structures for PV domains, to match processing of SHARED_INFO in xc_sr_restore_x86_pv.c
- Use additional option to domain_load_end() to ignore the record for HVM domains
v6:
- Only save compat_shared_info buffer if has_32bit_shinfo is set
- Validate flags field in load handler
v5:
- Addressed comments from Julien
v4:
- Addressed comments from Jan
v3:
- Actually dump some of the content of shared_info
v2:
- Drop the header change to define a 'Xen' page size and instead use a variable length struct now that the framework makes this is feasible
- Guard use of 'has_32bit_shinfo' in common code with CONFIG_COMPAT
--- tools/misc/xen-domctx.c | 78 ++++++++++++++++++++++++++ xen/common/domain.c | 114 ++++++++++++++++++++++++++++++++++++++ xen/include/public/save.h | 13 ++++- 3 files changed, 204 insertions(+), 1 deletion(-)
tools/misc/xen-domctx.c +78 -0
@@ -31,6 +31,7 @@#include <errno.h> #include <xenctrl.h> +#include <xen-tools/libs.h> #include <xen/xen.h> #include <xen/domctl.h> #include <xen/save.h>
@@ -61,6 +62,82 @@ static void dump_header(void) } +static void print_binary (const char *prefix,const void *val,size_t size, +const char *suffix) +{ +printf ("%s" , prefix); + +while ( size-- ) + { +uint8_t octet = *(const uint8_t *)val++; +unsigned int i; + +for ( i =0 ; i <8 ; i++ ) + { +printf ("%u" , octet &1 ); + octet >>=1 ; + } + } + +printf ("%s" , suffix); +} + +static void dump_shared_info (void ) +{ +DOMAIN_SAVE_TYPE (SHARED_INFO) *s; +bool has_32bit_shinfo; +shared_info_any_t *info; +unsigned int i, n; + +GET_PTR (s); + has_32bit_shinfo = s->flags &DOMAIN_SAVE_32BIT_SHINFO ; + +printf (" SHARED_INFO: has_32bit_shinfo: %s buffer_size: %u\n" , + has_32bit_shinfo ?"true" :"false" , s->buffer_size); + + info = (shared_info_any_t *)s->buffer; + +#define GET_FIELD_PTR (_f) \ + (has_32bit_shinfo ? \ + (const void *)&(info->x32._f) : \ + (const void *)&(info->x64._f)) +#define GET_FIELD_SIZE (_f) \ + (has_32bit_shinfo ? sizeof(info->x32._f) : sizeof(info->x64._f)) +#define GET_FIELD (_f) \ + (has_32bit_shinfo ? info->x32._f : info->x64._f) + + n = has_32bit_shinfo ? +ARRAY_SIZE (info->x32.evtchn_pending) : +ARRAY_SIZE (info->x64.evtchn_pending); + +for ( i =0 ; i < n; i++ ) + { +const char *prefix = !i ? +" evtchn_pending: " : +" " ; + +print_binary (prefix,GET_FIELD_PTR (evtchn_pending[0 ]), +GET_FIELD_SIZE (evtchn_pending[0 ]),"\n" ); + } + +for ( i =0 ; i < n; i++ ) + { +const char *prefix = !i ? +" evtchn_mask: " : +" " ; + +print_binary (prefix,GET_FIELD_PTR (evtchn_mask[0 ]), +GET_FIELD_SIZE (evtchn_mask[0 ]),"\n" ); + } + +printf (" wc: version: %u sec: %u nsec: %u\n" , +GET_FIELD (wc_version),GET_FIELD (wc_sec),GET_FIELD (wc_nsec)); + +#undef GET_FIELD +#undef GET_FIELD_SIZE +#undef GET_FIELD_PTR +} +static void dump_end (void ) {DOMAIN_SAVE_TYPE (END) *e;
@@ -173,6 +250,7 @@ int main(int argc, char **argv)switch (desc->typecode) {case DOMAIN_SAVE_CODE (HEADER ):dump_header ();break ; +case DOMAIN_SAVE_CODE (SHARED_INFO ):dump_shared_info ();break ;case DOMAIN_SAVE_CODE (END ):dump_end ();break ;default :printf ("Unknown type %u: skipping\n" , desc->typecode);
xen/common/domain.c +114 -0
126 lines not on this page · raw message
xen/include/public/save.h +12 -1
@@ -82,7 +82,18 @@ struct domain_save_header { }; DECLARE_DOMAIN_SAVE_TYPE(HEADER, 1, struct domain_save_header); -#define DOMAIN_SAVE_CODE_MAX 1 +struct domain_shared_info_context { +uint32_t flags; + +#define DOMAIN_SAVE_32BIT_SHINFO 0x00000001 + +uint32_t buffer_size; +uint8_t buffer[XEN_FLEX_ARRAY_DIM ];/* Implementation specific size */ +}; + +DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context); + +#define DOMAIN_SAVE_CODE_MAX 2#endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */
Signature · 1 line
2.20.1
Re: [PATCH v7 7/9] common/domain: add a domain context record for shared_info...
Quoted 3 lines · Paul Durrant
v7:
- Only restorevcpu_infoand arch sub-structures for PV domains, to match processing ofSHARED_INFOinxc_sr_restore_x86_pv.c
Since you point out this original logic, ...
Quoted 59 lines
xen/common/domain.c +59
-0+static int load_shared_info(struct domain *d, struct domain_context *c) +{ + struct domain_shared_info_context ctxt; + size_t hdr_size = offsetof(typeof(ctxt), buffer); + unsigned int i; + int rc; + + rc = DOMAIN_LOAD_BEGIN(SHARED_INFO, c, &i); + if ( rc ) + return rc; + + if ( i ) /* expect only a single instance */ + return -ENXIO; + + rc = domain_load_data(c, &ctxt, hdr_size); + if ( rc ) + return rc; + + if ( ctxt.buffer_size > sizeof(shared_info_t) || + (ctxt.flags & ~DOMAIN_SAVE_32BIT_SHINFO) ) + return -EINVAL; + + if ( ctxt.flags & DOMAIN_SAVE_32BIT_SHINFO ) + { +#ifdef CONFIG_COMPAT + has_32bit_shinfo(d) = true; +#else + return -EINVAL; +#endif + } + + if ( is_pv_domain(d) ) + { + shared_info_t *shinfo = xmalloc(shared_info_t); + + rc = domain_load_data(c, shinfo, sizeof(*shinfo)); + if ( rc ) + { + xfree(shinfo); + return rc; + } + +#ifdef CONFIG_COMPAT + if ( has_32bit_shinfo(d) ) + { + memcpy(&d->shared_info->compat.vcpu_info, + &shinfo->compat.vcpu_info, + sizeof(d->shared_info->compat.vcpu_info)); + memcpy(&d->shared_info->compat.arch, + &shinfo->compat.arch, + sizeof(d->shared_info->compat.vcpu_info)); + } + else + { + memcpy(&d->shared_info->native.vcpu_info, + &shinfo->native.vcpu_info, + sizeof(d->shared_info->native.vcpu_info)); + memcpy(&d->shared_info->native.arch, + &shinfo->native.arch,
xen/common/domain.c int continue_hypercall_on_cpu( · line 1762
Quoted 10 lines
xen/common/domain.c +10
-0+ sizeof(d->shared_info->native.arch)); + } +#else +memcpy (&d->shared_info->vcpu_info, + &shinfo->vcpu_info, +sizeof (d->shared_info->vcpu_info)); +memcpy (&d->shared_info->arch, + &shinfo->arch, +sizeof (d->shared_info->shared)); +#endif
... where does the rest of that logic (resetting of arch.pfn_to_mfn_frame_list_list, evtchn_pending, evtchn_mask, and evtchn_pending_sel) get done? Or why is it not needed anymore?
Jan
RE: [PATCH v7 7/9] common/domain: add a domain context record for shared_info...
Quoted 82 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 14:58
To: Paul Durrant <[email protected]>
Cc: [email protected]; Paul Durrant <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Julien Grall <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH v7 7/9] common/domain: add a domain context record forshared_info...Quoted 3 lines · Paul Durrant
v7:
- Only restorevcpu_infoand arch sub-structures for PV domains, to match processing ofSHARED_INFOinxc_sr_restore_x86_pv.cSince you point out this original logic, ...
Quoted 65 lines
xen/common/domain.c +65
-0+static int load_shared_info (struct domain *d,struct domain_context *c) +{ +struct domain_shared_info_context ctxt; +size_t hdr_size = offsetof(typeof (ctxt ), buffer); +unsigned int i; +int rc; + + rc =DOMAIN_LOAD_BEGIN (SHARED_INFO , c, &i); +if ( rc ) +return rc; + +if ( i )/* expect only a single instance */ +return -ENXIO ; + + rc =domain_load_data (c, &ctxt, hdr_size); +if ( rc ) +return rc; + +if ( ctxt.buffer_size >sizeof (shared_info_t) || + (ctxt.flags & ~DOMAIN_SAVE_32BIT_SHINFO ) ) +return -EINVAL ; + +if ( ctxt.flags &DOMAIN_SAVE_32BIT_SHINFO ) + { +#ifdef CONFIG_COMPAT +has_32bit_shinfo (d) = true; +#else +return -EINVAL ; +#endif + } + +if (is_pv_domain (d) ) + { +shared_info_t *shinfo =xmalloc (shared_info_t); + + rc =domain_load_data (c, shinfo,sizeof (*shinfo)); +if ( rc ) + { +xfree (shinfo); +return rc; + } + +#ifdef CONFIG_COMPAT +if (has_32bit_shinfo (d) ) + { +memcpy (&d->shared_info->compat.vcpu_info, + &shinfo->compat.vcpu_info, +sizeof (d->shared_info->compat.vcpu_info)); +memcpy (&d->shared_info->compat.arch, + &shinfo->compat.arch, +sizeof (d->shared_info->compat.vcpu_info)); + } +else + { +memcpy (&d->shared_info->native.vcpu_info, + &shinfo->native.vcpu_info, +sizeof (d->shared_info->native.vcpu_info)); +memcpy (&d->shared_info->native.arch, + &shinfo->native.arch, +sizeof (d->shared_info->native.arch)); + } +#else +memcpy (&d->shared_info->vcpu_info, + &shinfo->vcpu_info, +sizeof (d->shared_info->vcpu_info));
Quoted 8 lines
xen/common/domain.c int continue_hypercall_on_cpu( · line 1762
Quoted 4 lines
xen/common/domain.c +4
-0+memcpy (&d->shared_info->arch, + &shinfo->arch, +sizeof (d->shared_info->shared)); +#endif ... where does the rest of that logic (resetting of arch.pfn_to_mfn_frame_list_list,
evtchn_pending,evtchn_mask, andevtchn_pending_sel) get done? Or why is it not needed anymore?
The resetting logic is still in xc_sr_restore_x86_pv.c (see patch #6). It's going to need to stay there anyway to deal with older streams so I made it common to both cases; it seems slightly separate from restoring the shared info.
Paul
Quoted 2 lines
Jan
Re: [PATCH v7 7/9] common/domain: add a domain context record for shared_info...
Quoted 84 lines · Paul Durrant
Quoted 75 lines
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 14:58Quoted 3 lines · Paul Durrant
v7:
- Only restorevcpu_infoand arch sub-structures for PV domains, to match processing ofSHARED_INFOinxc_sr_restore_x86_pv.cSince you point out this original logic, ...
Quoted 65 lines
xen/common/domain.c +65
-0+static int load_shared_info (struct domain *d,struct domain_context *c) +{ +struct domain_shared_info_context ctxt; +size_t hdr_size = offsetof(typeof (ctxt ), buffer); +unsigned int i; +int rc; + + rc =DOMAIN_LOAD_BEGIN (SHARED_INFO , c, &i); +if ( rc ) +return rc; + +if ( i )/* expect only a single instance */ +return -ENXIO ; + + rc =domain_load_data (c, &ctxt, hdr_size); +if ( rc ) +return rc; + +if ( ctxt.buffer_size >sizeof (shared_info_t) || + (ctxt.flags & ~DOMAIN_SAVE_32BIT_SHINFO ) ) +return -EINVAL ; + +if ( ctxt.flags &DOMAIN_SAVE_32BIT_SHINFO ) + { +#ifdef CONFIG_COMPAT +has_32bit_shinfo (d) = true; +#else +return -EINVAL ; +#endif + } + +if (is_pv_domain (d) ) + { +shared_info_t *shinfo =xmalloc (shared_info_t); + + rc =domain_load_data (c, shinfo,sizeof (*shinfo)); +if ( rc ) + { +xfree (shinfo); +return rc; + } + +#ifdef CONFIG_COMPAT +if (has_32bit_shinfo (d) ) + { +memcpy (&d->shared_info->compat.vcpu_info, + &shinfo->compat.vcpu_info, +sizeof (d->shared_info->compat.vcpu_info)); +memcpy (&d->shared_info->compat.arch, + &shinfo->compat.arch, +sizeof (d->shared_info->compat.vcpu_info)); + } +else + { +memcpy (&d->shared_info->native.vcpu_info, + &shinfo->native.vcpu_info, +sizeof (d->shared_info->native.vcpu_info)); +memcpy (&d->shared_info->native.arch, + &shinfo->native.arch, +sizeof (d->shared_info->native.arch)); + } +#else +memcpy (&d->shared_info->vcpu_info, + &shinfo->vcpu_info, +sizeof (d->shared_info->vcpu_info));Quoted 8 lines
xen/common/domain.c int continue_hypercall_on_cpu( · line 1762
Quoted 4 lines
xen/common/domain.c +4
-0+memcpy (&d->shared_info->arch, + &shinfo->arch, +sizeof (d->shared_info->shared)); +#endif ... where does the rest of that logic (resetting of arch.pfn_to_mfn_frame_list_list,
evtchn_pending,evtchn_mask, andevtchn_pending_sel) get done? Or why is it not needed anymore?
Quoted 4 lines
The resetting logic is still in
xc_sr_restore_x86_pv.c(see patch #6). It's going to need to stay there anyway to deal with older streams so I made it common to both cases; it seems slightly separate from restoring the shared info.
I guess I at least don't fully agree: The resetting is part of restoring, as it effectively determines which parts are restored and which parts are simply set (not truly reset, but I agree the perception may change depending on whose position you take). Hence at the very least this aspect wants clearly spelling out in the description, I think. But really I'd prefer if for old streams libxc took care of (all of) it, and if for new streams all logic lived in the hypervisor.
Jan
RE: [EXTERNAL] [PATCH v7 7/9] common/domain: add a domain context record for shared_info...
Quoted 105 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 07 September 2020 08:01
To: [email protected]
Cc: [email protected]; Durrant, Paul <[email protected]>; 'Ian Jackson' <[email protected]>; 'Wei Liu' <[email protected]>; 'Andrew Cooper' <[email protected]>; 'George Dunlap' <[email protected]>; 'Julien Grall' <[email protected]>; 'Stefano Stabellini' <[email protected]>
Subject: RE: [EXTERNAL] [PATCH v7 7/9] common/domain: add a domain context record forshared_info...CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Quoted 84 lines · Paul Durrant
Quoted 75 lines
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 14:58Quoted 3 lines · Paul Durrant
v7:
- Only restorevcpu_infoand arch sub-structures for PV domains, to match processing ofSHARED_INFOinxc_sr_restore_x86_pv.cSince you point out this original logic, ...
Quoted 65 lines
xen/common/domain.c +65
-0+static int load_shared_info (struct domain *d,struct domain_context *c) +{ +struct domain_shared_info_context ctxt; +size_t hdr_size = offsetof(typeof (ctxt ), buffer); +unsigned int i; +int rc; + + rc =DOMAIN_LOAD_BEGIN (SHARED_INFO , c, &i); +if ( rc ) +return rc; + +if ( i )/* expect only a single instance */ +return -ENXIO ; + + rc =domain_load_data (c, &ctxt, hdr_size); +if ( rc ) +return rc; + +if ( ctxt.buffer_size >sizeof (shared_info_t) || + (ctxt.flags & ~DOMAIN_SAVE_32BIT_SHINFO ) ) +return -EINVAL ; + +if ( ctxt.flags &DOMAIN_SAVE_32BIT_SHINFO ) + { +#ifdef CONFIG_COMPAT +has_32bit_shinfo (d) = true; +#else +return -EINVAL ; +#endif + } + +if (is_pv_domain (d) ) + { +shared_info_t *shinfo =xmalloc (shared_info_t); + + rc =domain_load_data (c, shinfo,sizeof (*shinfo)); +if ( rc ) + { +xfree (shinfo); +return rc; + } + +#ifdef CONFIG_COMPAT +if (has_32bit_shinfo (d) ) + { +memcpy (&d->shared_info->compat.vcpu_info, + &shinfo->compat.vcpu_info, +sizeof (d->shared_info->compat.vcpu_info)); +memcpy (&d->shared_info->compat.arch, + &shinfo->compat.arch, +sizeof (d->shared_info->compat.vcpu_info)); + } +else + { +memcpy (&d->shared_info->native.vcpu_info, + &shinfo->native.vcpu_info, +sizeof (d->shared_info->native.vcpu_info)); +memcpy (&d->shared_info->native.arch, + &shinfo->native.arch, +sizeof (d->shared_info->native.arch)); + } +#else +memcpy (&d->shared_info->vcpu_info, + &shinfo->vcpu_info, +sizeof (d->shared_info->vcpu_info));Quoted 8 lines
xen/common/domain.c int continue_hypercall_on_cpu( · line 1762
Quoted 4 lines
xen/common/domain.c +4
-0+memcpy (&d->shared_info->arch, + &shinfo->arch, +sizeof (d->shared_info->shared)); +#endif ... where does the rest of that logic (resetting of arch.pfn_to_mfn_frame_list_list,
evtchn_pending,evtchn_mask, andevtchn_pending_sel) get done? Or why is it not needed anymore?Quoted 4 lines
The resetting logic is still in
xc_sr_restore_x86_pv.c(see patch #6). It's going to need to stay there anyway to deal with older streams so I made it common to both cases; it seems slightly separate from restoring the shared info.
Quoted 8 lines
I guess I at least don't fully agree: The resetting is part of restoring, as it effectively determines which parts are restored and which parts are simply set (not truly reset, but I agree the perception may change depending on whose position you take). Hence at the very least this aspect wants clearly spelling out in the description, I think. But really I'd prefer if for old streams libxc took care of (all of) it, and if for new streams all logic lived in the hypervisor.
Andrew, do you have an opinion either way?
Paul
Quoted 1 line
Jan
RE: [PATCH v7 7/9] common/domain: add a domain context record for shared_info...
In reply to Paul DurrantQuoted 106 lines
-----Original Message-----
From: Paul Durrant <[email protected]>
Sent: 04 September 2020 18:30
To: 'Jan Beulich' <[email protected]>
Cc: [email protected]; Durrant, Paul <[email protected]>; 'Ian Jackson' <[email protected]>; 'Wei Liu' <[email protected]>; 'Andrew Cooper' <[email protected]>; 'George Dunlap' <[email protected]>; 'Julien Grall' <[email protected]>; 'Stefano Stabellini' <[email protected]>
Subject: RE: [EXTERNAL] [PATCH v7 7/9] common/domain: add a domain context record forshared_info...CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Quoted 82 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 14:58
To: Paul Durrant <[email protected]>
Cc: [email protected]; Paul Durrant <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Julien Grall <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH v7 7/9] common/domain: add a domain context record forshared_info...Quoted 3 lines · Paul Durrant
v7:
- Only restorevcpu_infoand arch sub-structures for PV domains, to match processing ofSHARED_INFOinxc_sr_restore_x86_pv.cSince you point out this original logic, ...
Quoted 65 lines
xen/common/domain.c +65
-0+static int load_shared_info (struct domain *d,struct domain_context *c) +{ +struct domain_shared_info_context ctxt; +size_t hdr_size = offsetof(typeof (ctxt ), buffer); +unsigned int i; +int rc; + + rc =DOMAIN_LOAD_BEGIN (SHARED_INFO , c, &i); +if ( rc ) +return rc; + +if ( i )/* expect only a single instance */ +return -ENXIO ; + + rc =domain_load_data (c, &ctxt, hdr_size); +if ( rc ) +return rc; + +if ( ctxt.buffer_size >sizeof (shared_info_t) || + (ctxt.flags & ~DOMAIN_SAVE_32BIT_SHINFO ) ) +return -EINVAL ; + +if ( ctxt.flags &DOMAIN_SAVE_32BIT_SHINFO ) + { +#ifdef CONFIG_COMPAT +has_32bit_shinfo (d) = true; +#else +return -EINVAL ; +#endif + } + +if (is_pv_domain (d) ) + { +shared_info_t *shinfo =xmalloc (shared_info_t); + + rc =domain_load_data (c, shinfo,sizeof (*shinfo)); +if ( rc ) + { +xfree (shinfo); +return rc; + } + +#ifdef CONFIG_COMPAT +if (has_32bit_shinfo (d) ) + { +memcpy (&d->shared_info->compat.vcpu_info, + &shinfo->compat.vcpu_info, +sizeof (d->shared_info->compat.vcpu_info)); +memcpy (&d->shared_info->compat.arch, + &shinfo->compat.arch, +sizeof (d->shared_info->compat.vcpu_info)); + } +else + { +memcpy (&d->shared_info->native.vcpu_info, + &shinfo->native.vcpu_info, +sizeof (d->shared_info->native.vcpu_info)); +memcpy (&d->shared_info->native.arch, + &shinfo->native.arch, +sizeof (d->shared_info->native.arch)); + } +#else +memcpy (&d->shared_info->vcpu_info, + &shinfo->vcpu_info, +sizeof (d->shared_info->vcpu_info));Quoted 8 lines
xen/common/domain.c int continue_hypercall_on_cpu( · line 1762
Quoted 4 lines
xen/common/domain.c +4
-0+memcpy (&d->shared_info->arch, + &shinfo->arch, +sizeof (d->shared_info->shared)); +#endif ... where does the rest of that logic (resetting of arch.pfn_to_mfn_frame_list_list,
evtchn_pending,evtchn_mask, andevtchn_pending_sel) get done? Or why is it not needed anymore?
Quoted 4 lines
The resetting logic is still in
xc_sr_restore_x86_pv.c(see patch #6). It's going to need to stay there anyway to deal with older streams so I made it common to both cases; it seems slightly separate from restoring the shared info.
Actually I've changed my mind on this one; I'll do the zeroing here and leave the old SHARED_INFO record restore code alone.
Paul
Quoted 4 lines
Paul
Quoted 2 lines
Jan
PATCH v7 8/9 x86/time: add a domain context record for tsc_info...
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
... and update xen-domctx to dump some information describing the record.
NOTE: Whilst the record definition is x86 specific, it is visible directly
in the common header as context record numbers should be unique across all architectures.
---
Cc: Ian Jackson <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v7:
- New in v7
--- tools/misc/xen-domctx.c | 12 ++++++++++++ xen/arch/x86/time.c | 34 +++++++++++++++++++++++++++++++++- xen/include/asm-x86/time.h | 5 +++-- xen/include/public/save.h | 13 ++++++++++++- 4 files changed, 60 insertions(+), 4 deletions(-)
tools/misc/xen-domctx.c +12 -0
@@ -59,9 +59,20 @@ static void dump_header(void)printf (" HEADER: magic %#x, version %u\n" , h->magic, h->version); +} + +static void dump_tsc_info (void ) +{ +DOMAIN_SAVE_TYPE (TSC_INFO) *t; +GET_PTR (t); + +printf (" TSC_INFO: mode: %u incarnation: %u\n" +" khz %u elapsed_nsec: %" PRIu64"\n" , + t->mode, t->incarnation, t->khz, t->elapsed_nsec); } +static void print_binary (const char *prefix,const void *val,size_t size,const char *suffix) {
@@ -251,6 +262,7 @@ int main(int argc, char **argv) {case DOMAIN_SAVE_CODE (HEADER ):dump_header ();break ;case DOMAIN_SAVE_CODE (SHARED_INFO ):dump_shared_info ();break ; +case DOMAIN_SAVE_CODE (TSC_INFO ):dump_tsc_info ();break ;case DOMAIN_SAVE_CODE (END ):dump_end ();break ;default :printf ("Unknown type %u: skipping\n" , desc->typecode);
xen/arch/x86/time.c +33 -1
@@ -26,6 +26,7 @@#include <xen/symbols.h> #include <xen/keyhandler.h> #include <xen/guest_access.h> +#include <xen/save.h> #include <asm/io.h> #include <asm/iocap.h> #include <asm/msr.h>
@@ -2334,7 +2335,7 @@ int host_tsc_is_safe(void)* called to collect tsc-related data only for save file or live * migrate; called after last rdtsc is done on this incarnation */ -void tsc_get_info (struct domain *d,uint32_t *tsc_mode, +void tsc_get_info (const struct domain *d,uint32_t *tsc_mode,uint64_t *elapsed_nsec,uint32_t *gtsc_khz,uint32_t *incarnation) {
@@ -2453,6 +2454,37 @@ int tsc_set_info(struct domain *d,return 0 ; } +static int save_tsc_info (const struct domain *d,struct domain_context *c, +bool dry_run) +{ +struct domain_tsc_info_context ctxt; + +if ( !dry_run ) +tsc_get_info (d, &ctxt.mode, &ctxt.elapsed_nsec, &ctxt.khz, + &ctxt.incarnation); + +return DOMAIN_SAVE_ENTRY (TSC_INFO , c,0 , &ctxt,sizeof (ctxt)); +} + +static int load_tsc_info (struct domain *d,struct domain_context *c) +{ +struct domain_tsc_info_context ctxt; +unsigned int i; +int rc; + + rc =DOMAIN_LOAD_ENTRY (TSC_INFO , c, &i, &ctxt,sizeof (ctxt)); +if ( rc ) +return rc; + +if ( i )/* expect only a single instance */ +return -ENXIO ; + +return tsc_set_info (d, ctxt.mode, ctxt.elapsed_nsec, ctxt.khz, + ctxt.incarnation); +} + +DOMAIN_REGISTER_SAVE_LOAD (TSC_INFO , save_tsc_info, load_tsc_info); +/* vtsc may incur measurable performance degradation, diagnose with this */ static void dump_softtsc (unsigned char key) {
xen/include/asm-x86/time.h +3 -2
@@ -59,8 +59,9 @@ u64 gtsc_to_gtime(struct domain *d, u64 tsc);int tsc_set_info (struct domain *d,uint32_t tsc_mode,uint64_t elapsed_nsec,uint32_t gtsc_khz,uint32_t incarnation); -void tsc_get_info (struct domain *d,uint32_t *tsc_mode,uint64_t *elapsed_nsec, -uint32_t *gtsc_khz,uint32_t *incarnation); +void tsc_get_info (const struct domain *d,uint32_t *tsc_mode, +uint64_t *elapsed_nsec,uint32_t *gtsc_khz, +uint32_t *incarnation);void force_update_vcpu_system_time (struct vcpu *v);
xen/include/public/save.h +12 -1
@@ -93,7 +93,18 @@ struct domain_shared_info_context { DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context); -#define DOMAIN_SAVE_CODE_MAX 2 +#if defined(__i386__) || defined(__x86_64__) +struct domain_tsc_info_context { +uint32_t mode; +uint32_t incarnation; +uint64_t elapsed_nsec; +uint32_t khz; +}; + +DECLARE_DOMAIN_SAVE_TYPE(TSC_INFO, 3, struct domain_tsc_info_context); +#endif + +#define DOMAIN_SAVE_CODE_MAX 3#endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */
Signature · 1 line
2.20.1
Re: [PATCH v7 8/9] x86/time: add a domain context record for tsc_info...
Quoted 4 lines · Paul Durrant
xen/include/public/save.h +0
-0@@ -93,7 +93,18 @@ struct domain_shared_info_context {
xen/include/public/save.h struct domain_shared_info_context { · line 102
Quoted 10 lines
xen/include/public/save.h +7
-1DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context); -#define DOMAIN_SAVE_CODE_MAX 2 +#if defined(__i386__) || defined(__x86_64__) +struct domain_tsc_info_context { +uint32_t mode; +uint32_t incarnation; +uint64_t elapsed_nsec; +uint32_t khz; +};
sizeof() for this struct varies between 32-bit and 64-bit - is this not a problem? (alignof() varies too, but there I think it's indeed not a problem, albeit it could still be taken care of by using uint64_aligned_t, alongside the addition of an explicit padding field).
Jan
RE: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record for tsc_info...
Quoted 22 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 15:03
To: Paul Durrant <[email protected]>
Cc: [email protected]; Durrant, Paul <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Julien Grall <[email protected]>; Stefano Stabellini <[email protected]>; Roger Pau Monné <[email protected]>
Subject: RE: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record fortsc_info...CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Quoted 6 lines · Paul Durrant
xen/include/public/save.h +0
-0@@ -93,7 +93,18 @@ struct domain_shared_info_context { DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context);
Quoted 14 lines
xen/include/public/save.h struct domain_shared_info_context { · line 102
Quoted 8 lines
xen/include/public/save.h +7
-1-#define DOMAIN_SAVE_CODE_MAX 2 +#if defined(__i386__) || defined(__x86_64__) +struct domain_tsc_info_context { +uint32_t mode; +uint32_t incarnation; +uint64_t elapsed_nsec; +uint32_t khz; +};
sizeof()for this struct varies between 32-bit and 64-bit - is this not a problem? (alignof()varies too, but there I think it's indeed not a problem, albeit it could still be taken care of by usinguint64_aligned_t, alongside the addition of an explicit padding field).
I don't think it should matter because domain context records have implicit padding to align up to the next 64-bit boundary, so as long as fields within the struct don't move (which I think is true in this case) then we should be ok.
Paul
Quoted 2 lines
Jan
Re: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record for tsc_info...
Quoted 37 lines · Paul Durrant
Quoted 22 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 15:03
To: Paul Durrant <[email protected]>
Cc: [email protected]; Durrant, Paul <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Julien Grall <[email protected]>; Stefano Stabellini <[email protected]>; Roger Pau Monné <[email protected]>
Subject: RE: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record fortsc_info...CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Quoted 6 lines · Paul Durrant
xen/include/public/save.h +0
-0@@ -93,7 +93,18 @@ struct domain_shared_info_context { DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context);Quoted 14 lines
xen/include/public/save.h struct domain_shared_info_context { · line 102
Quoted 8 lines
xen/include/public/save.h +7
-1-#define DOMAIN_SAVE_CODE_MAX 2 +#if defined(__i386__) || defined(__x86_64__) +struct domain_tsc_info_context { +uint32_t mode; +uint32_t incarnation; +uint64_t elapsed_nsec; +uint32_t khz; +};
sizeof()for this struct varies between 32-bit and 64-bit - is this not a problem? (alignof()varies too, but there I think it's indeed not a problem, albeit it could still be taken care of by usinguint64_aligned_t, alongside the addition of an explicit padding field).
Quoted 2 lines
I don't think it should matter because domain context records have implicit padding to align up to the next 64-bit boundary,
Could you remind me where this is written down and enforced?
Quoted 2 lines
so as long as fields within the struct don't move (which I think is true in this case) then we should be ok.
Right.
Jan
RE: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record for tsc_info...
Quoted 49 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 28 August 2020 16:53
To: [email protected]
Cc: [email protected]; 'Ian Jackson' <[email protected]>; 'Wei Liu' <[email protected]>; 'Andrew Cooper' <[email protected]>; 'George Dunlap' <[email protected]>; 'Julien Grall' <[email protected]>; 'Stefano Stabellini' <[email protected]>; 'Roger Pau Monné' <[email protected]>
Subject: Re: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record fortsc_info...Quoted 6 lines · Paul Durrant
Quoted 6 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 15:03
To: Paul Durrant <[email protected]>
Cc: [email protected]; Durrant, Paul <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Andrew Cooper <[email protected]>;George
Quoted 31 lines
Quoted 30 lines
Dunlap <[email protected]>; Julien Grall <[email protected]>; Stefano Stabellini <[email protected]>; Roger Pau Monné <[email protected]>
Subject: RE: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record fortsc_info...CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Quoted 4 lines · Paul Durrant
xen/include/public/save.h +0
-0@@ -93,7 +93,18 @@ struct domain_shared_info_context {xen/include/public/save.h struct domain_shared_info_context { · line 102
Quoted 10 lines
xen/include/public/save.h +7
-1DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context); -#define DOMAIN_SAVE_CODE_MAX 2 +#if defined(__i386__) || defined(__x86_64__) +struct domain_tsc_info_context { +uint32_t mode; +uint32_t incarnation; +uint64_t elapsed_nsec; +uint32_t khz; +};
sizeof()for this struct varies between 32-bit and 64-bit - is this not a problem? (alignof()varies too, but there I think it's indeed not a problem, albeit it could still be taken care of by usinguint64_aligned_t, alongside the addition of an explicit padding field).
Quoted 5 lines
Quoted 2 lines
I don't think it should matter because domain context records have implicit padding to align up to the next 64-bit boundary,
Could you remind me where this is written down and enforced?
With the series fully applied, see xen/include/public/save.h line 62-68 for the comment and then see domain_save_end() in xen/common/save.c for where the padding is applied.
Paul
Quoted 6 lines
Quoted 2 lines
so as long as fields within the struct don't move (which I think is true in this case) then we should be ok.
Right.
Jan
Jan Beulich
Reviewed-byRe: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record for tsc_info...
Quoted 52 lines · Paul Durrant
Quoted 52 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 28 August 2020 16:53
To: [email protected]
Cc: [email protected]; 'Ian Jackson' <[email protected]>; 'Wei Liu' <[email protected]>; 'Andrew Cooper' <[email protected]>; 'George Dunlap' <[email protected]>; 'Julien Grall' <[email protected]>; 'Stefano Stabellini' <[email protected]>; 'Roger Pau Monné' <[email protected]>
Subject: Re: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record fortsc_info...Quoted 6 lines · Paul Durrant
Quoted 6 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 26 August 2020 15:03
To: Paul Durrant <[email protected]>
Cc: [email protected]; Durrant, Paul <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Andrew Cooper <[email protected]>;George
Quoted 33 lines
Quoted 16 lines
Dunlap <[email protected]>; Julien Grall <[email protected]>; Stefano Stabellini <[email protected]>; Roger Pau Monné <[email protected]>
Subject: RE: [EXTERNAL] [PATCH v7 8/9] x86/time: add a domain context record fortsc_info...CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Quoted 6 lines · Paul Durrant
xen/include/public/save.h +0
-0@@ -93,7 +93,18 @@ struct domain_shared_info_context { DECLARE_DOMAIN_SAVE_TYPE(SHARED_INFO, 2, struct domain_shared_info_context);Quoted 14 lines
xen/include/public/save.h struct domain_shared_info_context { · line 102
Quoted 8 lines
xen/include/public/save.h +7
-1-#define DOMAIN_SAVE_CODE_MAX 2 +#if defined(__i386__) || defined(__x86_64__) +struct domain_tsc_info_context { +uint32_t mode; +uint32_t incarnation; +uint64_t elapsed_nsec; +uint32_t khz; +};
sizeof()for this struct varies between 32-bit and 64-bit - is this not a problem? (alignof()varies too, but there I think it's indeed not a problem, albeit it could still be taken care of by usinguint64_aligned_t, alongside the addition of an explicit padding field).I don't think it should matter because domain context records have implicit padding to align up to the next 64-bit boundary,
Quoted 6 lines
Quoted 2 lines
Could you remind me where this is written down and enforced?
With the series fully applied, see
xen/include/public/save.hline 62-68 for the comment and then seedomain_save_end()inxen/common/save.cfor where the padding is applied.
Ah, yes, this helped find the places in the patches. Therefore with the stray blank line addition removed from tools/misc/xen-domctx.c
Jan
PATCH v7 9/9 tools/libxc: add DOMAIN_CONTEXT records to the migration stream...
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
... and bump the version.
This patch implements version 4 of the migration stream by adding the code necessary to save and restore DOMAIN_CONTEXT records, and removing the code to save the SHARED_INFO and TSC_INFO records (as these are deprecated in version 4).
---
Cc: Ian Jackson <[email protected]>
Cc: Wei Liu <[email protected]>
v7:
- New in v7
--- tools/libxc/xc_sr_common.h | 3 ++ tools/libxc/xc_sr_common_x86.c | 20 ------------ tools/libxc/xc_sr_common_x86.h | 6 ---- tools/libxc/xc_sr_restore.c | 45 +++++++++++++++++++++++++-- tools/libxc/xc_sr_save.c | 52 +++++++++++++++++++++++++++++++- tools/libxc/xc_sr_save_x86_hvm.c | 5 --- tools/libxc/xc_sr_save_x86_pv.c | 22 -------------- 7 files changed, 97 insertions(+), 56 deletions(-)
tools/libxc/xc_sr_common.h +3 -0
@@ -298,6 +298,9 @@ struct xc_sr_context/* Sender has invoked verify mode on the stream. */ bool verify; + +/* Domain context blob. */ +struct xc_sr_blob context; } restore; };
tools/libxc/xc_sr_common_x86.c +0 -20
@@ -1,25 +1,5 @@#include "xc_sr_common_x86.h" -int write_x86_tsc_info (struct xc_sr_context *ctx) -{ -xc_interface *xch = ctx->xch; -struct xc_sr_rec_x86_tsc_info tsc = {}; -struct xc_sr_record rec = { - .type =REC_TYPE_X86_TSC_INFO , - .length =sizeof (tsc), - .data = &tsc, - }; - -if (xc_domain_get_tsc_info (xch, ctx->domid, &tsc.mode, - &tsc.nsec, &tsc.khz, &tsc.incarnation) <0 ) - { -PERROR ("Unable to obtain TSC information" ); -return -1 ; - } - -return write_record (ctx, &rec); -} -int handle_x86_tsc_info (struct xc_sr_context *ctx,struct xc_sr_record *rec) {xc_interface *xch = ctx->xch;
tools/libxc/xc_sr_common_x86.h +0 -6
@@ -3,12 +3,6 @@#include "xc_sr_common.h" -/* -* Obtains a domains TSC information from Xen and writes a X86_TSC_INFO record -* into the stream. -*/ -int write_x86_tsc_info (struct xc_sr_context *ctx); -/* * Parses a X86_TSC_INFO record and applies the result to the domain. */
tools/libxc/xc_sr_restore.c +43 -2
82 lines not on this page · raw message
tools/libxc/xc_sr_save.c +51 -1
70 lines not on this page · raw message
tools/libxc/xc_sr_save_x86_hvm.c +0 -5
@@ -193,11 +193,6 @@ static int x86_hvm_end_of_checkpoint(struct xc_sr_context *ctx) {int rc; -/* Write the TSC record. */ - rc =write_x86_tsc_info (ctx); -if ( rc ) -return rc; -/* Write the HVM_CONTEXT record. */ rc =write_hvm_context (ctx);if ( rc )
tools/libxc/xc_sr_save_x86_pv.c +0 -22
@@ -849,20 +849,6 @@ static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx)return rc; } -/* -* Writes an SHARED_INFO record into the stream. -*/ -static int write_shared_info (struct xc_sr_context *ctx) -{ -struct xc_sr_record rec = { - .type =REC_TYPE_SHARED_INFO , - .length =PAGE_SIZE , - .data = ctx->x86.pv.shinfo, - }; - -return write_record (ctx, &rec); -} -/* * Normalise a pagetable for the migration stream. Performs mfn->pfn * conversions on the ptes.
@@ -1093,14 +1079,6 @@ static int x86_pv_end_of_checkpoint(struct xc_sr_context *ctx) {int rc; - rc =write_x86_tsc_info (ctx); -if ( rc ) -return rc; - - rc =write_shared_info (ctx); -if ( rc ) -return rc; - rc =write_all_vcpu_information (ctx);if ( rc )return rc;
Signature · 1 line
2.20.1