VM forking
v5 cover VM forking
The following series implements VM forking for Intel HVM guests to allow for the fast creation of identical VMs without the assosciated high startup costs of booting or restoring the VM from a savefile.
JIRA issue: https://xenproject.atlassian.net/browse/XEN-89
The fork operation is implemented as part of the "xl fork-vm" command:
xl fork-vm -C <config_file_for_fork> -Q <qemu_save_file> <parent_domid>
By default a fully functional fork is created. The user is in charge however to create the appropriate config file for the fork and to generate the QEMU save file before the fork-vm call is made. The config file needs to give the fork a new name at minimum but other settings may also require changes.
The interface also allows to split the forking into two steps:
xl fork-vm --launch-dm no \
-p <parent_domid>
xl fork-vm --launch-dm late \
-C <config_file_for_fork> \
-Q <qemu_save_file> \
<fork_domid>The split creation model is useful when the VM needs to be created as fast as possible. The forked VM can be unpaused without the device model being launched to be monitored and accessed via VMI. Note however that without its device model running (depending on what is executing in the VM) it is bound to misbehave or even crash when its trying to access devices that would be emulated by QEMU. We anticipate that for certain use-cases this would be an acceptable situation, in case for example when fuzzing is performed of code segments that don't access such devices.
Launching the device model requires the QEMU Xen savefile to be generated manually from the parent VM. This can be accomplished simply by connecting to its QMP socket and issuing the "xen-save-devices-state" command. For example using the standard tool socat these commands can be used to generate the file:
socat - UNIX-CONNECT:/var/run/xen/qmp-libxl-<parent_domid>
{ "execute": "qmp_capabilities" }
{ "execute": "xen-save-devices-state", \
"arguments": { "filename": "/path/to/save/qemu_state", \
"live": false} }At runtime the forked VM starts running with an empty p2m which gets lazily populated when the VM generates EPT faults, similar to how altp2m views are populated. If the memory access is a read-only access, the p2m entry is populated with a memory shared entry with its parent. For write memory accesses or in case memory sharing wasn't possible (for example in case a reference is held by a third party), a new page is allocated and the page contents are copied over from the parent VM. Forks can be further forked if needed, thus allowing for further memory savings.
A VM fork reset hypercall is also added that allows the fork to be reset to the state it was just after a fork, also accessible via xl:
xl fork-vm --fork-reset -p <fork_domid>
This is an optimization for cases where the forks are very short-lived and run without a device model, so resetting saves some time compared to creating a brand new fork provided the fork has not aquired a lot of memory. If the fork has a lot of memory deduplicated it is likely going to be faster to create a new fork from scratch and asynchronously destroying the old one.
The series has been tested with both Linux and Windows VMs and functions as expected. VM forking time has been measured to be 0.0007s, device model launch to be around 1s depending largely on the number of devices being emulated. Fork resets have been measured to be 0.0001s under the optimal circumstances.
Patches 1-2 implement changes to existing internal Xen APIs to make VM forking possible.
Patches 3-14 are code-cleanups and adjustments of to Xen memory sharing subsystem with no functional changes.
Patch 15 adds the hypervisor-side code implementing VM forking.
Patch 16 is integration of mem_access with forked VMs.
Patch 17 implements the VM fork reset operation hypervisor side bits.
Patch 18 adds the toolstack-side code implementing VM forking and reset.
New in v5: Patch 3 and 14.
Paches without Acks or Reviews:
1, 2, 3, 11, 12, 14, 15, 16, 17, 18
Tamas K Lengyel (18):
x86/hvm: introduce hvm_copy_context_and_params xen/x86: Make hap_get_allocation accessible
x86/p2m: Allow p2m_get_page_from_gfn to return shared entries x86/mem_sharing: make get_two_gfns take locks conditionally x86/mem_sharing: drop flags from mem_sharing_unshare_page x86/mem_sharing: don't try to unshare twice during page fault x86/mem_sharing: define mem_sharing_domain to hold some scattered
variables
x86/mem_sharing: Use INVALID_MFN and p2m_is_shared in
relinquish_shared_pages
x86/mem_sharing: Make add_to_physmap static and shorten name x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool x86/mem_sharing: Replace MEM_SHARING_DEBUG with gdprintk x86/mem_sharing: Enable mem_sharing on first memop x86/mem_sharing: Skip xen heap pages in memshr nominate x86/mem_sharing: use default_access in add_to_physmap xen/mem_sharing: VM forking
xen/mem_access: Use __get_gfn_type_access in set_mem_access x86/mem_sharing: reset a fork
xen/tools: VM forking toolstack side
docs/man/xl.1.pod.in | 36 +++ tools/libxc/include/xenctrl.h | 13 + tools/libxc/xc_memshr.c | 22 ++ tools/libxl/libxl.h | 7 + tools/libxl/libxl_create.c | 237 ++++++++++----- tools/libxl/libxl_dm.c | 2 +- tools/libxl/libxl_dom.c | 83 ++++-- tools/libxl/libxl_internal.h | 1 + tools/libxl/libxl_types.idl | 1 + tools/xl/xl.h | 5 + tools/xl/xl_cmdtable.c | 12 + tools/xl/xl_saverestore.c | 96 ++++++ tools/xl/xl_vmcontrol.c | 8 + xen/arch/x86/domain.c | 9 + xen/arch/x86/hvm/hvm.c | 267 ++++++++++------- xen/arch/x86/mm/hap/hap.c | 3 +- xen/arch/x86/mm/mem_access.c | 5 +- xen/arch/x86/mm/mem_sharing.c | 474 ++++++++++++++++++++++++------ xen/arch/x86/mm/p2m.c | 21 +- xen/common/memory.c | 2 +- xen/drivers/passthrough/pci.c | 3 +- xen/include/asm-x86/hap.h | 1 + xen/include/asm-x86/hvm/domain.h | 6 +- xen/include/asm-x86/hvm/hvm.h | 2 + xen/include/asm-x86/mem_sharing.h | 42 ++- xen/include/asm-x86/p2m.h | 14 +- xen/include/public/memory.h | 6 + xen/include/xen/sched.h | 2 + 28 files changed, 1051 insertions(+), 329 deletions(-)
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 1/18 x86/hvm: introduce hvm_copy_context_and_params
Currently the hvm parameters are only accessible via the HVMOP hypercalls. In this patch we introduce a new function that can copy both the hvm context and parameters directly into a target domain. No functional changes in existing code.
--- xen/arch/x86/hvm/hvm.c | 240 +++++++++++++++++++++------------- xen/include/asm-x86/hvm/hvm.h | 2 + 2 files changed, 151 insertions(+), 91 deletions(-)
xen/arch/x86/hvm/hvm.c +149 -91
@@ -4067,16 +4067,17 @@ static int hvmop_set_evtchn_upcall_vector( }static int hvm_allow_set_param (struct domain *d, -const struct xen_hvm_param *a) +uint32_t index, +uint64_t new_value) { -uint64_t value = d->arch.hvm.params[a->index]; +uint64_t value = d->arch.hvm.params[index];int rc; rc =xsm_hvm_param (XSM_TARGET , d, HVMOP_set_param);if ( rc )return rc; -switch ( a->index ) +switch ( index ) {/* The following parameters can be set by the guest. */ case HVM_PARAM_CALLBACK_IRQ :
@@ -4109,7 +4110,7 @@ static int hvm_allow_set_param(struct domain *d,if ( rc )return rc; -switch ( a->index ) +switch ( index ) {/* The following parameters should only be changed once. */ case HVM_PARAM_VIRIDIAN :
@@ -4119,7 +4120,7 @@ static int hvm_allow_set_param(struct domain *d,case HVM_PARAM_NR_IOREQ_SERVER_PAGES :case HVM_PARAM_ALTP2M :case HVM_PARAM_MCA_CAP : -if ( value !=0 && a->value != value ) +if ( value !=0 && new_value != value ) rc = -EEXIST ;break ;default :
@@ -4129,49 +4130,32 @@ static int hvm_allow_set_param(struct domain *d,return rc; } -static int hvmop_set_param ( -XEN_GUEST_HANDLE_PARAM (xen_hvm_param_t) arg) +static int hvm_set_param (struct domain *d,uint32_t index,uint64_t value) {struct domain *curr_d = current->domain; -struct xen_hvm_param a; -struct domain *d; -struct vcpu *v;int rc; +struct vcpu *v; -if (copy_from_guest (&a, arg,1 ) ) -return -EFAULT ; - -if ( a.index >=HVM_NR_PARAMS ) +if ( index >=HVM_NR_PARAMS )return -EINVAL ; -/* Make sure the above bound check is not bypassed during speculation. */ -block_speculation (); - - d =rcu_lock_domain_by_any_id (a.domid); -if ( d ==NULL ) -return -ESRCH ; - - rc = -EINVAL ; -if ( !is_hvm_domain (d) ) - goto out; - - rc =hvm_allow_set_param (d, &a); + rc =hvm_allow_set_param (d, index, value);if ( rc ) goto out; -switch ( a.index ) +switch ( index ) {case HVM_PARAM_CALLBACK_IRQ : -hvm_set_callback_via (d, a.value); +hvm_set_callback_via (d, value);hvm_latch_shinfo_size (d);break ;case HVM_PARAM_TIMER_MODE : -if ( a.value > HVMPTM_one_missed_tick_pending ) +if ( value > HVMPTM_one_missed_tick_pending ) rc = -EINVAL ;break ;case HVM_PARAM_VIRIDIAN : -if ( (a.value & ~HVMPV_feature_mask) || - !(a.value & HVMPV_base_freq) ) +if ( (value & ~HVMPV_feature_mask) || + !(value & HVMPV_base_freq) ) rc = -EINVAL ;break ;case HVM_PARAM_IDENT_PT :
@@ -4181,7 +4165,7 @@ static int hvmop_set_param(*/ if ( !paging_mode_hap (d) || !cpu_has_vmx ) { - d->arch.hvm.params[a.index] = a.value; + d->arch.hvm.params[index] = value;break ; }
@@ -4196,7 +4180,7 @@ static int hvmop_set_param( rc =0 ;domain_pause (d); - d->arch.hvm.params[a.index] = a.value; + d->arch.hvm.params[index] = value; for_each_vcpu ( d, v )paging_update_cr3 (v, false);domain_unpause (d);
@@ -4205,23 +4189,23 @@ static int hvmop_set_param(break ;case HVM_PARAM_DM_DOMAIN :/* The only value this should ever be set to is DOMID_SELF */ -if ( a.value !=DOMID_SELF ) +if ( value !=DOMID_SELF ) rc = -EINVAL ; - a.value = curr_d->domain_id; + value = curr_d->domain_id;break ;case HVM_PARAM_ACPI_S_STATE : rc =0 ; -if ( a.value ==3 ) +if ( value ==3 )hvm_s3_suspend (d); -else if ( a.value ==0 ) +else if ( value ==0 )hvm_s3_resume (d);else rc = -EINVAL ;break ;case HVM_PARAM_ACPI_IOPORTS_LOCATION : - rc =pmtimer_change_ioport (d, a.value); + rc =pmtimer_change_ioport (d, value);break ;case HVM_PARAM_MEMORY_EVENT_CR0 :case HVM_PARAM_MEMORY_EVENT_CR3 :
@@ -4236,24 +4220,24 @@ static int hvmop_set_param( rc =xsm_hvm_param_nested (XSM_PRIV , d);if ( rc )break ; -if ( a.value >1 ) +if ( value >1 ) rc = -EINVAL ;/* * Remove the check below once we have * shadow-on-shadow. */ -if ( !paging_mode_hap (d) && a.value ) +if ( !paging_mode_hap (d) && value ) rc = -EINVAL ; -if ( a.value && +if ( value && d->arch.hvm.params[HVM_PARAM_ALTP2M ] ) rc = -EINVAL ;/* Set up NHVM state for any vcpus that are already up. */ -if ( a.value && +if ( value && !d->arch.hvm.params[HVM_PARAM_NESTEDHVM ] )for_each_vcpu (d, v)if ( rc ==0 ) rc =nestedhvm_vcpu_initialise (v); -if ( !a.value || rc ) +if ( !value || rc )for_each_vcpu (d, v)nestedhvm_vcpu_destroy (v);break ;
@@ -4261,30 +4245,30 @@ static int hvmop_set_param( rc =xsm_hvm_param_altp2mhvm (XSM_PRIV , d);if ( rc )break ; -if ( a.value > XEN_ALTP2M_limited ) +if ( value > XEN_ALTP2M_limited ) rc = -EINVAL ; -if ( a.value && +if ( value && d->arch.hvm.params[HVM_PARAM_NESTEDHVM ] ) rc = -EINVAL ;break ;case HVM_PARAM_TRIPLE_FAULT_REASON : -if ( a.value >SHUTDOWN_MAX ) +if ( value >SHUTDOWN_MAX ) rc = -EINVAL ;break ;case HVM_PARAM_IOREQ_SERVER_PFN : - d->arch.hvm.ioreq_gfn.base = a.value; + d->arch.hvm.ioreq_gfn.base = value;break ;case HVM_PARAM_NR_IOREQ_SERVER_PAGES : {unsigned int i; -if ( a.value ==0 || - a.value >sizeof (d->arch.hvm.ioreq_gfn.mask) *8 ) +if ( value ==0 || + value >sizeof (d->arch.hvm.ioreq_gfn.mask) *8 ) { rc = -EINVAL ;break ; } -for ( i =0 ; i < a.value; i++ ) +for ( i =0 ; i < value; i++ )set_bit (i, &d->arch.hvm.ioreq_gfn.mask);break ;
@@ -4296,35 +4280,35 @@ static int hvmop_set_param( sizeof(d->arch.hvm.ioreq_gfn.legacy_mask) * 8);BUILD_BUG_ON (HVM_PARAM_BUFIOREQ_PFN >sizeof (d->arch.hvm.ioreq_gfn.legacy_mask) *8 ); -if ( a.value ) -set_bit (a.index, &d->arch.hvm.ioreq_gfn.legacy_mask); +if ( value ) +set_bit (index, &d->arch.hvm.ioreq_gfn.legacy_mask);break ;case HVM_PARAM_X87_FIP_WIDTH : -if ( a.value !=0 && a.value !=4 && a.value !=8 ) +if ( value !=0 && value !=4 && value !=8 ) { rc = -EINVAL ;break ; } - d->arch.x87_fip_width = a.value; + d->arch.x87_fip_width = value;break ;case HVM_PARAM_VM86_TSS :/* Hardware would silently truncate high bits. */ -if ( a.value != (uint32_t )a.value ) +if ( value != (uint32_t )value ) {if ( d == curr_d )domain_crash (d); rc = -EINVAL ; }/* Old hvmloader binaries hardcode the size to 128 bytes. */ -if ( a.value ) - a.value |= (128ULL <<32 ) |VM86_TSS_UPDATED ; - a.index =HVM_PARAM_VM86_TSS_SIZED ; +if ( value ) + value |= (128ULL <<32 ) |VM86_TSS_UPDATED ; + index =HVM_PARAM_VM86_TSS_SIZED ;break ;case HVM_PARAM_VM86_TSS_SIZED : -if ( (a.value >>32 ) <sizeof (struct tss32 ) ) +if ( (value >>32 ) <sizeof (struct tss32 ) ) {if ( d == curr_d )domain_crash (d);
@@ -4335,26 +4319,56 @@ static int hvmop_set_param(* 256 bits interrupt redirection bitmap + 64k bits I/O bitmap * plus one padding byte). */ -if ( (a.value >>32 ) >sizeof (struct tss32 ) + +if ( (value >>32 ) >sizeof (struct tss32 ) + (0x100 /8 ) + (0x10000 /8 ) +1 ) - a.value = (uint32_t )a.value | + value = (uint32_t )value | ((sizeof (struct tss32 ) + (0x100 /8 ) + (0x10000 /8 ) +1 ) <<32 ); - a.value |=VM86_TSS_UPDATED ; + value |=VM86_TSS_UPDATED ;break ;case HVM_PARAM_MCA_CAP : - rc =vmce_enable_mca_cap (d, a.value); + rc =vmce_enable_mca_cap (d, value);break ; }if ( rc !=0 ) goto out; - d->arch.hvm.params[a.index] = a.value; + d->arch.hvm.params[index] = value;HVM_DBG_LOG (DBG_LEVEL_HCALL ,"set param %u = %" PRIx64, - a.index, a.value); + index, value); + + out: +return rc; +} + +int hvmop_set_param ( +XEN_GUEST_HANDLE_PARAM (xen_hvm_param_t) arg) +{ +struct xen_hvm_param a; +struct domain *d; +int rc; + +if (copy_from_guest (&a, arg,1 ) ) +return -EFAULT ; + +if ( a.index >=HVM_NR_PARAMS ) +return -EINVAL ; + +/* Make sure the above bound check is not bypassed during speculation. */ +block_speculation (); + + d =rcu_lock_domain_by_any_id (a.domid); +if ( d ==NULL ) +return -ESRCH ; + + rc = -EINVAL ; +if ( !is_hvm_domain (d) ) + goto out; + + rc =hvm_set_param (d, a.index, a.value); out:rcu_unlock_domain (d);
@@ -4362,7 +4376,7 @@ static int hvmop_set_param( }static int hvm_allow_get_param (struct domain *d, -const struct xen_hvm_param *a) +uint32_t index) {int rc;
@@ -4370,7 +4384,7 @@ static int hvm_allow_get_param(struct domain *d,if ( rc )return rc; -switch ( a->index ) +switch ( index ) {/* The following parameters can be read by the guest. */ case HVM_PARAM_CALLBACK_IRQ :
@@ -4400,6 +4414,43 @@ static int hvm_allow_get_param(struct domain *d,return rc; } +static int hvm_get_param (struct domain *d,uint32_t index,uint64_t *value) +{ +int rc; + +if ( index >=HVM_NR_PARAMS || !value ) +return -EINVAL ; + + rc =hvm_allow_get_param (d, index); +if ( rc ) +return rc; + +switch ( index ) + { +case HVM_PARAM_ACPI_S_STATE : + *value = d->arch.hvm.is_s3_suspended ?3 :0 ; +break ; + +case HVM_PARAM_VM86_TSS : + *value = (uint32_t )d->arch.hvm.params[HVM_PARAM_VM86_TSS_SIZED ]; +break ; + +case HVM_PARAM_VM86_TSS_SIZED : + *value = d->arch.hvm.params[HVM_PARAM_VM86_TSS_SIZED ] & + ~VM86_TSS_UPDATED ; +break ; + +case HVM_PARAM_X87_FIP_WIDTH : + *value = d->arch.x87_fip_width; +break ; +default : + *value = d->arch.hvm.params[index]; +break ; + } + +return 0 ; +}; +static int hvmop_get_param (XEN_GUEST_HANDLE_PARAM (xen_hvm_param_t) arg) {
@@ -4424,33 +4475,10 @@ static int hvmop_get_param(if ( !is_hvm_domain (d) ) goto out; - rc =hvm_allow_get_param (d, &a); + rc =hvm_get_param (d, a.index, &a.value);if ( rc ) goto out; -switch ( a.index ) - { -case HVM_PARAM_ACPI_S_STATE : - a.value = d->arch.hvm.is_s3_suspended ?3 :0 ; -break ; - -case HVM_PARAM_VM86_TSS : - a.value = (uint32_t )d->arch.hvm.params[HVM_PARAM_VM86_TSS_SIZED ]; -break ; - -case HVM_PARAM_VM86_TSS_SIZED : - a.value = d->arch.hvm.params[HVM_PARAM_VM86_TSS_SIZED ] & - ~VM86_TSS_UPDATED ; -break ; - -case HVM_PARAM_X87_FIP_WIDTH : - a.value = d->arch.x87_fip_width; -break ; -default : - a.value = d->arch.hvm.params[a.index]; -break ; - } - rc =__copy_to_guest (arg, &a,1 ) ? -EFAULT :0 ; HVM_DBG_LOG(DBG_LEVEL_HCALL, "get param %u = %"PRIx64,
@@ -5266,6 +5294,36 @@ void hvm_set_segment_register(struct vcpu *v, enum x86_segment seg,alternative_vcall (hvm_funcs.set_segment_register, v, seg, reg); } +int hvm_copy_context_and_params (struct domain *src,struct domain *dst) +{ +int rc, i; +struct hvm_domain_context c = { }; + +for ( i =0 ; i <HVM_NR_PARAMS ; i++ ) + { +uint64_t value =0 ; + +if (hvm_get_param (src, i, &value) || !value ) +continue ; + +if ( (rc =hvm_set_param (dst, i, value)) ) +return rc; + } + + c.size =hvm_save_size (src); +if ( (c.data =vmalloc (c.size)) ==NULL ) +return -ENOMEM ; + +if ( !(rc =hvm_save (src, &c)) ) + { + c.cur =0 ; + rc =hvm_load (dst, &c); + } + +vfree (c.data); +return rc; +} +/* * Local variables: * mode: C
xen/include/asm-x86/hvm/hvm.h +2 -0
@@ -336,6 +336,8 @@ unsigned long hvm_cr4_guest_valid_bits(const struct domain *d, bool restore);bool hvm_flush_vcpu_tlb (bool (*flush_vcpu)(void *ctxt,struct vcpu *v),void *ctxt); +int hvm_copy_context_and_params (struct domain *src,struct domain *dst); +#ifdef CONFIG_HVM #define hvm_get_guest_tsc (v) hvm_get_guest_tsc_fixed(v, 0)
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 01/18] x86/hvm: introduce hvm_copy_context_and_params
Quoted 6 lines · Tamas K Lengyel
Currently the hvm parameters are only accessible via the HVMOP hypercalls. In this patch we introduce a new function that can copy both the hvm context and parameters directly into a target domain. No functional changes in existing code.
Signed-off-by Tamas K Lengyel <[email protected]>
In reply to my v4 comments you said "I don't have any objections to the things you pointed out." Yet only one aspect was actually changed here. It also doesn't help that there's no brief summary of the changes done for v5. I guess I'm confused.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 01/18] x86/hvm: introduce hvm_copy_context_and_params
Quoted 7 lines · Jan Beulich
Quoted 5 lines · Tamas K Lengyel
Currently the hvm parameters are only accessible via the HVMOP hypercalls. In this patch we introduce a new function that can copy both the hvm context and parameters directly into a target domain. No functional changes in existing code.
Quoted 6 lines
Quoted 1 line
Signed-off-by Tamas K Lengyel <[email protected]>In reply to my v4 comments you said "I don't have any objections to the things you pointed out." Yet only one aspect was actually changed here. It also doesn't help that there's no brief summary of the changes done for v5. I guess I'm confused.
Indeed it seems I missed some of your previous requests. I was halfway through making the modifications but simply forgot to do the rest after I stepped away. I still don't have any objections to them though, so will catch up on it in v6.
Thanks,
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 2/18 xen/x86: Make hap_get_allocation accessible
In reply to Tamas K LengyelDuring VM forking we'll copy the parent domain's parameters to the client, including the HAP shadow memory setting that is used for storing the domain's EPT. We'll copy this in the hypervisor instead doing it during toolstack launch to allow the domain to start executing and unsharing memory before (or even completely without) the toolstack.
--- xen/arch/x86/mm/hap/hap.c | 3 +-- xen/include/asm-x86/hap.h | 1 + 2 files changed, 2 insertions(+), 2 deletions(-)
xen/arch/x86/mm/hap/hap.c +1 -2
@@ -321,8 +321,7 @@ static void hap_free_p2m_page(struct domain *d, struct page_info *pg) }/* Return the size of the pool, rounded up to the nearest MB */ -static unsigned int -hap_get_allocation (struct domain *d) +unsigned int hap_get_allocation (struct domain *d) {unsigned int pg = d->arch.paging.hap.total_pages + d->arch.paging.hap.p2m_pages;
xen/include/asm-x86/hap.h +1 -0
@@ -45,6 +45,7 @@ int hap_track_dirty_vram(struct domain *d,extern const struct paging_mode *hap_paging_get_mode (struct vcpu *);int hap_set_allocation (struct domain *d,unsigned int pages,bool *preempted); +unsigned int hap_get_allocation (struct domain *d);#endif /* XEN_HAP_H */
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 3/18 x86/p2m: Allow p2m_get_page_from_gfn to return shared entries
In reply to Tamas K LengyelThe owner domain of shared pages is dom_cow, use that for get_page otherwise the function fails to return the correct page.
--- xen/arch/x86/mm/p2m.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
xen/arch/x86/mm/p2m.c +4 -1
@@ -594,7 +594,10 @@ struct page_info *p2m_get_page_from_gfn(if (p2m_is_ram (*t) &&mfn_valid (mfn) ) { page =mfn_to_page (mfn); -if ( !get_page (page, p2m->domain) ) +if ( !get_page (page, p2m->domain) && +/* Page could be shared */ + (!dom_cow || !p2m_is_shared (*t) || + !get_page (page, dom_cow)) ) page =NULL ; }put_gfn (p2m->domain,gfn_x (gfn));
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 03/18] x86/p2m: Allow p2m_get_page_from_gfn to return shared entries
Quoted 2 lines · Tamas K Lengyel
The owner domain of shared pages is
dom_cow, use that forget_pageotherwise the function fails to return the correct page.
I think this description needs improvement: The function does the special shared page dance in one place (on the "fast path") already. This wants mentioning, either because it was a mistake to have it just there, or because a new need has appeared to also have it on the "slow path".
xen/arch/x86/mm/p2m.c struct page_info *p2m_get_page_from_gfn( · line 600
Quoted 11 lines
xen/arch/x86/mm/p2m.c +4
-1@@ -594,7 +594,10 @@ struct page_info *p2m_get_page_from_gfn(if (p2m_is_ram (*t) &&mfn_valid (mfn) ) { page =mfn_to_page (mfn); -if ( !get_page (page, p2m->domain) ) +if ( !get_page (page, p2m->domain) && +/* Page could be shared */ + (!dom_cow || !p2m_is_shared (*t) || + !get_page (page, dom_cow)) )
While there may be a reason why on the fast path two get_page() invocations are be necessary, couldn't you get away with just one
if ( !get_page(page, !dom_cow || !p2m_is_shared(*t) ? p2m->domain
: dom_cow) )at least here? It's also not really clear to me why here and there we need "!dom_cow || !p2m_is_shared(*t)" - wouldn't p2m_is_shared() return consistently "false" when !dom_cow ?
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 03/18] x86/p2m: Allow p2m_get_page_from_gfn to return shared entries
Quoted 10 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
The owner domain of shared pages is
dom_cow, use that forget_pageotherwise the function fails to return the correct page.I think this description needs improvement: The function does the special shared page dance in one place (on the "fast path") already. This wants mentioning, either because it was a mistake to have it just there, or because a new need has appeared to also have it on the "slow path".
It was a pre-existing error not to get the page from dom_cow for a shared entry in the slow path. I only ran into it now because the erroneous type_count check move in the previous version of the series was resulting in all pages being fully deduplicated instead of mostly being shared. Now that the pages are properly shared running LibVMI on the fork resulted in failures do to this bug.
Quoted 16 lines
xen/arch/x86/mm/p2m.c struct page_info *p2m_get_page_from_gfn( · line 600
Quoted 11 lines
xen/arch/x86/mm/p2m.c +4
-1@@ -594,7 +594,10 @@ struct page_info *p2m_get_page_from_gfn(if (p2m_is_ram (*t) &&mfn_valid (mfn) ) { page =mfn_to_page (mfn); -if ( !get_page (page, p2m->domain) ) +if ( !get_page (page, p2m->domain) && +/* Page could be shared */ + (!dom_cow || !p2m_is_shared (*t) || + !get_page (page, dom_cow)) )While there may be a reason why on the fast path two
get_page()invocations are be necessary, couldn't you get away with just one
Quoted 6 lines
if ( !get_page(page, !dom_cow || !p2m_is_shared(*t) ? p2m->domain : dom_cow) )at least here? It's also not really clear to me why here and there we need "!dom_cow || !p2m_is_shared(*t)" - wouldn't
p2m_is_shared()return consistently "false" when !dom_cow ?
I simply copied the existing code from the slow_path as-is. IMHO it would suffice to do a single get_page(page, !p2m_is_shared(*t) ? p2m->domain : dom_cow); since we can't have any entries that are shared when dom_cow is NULL so this is safe, no need for the extra !dom_cow check. If you prefer I can make the change for both locations.
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 03/18] x86/p2m: Allow p2m_get_page_from_gfn to return shared entries
Quoted 42 lines · Tamas K Lengyel
Quoted 10 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
The owner domain of shared pages is
dom_cow, use that forget_pageotherwise the function fails to return the correct page.I think this description needs improvement: The function does the special shared page dance in one place (on the "fast path") already. This wants mentioning, either because it was a mistake to have it just there, or because a new need has appeared to also have it on the "slow path".
It was a pre-existing error not to get the page from
dom_cowfor a shared entry in the slow path. I only ran into it now because the erroneoustype_countcheck move in the previous version of the series was resulting in all pages being fully deduplicated instead of mostly being shared. Now that the pages are properly shared running LibVMI on the fork resulted in failures do to this bug.Quoted 16 lines
xen/arch/x86/mm/p2m.c struct page_info *p2m_get_page_from_gfn( · line 600
Quoted 11 lines
xen/arch/x86/mm/p2m.c +4
-1@@ -594,7 +594,10 @@ struct page_info *p2m_get_page_from_gfn(if (p2m_is_ram (*t) &&mfn_valid (mfn) ) { page =mfn_to_page (mfn); -if ( !get_page (page, p2m->domain) ) +if ( !get_page (page, p2m->domain) && +/* Page could be shared */ + (!dom_cow || !p2m_is_shared (*t) || + !get_page (page, dom_cow)) )While there may be a reason why on the fast path two
get_page()invocations are be necessary, couldn't you get away with just oneQuoted 6 lines
if ( !get_page(page, !dom_cow || !p2m_is_shared(*t) ? p2m->domain : dom_cow) )at least here? It's also not really clear to me why here and there we need "!dom_cow || !p2m_is_shared(*t)" - wouldn't
p2m_is_shared()return consistently "false" when !dom_cow ?
Quoted 6 lines
I simply copied the existing code from the
slow_pathas-is. IMHO it would suffice to do a singleget_page(page, !p2m_is_shared(*t) ? p2m->domain : dom_cow); since we can't have any entries that are shared whendom_cowis NULL so this is safe, no need for the extra !dom_cow check. If you prefer I can make the change for both locations.
If the change is correct to make also in the other place, I'd definitely prefer you doing so.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 03/18] x86/p2m: Allow p2m_get_page_from_gfn to return shared entries
Quoted 50 lines · Jan Beulich
Quoted 42 lines · Tamas K Lengyel
Quoted 10 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
The owner domain of shared pages is
dom_cow, use that forget_pageotherwise the function fails to return the correct page.I think this description needs improvement: The function does the special shared page dance in one place (on the "fast path") already. This wants mentioning, either because it was a mistake to have it just there, or because a new need has appeared to also have it on the "slow path".
It was a pre-existing error not to get the page from
dom_cowfor a shared entry in the slow path. I only ran into it now because the erroneoustype_countcheck move in the previous version of the series was resulting in all pages being fully deduplicated instead of mostly being shared. Now that the pages are properly shared running LibVMI on the fork resulted in failures do to this bug.Quoted 16 lines
xen/arch/x86/mm/p2m.c struct page_info *p2m_get_page_from_gfn( · line 600
Quoted 11 lines
xen/arch/x86/mm/p2m.c +4
-1@@ -594,7 +594,10 @@ struct page_info *p2m_get_page_from_gfn(if (p2m_is_ram (*t) &&mfn_valid (mfn) ) { page =mfn_to_page (mfn); -if ( !get_page (page, p2m->domain) ) +if ( !get_page (page, p2m->domain) && +/* Page could be shared */ + (!dom_cow || !p2m_is_shared (*t) || + !get_page (page, dom_cow)) )While there may be a reason why on the fast path two
get_page()invocations are be necessary, couldn't you get away with just oneQuoted 6 lines
if ( !get_page(page, !dom_cow || !p2m_is_shared(*t) ? p2m->domain : dom_cow) )at least here? It's also not really clear to me why here and there we need "!dom_cow || !p2m_is_shared(*t)" - wouldn't
p2m_is_shared()return consistently "false" when !dom_cow ?Quoted 6 lines
I simply copied the existing code from the
slow_pathas-is. IMHO it would suffice to do a singleget_page(page, !p2m_is_shared(*t) ? p2m->domain : dom_cow); since we can't have any entries that are shared whendom_cowis NULL so this is safe, no need for the extra !dom_cow check. If you prefer I can make the change for both locations.
Quoted 2 lines
If the change is correct to make also in the other place, I'd definitely prefer you doing so.
I agree with everything Jan said. :-)
Also, since this is a general bugfix, you might consider moving it to the top of your series, so it can be checked in as soon as it's ready.
-George
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 03/18] x86/p2m: Allow p2m_get_page_from_gfn to return shared entries
Quoted 52 lines · George Dunlap
Quoted 50 lines · Jan Beulich
Quoted 48 lines · Tamas K Lengyel
Quoted 10 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
The owner domain of shared pages is
dom_cow, use that forget_pageotherwise the function fails to return the correct page.I think this description needs improvement: The function does the special shared page dance in one place (on the "fast path") already. This wants mentioning, either because it was a mistake to have it just there, or because a new need has appeared to also have it on the "slow path".
It was a pre-existing error not to get the page from
dom_cowfor a shared entry in the slow path. I only ran into it now because the erroneoustype_countcheck move in the previous version of the series was resulting in all pages being fully deduplicated instead of mostly being shared. Now that the pages are properly shared running LibVMI on the fork resulted in failures do to this bug.Quoted 16 lines
xen/arch/x86/mm/p2m.c struct page_info *p2m_get_page_from_gfn( · line 600
Quoted 11 lines
xen/arch/x86/mm/p2m.c +4
-1@@ -594,7 +594,10 @@ struct page_info *p2m_get_page_from_gfn(if (p2m_is_ram (*t) &&mfn_valid (mfn) ) { page =mfn_to_page (mfn); -if ( !get_page (page, p2m->domain) ) +if ( !get_page (page, p2m->domain) && +/* Page could be shared */ + (!dom_cow || !p2m_is_shared (*t) || + !get_page (page, dom_cow)) )While there may be a reason why on the fast path two
get_page()invocations are be necessary, couldn't you get away with just oneQuoted 6 lines
if ( !get_page(page, !dom_cow || !p2m_is_shared(*t) ? p2m->domain : dom_cow) )at least here? It's also not really clear to me why here and there we need "!dom_cow || !p2m_is_shared(*t)" - wouldn't
p2m_is_shared()return consistently "false" when !dom_cow ?I simply copied the existing code from the
slow_pathas-is. IMHO it would suffice to do a singleget_page(page, !p2m_is_shared(*t) ? p2m->domain : dom_cow); since we can't have any entries that are shared whendom_cowis NULL so this is safe, no need for the extra !dom_cow check. If you prefer I can make the change for both locations.
Quoted 7 lines
Quoted 2 lines
If the change is correct to make also in the other place, I'd definitely prefer you doing so.
I agree with everything Jan said. :-)
Also, since this is a general bugfix, you might consider moving it to the top of your series, so it can be checked in as soon as it's ready.
Sure - although it can be checked in before patch 1 & 2, it's not dependent on them. In fact, all the patches in this series up to and including Patch 14 could be checked in out-of-order.
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 4/18 x86/mem_sharing: make get_two_gfns take locks conditionally
In reply to Tamas K LengyelDuring VM forking the client lock will already be taken.
--- xen/arch/x86/mm/mem_sharing.c | 11 ++++++----- xen/include/asm-x86/p2m.h | 10 +++++----- 2 files changed, 11 insertions(+), 10 deletions(-)
xen/arch/x86/mm/mem_sharing.c +6 -5
@@ -943,7 +943,7 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh,unsigned long put_count =0 ;get_two_gfns (sd, sgfn, &smfn_type,NULL , &smfn, - cd, cgfn, &cmfn_type,NULL , &cmfn,0 , &tg); + cd, cgfn, &cmfn_type,NULL , &cmfn,0 , &tg, true);/* * This tricky business is to avoid two callers deadlocking if
@@ -1061,7 +1061,7 @@ err_out: }int mem_sharing_add_to_physmap (struct domain *sd,unsigned long sgfn,shr_handle_t sh, -struct domain *cd,unsigned long cgfn) +struct domain *cd,unsigned long cgfn,bool lock) {struct page_info *spage;int ret = -EINVAL ;
@@ -1073,7 +1073,7 @@ int mem_sharing_add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handlestruct two_gfns tg;get_two_gfns (sd,_gfn (sgfn), &smfn_type,NULL , &smfn, - cd,_gfn (cgfn), &cmfn_type, &a, &cmfn,0 , &tg); + cd,_gfn (cgfn), &cmfn_type, &a, &cmfn,0 , &tg, lock);/* Get the source shared page, check and lock */ ret =XENMEM_SHARING_OP_S_HANDLE_INVALID ;
@@ -1150,7 +1150,8 @@ int mem_sharing_add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle err_unlock:mem_sharing_page_unlock (spage); err_out: -put_two_gfns (&tg); +if ( lock ) +put_two_gfns (&tg);return ret; }
@@ -1571,7 +1572,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) sh = mso.u.share.source_handle; cgfn = mso.u.share.client_gfn; - rc =mem_sharing_add_to_physmap (d, sgfn, sh, cd, cgfn); + rc =mem_sharing_add_to_physmap (d, sgfn, sh, cd, cgfn, true);rcu_unlock_domain (cd); }
xen/include/asm-x86/p2m.h +5 -5
@@ -539,7 +539,7 @@ struct two_gfns {static inline void get_two_gfns (struct domain *rd,gfn_t rgfn,p2m_type_t *rt,p2m_access_t *ra,mfn_t *rmfn,struct domain *ld,gfn_t lgfn,p2m_type_t *lt,p2m_access_t *la,mfn_t *lmfn, -p2m_query_t q,struct two_gfns *rval) +p2m_query_t q,struct two_gfns *rval,bool lock) {mfn_t *first_mfn, *second_mfn, scratch_mfn;p2m_access_t *first_a, *second_a, scratch_a;
@@ -569,10 +569,10 @@ do { \#undef assign_pointers/* Now do the gets */ - *first_mfn =get_gfn_type_access (p2m_get_hostp2m (rval->first_domain), -gfn_x (rval->first_gfn), first_t, first_a, q,NULL ); - *second_mfn =get_gfn_type_access (p2m_get_hostp2m (rval->second_domain), -gfn_x (rval->second_gfn), second_t, second_a, q,NULL ); + *first_mfn =__get_gfn_type_access (p2m_get_hostp2m (rval->first_domain), +gfn_x (rval->first_gfn), first_t, first_a, q,NULL , lock); + *second_mfn =__get_gfn_type_access (p2m_get_hostp2m (rval->second_domain), +gfn_x (rval->second_gfn), second_t, second_a, q,NULL , lock); }static inline void put_two_gfns (struct two_gfns *arg)
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 04/18] x86/mem_sharing: make get_two_gfns take locks conditionally
Quoted 4 lines · Tamas K Lengyel
During VM forking the client lock will already be taken.
Signed-off-by Tamas K Lengyel <[email protected]>Acked-by Andrew Coopers <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 5/18 x86/mem_sharing: drop flags from mem_sharing_unshare_page
In reply to Tamas K LengyelAll callers pass 0 in.
--- xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/mm/p2m.c | 5 ++--- xen/common/memory.c | 2 +- xen/include/asm-x86/mem_sharing.h | 8 +++----- 4 files changed, 7 insertions(+), 10 deletions(-)
xen/arch/x86/hvm/hvm.c +1 -1
@@ -1898,7 +1898,7 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,if ( npfec.write_access && (p2mt == p2m_ram_shared) ) {ASSERT (p2m_is_hostp2m (p2m)); - sharing_enomem =mem_sharing_unshare_page (currd, gfn,0 ); + sharing_enomem =mem_sharing_unshare_page (currd, gfn); rc =1 ; goto out_put_gfn; }
xen/arch/x86/mm/p2m.c +2 -3
@@ -515,7 +515,7 @@ mfn_t __get_gfn_type_access(struct p2m_domain *p2m, unsigned long gfn_l,* Try to unshare. If we fail, communicate ENOMEM without * sleeping. */ -if (mem_sharing_unshare_page (p2m->domain, gfn_l,0 ) <0 ) +if (mem_sharing_unshare_page (p2m->domain, gfn_l) <0 )mem_sharing_notify_enomem (p2m->domain, gfn_l, false); mfn = p2m->get_entry (p2m, gfn, t, a, q, page_order,NULL ); }
@@ -899,8 +899,7 @@ guest_physmap_add_entry(struct domain *d, gfn_t gfn, mfn_t mfn, {/* Do an unshare to cleanly take care of all corner cases. */ int rc; - rc =mem_sharing_unshare_page (p2m->domain, -gfn_x (gfn_add (gfn, i)),0 ); + rc =mem_sharing_unshare_page (p2m->domain,gfn_x (gfn_add (gfn, i)));if ( rc ) {p2m_unlock (p2m);
xen/common/memory.c +1 -1
@@ -352,7 +352,7 @@ int guest_remove_page(struct domain *d, unsigned long gmfn)* might be the only one using this shared page, and we need to * trigger proper cleanup. Once done, this is like any other page. */ - rc =mem_sharing_unshare_page (d, gmfn,0 ); + rc =mem_sharing_unshare_page (d, gmfn);if ( rc ) {mem_sharing_notify_enomem (d, gmfn, false);
xen/include/asm-x86/mem_sharing.h +3 -5
@@ -69,10 +69,9 @@ int __mem_sharing_unshare_page(struct domain *d, uint16_t flags);static inline int mem_sharing_unshare_page (struct domain *d, -unsigned long gfn, -uint16_t flags) +unsigned long gfn) { -int rc =__mem_sharing_unshare_page (d, gfn, flags); +int rc =__mem_sharing_unshare_page (d, gfn,0 );BUG_ON (rc && (rc != -ENOMEM ));return rc; }
@@ -115,8 +114,7 @@ static inline unsigned int mem_sharing_get_nr_shared_mfns(void)return 0 ; } -static inline int mem_sharing_unshare_page (struct domain *d,unsigned long gfn, -uint16_t flags) +static inline int mem_sharing_unshare_page (struct domain *d,unsigned long gfn) {ASSERT_UNREACHABLE ();return -EOPNOTSUPP ;
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 05/18] x86/mem_sharing: drop flags from mem_sharing_unshare_page
Quoted 4 lines · Tamas K Lengyel
All callers pass 0 in.
Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Wei Liu <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 6/18 x86/mem_sharing: don't try to unshare twice during page fault
In reply to Tamas K LengyelThe page was already tried to be unshared in get_gfn_type_access. If that didn't work, then trying again is pointless. Don't try to send vm_event again either, simply check if there is a ring or not.
--- xen/arch/x86/hvm/hvm.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-)
xen/arch/x86/hvm/hvm.c +15 -10
@@ -38,6 +38,7 @@#include <xen/warning.h> #include <xen/vpci.h> #include <xen/nospec.h> +#include <xen/vm_event.h> #include <asm/shadow.h> #include <asm/hap.h> #include <asm/current.h>
@@ -1702,7 +1703,7 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,struct domain *currd = curr->domain;struct p2m_domain *p2m, *hostp2m;int rc, fall_through =0 , paged =0 ; -int sharing_enomem =0 ; +bool sharing_enomem = false;vm_event_request_t *req_ptr =NULL ;bool sync = false;unsigned int page_order;
@@ -1894,14 +1895,16 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,if (p2m_is_paged (p2mt) || (p2mt == p2m_ram_paging_out) ) paged =1 ; -/* Mem sharing: unshare the page and try again */ -if ( npfec.write_access && (p2mt == p2m_ram_shared) ) +#ifdef CONFIG_MEM_SHARING +/* Mem sharing: if still shared on write access then its enomem */ +if ( npfec.write_access &&p2m_is_shared (p2mt) ) {ASSERT (p2m_is_hostp2m (p2m)); - sharing_enomem =mem_sharing_unshare_page (currd, gfn); + sharing_enomem = true; rc =1 ; goto out_put_gfn; } +#endif /* Spurious fault? PoD and log-dirty also take this path. */ if (p2m_is_ram (p2mt) )
@@ -1955,19 +1958,21 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla,*/ if ( paged )p2m_mem_paging_populate (currd, gfn); +if ( sharing_enomem ) { -int rv; - -if ( (rv =mem_sharing_notify_enomem (currd, gfn, true)) <0 ) +#ifdef CONFIG_MEM_SHARING +if ( !vm_event_check_ring (currd->vm_event_share) ) { -gdprintk (XENLOG_ERR ,"Domain %hu attempt to unshare " -"gfn %lx, ENOMEM and no helper (rc %d)\n" , - currd->domain_id, gfn, rv); +gprintk (XENLOG_ERR ,"Domain %pd attempt to unshare " +"gfn %lx, ENOMEM and no helper\n" , + currd, gfn);/* Crash the domain */ rc =0 ; } +#endif } +if ( req_ptr ) {if (monitor_traps (curr, sync, req_ptr) <0 )
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 7/18 x86/mem_sharing: define mem_sharing_domain to hold some scattered variables
In reply to Tamas K LengyelCreate struct mem_sharing_domain under hvm_domain and move mem sharing variables into it from p2m_domain and hvm_domain.
Expose the mem_sharing_enabled macro to be used consistently across Xen.
Remove some duplicate calls to mem_sharing_enabled in mem_sharing.c
---
v5: remove hap_enabled check() from mem_sharing_enabled as suggested by Jan
--- xen/arch/x86/mm/mem_sharing.c | 10 ++++------ xen/drivers/passthrough/pci.c | 3 +-- xen/include/asm-x86/hvm/domain.h | 6 +++++- xen/include/asm-x86/mem_sharing.h | 15 +++++++++++++++ xen/include/asm-x86/p2m.h | 4 ---- 5 files changed, 25 insertions(+), 13 deletions(-)
xen/arch/x86/mm/mem_sharing.c +4 -6
@@ -197,9 +197,6 @@ static shr_handle_t get_next_handle(void)return x +1 ; } -#define mem_sharing_enabled (d) \ - (is_hvm_domain(d) && (d)->arch.hvm.mem_sharing_enabled) -static atomic_t nr_saved_mfns =ATOMIC_INIT (0 );static atomic_t nr_shared_mfns =ATOMIC_INIT (0 );
@@ -1297,6 +1294,7 @@ int __mem_sharing_unshare_page(struct domain *d,int relinquish_shared_pages (struct domain *d) {int rc =0 ; +struct mem_sharing_domain *msd = &d->arch.hvm.mem_sharing;struct p2m_domain *p2m =p2m_get_hostp2m (d);unsigned long gfn, count =0 ;
@@ -1304,7 +1302,7 @@ int relinquish_shared_pages(struct domain *d)return 0 ;p2m_lock (p2m); -for ( gfn = p2m->next_shared_gfn_to_relinquish; +for ( gfn = msd->next_shared_gfn_to_relinquish; gfn <= p2m->max_mapped_pfn; gfn++ ) {p2m_access_t a;
@@ -1339,7 +1337,7 @@ int relinquish_shared_pages(struct domain *d) {if (hypercall_preempt_check () ) { - p2m->next_shared_gfn_to_relinquish = gfn +1 ; + msd->next_shared_gfn_to_relinquish = gfn +1 ; rc = -ERESTART ;break ; }
@@ -1425,7 +1423,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)/* Only HAP is supported */ rc = -ENODEV ; -if ( !hap_enabled (d) || !d->arch.hvm.mem_sharing_enabled ) +if ( !mem_sharing_enabled (d) ) goto out; switch ( mso.op )
xen/drivers/passthrough/pci.c +1 -2
@@ -1498,8 +1498,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)/* Prevent device assign if mem paging or mem sharing have been * enabled for this domain */ if ( d != dom_io && -unlikely ((is_hvm_domain (d) && - d->arch.hvm.mem_sharing_enabled) || +unlikely (mem_sharing_enabled (d) ||vm_event_check_ring (d->vm_event_paging) ||p2m_get_hostp2m (d)->global_logdirty) )return -EXDEV ;
xen/include/asm-x86/hvm/domain.h +5 -1
@@ -29,6 +29,7 @@#include <asm/hvm/viridian.h> #include <asm/hvm/vmx/vmcs.h> #include <asm/hvm/svm/vmcb.h> +#include <asm/mem_sharing.h> #include <public/grant_table.h> #include <public/hvm/params.h> #include <public/hvm/save.h>
@@ -156,7 +157,6 @@ struct hvm_domain {struct viridian_domain *viridian; -bool_t mem_sharing_enabled;bool_t qemu_mapcache_invalidate;bool_t is_s3_suspended;
@@ -192,6 +192,10 @@ struct hvm_domain {struct vmx_domain vmx; struct svm_domain svm; }; + +#ifdef CONFIG_MEM_SHARING +struct mem_sharing_domain mem_sharing; +#endif };#endif /* __ASM_X86_HVM_DOMAIN_H__ */
xen/include/asm-x86/mem_sharing.h +15 -0
@@ -26,6 +26,19 @@#ifdef CONFIG_MEM_SHARING +struct mem_sharing_domain +{ +bool enabled; + +/* +* When releasing shared gfn's in a preemptible manner, recall where +* to resume the search. +*/ +unsigned long next_shared_gfn_to_relinquish; +}; + +#define mem_sharing_enabled (d) ((d)->arch.hvm.mem_sharing.enabled) +/* Auditing of memory sharing code? */ #ifndef NDEBUG #define MEM_SHARING_AUDIT 1
@@ -104,6 +117,8 @@ int relinquish_shared_pages(struct domain *d);#else +#define mem_sharing_enabled (d) false +static inline unsigned int mem_sharing_get_nr_saved_mfns (void ) {return 0 ;
xen/include/asm-x86/p2m.h +0 -4
@@ -305,10 +305,6 @@ struct p2m_domain {unsigned long min_remapped_gfn;unsigned long max_remapped_gfn; -/* When releasing shared gfn's in a preemptible manner, recall where -* to resume the search */ -unsigned long next_shared_gfn_to_relinquish; -#ifdef CONFIG_HVM /* Populate-on-demand variables * All variables are protected with the pod lock. We cannot rely on
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 07/18] x86/mem_sharing: define mem_sharing_domain to hold some scattered variables
Quoted 9 lines · Tamas K Lengyel
Create struct
mem_sharing_domainunderhvm_domainand move mem sharing variables into it fromp2m_domainandhvm_domain.Expose the
mem_sharing_enabledmacro to be used consistently across Xen.Remove some duplicate calls to
mem_sharing_enabledinmem_sharing.cSigned-off-by Tamas K Lengyel <[email protected]>Acked-by Jan Beulich <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 8/18 x86/mem_sharing: Use INVALID_MFN and p2m_is_shared in relinquish_shared_pages
In reply to Tamas K LengyelWhile using _mfn(0) is of no consequence during teardown, INVALID_MFN is the correct value that should be used.
--- xen/arch/x86/mm/mem_sharing.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
xen/arch/x86/mm/mem_sharing.c +2 -2
@@ -1314,7 +1314,7 @@ int relinquish_shared_pages(struct domain *d)break ; mfn = p2m->get_entry (p2m,_gfn (gfn), &t, &a,0 ,NULL ,NULL ); -if (mfn_valid (mfn) && t == p2m_ram_shared ) +if (mfn_valid (mfn) &&p2m_is_shared (t) ) {/* Does not fail with ENOMEM given the DESTROY flag */ BUG_ON(__mem_sharing_unshare_page(
@@ -1324,7 +1324,7 @@ int relinquish_shared_pages(struct domain *d)* unshare. Must succeed: we just read the old entry and * we hold the p2m lock. */ - set_rc = p2m->set_entry (p2m,_gfn (gfn),_mfn (0 ),PAGE_ORDER_4K , + set_rc = p2m->set_entry (p2m,_gfn (gfn),INVALID_MFN ,PAGE_ORDER_4K , p2m_invalid, p2m_access_rwx,-1 );ASSERT (!set_rc); count +=0x10 ;
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 9/18 x86/mem_sharing: Make add_to_physmap static and shorten name
In reply to Tamas K LengyelIt's not being called from outside mem_sharing.c
--- xen/arch/x86/mm/mem_sharing.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
xen/arch/x86/mm/mem_sharing.c +4 -3
@@ -1057,8 +1057,9 @@ err_out:return ret; } -int mem_sharing_add_to_physmap (struct domain *sd,unsigned long sgfn,shr_handle_t sh, -struct domain *cd,unsigned long cgfn,bool lock) +static +int add_to_physmap (struct domain *sd,unsigned long sgfn,shr_handle_t sh, +struct domain *cd,unsigned long cgfn,bool lock) {struct page_info *spage;int ret = -EINVAL ;
@@ -1570,7 +1571,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) sh = mso.u.share.source_handle; cgfn = mso.u.share.client_gfn; - rc =mem_sharing_add_to_physmap (d, sgfn, sh, cd, cgfn, true); + rc =add_to_physmap (d, sgfn, sh, cd, cgfn, true);rcu_unlock_domain (cd); }
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 10/18 x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
In reply to Tamas K LengyelMEM_SHARING_DESTROY_GFN is used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.
--- xen/arch/x86/mm/mem_sharing.c | 9 ++++----- xen/include/asm-x86/mem_sharing.h | 5 ++--- 2 files changed, 6 insertions(+), 8 deletions(-)
xen/arch/x86/mm/mem_sharing.c +4 -5
@@ -1170,7 +1170,7 @@ err_out:*/ int __mem_sharing_unshare_page (struct domain *d,unsigned long gfn, -uint16_t flags) +bool destroy) {p2m_type_t p2mt;mfn_t mfn;
@@ -1226,7 +1226,7 @@ int __mem_sharing_unshare_page(struct domain *d,* If the GFN is getting destroyed drop the references to MFN * (possibly freeing the page), and exit early. */ -if ( flags &MEM_SHARING_DESTROY_GFN ) +if ( destroy ) {if ( !last_gfn )mem_sharing_gfn_destroy (page, d, gfn_info);
@@ -1317,9 +1317,8 @@ int relinquish_shared_pages(struct domain *d) mfn = p2m->get_entry (p2m,_gfn (gfn), &t, &a,0 ,NULL ,NULL );if (mfn_valid (mfn) &&p2m_is_shared (t) ) { -/* Does not fail with ENOMEM given the DESTROY flag */ -BUG_ON (__mem_sharing_unshare_page ( - d, gfn,MEM_SHARING_DESTROY_GFN )); +/* Does not fail with ENOMEM given "destroy" is set to true */ +BUG_ON (__mem_sharing_unshare_page (d, gfn, true));/* * Clear out the p2m entry so no one else may try to * unshare. Must succeed: we just read the old entry and
xen/include/asm-x86/mem_sharing.h +2 -3
@@ -75,16 +75,15 @@ struct page_sharing_infounsigned int mem_sharing_get_nr_saved_mfns (void );unsigned int mem_sharing_get_nr_shared_mfns (void ); -#define MEM_SHARING_DESTROY_GFN (1<<1)/* Only fails with -ENOMEM. Enforce it with a BUG_ON wrapper. */ int __mem_sharing_unshare_page (struct domain *d,unsigned long gfn, -uint16_t flags); +bool destroy);static inline int mem_sharing_unshare_page (struct domain *d,unsigned long gfn) { -int rc =__mem_sharing_unshare_page (d, gfn,0 ); +int rc =__mem_sharing_unshare_page (d, gfn, false);BUG_ON (rc && (rc != -ENOMEM ));return rc; }
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 10/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
Quoted 6 lines · Tamas K Lengyel
MEM_SHARING_DESTROY_GFNis used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Jan Beulich <[email protected]>
But is there a particular advantage to getting rid of the bitfield?
You're the maintainer, so it's your decision of course. But if it were me I'd leave it as a bitfield so that all the bitfield code is there if it's needed in the future. Flipping it to a bool, with the risk of flipping it back to a bitfield later, seems like pointless churn to me.
(Although perhaps the reason will become evident by the time I get to the end of the series.)
-George
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 10/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
Quoted 11 lines · George Dunlap
Quoted 6 lines · Tamas K Lengyel
MEM_SHARING_DESTROY_GFNis used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Jan Beulich <[email protected]>But is there a particular advantage to getting rid of the bitfield?
Quoted 7 lines
You're the maintainer, so it's your decision of course. But if it were me I'd leave it as a bitfield so that all the bitfield code is there if it's needed in the future. Flipping it to a bool, with the risk of flipping it back to a bitfield later, seems like pointless churn to me.
(Although perhaps the reason will become evident by the time I get to the end of the series.)
Provided its been many years since this code has been added with no need for any extra bits, and with no expectations that this would change any time soon, I wouldn't worry about that. True, there is very little difference between keeping the code as-is vs converting it to bool, but IMHO it makes the code easier to follow without you wondering what might be those non-existent situations that warranted it to be a bitmap to begin with.
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 10/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
Quoted 20 lines · Tamas K Lengyel
Quoted 11 lines · George Dunlap
Quoted 6 lines · Tamas K Lengyel
MEM_SHARING_DESTROY_GFNis used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Jan Beulich <[email protected]>But is there a particular advantage to getting rid of the bitfield?
Quoted 7 lines
You're the maintainer, so it's your decision of course. But if it were me I'd leave it as a bitfield so that all the bitfield code is there if it's needed in the future. Flipping it to a bool, with the risk of flipping it back to a bitfield later, seems like pointless churn to me.
(Although perhaps the reason will become evident by the time I get to the end of the series.)
Quoted 7 lines
Provided its been many years since this code has been added with no need for any extra bits, and with no expectations that this would change any time soon, I wouldn't worry about that. True, there is very little difference between keeping the code as-is vs converting it to bool, but IMHO it makes the code easier to follow without you wondering what might be those non-existent situations that warranted it to be a bitmap to begin with.
It's definitely a judgement call, and I can see where you're coming
from. Like I said, if it were me I'd leave it, but it's not. :-) Just
wanted to raise the issue as I was going through. I'd Ack it but you've already got an R-b.
-George
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 10/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
Quoted 29 lines · George Dunlap
Quoted 20 lines · Tamas K Lengyel
Quoted 11 lines · George Dunlap
Quoted 6 lines · Tamas K Lengyel
MEM_SHARING_DESTROY_GFNis used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Jan Beulich <[email protected]>But is there a particular advantage to getting rid of the bitfield?
Quoted 7 lines
You're the maintainer, so it's your decision of course. But if it were me I'd leave it as a bitfield so that all the bitfield code is there if it's needed in the future. Flipping it to a bool, with the risk of flipping it back to a bitfield later, seems like pointless churn to me.
(Although perhaps the reason will become evident by the time I get to the end of the series.)
Quoted 7 lines
Provided its been many years since this code has been added with no need for any extra bits, and with no expectations that this would change any time soon, I wouldn't worry about that. True, there is very little difference between keeping the code as-is vs converting it to bool, but IMHO it makes the code easier to follow without you wondering what might be those non-existent situations that warranted it to be a bitmap to begin with.
Quoted 4 lines
It's definitely a judgement call, and I can see where you're coming
from. Like I said, if it were me I'd leave it, but it's not. :-) Just
wanted to raise the issue as I was going through. I'd Ack it but you've already got an R-b.
Until your proposal gets accepted, isn't it that your ack is needed despite the R-b? This is also why e.g. for patch 2 I didn't see a point in sending any R-b, as the patch will need your ack anyway, and it's so simple that "reviewed" would be an overstatement.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 10/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
Quoted 35 lines · Jan Beulich
Quoted 29 lines · George Dunlap
Quoted 20 lines · Tamas K Lengyel
Quoted 11 lines · George Dunlap
Quoted 6 lines · Tamas K Lengyel
MEM_SHARING_DESTROY_GFNis used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Jan Beulich <[email protected]>But is there a particular advantage to getting rid of the bitfield?
Quoted 7 lines
You're the maintainer, so it's your decision of course. But if it were me I'd leave it as a bitfield so that all the bitfield code is there if it's needed in the future. Flipping it to a bool, with the risk of flipping it back to a bitfield later, seems like pointless churn to me.
(Although perhaps the reason will become evident by the time I get to the end of the series.)
Quoted 7 lines
Provided its been many years since this code has been added with no need for any extra bits, and with no expectations that this would change any time soon, I wouldn't worry about that. True, there is very little difference between keeping the code as-is vs converting it to bool, but IMHO it makes the code easier to follow without you wondering what might be those non-existent situations that warranted it to be a bitmap to begin with.
Quoted 4 lines
It's definitely a judgement call, and I can see where you're coming
from. Like I said, if it were me I'd leave it, but it's not. :-) Just
wanted to raise the issue as I was going through. I'd Ack it but you've already got an R-b.
Quoted 4 lines
Until your proposal gets accepted, isn't it that your ack is needed despite the R-b? This is also why e.g. for patch 2 I didn't see a point in sending any R-b, as the patch will need your ack anyway, and it's so simple that "reviewed" would be an overstatement.
I don't think I'm co-maintainer of this; and you're a less-specific maintainer than Tamas, right? Do you need my Ack?
I'm happy to go back and Ack all the mm/ ones you've given an R-b to up until this point in the series; I just didn't think it was necessary.
I'll probably Ack patch 2 once I become convinced the change in that patch is necessary (which I'm guessing might be when I get to 15/18).
-George
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 10/18] x86/mem_sharing: Convert MEM_SHARING_DESTROY_GFN to a bool
Quoted 36 lines · George Dunlap
Quoted 35 lines · Jan Beulich
Quoted 33 lines · George Dunlap
Quoted 20 lines · Tamas K Lengyel
Quoted 11 lines · George Dunlap
Quoted 6 lines · Tamas K Lengyel
MEM_SHARING_DESTROY_GFNis used on the 'flags' bitfield during unsharing. However, the bitfield is not used for anything else, so just convert it to a bool instead.Signed-off-by Tamas K Lengyel <[email protected]>Reviewed-by Jan Beulich <[email protected]>But is there a particular advantage to getting rid of the bitfield?
Quoted 7 lines
You're the maintainer, so it's your decision of course. But if it were me I'd leave it as a bitfield so that all the bitfield code is there if it's needed in the future. Flipping it to a bool, with the risk of flipping it back to a bitfield later, seems like pointless churn to me.
(Although perhaps the reason will become evident by the time I get to the end of the series.)
Quoted 7 lines
Provided its been many years since this code has been added with no need for any extra bits, and with no expectations that this would change any time soon, I wouldn't worry about that. True, there is very little difference between keeping the code as-is vs converting it to bool, but IMHO it makes the code easier to follow without you wondering what might be those non-existent situations that warranted it to be a bitmap to begin with.
It's definitely a judgement call, and I can see where you're coming
from. Like I said, if it were me I'd leave it, but it's not. :-) Just
wanted to raise the issue as I was going through. I'd Ack it but you've already got an R-b.
Quoted 7 lines
Quoted 4 lines
Until your proposal gets accepted, isn't it that your ack is needed despite the R-b? This is also why e.g. for patch 2 I didn't see a point in sending any R-b, as the patch will need your ack anyway, and it's so simple that "reviewed" would be an overstatement.
I don't think I'm co-maintainer of this; and you're a less-specific maintainer than Tamas, right? Do you need my Ack?
Well, I was under the impression that the maintainership nesting was hierarchical, i.e. the next level up would become the relevant one in cases like this one. But I'm of course happy to commit the parts of this series which are ready, if just any less specific maintainer's ack is sufficient. Probably tomorrow morning then.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 11/18 x86/mem_sharing: Replace MEM_SHARING_DEBUG with gdprintk
In reply to Tamas K LengyelUsing XENLOG_ERR level since this is only used in debug paths (ie. it's expected the user already has loglvl=all set). Also use %pd to print the domain ids.
--- xen/arch/x86/mm/mem_sharing.c | 82 +++++++++++++++++------------------ 1 file changed, 41 insertions(+), 41 deletions(-)
xen/arch/x86/mm/mem_sharing.c +41 -41
@@ -49,9 +49,6 @@ typedef struct pg_lock_data { static DEFINE_PER_CPU(pg_lock_data_t, __pld); -#define MEM_SHARING_DEBUG(_f, _a...) \ - debugtrace_printk("mem_sharing_debug: %s(): " _f, __func__, ##_a) -/* Reverse map defines */ #define RMAP_HASHTAB_ORDER 0#define RMAP_HASHTAB_SIZE \
@@ -482,9 +479,9 @@ static int audit(void)/* If we can't lock it, it's definitely not a shared page */ if ( !mem_sharing_page_lock (pg) ) { -MEM_SHARING_DEBUG ( -"mfn %lx in audit list, but cannot be locked (%lx)!\n" , -mfn_x (mfn), pg->u.inuse.type_info); +gdprintk (XENLOG_ERR , +"mfn %lx in audit list, but cannot be locked (%lx)!\n" , +mfn_x (mfn), pg->u.inuse.type_info); errors++;continue ; }
@@ -492,9 +489,9 @@ static int audit(void)/* Check if the MFN has correct type, owner and handle. */ if ( (pg->u.inuse.type_info & PGT_type_mask) != PGT_shared_page ) { -MEM_SHARING_DEBUG ( -"mfn %lx in audit list, but not PGT_shared_page (%lx)!\n" , -mfn_x (mfn), pg->u.inuse.type_info & PGT_type_mask); +gdprintk (XENLOG_ERR , +"mfn %lx in audit list, but not PGT_shared_page (%lx)!\n" , +mfn_x (mfn), pg->u.inuse.type_info & PGT_type_mask); errors++;continue ; }
@@ -502,24 +499,24 @@ static int audit(void)/* Check the page owner. */ if (page_get_owner (pg) != dom_cow ) { -MEM_SHARING_DEBUG ("mfn %lx shared, but wrong owner %pd!\n" , -mfn_x (mfn),page_get_owner (pg)); +gdprintk (XENLOG_ERR ,"mfn %lx shared, but wrong owner (%pd)!\n" , +mfn_x (mfn),page_get_owner (pg)); errors++; }/* Check the m2p entry */ if ( !SHARED_M2P (get_gpfn_from_mfn (mfn_x (mfn))) ) { -MEM_SHARING_DEBUG ("mfn %lx shared, but wrong m2p entry (%lx)!\n" , -mfn_x (mfn),get_gpfn_from_mfn (mfn_x (mfn))); +gdprintk (XENLOG_ERR ,"mfn %lx shared, but wrong m2p entry (%lx)!\n" , +mfn_x (mfn),get_gpfn_from_mfn (mfn_x (mfn))); errors++; }/* Check we have a list */ if ( (!pg->sharing) ||rmap_count (pg) ==0 ) { -MEM_SHARING_DEBUG ("mfn %lx shared, but empty gfn list!\n" , -mfn_x (mfn)); +gdprintk (XENLOG_ERR ,"mfn %lx shared, but empty gfn list!\n" , +mfn_x (mfn)); errors++;continue ; }
@@ -538,24 +535,26 @@ static int audit(void) d =get_domain_by_id (g->domain);if ( d ==NULL ) { -MEM_SHARING_DEBUG ("Unknown dom: %hu, for PFN=%lx, MFN=%lx\n" , - g->domain, g->gfn,mfn_x (mfn)); +gdprintk (XENLOG_ERR , +"Unknown dom: %pd, for PFN=%lx, MFN=%lx\n" , + d, g->gfn,mfn_x (mfn)); errors++;continue ; } o_mfn =get_gfn_query_unlocked (d, g->gfn, &t);if ( !mfn_eq (o_mfn, mfn) ) { -MEM_SHARING_DEBUG ("Incorrect P2M for d=%hu, PFN=%lx." -"Expecting MFN=%lx, got %lx\n" , - g->domain, g->gfn,mfn_x (mfn),mfn_x (o_mfn)); +gdprintk (XENLOG_ERR ,"Incorrect P2M for d=%pd, PFN=%lx." +"Expecting MFN=%lx, got %lx\n" , + d, g->gfn,mfn_x (mfn),mfn_x (o_mfn)); errors++; }if ( t != p2m_ram_shared ) { -MEM_SHARING_DEBUG ("Incorrect P2M type for d=%hu, PFN=%lx MFN=%lx." -"Expecting t=%d, got %d\n" , - g->domain, g->gfn,mfn_x (mfn), p2m_ram_shared, t); +gdprintk (XENLOG_ERR , +"Incorrect P2M type for d=%pd, PFN=%lx MFN=%lx." +"Expecting t=%d, got %d\n" , + d, g->gfn,mfn_x (mfn), p2m_ram_shared, t); errors++; }put_domain (d);
@@ -564,10 +563,10 @@ static int audit(void)/* The type count has an extra ref because we have locked the page */ if ( (nr_gfns +1 ) != (pg->u.inuse.type_info & PGT_count_mask) ) { -MEM_SHARING_DEBUG ("Mismatched counts for MFN=%lx." -"nr_gfns in list %lu, in type_info %lx\n" , -mfn_x (mfn), nr_gfns, - (pg->u.inuse.type_info & PGT_count_mask)); +gdprintk (XENLOG_ERR ,"Mismatched counts for MFN=%lx." +"nr_gfns in list %lu, in type_info %lx\n" , +mfn_x (mfn), nr_gfns, + (pg->u.inuse.type_info & PGT_count_mask)); errors++; }
@@ -578,8 +577,8 @@ static int audit(void)if ( count_found != count_expected ) { -MEM_SHARING_DEBUG ("Expected %ld shared mfns, found %ld." , - count_expected, count_found); +gdprintk (XENLOG_ERR ,"Expected %ld shared mfns, found %ld." , + count_expected, count_found); errors++; }
@@ -757,10 +756,10 @@ static int debug_mfn(mfn_t mfn)return -EINVAL ; } -MEM_SHARING_DEBUG ( -"Debug page: MFN=%lx is ci=%lx, ti=%lx, owner=%pd\n" , -mfn_x (page_to_mfn (page)), page->count_info, - page->u.inuse.type_info,page_get_owner (page)); +gdprintk (XENLOG_ERR , +"Debug page: MFN=%lx is ci=%lx, ti=%lx, owner_id=%d\n" , +mfn_x (page_to_mfn (page)), page->count_info, + page->u.inuse.type_info,page_get_owner (page)->domain_id);/* -1 because the page is locked and that's an additional type ref */ num_refs = ((int ) (page->u.inuse.type_info & PGT_count_mask)) -1 ;
@@ -776,8 +775,9 @@ static int debug_gfn(struct domain *d, gfn_t gfn) mfn =get_gfn_query (d,gfn_x (gfn), &p2mt); -MEM_SHARING_DEBUG ("Debug for dom%d, gfn=%" PRI_gfn"\n" , - d->domain_id,gfn_x (gfn)); +gdprintk (XENLOG_ERR ,"Debug for dom%d, gfn=%" PRI_gfn"\n" , + d->domain_id,gfn_x (gfn)); + num_refs =debug_mfn (mfn);put_gfn (d,gfn_x (gfn));
@@ -793,13 +793,13 @@ static int debug_gref(struct domain *d, grant_ref_t ref) rc =mem_sharing_gref_to_gfn (d->grant_table, ref, &gfn, &status);if ( rc ) { -MEM_SHARING_DEBUG ("Asked to debug [dom=%d,gref=%u]: error %d.\n" , - d->domain_id, ref, rc); +gdprintk (XENLOG_ERR ,"Asked to debug [dom=%d,gref=%u]: error %d.\n" , + d->domain_id, ref, rc);return rc; } -MEM_SHARING_DEBUG ("==> Grant [dom=%d,ref=%d], status=%x. " , - d->domain_id, ref, status); +gdprintk (XENLOG_ERR ,"==> Grant [dom=%d,ref=%d], status=%x. " , + d->domain_id, ref, status);return debug_gfn (d, gfn); }
@@ -1274,8 +1274,8 @@ int __mem_sharing_unshare_page(struct domain *d, private_page_found:if (p2m_change_type_one (d, gfn, p2m_ram_shared, p2m_ram_rw) ) { -gdprintk (XENLOG_ERR ,"Could not change p2m type d %hu gfn %lx.\n" , - d->domain_id, gfn); +gdprintk (XENLOG_ERR ,"Could not change p2m type d %pd gfn %lx.\n" , + d, gfn);BUG (); }
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 11/18] x86/mem_sharing: Replace MEM_SHARING_DEBUG with gdprintk
xen/arch/x86/mm/mem_sharing.c static int audit(void) · line 540
Quoted 9 lines · Tamas K Lengyel
xen/arch/x86/mm/mem_sharing.c +3
-2@@ -538,24 +535,26 @@ static int audit(void) d =get_domain_by_id (g->domain);if ( d ==NULL ) { -MEM_SHARING_DEBUG ("Unknown dom: %hu, for PFN=%lx, MFN=%lx\n" , - g->domain, g->gfn,mfn_x (mfn)); +gdprintk (XENLOG_ERR , +"Unknown dom: %pd, for PFN=%lx, MFN=%lx\n" , + d, g->gfn,mfn_x (mfn));
With "if ( d == NULL )" around this you hardly mean to pass d to the function here. This is a case where you really need to stick to logging a raw number.
xen/arch/x86/mm/mem_sharing.c static int audit(void) · line 547
Quoted 10 lines
xen/arch/x86/mm/mem_sharing.c +1
-3errors++;continue ; } o_mfn =get_gfn_query_unlocked (d, g->gfn, &t);if ( !mfn_eq (o_mfn, mfn) ) { -MEM_SHARING_DEBUG ("Incorrect P2M for d=%hu, PFN=%lx." -"Expecting MFN=%lx, got %lx\n" , - g->domain, g->gfn,mfn_x (mfn),mfn_x (o_mfn)); + gdprintk(XENLOG_ERR, "Incorrect P2M for d=%pd, PFN=%lx."
Here and elsewhere may I recommend dropping d= (or dom= further down)?
xen/arch/x86/mm/mem_sharing.c static int debug_mfn(mfn_t mfn) · line 762
Quoted 12 lines
xen/arch/x86/mm/mem_sharing.c +4
-4@@ -757,10 +756,10 @@ static int debug_mfn(mfn_t mfn)return -EINVAL ; } -MEM_SHARING_DEBUG ( -"Debug page: MFN=%lx is ci=%lx, ti=%lx, owner=%pd\n" , -mfn_x (page_to_mfn (page)), page->count_info, - page->u.inuse.type_info,page_get_owner (page)); +gdprintk (XENLOG_ERR , +"Debug page: MFN=%lx is ci=%lx, ti=%lx, owner_id=%d\n" , +mfn_x (page_to_mfn (page)), page->count_info, + page->u.inuse.type_info,page_get_owner (page)->domain_id);
As indicated before (I think), please prefer %pd and a struct domain pointer over passing ->domain_id (at least one more instance further down).
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 11/18] x86/mem_sharing: Replace MEM_SHARING_DEBUG with gdprintk
Quoted 15 lines · Jan Beulich
xen/arch/x86/mm/mem_sharing.c static int audit(void) · line 540
Quoted 9 lines · Tamas K Lengyel
xen/arch/x86/mm/mem_sharing.c +3
-2@@ -538,24 +535,26 @@ static int audit(void) d =get_domain_by_id (g->domain);if ( d ==NULL ) { -MEM_SHARING_DEBUG ("Unknown dom: %hu, for PFN=%lx, MFN=%lx\n" , - g->domain, g->gfn,mfn_x (mfn)); +gdprintk (XENLOG_ERR , +"Unknown dom: %pd, for PFN=%lx, MFN=%lx\n" , + d, g->gfn,mfn_x (mfn));With "if ( d == NULL )" around this you hardly mean to pass d to the function here. This is a case where you really need to stick to logging a raw number.
Indeed..
Quoted 12 lines
xen/arch/x86/mm/mem_sharing.c static int audit(void) · line 547
Quoted 11 lines
xen/arch/x86/mm/mem_sharing.c +1
-3errors++;continue ; } o_mfn =get_gfn_query_unlocked (d, g->gfn, &t);if ( !mfn_eq (o_mfn, mfn) ) { -MEM_SHARING_DEBUG ("Incorrect P2M for d=%hu, PFN=%lx." -"Expecting MFN=%lx, got %lx\n" , - g->domain, g->gfn,mfn_x (mfn),mfn_x (o_mfn)); + gdprintk(XENLOG_ERR, "Incorrect P2M for d=%pd, PFN=%lx."
Quoted 2 lines
Here and elsewhere may I recommend dropping d= (or dom= further down)?
SGTM
Quoted 5 lines
Quoted 5 lines
xen/arch/x86/mm/mem_sharing.c +0
-0@@ -757,10 +756,10 @@ static int debug_mfn(mfn_t mfn)return -EINVAL ; }
Quoted 12 lines
xen/arch/x86/mm/mem_sharing.c static int debug_mfn(mfn_t mfn) · line 762
Quoted 8 lines
xen/arch/x86/mm/mem_sharing.c +4
-4-MEM_SHARING_DEBUG ( -"Debug page: MFN=%lx is ci=%lx, ti=%lx, owner=%pd\n" , -mfn_x (page_to_mfn (page)), page->count_info, - page->u.inuse.type_info,page_get_owner (page)); +gdprintk (XENLOG_ERR , +"Debug page: MFN=%lx is ci=%lx, ti=%lx, owner_id=%d\n" , +mfn_x (page_to_mfn (page)), page->count_info, + page->u.inuse.type_info,page_get_owner (page)->domain_id);As indicated before (I think), please prefer %pd and a struct domain pointer over passing ->domain_id (at least one more instance further down).
I thought I fixed them all but evidently some remained.
Thanks,
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 12/18 x86/mem_sharing: Enable mem_sharing on first memop
In reply to Tamas K LengyelIt is wasteful to require separate hypercalls to enable sharing on both the parent and the client domain during VM forking. To speed things up we enable sharing on the first memop in case it wasn't already enabled.
--- xen/arch/x86/mm/mem_sharing.c | 52 ++++++++++++++--------------------- 1 file changed, 21 insertions(+), 31 deletions(-)
xen/arch/x86/mm/mem_sharing.c +21 -31
@@ -1400,6 +1400,24 @@ static int range_share(struct domain *d, struct domain *cd,return rc; } +static inline int mem_sharing_control (struct domain *d,bool enable) +{ +if ( enable ) + { +if (unlikely (!is_hvm_domain (d)) ) +return -EOPNOTSUPP ; + +if (unlikely (!hap_enabled (d)) ) +return -ENODEV ; + +if (unlikely (is_iommu_enabled (d)) ) +return -EXDEV ; + } + + d->arch.hvm.mem_sharing.enabled = enable; +return 0 ; +} +int mem_sharing_memop (XEN_GUEST_HANDLE_PARAM (xen_mem_sharing_op_t) arg) {int rc;
@@ -1421,10 +1439,8 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)if ( rc ) goto out; -/* Only HAP is supported */ - rc = -ENODEV ; -if ( !mem_sharing_enabled (d) ) - goto out; +if ( !mem_sharing_enabled (d) && (rc =mem_sharing_control (d, true)) ) +return rc;switch ( mso.op ) {
@@ -1432,10 +1448,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) {shr_handle_t handle; - rc = -EINVAL ; -if ( !mem_sharing_enabled (d) ) - goto out; - rc =nominate_page (d,_gfn (mso.u.nominate.u.gfn),0 , &handle); mso.u.nominate.handle = handle; }
@@ -1447,9 +1459,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)gfn_t gfn;shr_handle_t handle; - rc = -EINVAL ; -if ( !mem_sharing_enabled (d) ) - goto out; rc =mem_sharing_gref_to_gfn (d->grant_table, gref, &gfn,NULL );if ( rc <0 ) goto out;
@@ -1465,10 +1474,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)struct domain *cd;shr_handle_t sh, ch; - rc = -EINVAL ; -if ( !mem_sharing_enabled (d) ) - goto out; - rc =rcu_lock_live_remote_domain_by_id (mso.u.share.client_domain, &cd);if ( rc )
@@ -1535,10 +1540,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)struct domain *cd;shr_handle_t sh; - rc = -EINVAL ; -if ( !mem_sharing_enabled (d) ) - goto out; - rc =rcu_lock_live_remote_domain_by_id (mso.u.share.client_domain, &cd);if ( rc )
@@ -1597,9 +1598,6 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) mso.u.range.opaque > mso.u.range.last_gfn) ) goto out; -if ( !mem_sharing_enabled (d) ) - goto out; - rc =rcu_lock_live_remote_domain_by_id (mso.u.range.client_domain, &cd);if ( rc )
@@ -1691,18 +1689,10 @@ int mem_sharing_domctl(struct domain *d, struct xen_domctl_mem_sharing_op *mec) {int rc; -/* Only HAP is supported */ -if ( !hap_enabled (d) ) -return -ENODEV ; -switch ( mec->op ) {case XEN_DOMCTL_MEM_SHARING_CONTROL : - rc =0 ; -if (unlikely (is_iommu_enabled (d) && mec->u.enable) ) - rc = -EXDEV ; -else - d->arch.hvm.mem_sharing_enabled = mec->u.enable; + rc =mem_sharing_control (d, mec->u.enable);break ;default :
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 12/18] x86/mem_sharing: Enable mem_sharing on first memop
Quoted 5 lines · Tamas K Lengyel
It is wasteful to require separate hypercalls to enable sharing on both the parent and the client domain during VM forking. To speed things up we enable sharing on the first memop in case it wasn't already enabled.
Signed-off-by Tamas K Lengyel <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 13/18 x86/mem_sharing: Skip xen heap pages in memshr nominate
In reply to Tamas K LengyelTrying to share these would fail anyway, better to skip them early.
--- xen/arch/x86/mm/mem_sharing.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
xen/arch/x86/mm/mem_sharing.c +5 -1
@@ -840,6 +840,11 @@ static int nominate_page(struct domain *d, gfn_t gfn,if ( !p2m_is_sharable (p2mt) ) goto out; +/* Skip xen heap pages */ + page =mfn_to_page (mfn); +if ( !page ||is_xen_heap_page (page) ) + goto out; +/* Check if there are mem_access/remapped altp2m entries for this page */ if (altp2m_active (d) ) {
@@ -870,7 +875,6 @@ static int nominate_page(struct domain *d, gfn_t gfn, }/* Try to convert the mfn to the sharable type */ - page =mfn_to_page (mfn); ret =page_make_sharable (d, page, expected_refcnt);if ( ret ) goto out;
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 14/18 x86/mem_sharing: use default_access in add_to_physmap
In reply to Tamas K LengyelWhen plugging a hole in the target physmap don't use the access permission returned by __get_gfn_type_access as it can be non-sensical, leading to spurious vm_events being sent out for access violations at unexpected locations. Make use of p2m->default_access instead.
--- xen/arch/x86/mm/mem_sharing.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
xen/arch/x86/mm/mem_sharing.c +2 -3
@@ -1071,11 +1071,10 @@ int add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle_t sh,p2m_type_t smfn_type, cmfn_type;struct gfn_info *gfn_info;struct p2m_domain *p2m =p2m_get_hostp2m (cd); -p2m_access_t a;struct two_gfns tg;get_two_gfns (sd,_gfn (sgfn), &smfn_type,NULL , &smfn, - cd,_gfn (cgfn), &cmfn_type, &a, &cmfn,0 , &tg, lock); + cd,_gfn (cgfn), &cmfn_type,NULL , &cmfn,0 , &tg, lock);/* Get the source shared page, check and lock */ ret =XENMEM_SHARING_OP_S_HANDLE_INVALID ;
@@ -1110,7 +1109,7 @@ int add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle_t sh, } ret =p2m_set_entry (p2m,_gfn (cgfn), smfn,PAGE_ORDER_4K , - p2m_ram_shared, a); + p2m_ram_shared, p2m->default_access);/* Tempted to turn this into an assert */ if ( ret )
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 14/18] x86/mem_sharing: use default_access in add_to_physmap
Quoted 2 lines · Tamas K Lengyel
When plugging a hole in the target physmap don't use the access permission returned by
__get_gfn_type_accessas it can be non-sensical,
"can be" is too vague for my taste - it suggests there may also be cases where a sensible value is returned, and hence it should be used. Could you clarify this please? (The code change itself of course is simple and mechanical enough to look okay.)
Jan
Quoted 28 lines
leading to
spuriousvm_eventsbeing sent out for access violations at unexpected locations. Make use ofp2m->default_accessinstead.Signed-off-by Tamas K Lengyel <[email protected]>--- xen/arch/x86/mm/mem_sharing.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)xen/arch/x86/mm/mem_sharing.c +1
-2@@ -1071,11 +1071,10 @@ int add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle_t sh,p2m_type_t smfn_type, cmfn_type;struct gfn_info *gfn_info;struct p2m_domain *p2m =p2m_get_hostp2m (cd); -p2m_access_t a;struct two_gfns tg;get_two_gfns (sd,_gfn (sgfn), &smfn_type,NULL , &smfn, - cd,_gfn (cgfn), &cmfn_type, &a, &cmfn,0 , &tg, lock); + cd,_gfn (cgfn), &cmfn_type,NULL , &cmfn,0 , &tg, lock);/* Get the source shared page, check and lock */ ret =XENMEM_SHARING_OP_S_HANDLE_INVALID ;
xen/arch/x86/mm/mem_sharing.c int add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle_t sh, · line 1115
Quoted 8 lines
xen/arch/x86/mm/mem_sharing.c +1
-1@@ -1110,7 +1109,7 @@ int add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle_t sh, } ret =p2m_set_entry (p2m,_gfn (cgfn), smfn,PAGE_ORDER_4K , - p2m_ram_shared, a); + p2m_ram_shared, p2m->default_access);/* Tempted to turn this into an assert */ if ( ret )
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 14/18] x86/mem_sharing: use default_access in add_to_physmap
Quoted 9 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
When plugging a hole in the target physmap don't use the access permission returned by
__get_gfn_type_accessas it can be non-sensical,"can be" is too vague for my taste - it suggests there may also be cases where a sensible value is returned, and hence it should be used. Could you clarify this please? (The code change itself of course is simple and mechanical enough to look okay.)
Well, I can only speak of what I observed. The case seems to be that most of the time the function actually returns p2m_access_rwx (which is sensible), but occasionally something else. I didn't investigate where that value actually comes from, but when populating a physmap like this only the default_access is sane.
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 14/18] x86/mem_sharing: use default_access in add_to_physmap
Quoted 11 lines · Tamas K Lengyel
Quoted 9 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
When plugging a hole in the target physmap don't use the access permission returned by
__get_gfn_type_accessas it can be non-sensical,"can be" is too vague for my taste - it suggests there may also be cases where a sensible value is returned, and hence it should be used. Could you clarify this please? (The code change itself of course is simple and mechanical enough to look okay.)
Quoted 5 lines
Well, I can only speak of what I observed. The case seems to be that most of the time the function actually returns
p2m_access_rwx(which is sensible), but occasionally something else. I didn't investigate where that value actually comes from, but when populating a physmap like this only thedefault_accessis sane.
It would be good to get to the bottom of this. Is it possible that your dom0 agent (or whatever it's called) is calling add_to_physmap() on gfns that have already been populated? Is that something you want to catch?
-George
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 14/18] x86/mem_sharing: use default_access in add_to_physmap
Quoted 19 lines · George Dunlap
Quoted 11 lines · Tamas K Lengyel
Quoted 9 lines · Jan Beulich
Quoted 2 lines · Tamas K Lengyel
When plugging a hole in the target physmap don't use the access permission returned by
__get_gfn_type_accessas it can be non-sensical,"can be" is too vague for my taste - it suggests there may also be cases where a sensible value is returned, and hence it should be used. Could you clarify this please? (The code change itself of course is simple and mechanical enough to look okay.)
Quoted 5 lines
Well, I can only speak of what I observed. The case seems to be that most of the time the function actually returns
p2m_access_rwx(which is sensible), but occasionally something else. I didn't investigate where that value actually comes from, but when populating a physmap like this only thedefault_accessis sane.
Quoted 4 lines
It would be good to get to the bottom of this. Is it possible that your dom0 agent (or whatever it's called) is calling
add_to_physmap()on gfns that have already been populated? Is that something you want to catch?
OK, I went back and deciphered why sometimes I saw different permissions here.
In the context I ran into this issue there is no dom0 agent calling add_to_physmap. We wind up in this path with a VM fork where the MMU faulted. The fault handler is trying to determine whether the page needs to be forked from its parent: populated with a shared entry if it's R/X access, or deduplicated if it's a W. This forking only actually happens if the page type that is returned by ept_get_entry is of a hole type. When it's a hole type, ept_get_entry always returns p2m_access_n as the access permission. Copying that access permission to the newly populated entry is bad - that's what this patch fixes.
But this path also gets hit when the MMU faults for other reasons. In those cases will get permissions other then p2m_access_n since the type is not a hole. But when it's not a hole, this function bails as that's a clear signal that the page doesn't need forking. So I was seeing p2m_access_rwx permission for page accesses that triggered the MMU fault for reasons other then mem_access. For example, when a previously shared entry needs unsharing.
So there is no mystery after all, I was just printing my debug lines with the mem_access permissions irrespective of the page type before the path bails due to the type check.
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 15/18 xen/mem_sharing: VM forking
In reply to Tamas K LengyelVM forking is the process of creating a domain with an empty memory space and a parent domain specified from which to populate the memory when necessary. For the new domain to be functional the VM state is copied over as part of the fork operation (HVM params, hap allocation, etc).
--- xen/arch/x86/domain.c | 9 ++ xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/mm/mem_sharing.c | 220 ++++++++++++++++++++++++++++++ xen/arch/x86/mm/p2m.c | 11 +- xen/include/asm-x86/mem_sharing.h | 20 ++- xen/include/public/memory.h | 5 + xen/include/xen/sched.h | 2 + 7 files changed, 265 insertions(+), 4 deletions(-)
xen/arch/x86/domain.c +9 -0
@@ -2198,6 +2198,15 @@ int domain_relinquish_resources(struct domain *d) ret =relinquish_shared_pages (d);if ( ret )return ret; + +/* +* If the domain is forked, decrement the parent's pause count. +*/ +if ( d->parent ) + { +domain_unpause (d->parent); + d->parent =NULL ; + } }#endif
xen/arch/x86/hvm/hvm.c +1 -1
@@ -1906,7 +1906,7 @@ int hvm_hap_nested_page_fault(paddr_t gpa, unsigned long gla, }#endif -/* Spurious fault? PoD and log-dirty also take this path. */ +/* Spurious fault? PoD, log-dirty and VM forking also take this path. */ if (p2m_is_ram (p2mt) ) { rc =1 ;
xen/arch/x86/mm/mem_sharing.c +220 -0
249 lines not on this page · raw message
xen/arch/x86/mm/p2m.c +10 -1
@@ -508,6 +508,14 @@ mfn_t __get_gfn_type_access(struct p2m_domain *p2m, unsigned long gfn_l, mfn = p2m->get_entry (p2m, gfn, t, a, q, page_order,NULL ); +/* Check if we need to fork the page */ +if ( (q &P2M_ALLOC ) &&p2m_is_hole (*t) && + !mem_sharing_fork_page (p2m->domain, gfn, !!(q &P2M_UNSHARE )) ) + { + mfn = p2m->get_entry (p2m, gfn, t, a, q, page_order,NULL ); + } + +/* Check if we need to unshare the page */ if ( (q &P2M_UNSHARE ) &&p2m_is_shared (*t) ) {ASSERT (p2m_is_hostp2m (p2m));
@@ -585,7 +593,8 @@ struct page_info *p2m_get_page_from_gfn(return page;/* Error path: not a suitable GFN at all */ -if ( !p2m_is_ram (*t) && !p2m_is_paging (*t) && !p2m_is_pod (*t) ) +if ( !p2m_is_ram (*t) && !p2m_is_paging (*t) && !p2m_is_pod (*t) && + !mem_sharing_is_fork (p2m->domain) )return NULL ; }
xen/include/asm-x86/mem_sharing.h +18 -2
@@ -26,8 +26,7 @@ #ifdef CONFIG_MEM_SHARING -struct mem_sharing_domain -{ +struct mem_sharing_domain { bool enabled; /*
@@ -39,6 +38,9 @@ struct mem_sharing_domain#define mem_sharing_enabled (d) ((d)->arch.hvm.mem_sharing.enabled) +#define mem_sharing_is_fork (d) \ + (mem_sharing_enabled(d) && !!((d)->parent)) +/* Auditing of memory sharing code? */ #ifndef NDEBUG #define MEM_SHARING_AUDIT 1
@@ -88,6 +90,9 @@ static inline int mem_sharing_unshare_page(struct domain *d,return rc; } +int mem_sharing_fork_page (struct domain *d,gfn_t gfn, +bool unsharing); +/* * If called by a foreign domain, possible errors are * -EBUSY -> ring full
@@ -117,6 +122,7 @@ int relinquish_shared_pages(struct domain *d);#else #define mem_sharing_enabled (d) false +#define mem_sharing_is_fork (p2m) falsestatic inline unsigned int mem_sharing_get_nr_saved_mfns (void ) {
@@ -141,6 +147,16 @@ static inline int mem_sharing_notify_enomem(struct domain *d, unsigned long gfn,return -EOPNOTSUPP ; } +static inline int mem_sharing_fork (struct domain *d,struct domain *cd,bool vcpu) +{ +return -EOPNOTSUPP ; +} + +static inline int mem_sharing_fork_page (struct domain *d,gfn_t gfn,bool lock) +{ +return -EOPNOTSUPP ; +} +#endif #endif /* __MEM_SHARING_H__ */
xen/include/public/memory.h +5 -0
@@ -482,6 +482,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_mem_access_op_t);#define XENMEM_sharing_op_add_physmap 6#define XENMEM_sharing_op_audit 7#define XENMEM_sharing_op_range_share 8 +#define XENMEM_sharing_op_fork 9#define XENMEM_SHARING_OP_S_HANDLE_INVALID (-10)#define XENMEM_SHARING_OP_C_HANDLE_INVALID (-9)
@@ -532,6 +533,10 @@ struct xen_mem_sharing_op {uint32_t gref;/* IN: gref to debug */ } u; } debug; +struct mem_sharing_op_fork { +domid_t parent_domain; +uint16_t _pad[3 ];/* Must be set to 0 */ + } fork; } u; };typedef struct xen_mem_sharing_op xen_mem_sharing_op_t ;
xen/include/xen/sched.h +2 -0
@@ -504,6 +504,8 @@ struct domain/* Memory sharing support */ #ifdef CONFIG_MEM_SHARING struct vm_event_domain *vm_event_share; +struct domain *parent;/* VM fork parent */ +bool parent_paused;#endif /* Memory paging support */ #ifdef CONFIG_HAS_MEM_PAGING
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 15/18] xen/mem_sharing: VM forking
Quoted 6 lines · Tamas K Lengyel
VM forking is the process of creating a domain with an empty memory space and a parent domain specified from which to populate the memory when necessary. For the new domain to be functional the VM state is copied over as part of the fork operation (HVM params, hap allocation, etc).
Signed-off-by Tamas K Lengyel <[email protected]>
Overall this looks really good. Just a few questions.
Quoted 16 lines
--- xen/arch/x86/domain.c | 9 ++ xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/mm/mem_sharing.c | 220 ++++++++++++++++++++++++++++++ xen/arch/x86/mm/p2m.c | 11 +- xen/include/asm-x86/mem_sharing.h | 20 ++- xen/include/public/memory.h | 5 + xen/include/xen/sched.h | 2 + 7 files changed, 265 insertions(+), 4 deletions(-)xen/arch/x86/domain.c +0
-0@@ -2198,6 +2198,15 @@ int domain_relinquish_resources(struct domain *d) ret =relinquish_shared_pages (d);
xen/arch/x86/domain.c int domain_relinquish_resources(struct domain *d) · line 2208
Quoted 10 lines
xen/arch/x86/domain.c +8
-0if ( ret )return ret; + +/* +* If the domain is forked, decrement the parent's pause count. +*/ +if ( d->parent ) + { +domain_unpause (d->parent); + d->parent =NULL ;
Did this want to be if ( d->parent_paused )?
Quoted 5 lines
xen/arch/x86/domain.c +5
-0+static int bring_up_vcpus (struct domain *cd,struct cpupool *cpupool) +{ +int ret; +unsigned int i; +
xen/arch/x86/domain.c int domain_relinquish_resources(struct domain *d)
Quoted 10 lines
xen/arch/x86/domain.c +10
-0+if ( (ret =cpupool_move_domain (cd, cpupool)) ) +return ret; + +for ( i =0 ; i < cd->max_vcpus; i++ ) + { +if ( cd->vcpu[i] ) +continue ; + +if ( !vcpu_create (cd, i) ) +return -EINVAL ;
You're not copying the contents of the vcpu registers or anything here -- is that something you're leaving to your dom0 agent?
Quoted 23 lines
xen/arch/x86/domain.c +23
-0+static int mem_sharing_fork (struct domain *d,struct domain *cd) +{ +int rc = -EINVAL ; + +if ( !cd->controller_pause_count ) +return rc; + +/* +* We only want to pause the parent once, not each time this +* operation is restarted due to preemption. +*/ +if ( !cd->parent_paused ) + { +domain_pause (d); + cd->parent_paused = true; + } + + cd->max_pages = d->max_pages; + cd->max_vcpus = d->max_vcpus; + +/* this is preemptible so it's the first to get done */ +if ( (rc =fork_hap_allocation (d, cd)) ) + goto done;
xen/arch/x86/domain.c
Quoted 10 lines
xen/arch/x86/domain.c +10
-0+ +if ( (rc =bring_up_vcpus (cd, d->cpupool)) ) + goto done; + +if ( (rc =hvm_copy_context_and_params (d, cd)) ) + goto done; + +fork_tsc (d, cd); + + cd->parent = d;
What happens if the parent dies?
It seems like we might want to do get_domain(parent) here, and put_domain(parent) in domain_destroy.
I'll probably need to come back to this, at which point I may have more questions.
-George
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v5 15/18] xen/mem_sharing: VM forking
Quoted 31 lines · George Dunlap
Quoted 6 lines · Tamas K Lengyel
VM forking is the process of creating a domain with an empty memory space and a parent domain specified from which to populate the memory when necessary. For the new domain to be functional the VM state is copied over as part of the fork operation (HVM params, hap allocation, etc).
Signed-off-by Tamas K Lengyel <[email protected]>Overall this looks really good. Just a few questions.
Quoted 20 lines
--- xen/arch/x86/domain.c | 9 ++ xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/mm/mem_sharing.c | 220 ++++++++++++++++++++++++++++++ xen/arch/x86/mm/p2m.c | 11 +- xen/include/asm-x86/mem_sharing.h | 20 ++- xen/include/public/memory.h | 5 + xen/include/xen/sched.h | 2 + 7 files changed, 265 insertions(+), 4 deletions(-)xen/arch/x86/domain.c +2
-0@@ -2198,6 +2198,15 @@ int domain_relinquish_resources(struct domain *d) ret =relinquish_shared_pages (d);if ( ret )return ret; + +/*
Quoted 8 lines
xen/arch/x86/domain.c int domain_relinquish_resources(struct domain *d) · line 2208
Quoted 6 lines
xen/arch/x86/domain.c +6
-0+* If the domain is forked, decrement the parent's pause count. +*/ +if ( d->parent ) + { +domain_unpause (d->parent); + d->parent =NULL ;Did this want to be
if ( d->parent_paused )?
If the domain has the parent pointer set, it's guaranteed that the parent is paused. It's paused before the parent pointer is set during the fork hypercall.
Quoted 11 lines
Quoted 11 lines
xen/arch/x86/domain.c +10
-0+static int bring_up_vcpus (struct domain *cd,struct cpupool *cpupool) +{ +int ret; +unsigned int i; + +if ( (ret =cpupool_move_domain (cd, cpupool)) ) +return ret; + +for ( i =0 ; i < cd->max_vcpus; i++ ) + {
Quoted 8 lines
xen/arch/x86/domain.c
Quoted 5 lines
xen/arch/x86/domain.c +5
-0+if ( cd->vcpu[i] ) +continue ; + +if ( !vcpu_create (cd, i) ) +return -EINVAL ;You're not copying the contents of the vcpu registers or anything here -- is that something you're leaving to your dom0 agent?
The registers are being copied as part of the HVM contexts.
Quoted 37 lines
Quoted 24 lines
xen/arch/x86/domain.c +23
-0+static int mem_sharing_fork (struct domain *d,struct domain *cd) +{ +int rc = -EINVAL ; + +if ( !cd->controller_pause_count ) +return rc; + +/* +* We only want to pause the parent once, not each time this +* operation is restarted due to preemption. +*/ +if ( !cd->parent_paused ) + { +domain_pause (d); + cd->parent_paused = true; + } + + cd->max_pages = d->max_pages; + cd->max_vcpus = d->max_vcpus; + +/* this is preemptible so it's the first to get done */ +if ( (rc =fork_hap_allocation (d, cd)) ) + goto done;xen/arch/x86/domain.c
Quoted 10 lines
xen/arch/x86/domain.c +10
-0+ +if ( (rc =bring_up_vcpus (cd, d->cpupool)) ) + goto done; + +if ( (rc =hvm_copy_context_and_params (d, cd)) ) + goto done; + +fork_tsc (d, cd); + + cd->parent = d;What happens if the parent dies?
Quoted 2 lines
It seems like we might want to do
get_domain(parent)here, andput_domain(parent)indomain_destroy.
If forks are still active when someone destroys the parent than Xen will crash I assume. Right now it's a requirement that the parent remains in existence - and it's paused - while there are forks active. We enforce the pause state but making the parent undestroyable is not implemented right now. We just trust that the user of this experimental system won't do that.
But yes, doing the get_domain()/put_domain() dance would be an easy way to do that. Will add that and then we don't have to worry about the parent getting pulled from under our feat.
Quoted 3 lines
I'll probably need to come back to this, at which point I may have more questions.
Thanks!
Tamas
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 16/18 xen/mem_access: Use __get_gfn_type_access in set_mem_access
In reply to Tamas K LengyelUse __get_gfn_type_access instead of p2m->get_entry to trigger page-forking when the mem_access permission is being set on a page that has not yet been copied over from the parent.
--- xen/arch/x86/mm/mem_access.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
xen/arch/x86/mm/mem_access.c +2 -3
@@ -303,11 +303,10 @@ static int set_mem_access(struct domain *d, struct p2m_domain *p2m,ASSERT (!ap2m);#endif { -mfn_t mfn;p2m_access_t _a;p2m_type_t t; - - mfn = p2m->get_entry (p2m, gfn, &t, &_a,0 ,NULL ,NULL ); +mfn_t mfn =__get_gfn_type_access (p2m,gfn_x (gfn), &t, &_a, +P2M_ALLOC ,NULL , false); rc = p2m->set_entry (p2m, gfn, mfn,PAGE_ORDER_4K , t, a,-1 ); }
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 17/18 x86/mem_sharing: reset a fork
In reply to Tamas K LengyelImplement hypercall that allows a fork to shed all memory that got allocated for it during its execution and re-load its vCPU context from the parent VM. This allows the forked VM to reset into the same state the parent VM is in a faster way then creating a new fork would be. Measurements show about a 2x speedup during normal fuzzing operations. Performance may vary depending how much memory got allocated for the forked VM. If it has been completely deduplicated from the parent VM then creating a new fork would likely be more performant.
--- xen/arch/x86/mm/mem_sharing.c | 76 +++++++++++++++++++++++++++++++++++ xen/include/public/memory.h | 1 + 2 files changed, 77 insertions(+)
xen/arch/x86/mm/mem_sharing.c +76 -0
@@ -1611,6 +1611,59 @@ static int mem_sharing_fork(struct domain *d, struct domain *cd)return rc; } +/* +* The fork reset operation is intended to be used on short-lived forks only. +* There is no hypercall continuation operation implemented for this reason. +* For forks that obtain a larger memory footprint it is likely going to be +* more performant to create a new fork instead of resetting an existing one. +* +* TODO: In case this hypercall would become useful on forks with larger memory +* footprints the hypercall continuation should be implemented. +*/ +static int mem_sharing_fork_reset (struct domain *d,struct domain *cd) +{ +int rc; +struct p2m_domain * p2m =p2m_get_hostp2m (cd); +struct page_info *page, *tmp; + +domain_pause (cd); + +page_list_for_each_safe (page, tmp, &cd->page_list) + { +p2m_type_t p2mt; +p2m_access_t p2ma; +gfn_t gfn; +mfn_t mfn =page_to_mfn (page); + +if ( !mfn_valid (mfn) ) +continue ; + + gfn =mfn_to_gfn (cd, mfn); + mfn =__get_gfn_type_access (p2m,gfn_x (gfn), &p2mt, &p2ma, +0 ,NULL , false); + +if ( !p2m_is_ram (p2mt) ||p2m_is_shared (p2mt) ) +continue ; + +/* take an extra reference */ +if ( !get_page (page, cd) ) +continue ; + + rc = p2m->set_entry (p2m, gfn,INVALID_MFN ,PAGE_ORDER_4K , + p2m_invalid, p2m_access_rwx,-1 ); +ASSERT (!rc); + +put_page_alloc_ref (page); +put_page (page); + } + +if ( !(rc =hvm_copy_context_and_params (d, cd)) ) +fork_tsc (d, cd); + +domain_unpause (cd); +return rc; +} +int mem_sharing_memop (XEN_GUEST_HANDLE_PARAM (xen_mem_sharing_op_t) arg) {int rc;
@@ -1895,6 +1948,29 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)break ; } +case XENMEM_sharing_op_fork_reset: + { +struct domain *pd; + + rc = -EINVAL ; +if ( mso.u.fork._pad[0 ] || mso.u.fork._pad[1 ] || + mso.u.fork._pad[2 ] ) + goto out; + + rc = -ENOSYS ; +if ( !d->parent ) + goto out; + + rc =rcu_lock_live_remote_domain_by_id (d->parent->domain_id, &pd); +if ( rc ) + goto out; + + rc =mem_sharing_fork_reset (pd, d); + +rcu_unlock_domain (pd); +break ; + } +default : rc = -ENOSYS ;break ;
xen/include/public/memory.h +1 -0
@@ -483,6 +483,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_mem_access_op_t);#define XENMEM_sharing_op_audit 7#define XENMEM_sharing_op_range_share 8#define XENMEM_sharing_op_fork 9 +#define XENMEM_sharing_op_fork_reset 10#define XENMEM_SHARING_OP_S_HANDLE_INVALID (-10)#define XENMEM_SHARING_OP_C_HANDLE_INVALID (-9)
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 18/18 xen/tools: VM forking toolstack side
In reply to Tamas K LengyelAdd necessary bits to implement "xl fork-vm" commands. The command allows the user to specify how to launch the device model allowing for a late-launch model in which the user can execute the fork without the device model and decide to only later launch it.
--- docs/man/xl.1.pod.in | 36 ++++++ tools/libxc/include/xenctrl.h | 13 ++ tools/libxc/xc_memshr.c | 22 ++++ tools/libxl/libxl.h | 7 + tools/libxl/libxl_create.c | 237 +++++++++++++++++++++++----------- tools/libxl/libxl_dm.c | 2 +- tools/libxl/libxl_dom.c | 83 ++++++++---- tools/libxl/libxl_internal.h | 1 + tools/libxl/libxl_types.idl | 1 + tools/xl/xl.h | 5 + tools/xl/xl_cmdtable.c | 12 ++ tools/xl/xl_saverestore.c | 96 ++++++++++++++ tools/xl/xl_vmcontrol.c | 8 ++ 13 files changed, 419 insertions(+), 104 deletions(-)
docs/man/xl.1.pod.in +36 -0
@@ -694,6 +694,42 @@ Leave the domain paused after creating the snapshot. =back +=item B<fork-vm> [I<OPTIONS>] I<domain-id> + +Create a fork of a running VM. The domain will be paused after the operation +and needs to remain paused while forks of it exist. + +B<OPTIONS> + +=over 4 + +=item B<-p> + +Leave the fork paused after creating it. + +=item B<--launch-dm> + +Specify whether the device model (QEMU) should be launched for the fork. Late +launch allows to start the device model for an already running fork. + +=item B<-C> + +The config file to use when launching the device model. Currently required when +launching the device model. + +=item B<-Q> + +The qemu save file to use when launching the device model. Currently required +when launching the device model. + +=item B<--fork-reset> + +Perform a reset operation of an already running fork. Note that resetting may +be less performant then creating a new fork depending on how much memory the +fork has deduplicated during its runtime. + +=back + =item B<sharing> [I<domain-id>] Display the number of shared pages for a specified domain. If no domain is
tools/libxc/include/xenctrl.h +13 -0
@@ -2221,6 +2221,19 @@ int xc_memshr_range_share(xc_interface *xch,uint64_t first_gfn, uint64_t last_gfn); +int xc_memshr_fork (xc_interface *xch, +uint32_t source_domain, +uint32_t client_domain); + +/* +* Note: this function is only intended to be used on short-lived forks that +* haven't yet aquired a lot of memory. In case the fork has a lot of memory +* it is likely more performant to create a new fork with xc_memshr_fork. +* +* With VMs that have a lot of memory this call may block for a long time. +*/ +int xc_memshr_fork_reset (xc_interface *xch,uint32_t forked_domain); +/* Debug calls: return the number of pages referencing the shared frame backing * the input argument. Should be one or greater. *
tools/libxc/xc_memshr.c +22 -0
@@ -239,6 +239,28 @@ int xc_memshr_debug_gref(xc_interface *xch,return xc_memshr_memop (xch, domid, &mso); } +int xc_memshr_fork (xc_interface *xch,uint32_t pdomid,uint32_t domid) +{ +xen_mem_sharing_op_t mso; + +memset (&mso,0 ,sizeof (mso)); + + mso.op = XENMEM_sharing_op_fork; + mso.u.fork.parent_domain = pdomid; + +return xc_memshr_memop (xch, domid, &mso); +} + +int xc_memshr_fork_reset (xc_interface *xch,uint32_t domid) +{ +xen_mem_sharing_op_t mso; + +memset (&mso,0 ,sizeof (mso)); + mso.op = XENMEM_sharing_op_fork_reset; + +return xc_memshr_memop (xch, domid, &mso); +} +int xc_memshr_audit (xc_interface *xch) {xen_mem_sharing_op_t mso;
tools/libxl/libxl.h +7 -0
@@ -1536,6 +1536,13 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config,const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how)LIBXL_EXTERNAL_CALLERS_ONLY ; +int libxl_domain_fork_vm(libxl_ctx *ctx, uint32_t pdomid, uint32_t *domid) + LIBXL_EXTERNAL_CALLERS_ONLY; +int libxl_domain_fork_launch_dm(libxl_ctx *ctx, libxl_domain_config *d_config, + uint32_t domid, + const libxl_asyncprogress_how *aop_console_how) + LIBXL_EXTERNAL_CALLERS_ONLY; +int libxl_domain_fork_reset (libxl_ctx *ctx,uint32_t domid); int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, uint32_t *domid, int restore_fd, int send_back_fd,
tools/libxl/libxl_create.c +161 -76
350 lines not on this page · raw message
tools/libxl/libxl_dm.c +1 -1
@@ -2788,7 +2788,7 @@ static void device_model_spawn_outcome(libxl__egc *egc,libxl__domain_build_state *state = dmss->build_state; -if (state->saved_state) { +if (state->saved_state && !state->forked_vm) { ret2 =unlink (state->saved_state);if (ret2) { LOGED(ERROR, dmss->guest_domid, "%s: failed to remove device-model state %s",
tools/libxl/libxl_dom.c +56 -27
141 lines not on this page · raw message
tools/libxl/libxl_internal.h +1 -0
@@ -1360,6 +1360,7 @@ typedef struct {char *saved_state;int dm_monitor_fd; +bool forked_vm;libxl__file_reference pv_kernel;libxl__file_reference pv_ramdisk;
tools/libxl/libxl_types.idl +1 -0
@@ -956,6 +956,7 @@ libxl_domain_config = Struct("domain_config", [ ("on_watchdog", libxl_action_on_shutdown), ("on_crash", libxl_action_on_shutdown), ("on_soft_reset", libxl_action_on_shutdown), + ("dm_restore_file", string, {'const': True}), ], dir=DIR_IN) libxl_diskinfo = Struct("diskinfo", [
tools/xl/xl.h +5 -0
@@ -31,6 +31,7 @@ struct cmd_spec { };struct domain_create { +uint32_t ddomid;/* fork launch dm for this domid */ int debug;int daemonize;int monitor;/* handle guest reboots etc */
@@ -45,6 +46,7 @@ struct domain_create {const char *config_file;char *extra_config;/* extra config string */ const char *restore_file; +const char *dm_restore_file;char *colo_proxy_script;bool userspace_colo_proxy;int migrate_fd;/* -1 means none */
@@ -127,6 +129,9 @@ int main_pciassignable_remove(int argc, char **argv);int main_pciassignable_list (int argc,char **argv);#ifndef LIBXL_HAVE_NO_SUSPEND_RESUME int main_restore (int argc,char **argv); +int main_fork_vm (int argc,char **argv); +int main_fork_launch_dm (int argc,char **argv); +int main_fork_reset (int argc,char **argv);int main_migrate_receive (int argc,char **argv);int main_save (int argc,char **argv);int main_migrate (int argc,char **argv);
tools/xl/xl_cmdtable.c +12 -0
@@ -185,6 +185,18 @@ struct cmd_spec cmd_table[] = { "Restore a domain from a saved state", "- for internal use only", }, + { "fork-vm", + &main_fork_vm, 0, 1, + "Fork a domain from the running parent domid", + "[options] <Domid>", + "-h Print this help.\n" + "-C <config> Use config file for VM fork.\n" + "-Q <qemu-save-file> Use qemu save file for VM fork.\n" + "--launch-dm <yes|no|late> Launch device model (QEMU) for VM fork.\n" + "--fork-reset Reset VM fork.\n" + "-p Do not unpause fork VM after operation.\n" + "-d Enable debug messages.\n" + }, #endif { "dump-core", &main_dump_core, 0, 1,
tools/xl/xl_saverestore.c +96 -0
102 lines not on this page · raw message
tools/xl/xl_vmcontrol.c +8 -0
@@ -645,6 +645,7 @@ int create_domain(struct domain_create *dom_info)libxl_domain_config d_config; +uint32_t ddomid = dom_info->ddomid;// launch dm for this domain iff set int debug = dom_info->debug;int daemonize = dom_info->daemonize;int monitor = dom_info->monitor;
@@ -655,6 +656,7 @@ int create_domain(struct domain_create *dom_info)const char *restore_file = dom_info->restore_file;const char *config_source =NULL ;const char *restore_source =NULL ; +const char *dm_restore_file = dom_info->dm_restore_file;int migrate_fd = dom_info->migrate_fd;bool config_in_json;
@@ -923,6 +925,12 @@ start:* restore/migrate-receive it again. */ restoring =0 ; + }else if ( ddomid ) { + d_config.dm_restore_file = dm_restore_file; + ret =libxl_domain_fork_launch_dm (ctx, &d_config, ddomid, + autoconnect_console_how); + domid = ddomid; + ddomid =INVALID_DOMID ; }else if (domid_soft_reset !=INVALID_DOMID ) {/* Do soft reset. */ ret = libxl_domain_soft_reset(ctx, &d_config, domid_soft_reset,
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel