viridian: implement more enlightenments
v8 cover viridian: implement more enlightenments
This series adds three new enlightenments:
- Synthetic timers, which depends on the...
- Synthetic interrupt controller (or SynIC)
- Synthetic cluster IPI
All these enlightenments are implemented in current versions of QEMU/KVM so this series closes the gap.
Paul Durrant (11):
viridian: add init hooks
viridian: separately allocate domain and vcpu structures
viridian: use stack variables for viridian_vcpu and viridian_domain...
viridian: make 'fields' struct anonymous...
viridian: extend init/deinit hooks into synic and time modules
viridian: add missing context save helpers into synic and time modules
viridian: use viridian_map/unmap_guest_page() for reference tsc page
viridian: stop directly calling
viridian_time_ref_count_freeze/thaw()...
viridian: add implementation of synthetic interrupt MSRs
viridian: add implementation of synthetic timers
viridian: add implementation of the HvSendSyntheticClusterIpi
hypercall
docs/man/xl.cfg.5.pod.in | 18 +- tools/libxl/libxl.h | 18 + tools/libxl/libxl_dom.c | 10 + tools/libxl/libxl_types.idl | 3 + xen/arch/x86/domain.c | 12 +- xen/arch/x86/hvm/hvm.c | 10 + xen/arch/x86/hvm/viridian/private.h | 31 +- xen/arch/x86/hvm/viridian/synic.c | 376 ++++++++++++++++-- xen/arch/x86/hvm/viridian/time.c | 515 ++++++++++++++++++++++--- xen/arch/x86/hvm/viridian/viridian.c | 229 +++++++++-- xen/arch/x86/hvm/vlapic.c | 20 +- xen/include/asm-x86/hvm/domain.h | 2 +- xen/include/asm-x86/hvm/hvm.h | 7 + xen/include/asm-x86/hvm/vcpu.h | 2 +- xen/include/asm-x86/hvm/viridian.h | 75 +++- xen/include/public/arch-x86/hvm/save.h | 4 + xen/include/public/hvm/params.h | 17 +- 17 files changed, 1209 insertions(+), 140 deletions(-)
v8:
- Squash in follow-up series
v4:
- Add two cleanup patches (#3 and #4) and re-order #8 and #9
v3:
- Add the synthetic cluster IPI patch (#11)
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Wei Liu <[email protected]>
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 1/11 viridian: add init hooks
This patch adds domain and vcpu init hooks for viridian features. The init hooks do not yet do anything; the functionality will be added to by subsequent patches.
---
Cc: Andrew Cooper <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v5:
- Put the call to viridian_domain_deinit() back into hvm_domain_relinquish_resources() where it should be
v3:
- Re-instate call from domain deinit to vcpu deinit
- Move deinit calls to avoid introducing new labels
v2:
- Remove call from domain deinit to vcpu deinit
--- xen/arch/x86/hvm/hvm.c | 10 ++++++++++ xen/arch/x86/hvm/viridian/viridian.c | 10 ++++++++++ xen/include/asm-x86/hvm/viridian.h | 3 +++ 3 files changed, 23 insertions(+)
xen/arch/x86/hvm/hvm.c +10 -0
@@ -666,6 +666,10 @@ int hvm_domain_initialise(struct domain *d)if ( hvm_tsc_scaling_supported ) d->arch.hvm.tsc_scaling_ratio = hvm_default_tsc_scaling_ratio; + rc =viridian_domain_init (d); +if ( rc ) + goto fail2; + rc = hvm_funcs.domain_initialise (d);if ( rc !=0 ) goto fail2;
@@ -687,6 +691,7 @@ int hvm_domain_initialise(struct domain *d)hvm_destroy_cacheattr_region_list (d);destroy_perdomain_mapping (d,PERDOMAIN_VIRT_START ,0 ); fail: +viridian_domain_deinit (d);return rc; }
@@ -1526,6 +1531,10 @@ int hvm_vcpu_initialise(struct vcpu *v) && (rc = nestedhvm_vcpu_initialise(v)) < 0 ) /* teardown: nestedhvm_vcpu_destroy */ goto fail5; + rc =viridian_vcpu_init (v); +if ( rc ) + goto fail5; + rc =hvm_all_ioreq_servers_add_vcpu (d, v);if ( rc !=0 ) goto fail6;
@@ -1553,6 +1562,7 @@ int hvm_vcpu_initialise(struct vcpu *v) fail2:hvm_vcpu_cacheattr_destroy (v); fail1: +viridian_vcpu_deinit (v);return rc; }
xen/arch/x86/hvm/viridian/viridian.c +10 -0
@@ -417,6 +417,16 @@ int guest_rdmsr_viridian(const struct vcpu *v, uint32_t idx, uint64_t *val)return X86EMUL_OKAY ; } +int viridian_vcpu_init (struct vcpu *v) +{ +return 0 ; +} + +int viridian_domain_init (struct domain *d) +{ +return 0 ; +} +void viridian_vcpu_deinit (struct vcpu *v) {viridian_synic_wrmsr (v,HV_X64_MSR_VP_ASSIST_PAGE ,0 );
xen/include/asm-x86/hvm/viridian.h +3 -0
@@ -80,6 +80,9 @@ viridian_hypercall(struct cpu_user_regs *regs);void viridian_time_ref_count_freeze (struct domain *d);void viridian_time_ref_count_thaw (struct domain *d); +int viridian_vcpu_init (struct vcpu *v); +int viridian_domain_init (struct domain *d); +void viridian_vcpu_deinit (struct vcpu *v);void viridian_domain_deinit (struct domain *d);
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 2/11 viridian: separately allocate domain and vcpu structures
In reply to Paul DurrantCurrently the viridian_domain and viridian_vcpu structures are inline in the hvm_domain and hvm_vcpu structures respectively. Subsequent patches will need to add sizable extra fields to the viridian structures which will cause the PAGE_SIZE limit of the overall vcpu structure to be exceeded. This patch, therefore, uses the new init hooks to separately allocate the structures and converts the 'viridian' fields in hvm_domain and hvm_cpu to be pointers to these allocations. These separate allocations also allow some vcpu and domain pointers to become const.
Ideally, now that they are no longer inline, the allocations of the viridian structures could be made conditional on whether the toolstack is going to configure the viridian enlightenments. However the toolstack is currently unable to convey this information to the domain creation code so such an enhancement is deferred until that becomes possible.
NOTE: The patch also introduced the 'is_viridian_vcpu' macro to avoid
introducing a second evaluation of 'is_viridian_domain' with an open-coded 'v->domain' argument. This macro will also be further used in a subsequent patch.
---
Cc: Andrew Cooper <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v4:
- Const-ify some vcpu and domain pointers
v2:
- use XFREE()
- expand commit comment to point out why allocations are unconditional
--- xen/arch/x86/hvm/viridian/private.h | 2 +- xen/arch/x86/hvm/viridian/synic.c | 46 ++++++++--------- xen/arch/x86/hvm/viridian/time.c | 38 +++++++------- xen/arch/x86/hvm/viridian/viridian.c | 75 ++++++++++++++++++---------- xen/include/asm-x86/hvm/domain.h | 2 +- xen/include/asm-x86/hvm/hvm.h | 4 ++ xen/include/asm-x86/hvm/vcpu.h | 2 +- xen/include/asm-x86/hvm/viridian.h | 10 ++-- 8 files changed, 101 insertions(+), 78 deletions(-)
xen/arch/x86/hvm/viridian/private.h +1 -1
@@ -89,7 +89,7 @@ void viridian_time_load_domain_ctxt(void viridian_dump_guest_page (const struct vcpu *v,const char *name,const struct viridian_page *vp); -void viridian_map_guest_page (struct vcpu *v,struct viridian_page *vp); +void viridian_map_guest_page (const struct vcpu *v,struct viridian_page *vp);void viridian_unmap_guest_page (struct viridian_page *vp);#endif /* X86_HVM_VIRIDIAN_PRIVATE_H */
xen/arch/x86/hvm/viridian/synic.c +23 -23
109 lines not on this page · raw message
xen/arch/x86/hvm/viridian/time.c +18 -20
@@ -27,7 +27,7 @@ typedef struct _HV_REFERENCE_TSC_PAGEstatic void dump_reference_tsc (const struct domain *d) { -const union viridian_page_msr *rt = &d->arch.hvm.viridian.reference_tsc; +const union viridian_page_msr *rt = &d->arch.hvm.viridian->reference_tsc;if ( !rt->fields.enabled )return ;
@@ -38,7 +38,7 @@ static void dump_reference_tsc(const struct domain *d)static void update_reference_tsc (struct domain *d,bool initialize) { -unsigned long gmfn = d->arch.hvm.viridian.reference_tsc.fields.pfn; +unsigned long gmfn = d->arch.hvm.viridian->reference_tsc.fields.pfn;struct page_info *page =get_page_from_gfn (d, gmfn,NULL ,P2M_ALLOC );HV_REFERENCE_TSC_PAGE *p;
@@ -107,7 +107,7 @@ static void update_reference_tsc(struct domain *d, bool initialize)put_page_and_type (page); } -static int64_t raw_trc_val (struct domain *d) +static int64_t raw_trc_val (const struct domain *d) {uint64_t tsc;struct time_scale tsc_to_ns;
@@ -119,21 +119,19 @@ static int64_t raw_trc_val(struct domain *d)return scale_delta (tsc, &tsc_to_ns) /100ul ; } -void viridian_time_ref_count_freeze (struct domain *d) +void viridian_time_ref_count_freeze (const struct domain *d) { -struct viridian_time_ref_count *trc; - - trc = &d->arch.hvm.viridian.time_ref_count; +struct viridian_time_ref_count *trc = + &d->arch.hvm.viridian->time_ref_count;if (test_and_clear_bit (_TRC_running, &trc->flags) ) trc->val =raw_trc_val (d) + trc->off; } -void viridian_time_ref_count_thaw (struct domain *d) +void viridian_time_ref_count_thaw (const struct domain *d) { -struct viridian_time_ref_count *trc; - - trc = &d->arch.hvm.viridian.time_ref_count; +struct viridian_time_ref_count *trc = + &d->arch.hvm.viridian->time_ref_count;if ( !d->is_shutting_down && !test_and_set_bit (_TRC_running, &trc->flags) )
@@ -150,9 +148,9 @@ int viridian_time_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val)if ( !(viridian_feature_mask (d) & HVMPV_reference_tsc) )return X86EMUL_EXCEPTION ; - d->arch.hvm.viridian.reference_tsc.raw = val; + d->arch.hvm.viridian->reference_tsc.raw = val;dump_reference_tsc (d); -if ( d->arch.hvm.viridian.reference_tsc.fields.enabled ) +if ( d->arch.hvm.viridian->reference_tsc.fields.enabled )update_reference_tsc (d, true);break ;
@@ -189,13 +187,13 @@ int viridian_time_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)if ( !(viridian_feature_mask (d) & HVMPV_reference_tsc) )return X86EMUL_EXCEPTION ; - *val = d->arch.hvm.viridian.reference_tsc.raw; + *val = d->arch.hvm.viridian->reference_tsc.raw;break ;case HV_X64_MSR_TIME_REF_COUNT : {struct viridian_time_ref_count *trc = - &d->arch.hvm.viridian.time_ref_count; + &d->arch.hvm.viridian->time_ref_count;if ( !(viridian_feature_mask (d) & HVMPV_time_ref_count) )return X86EMUL_EXCEPTION ;
@@ -219,17 +217,17 @@ int viridian_time_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)void viridian_time_save_domain_ctxt (const struct domain *d,struct hvm_viridian_domain_context *ctxt) { - ctxt->time_ref_count = d->arch.hvm.viridian.time_ref_count.val; - ctxt->reference_tsc = d->arch.hvm.viridian.reference_tsc.raw; + ctxt->time_ref_count = d->arch.hvm.viridian->time_ref_count.val; + ctxt->reference_tsc = d->arch.hvm.viridian->reference_tsc.raw; }void viridian_time_load_domain_ctxt (struct domain *d,const struct hvm_viridian_domain_context *ctxt) { - d->arch.hvm.viridian.time_ref_count.val = ctxt->time_ref_count; - d->arch.hvm.viridian.reference_tsc.raw = ctxt->reference_tsc; + d->arch.hvm.viridian->time_ref_count.val = ctxt->time_ref_count; + d->arch.hvm.viridian->reference_tsc.raw = ctxt->reference_tsc; -if ( d->arch.hvm.viridian.reference_tsc.fields.enabled ) +if ( d->arch.hvm.viridian->reference_tsc.fields.enabled )update_reference_tsc (d, false); }
xen/arch/x86/hvm/viridian/viridian.c +48 -27
193 lines not on this page · raw message
xen/include/asm-x86/hvm/domain.h +1 -1
@@ -154,7 +154,7 @@ struct hvm_domain {/* hypervisor intercepted msix table */ struct list_head msixtbl_list; -struct viridian_domain viridian; +struct viridian_domain *viridian;bool_t hap_enabled;bool_t mem_sharing_enabled;
xen/include/asm-x86/hvm/hvm.h +4 -0
@@ -463,6 +463,9 @@ static inline bool hvm_get_guest_bndcfgs(struct vcpu *v, u64 *val)#define is_viridian_domain (d) \ (is_hvm_domain(d) && (viridian_feature_mask(d) & HVMPV_base_freq)) +#define is_viridian_vcpu (v) \ + is_viridian_domain((v)->domain) +#define has_viridian_time_ref_count (d) \ (is_viridian_domain(d) && (viridian_feature_mask(d) & HVMPV_time_ref_count))
@@ -762,6 +765,7 @@ static inline bool hvm_has_set_descriptor_access_exiting(void) }#define is_viridian_domain (d) ((void)(d), false) +#define is_viridian_vcpu (v) ((void)(v), false)#define has_viridian_time_ref_count (d) ((void)(d), false)#define hvm_long_mode_active (v) ((void)(v), false)#define hvm_get_guest_time (v) ((void)(v), 0)
xen/include/asm-x86/hvm/vcpu.h +1 -1
@@ -205,7 +205,7 @@ struct hvm_vcpu {/* Pending hw/sw interrupt (.vector = -1 means nothing pending). */ struct x86_event inject_event; - struct viridian_vcpu viridian; + struct viridian_vcpu *viridian; };#endif /* __ASM_X86_HVM_VCPU_H__ */
xen/include/asm-x86/hvm/viridian.h +5 -5
@@ -77,8 +77,8 @@ int guest_rdmsr_viridian(const struct vcpu *v, uint32_t idx, uint64_t *val);int viridian_hypercall (struct cpu_user_regs *regs); -void viridian_time_ref_count_freeze (struct domain *d); -void viridian_time_ref_count_thaw (struct domain *d); +void viridian_time_ref_count_freeze (const struct domain *d); +void viridian_time_ref_count_thaw (const struct domain *d);int viridian_vcpu_init (struct vcpu *v);int viridian_domain_init (struct domain *d);
@@ -86,9 +86,9 @@ int viridian_domain_init(struct domain *d);void viridian_vcpu_deinit (struct vcpu *v);void viridian_domain_deinit (struct domain *d); -void viridian_apic_assist_set (struct vcpu *v); -bool viridian_apic_assist_completed (struct vcpu *v); -void viridian_apic_assist_clear (struct vcpu *v); +void viridian_apic_assist_set (const struct vcpu *v); +bool viridian_apic_assist_completed (const struct vcpu *v); +void viridian_apic_assist_clear (const struct vcpu *v);#endif /* __ASM_X86_HVM_VIRIDIAN_H__ */
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 3/11 viridian: use stack variables for viridian_vcpu and viridian_domain...
In reply to Paul Durrant...where there is more than one dereference inside a function.
This shortens the code and makes it more readable. No functional change.
---
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v4:
- New in v4
--- xen/arch/x86/hvm/viridian/synic.c | 49 ++++++++++++++++------------ xen/arch/x86/hvm/viridian/time.c | 27 ++++++++------- xen/arch/x86/hvm/viridian/viridian.c | 47 +++++++++++++------------- 3 files changed, 69 insertions(+), 54 deletions(-)
xen/arch/x86/hvm/viridian/synic.c +28 -21
@@ -30,7 +30,8 @@ typedef union _HV_VP_ASSIST_PAGEvoid viridian_apic_assist_set (const struct vcpu *v) { -HV_VP_ASSIST_PAGE *ptr = v->arch.hvm.viridian->vp_assist.ptr; +struct viridian_vcpu *vv = v->arch.hvm.viridian; +HV_VP_ASSIST_PAGE *ptr = vv->vp_assist.ptr;if ( !ptr )return ;
@@ -40,25 +41,25 @@ void viridian_apic_assist_set(const struct vcpu *v)* wrong and the VM will most likely hang so force a crash now * to make the problem clear. */ -if ( v->arch.hvm.viridian->apic_assist_pending ) +if ( vv->apic_assist_pending )domain_crash (v->domain); - v->arch.hvm.viridian->apic_assist_pending = true; + vv->apic_assist_pending = true; ptr->ApicAssist.no_eoi =1 ; }bool viridian_apic_assist_completed (const struct vcpu *v) { -HV_VP_ASSIST_PAGE *ptr = v->arch.hvm.viridian->vp_assist.ptr; +struct viridian_vcpu *vv = v->arch.hvm.viridian; +HV_VP_ASSIST_PAGE *ptr = vv->vp_assist.ptr;if ( !ptr )return false; -if ( v->arch.hvm.viridian->apic_assist_pending && - !ptr->ApicAssist.no_eoi ) +if ( vv->apic_assist_pending && !ptr->ApicAssist.no_eoi ) {/* An EOI has been avoided */ - v->arch.hvm.viridian->apic_assist_pending = false; + vv->apic_assist_pending = false;return true; }
@@ -67,17 +68,20 @@ bool viridian_apic_assist_completed(const struct vcpu *v)void viridian_apic_assist_clear (const struct vcpu *v) { -HV_VP_ASSIST_PAGE *ptr = v->arch.hvm.viridian->vp_assist.ptr; +struct viridian_vcpu *vv = v->arch.hvm.viridian; +HV_VP_ASSIST_PAGE *ptr = vv->vp_assist.ptr;if ( !ptr )return ; ptr->ApicAssist.no_eoi =0 ; - v->arch.hvm.viridian->apic_assist_pending = false; + vv->apic_assist_pending = false; }int viridian_synic_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val) { +struct viridian_vcpu *vv = v->arch.hvm.viridian; +switch ( idx ) {case HV_X64_MSR_EOI :
@@ -95,12 +99,11 @@ int viridian_synic_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val)case HV_X64_MSR_VP_ASSIST_PAGE :/* release any previous mapping */ -viridian_unmap_guest_page (&v->arch.hvm.viridian->vp_assist); - v->arch.hvm.viridian->vp_assist.msr.raw = val; -viridian_dump_guest_page (v,"VP_ASSIST" , - &v->arch.hvm.viridian->vp_assist); -if ( v->arch.hvm.viridian->vp_assist.msr.fields.enabled ) -viridian_map_guest_page (v, &v->arch.hvm.viridian->vp_assist); +viridian_unmap_guest_page (&vv->vp_assist); + vv->vp_assist.msr.raw = val; +viridian_dump_guest_page (v,"VP_ASSIST" , &vv->vp_assist); +if ( vv->vp_assist.msr.fields.enabled ) +viridian_map_guest_page (v, &vv->vp_assist);break ;default :
@@ -146,18 +149,22 @@ int viridian_synic_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)void viridian_synic_save_vcpu_ctxt (const struct vcpu *v,struct hvm_viridian_vcpu_context *ctxt) { - ctxt->apic_assist_pending = v->arch.hvm.viridian->apic_assist_pending; - ctxt->vp_assist_msr = v->arch.hvm.viridian->vp_assist.msr.raw; +const struct viridian_vcpu *vv = v->arch.hvm.viridian; + + ctxt->apic_assist_pending = vv->apic_assist_pending; + ctxt->vp_assist_msr = vv->vp_assist.msr.raw; }void viridian_synic_load_vcpu_ctxt (struct vcpu *v,const struct hvm_viridian_vcpu_context *ctxt) { - v->arch.hvm.viridian->vp_assist.msr.raw = ctxt->vp_assist_msr; -if ( v->arch.hvm.viridian->vp_assist.msr.fields.enabled ) -viridian_map_guest_page (v, &v->arch.hvm.viridian->vp_assist); +struct viridian_vcpu *vv = v->arch.hvm.viridian; + + vv->vp_assist.msr.raw = ctxt->vp_assist_msr; +if ( vv->vp_assist.msr.fields.enabled ) +viridian_map_guest_page (v, &vv->vp_assist); - v->arch.hvm.viridian->apic_assist_pending = ctxt->apic_assist_pending; + vv->apic_assist_pending = ctxt->apic_assist_pending; }/*
xen/arch/x86/hvm/viridian/time.c +16 -11
@@ -141,6 +141,7 @@ void viridian_time_ref_count_thaw(const struct domain *d)int viridian_time_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val) {struct domain *d = v->domain; +struct viridian_domain *vd = d->arch.hvm.viridian;switch ( idx ) {
@@ -148,9 +149,9 @@ int viridian_time_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val)if ( !(viridian_feature_mask (d) & HVMPV_reference_tsc) )return X86EMUL_EXCEPTION ; - d->arch.hvm.viridian->reference_tsc.raw = val; + vd->reference_tsc.raw = val;dump_reference_tsc (d); -if ( d->arch.hvm.viridian->reference_tsc.fields.enabled ) +if ( vd->reference_tsc.fields.enabled )update_reference_tsc (d, true);break ;
@@ -165,7 +166,8 @@ int viridian_time_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val)int viridian_time_rdmsr (const struct vcpu *v,uint32_t idx,uint64_t *val) { -struct domain *d = v->domain; +const struct domain *d = v->domain; +struct viridian_domain *vd = d->arch.hvm.viridian;switch ( idx ) {
@@ -187,13 +189,12 @@ int viridian_time_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)if ( !(viridian_feature_mask (d) & HVMPV_reference_tsc) )return X86EMUL_EXCEPTION ; - *val = d->arch.hvm.viridian->reference_tsc.raw; + *val = vd->reference_tsc.raw;break ;case HV_X64_MSR_TIME_REF_COUNT : { -struct viridian_time_ref_count *trc = - &d->arch.hvm.viridian->time_ref_count; +struct viridian_time_ref_count *trc = &vd->time_ref_count;if ( !(viridian_feature_mask (d) & HVMPV_time_ref_count) )return X86EMUL_EXCEPTION ;
@@ -217,17 +218,21 @@ int viridian_time_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)void viridian_time_save_domain_ctxt (const struct domain *d,struct hvm_viridian_domain_context *ctxt) { - ctxt->time_ref_count = d->arch.hvm.viridian->time_ref_count.val; - ctxt->reference_tsc = d->arch.hvm.viridian->reference_tsc.raw; +const struct viridian_domain *vd = d->arch.hvm.viridian; + + ctxt->time_ref_count = vd->time_ref_count.val; + ctxt->reference_tsc = vd->reference_tsc.raw; }void viridian_time_load_domain_ctxt (struct domain *d,const struct hvm_viridian_domain_context *ctxt) { - d->arch.hvm.viridian->time_ref_count.val = ctxt->time_ref_count; - d->arch.hvm.viridian->reference_tsc.raw = ctxt->reference_tsc; +struct viridian_domain *vd = d->arch.hvm.viridian; + + vd->time_ref_count.val = ctxt->time_ref_count; + vd->reference_tsc.raw = ctxt->reference_tsc; -if ( d->arch.hvm.viridian->reference_tsc.fields.enabled ) +if ( vd->reference_tsc.fields.enabled )update_reference_tsc (d, false); }
xen/arch/x86/hvm/viridian/viridian.c +25 -22
138 lines not on this page · raw message
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 4/11 viridian: make 'fields' struct anonymous...
In reply to Paul Durrant...inside viridian_page_msr and viridian_guest_os_id_msr unions.
There's no need to name it and the code is shortened by not doing so. No functional change.
---
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v4:
- New in v4
--- xen/arch/x86/hvm/viridian/synic.c | 4 ++-- xen/arch/x86/hvm/viridian/time.c | 10 +++++----- xen/arch/x86/hvm/viridian/viridian.c | 20 +++++++++----------- xen/include/asm-x86/hvm/viridian.h | 4 ++-- 4 files changed, 18 insertions(+), 20 deletions(-)
xen/arch/x86/hvm/viridian/synic.c +2 -2
@@ -102,7 +102,7 @@ int viridian_synic_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val)viridian_unmap_guest_page (&vv->vp_assist); vv->vp_assist.msr.raw = val;viridian_dump_guest_page (v,"VP_ASSIST" , &vv->vp_assist); -if ( vv->vp_assist.msr.fields.enabled ) +if ( vv->vp_assist.msr.enabled )viridian_map_guest_page (v, &vv->vp_assist);break ;
@@ -161,7 +161,7 @@ void viridian_synic_load_vcpu_ctxt(struct viridian_vcpu *vv = v->arch.hvm.viridian; vv->vp_assist.msr.raw = ctxt->vp_assist_msr; -if ( vv->vp_assist.msr.fields.enabled ) +if ( vv->vp_assist.msr.enabled )viridian_map_guest_page (v, &vv->vp_assist); vv->apic_assist_pending = ctxt->apic_assist_pending;
xen/arch/x86/hvm/viridian/time.c +5 -5
@@ -29,16 +29,16 @@ static void dump_reference_tsc(const struct domain *d) {const union viridian_page_msr *rt = &d->arch.hvm.viridian->reference_tsc; -if ( !rt->fields.enabled ) +if ( !rt->enabled )return ;printk (XENLOG_G_INFO "d%d: VIRIDIAN REFERENCE_TSC: pfn: %lx\n" , - d->domain_id, (unsigned long )rt->fields.pfn); + d->domain_id, (unsigned long )rt->pfn); }static void update_reference_tsc (struct domain *d,bool initialize) { -unsigned long gmfn = d->arch.hvm.viridian->reference_tsc.fields.pfn; +unsigned long gmfn = d->arch.hvm.viridian->reference_tsc.pfn;struct page_info *page =get_page_from_gfn (d, gmfn,NULL ,P2M_ALLOC );HV_REFERENCE_TSC_PAGE *p;
@@ -151,7 +151,7 @@ int viridian_time_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val) vd->reference_tsc.raw = val;dump_reference_tsc (d); -if ( vd->reference_tsc.fields.enabled ) +if ( vd->reference_tsc.enabled )update_reference_tsc (d, true);break ;
@@ -232,7 +232,7 @@ void viridian_time_load_domain_ctxt( vd->time_ref_count.val = ctxt->time_ref_count; vd->reference_tsc.raw = ctxt->reference_tsc; -if ( vd->reference_tsc.fields.enabled ) +if ( vd->reference_tsc.enabled )update_reference_tsc (d, false); }
xen/arch/x86/hvm/viridian/viridian.c +9 -11
@@ -192,7 +192,7 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,case 4 :/* Recommended hypercall usage. */ -if ( vd->guest_os_id.raw ==0 || vd->guest_os_id.fields.os <4 ) +if ( vd->guest_os_id.raw ==0 || vd->guest_os_id.os <4 )break ; res->a =CPUID4A_RELAX_TIMER_INT ;if (viridian_feature_mask (d) & HVMPV_hcall_remote_tlb_flush )
@@ -228,10 +228,8 @@ static void dump_guest_os_id(const struct domain *d)printk (XENLOG_G_INFO "d%d: VIRIDIAN GUEST_OS_ID: vendor: %x os: %x major: %x minor: %x sp: %x build: %x\n" , - d->domain_id, - goi->fields.vendor, goi->fields.os, - goi->fields.major, goi->fields.minor, - goi->fields.service_pack, goi->fields.build_number); + d->domain_id, goi->vendor, goi->os, goi->major, goi->minor, + goi->service_pack, goi->build_number); }static void dump_hypercall (const struct domain *d)
@@ -242,12 +240,12 @@ static void dump_hypercall(const struct domain *d)printk (XENLOG_G_INFO "d%d: VIRIDIAN HYPERCALL: enabled: %x pfn: %lx\n" , d->domain_id, - hg->fields.enabled, (unsigned long )hg->fields.pfn); + hg->enabled, (unsigned long )hg->pfn); }static void enable_hypercall_page (struct domain *d) { -unsigned long gmfn = d->arch.hvm.viridian->hypercall_gpa.fields.pfn; +unsigned long gmfn = d->arch.hvm.viridian->hypercall_gpa.pfn;struct page_info *page =get_page_from_gfn (d, gmfn,NULL ,P2M_ALLOC );uint8_t *p;
@@ -297,7 +295,7 @@ int guest_wrmsr_viridian(struct vcpu *v, uint32_t idx, uint64_t val)case HV_X64_MSR_HYPERCALL : vd->hypercall_gpa.raw = val;dump_hypercall (d); -if ( vd->hypercall_gpa.fields.enabled ) +if ( vd->hypercall_gpa.enabled )enable_hypercall_page (d);break ;
@@ -606,17 +604,17 @@ out:void viridian_dump_guest_page (const struct vcpu *v,const char *name,const struct viridian_page *vp) { -if ( !vp->msr.fields.enabled ) +if ( !vp->msr.enabled )return ;printk (XENLOG_G_INFO "%pv: VIRIDIAN %s: pfn: %lx\n" , - v, name, (unsigned long )vp->msr.fields.pfn); + v, name, (unsigned long )vp->msr.pfn); }void viridian_map_guest_page (const struct vcpu *v,struct viridian_page *vp) {struct domain *d = v->domain; -unsigned long gmfn = vp->msr.fields.pfn; +unsigned long gmfn = vp->msr.pfn;struct page_info *page;if ( vp->ptr )
xen/include/asm-x86/hvm/viridian.h +2 -2
@@ -17,7 +17,7 @@ union viridian_page_msr uint64_t enabled:1; uint64_t reserved_preserved:11; uint64_t pfn:48; - } fields; + }; };struct viridian_page
@@ -44,7 +44,7 @@ union viridian_guest_os_id_msr uint64_t major:8; uint64_t os:8; uint64_t vendor:16; - } fields; + }; };struct viridian_time_ref_count
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 5/11 viridian: extend init/deinit hooks into synic and time modules
In reply to Paul DurrantThis patch simply adds domain and vcpu init/deinit hooks into the synic and time modules and wires them into viridian_[domain|vcpu]_[init|deinit](). Only one of the hooks is currently needed (to unmap the 'VP Assist' page) but subsequent patches will make use of the others.
NOTE: To perform the unmap of the VP Assist page,
viridian_unmap_guest_page() is now directly called in the new viridian_synic_vcpu_deinit() function (which is safe even if is_viridian_vcpu() evaluates to false). This replaces the slightly hacky mechanism of faking a zero write to the HV_X64_MSR_VP_ASSIST_PAGE MSR in viridian_cpu_deinit().
---
Cc: Andrew Cooper <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v4:
- Constify vcpu and domain pointers
v2:
- Pay attention to sync and time init hook return values
--- xen/arch/x86/hvm/viridian/private.h | 12 +++++++++ xen/arch/x86/hvm/viridian/synic.c | 19 ++++++++++++++ xen/arch/x86/hvm/viridian/time.c | 18 ++++++++++++++ xen/arch/x86/hvm/viridian/viridian.c | 37 ++++++++++++++++++++++++++-- 4 files changed, 84 insertions(+), 2 deletions(-)
xen/arch/x86/hvm/viridian/private.h +12 -0
@@ -74,6 +74,12 @@int viridian_synic_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val);int viridian_synic_rdmsr (const struct vcpu *v,uint32_t idx,uint64_t *val); +int viridian_synic_vcpu_init (const struct vcpu *v); +int viridian_synic_domain_init (const struct domain *d); + +void viridian_synic_vcpu_deinit (const struct vcpu *v); +void viridian_synic_domain_deinit (const struct domain *d); +void viridian_synic_save_vcpu_ctxt (const struct vcpu *v,struct hvm_viridian_vcpu_context *ctxt); void viridian_synic_load_vcpu_ctxt(
@@ -82,6 +88,12 @@ void viridian_synic_load_vcpu_ctxt(int viridian_time_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val);int viridian_time_rdmsr (const struct vcpu *v,uint32_t idx,uint64_t *val); +int viridian_time_vcpu_init (const struct vcpu *v); +int viridian_time_domain_init (const struct domain *d); + +void viridian_time_vcpu_deinit (const struct vcpu *v); +void viridian_time_domain_deinit (const struct domain *d); +void viridian_time_save_domain_ctxt (const struct domain *d,struct hvm_viridian_domain_context *ctxt); void viridian_time_load_domain_ctxt(
xen/arch/x86/hvm/viridian/synic.c +19 -0
@@ -146,6 +146,25 @@ int viridian_synic_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)return X86EMUL_OKAY ; } +int viridian_synic_vcpu_init (const struct vcpu *v) +{ +return 0 ; +} + +int viridian_synic_domain_init (const struct domain *d) +{ +return 0 ; +} + +void viridian_synic_vcpu_deinit (const struct vcpu *v) +{ +viridian_unmap_guest_page (&v->arch.hvm.viridian->vp_assist); +} + +void viridian_synic_domain_deinit (const struct domain *d) +{ +} +void viridian_synic_save_vcpu_ctxt (const struct vcpu *v,struct hvm_viridian_vcpu_context *ctxt) {
xen/arch/x86/hvm/viridian/time.c +18 -0
@@ -215,6 +215,24 @@ int viridian_time_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)return X86EMUL_OKAY ; } +int viridian_time_vcpu_init (const struct vcpu *v) +{ +return 0 ; +} + +int viridian_time_domain_init (const struct domain *d) +{ +return 0 ; +} + +void viridian_time_vcpu_deinit (const struct vcpu *v) +{ +} + +void viridian_time_domain_deinit (const struct domain *d) +{ +} +void viridian_time_save_domain_ctxt (const struct domain *d,struct hvm_viridian_domain_context *ctxt) {
xen/arch/x86/hvm/viridian/viridian.c +35 -2
@@ -418,22 +418,52 @@ int guest_rdmsr_viridian(const struct vcpu *v, uint32_t idx, uint64_t *val)int viridian_vcpu_init (struct vcpu *v) { +int rc; +ASSERT (!v->arch.hvm.viridian); v->arch.hvm.viridian = xzalloc(struct viridian_vcpu);if ( !v->arch.hvm.viridian )return -ENOMEM ; + rc =viridian_synic_vcpu_init (v); +if ( rc ) + goto fail; + + rc =viridian_time_vcpu_init (v); +if ( rc ) + goto fail; +return 0 ; + + fail: +viridian_vcpu_deinit (v); + +return rc; }int viridian_domain_init (struct domain *d) { +int rc; +ASSERT (!d->arch.hvm.viridian); d->arch.hvm.viridian = xzalloc(struct viridian_domain);if ( !d->arch.hvm.viridian )return -ENOMEM ; + rc =viridian_synic_domain_init (d); +if ( rc ) + goto fail; + + rc =viridian_time_domain_init (d); +if ( rc ) + goto fail; +return 0 ; + + fail: +viridian_domain_deinit (d); + +return rc; }void viridian_vcpu_deinit (struct vcpu *v)
@@ -441,8 +471,8 @@ void viridian_vcpu_deinit(struct vcpu *v)if ( !v->arch.hvm.viridian )return ; -if (is_viridian_vcpu (v) ) -viridian_synic_wrmsr (v,HV_X64_MSR_VP_ASSIST_PAGE ,0 ); +viridian_time_vcpu_deinit (v); +viridian_synic_vcpu_deinit (v);XFREE (v->arch.hvm.viridian); }
@@ -457,6 +487,9 @@ void viridian_domain_deinit(struct domain *d)if ( !d->arch.hvm.viridian )return ; +viridian_time_domain_deinit (d); +viridian_synic_domain_deinit (d); +XFREE (d->arch.hvm.viridian); }
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 6/11 viridian: add missing context save helpers into synic and time modules
In reply to Paul DurrantCurrently the time module lacks vcpu context save helpers and the synic module lacks domain context save helpers. These helpers are not yet required but subsequent patches will require at least some of them so this patch completes the set to avoid introducing them in an ad-hoc way.
---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v3:
- Add missing callers so that they are not added in an ad-hoc way
--- xen/arch/x86/hvm/viridian/private.h | 10 ++++++++++ xen/arch/x86/hvm/viridian/synic.c | 10 ++++++++++ xen/arch/x86/hvm/viridian/time.c | 10 ++++++++++ xen/arch/x86/hvm/viridian/viridian.c | 4 ++++ 4 files changed, 34 insertions(+)
xen/arch/x86/hvm/viridian/private.h +10 -0
@@ -85,6 +85,11 @@ void viridian_synic_save_vcpu_ctxt(const struct vcpu *v,void viridian_synic_load_vcpu_ctxt (struct vcpu *v,const struct hvm_viridian_vcpu_context *ctxt); +void viridian_synic_save_domain_ctxt ( +const struct domain *d,struct hvm_viridian_domain_context *ctxt); +void viridian_synic_load_domain_ctxt ( +struct domain *d,const struct hvm_viridian_domain_context *ctxt); +int viridian_time_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val);int viridian_time_rdmsr (const struct vcpu *v,uint32_t idx,uint64_t *val);
@@ -94,6 +99,11 @@ int viridian_time_domain_init(const struct domain *d);void viridian_time_vcpu_deinit (const struct vcpu *v);void viridian_time_domain_deinit (const struct domain *d); +void viridian_time_save_vcpu_ctxt ( +const struct vcpu *v,struct hvm_viridian_vcpu_context *ctxt); +void viridian_time_load_vcpu_ctxt ( +struct vcpu *v,const struct hvm_viridian_vcpu_context *ctxt); +void viridian_time_save_domain_ctxt (const struct domain *d,struct hvm_viridian_domain_context *ctxt); void viridian_time_load_domain_ctxt(
xen/arch/x86/hvm/viridian/synic.c +10 -0
@@ -186,6 +186,16 @@ void viridian_synic_load_vcpu_ctxt( vv->apic_assist_pending = ctxt->apic_assist_pending; } +void viridian_synic_save_domain_ctxt ( +const struct domain *d,struct hvm_viridian_domain_context *ctxt) +{ +} + +void viridian_synic_load_domain_ctxt ( +struct domain *d,const struct hvm_viridian_domain_context *ctxt) +{ +} +/* * Local variables: * mode: C
xen/arch/x86/hvm/viridian/time.c +10 -0
@@ -233,6 +233,16 @@ void viridian_time_domain_deinit(const struct domain *d) { } +void viridian_time_save_vcpu_ctxt ( +const struct vcpu *v,struct hvm_viridian_vcpu_context *ctxt) +{ +} + +void viridian_time_load_vcpu_ctxt ( +struct vcpu *v,const struct hvm_viridian_vcpu_context *ctxt) +{ +} +void viridian_time_save_domain_ctxt (const struct domain *d,struct hvm_viridian_domain_context *ctxt) {
xen/arch/x86/hvm/viridian/viridian.c +4 -0
@@ -707,6 +707,7 @@ static int viridian_save_domain_ctxt(struct vcpu *v,return 0 ;viridian_time_save_domain_ctxt (d, &ctxt); +viridian_synic_save_domain_ctxt (d, &ctxt);return (hvm_save_entry (VIRIDIAN_DOMAIN ,0 , h, &ctxt) !=0 ); }
@@ -723,6 +724,7 @@ static int viridian_load_domain_ctxt(struct domain *d, vd->hypercall_gpa.raw = ctxt.hypercall_gpa; vd->guest_os_id.raw = ctxt.guest_os_id; +viridian_synic_load_domain_ctxt (d, &ctxt);viridian_time_load_domain_ctxt (d, &ctxt);return 0 ;
@@ -738,6 +740,7 @@ static int viridian_save_vcpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)if ( !is_viridian_vcpu (v) )return 0 ; +viridian_time_save_vcpu_ctxt (v, &ctxt);viridian_synic_save_vcpu_ctxt (v, &ctxt);return hvm_save_entry (VIRIDIAN_VCPU , v->vcpu_id, h, &ctxt);
@@ -764,6 +767,7 @@ static int viridian_load_vcpu_ctxt(struct domain *d,return -EINVAL ;viridian_synic_load_vcpu_ctxt (v, &ctxt); +viridian_time_load_vcpu_ctxt (v, &ctxt);return 0 ; }
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 7/11 viridian: use viridian_map/unmap_guest_page() for reference tsc page
In reply to Paul DurrantWhilst the reference tsc page does not currently need to be kept mapped after it is initially set up (or updated after migrate), the code can be simplified by using the common guest page map/unmap and dump functions. New functionality added by a subsequent patch will also require the page to kept mapped for the lifetime of the domain.
NOTE: Because the reference tsc page is per-domain rather than per-vcpu
this patch also changes viridian_map_guest_page() to take a domain pointer rather than a vcpu pointer. The domain pointer cannot be const, unlike the vcpu pointer.
---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
--- xen/arch/x86/hvm/viridian/private.h | 2 +- xen/arch/x86/hvm/viridian/synic.c | 6 ++- xen/arch/x86/hvm/viridian/time.c | 56 +++++++++------------------- xen/arch/x86/hvm/viridian/viridian.c | 3 +- xen/include/asm-x86/hvm/viridian.h | 2 +- 5 files changed, 25 insertions(+), 44 deletions(-)
xen/arch/x86/hvm/viridian/private.h +1 -1
@@ -111,7 +111,7 @@ void viridian_time_load_domain_ctxt(void viridian_dump_guest_page (const struct vcpu *v,const char *name,const struct viridian_page *vp); -void viridian_map_guest_page (const struct vcpu *v,struct viridian_page *vp); +void viridian_map_guest_page (struct domain *d,struct viridian_page *vp);void viridian_unmap_guest_page (struct viridian_page *vp);#endif /* X86_HVM_VIRIDIAN_PRIVATE_H */
xen/arch/x86/hvm/viridian/synic.c +4 -2
@@ -81,6 +81,7 @@ void viridian_apic_assist_clear(const struct vcpu *v)int viridian_synic_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val) {struct viridian_vcpu *vv = v->arch.hvm.viridian; +struct domain *d = v->domain;switch ( idx ) {
@@ -103,7 +104,7 @@ int viridian_synic_wrmsr(struct vcpu *v, uint32_t idx, uint64_t val) vv->vp_assist.msr.raw = val;viridian_dump_guest_page (v,"VP_ASSIST" , &vv->vp_assist);if ( vv->vp_assist.msr.enabled ) -viridian_map_guest_page (v, &vv->vp_assist); +viridian_map_guest_page (d, &vv->vp_assist);break ;default :
@@ -178,10 +179,11 @@ void viridian_synic_load_vcpu_ctxt( struct vcpu *v, const struct hvm_viridian_vcpu_context *ctxt) { struct viridian_vcpu *vv = v->arch.hvm.viridian; +struct domain *d = v->domain; vv->vp_assist.msr.raw = ctxt->vp_assist_msr;if ( vv->vp_assist.msr.enabled ) -viridian_map_guest_page (v, &vv->vp_assist); +viridian_map_guest_page (d, &vv->vp_assist); vv->apic_assist_pending = ctxt->apic_assist_pending; }
xen/arch/x86/hvm/viridian/time.c +18 -38
109 lines not on this page · raw message
xen/arch/x86/hvm/viridian/viridian.c +1 -2
@@ -644,9 +644,8 @@ void viridian_dump_guest_page(const struct vcpu *v, const char *name, v, name, (unsigned long)vp->msr.pfn); } -void viridian_map_guest_page (const struct vcpu *v,struct viridian_page *vp) +void viridian_map_guest_page (struct domain *d,struct viridian_page *vp) { -struct domain *d = v->domain;unsigned long gmfn = vp->msr.pfn;struct page_info *page;
xen/include/asm-x86/hvm/viridian.h +1 -1
@@ -65,7 +65,7 @@ struct viridian_domainunion viridian_guest_os_id_msr guest_os_id;union viridian_page_msr hypercall_gpa;struct viridian_time_ref_count time_ref_count; -union viridian_page_msr reference_tsc; +struct viridian_page reference_tsc; }; void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 8/11 viridian: stop directly calling viridian_time_ref_count_freeze/thaw()...
In reply to Paul Durrant...from arch_domain_shutdown/pause/unpause().
A subsequent patch will introduce an implementaion of synthetic timers which will also need freeze/thaw hooks, so make the exported hooks more generic and call through to (re-named and static) time_ref_count_freeze/thaw functions.
NOTE: This patch also introduces a new time_ref_count() helper to return
the current counter value. This is currently only used by the MSR read handler but the synthetic timer code will also need to use it.
---
Cc: Andrew Cooper <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
--- xen/arch/x86/domain.c | 12 ++++++------ xen/arch/x86/hvm/viridian/time.c | 24 +++++++++++++++++++++--- xen/include/asm-x86/hvm/viridian.h | 4 ++-- 3 files changed, 29 insertions(+), 11 deletions(-)
xen/arch/x86/domain.c +6 -6
@@ -657,20 +657,20 @@ void arch_domain_destroy(struct domain *d)void arch_domain_shutdown (struct domain *d) { -if (has_viridian_time_ref_count (d) ) -viridian_time_ref_count_freeze (d); +if (is_viridian_domain (d) ) +viridian_time_domain_freeze (d); }void arch_domain_pause (struct domain *d) { -if (has_viridian_time_ref_count (d) ) -viridian_time_ref_count_freeze (d); +if (is_viridian_domain (d) ) +viridian_time_domain_freeze (d); }void arch_domain_unpause (struct domain *d) { -if (has_viridian_time_ref_count (d) ) -viridian_time_ref_count_thaw (d); +if (is_viridian_domain (d) ) +viridian_time_domain_thaw (d); }int arch_domain_soft_reset (struct domain *d)
xen/arch/x86/hvm/viridian/time.c +21 -3
@@ -91,7 +91,7 @@ static int64_t raw_trc_val(const struct domain *d)return scale_delta (tsc, &tsc_to_ns) /100ul ; } -void viridian_time_ref_count_freeze (const struct domain *d) +static void time_ref_count_freeze (const struct domain *d) {struct viridian_time_ref_count *trc = &d->arch.hvm.viridian->time_ref_count;
@@ -100,7 +100,7 @@ void viridian_time_ref_count_freeze(const struct domain *d) trc->val =raw_trc_val (d) + trc->off; } -void viridian_time_ref_count_thaw (const struct domain *d) +static void time_ref_count_thaw (const struct domain *d) {struct viridian_time_ref_count *trc = &d->arch.hvm.viridian->time_ref_count;
@@ -110,6 +110,24 @@ void viridian_time_ref_count_thaw(const struct domain *d) trc->off = (int64_t )trc->val -raw_trc_val (d); } +static int64_t time_ref_count (const struct domain *d) +{ +struct viridian_time_ref_count *trc = + &d->arch.hvm.viridian->time_ref_count; + +return raw_trc_val (d) + trc->off; +} + +void viridian_time_domain_freeze (const struct domain *d) +{ +time_ref_count_freeze (d); +} + +void viridian_time_domain_thaw (const struct domain *d) +{ +time_ref_count_thaw (d); +} +int viridian_time_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val) {struct domain *d = v->domain;
@@ -179,7 +197,7 @@ int viridian_time_rdmsr(const struct vcpu *v, uint32_t idx, uint64_t *val)printk (XENLOG_G_INFO "d%d: VIRIDIAN MSR_TIME_REF_COUNT: accessed\n" , d->domain_id); - *val =raw_trc_val (d) + trc->off; + *val =time_ref_count (d);break ; }
xen/include/asm-x86/hvm/viridian.h +2 -2
@@ -77,8 +77,8 @@ int guest_rdmsr_viridian(const struct vcpu *v, uint32_t idx, uint64_t *val);int viridian_hypercall (struct cpu_user_regs *regs); -void viridian_time_ref_count_freeze (const struct domain *d); -void viridian_time_ref_count_thaw (const struct domain *d); +void viridian_time_domain_freeze (const struct domain *d); +void viridian_time_domain_thaw (const struct domain *d);int viridian_vcpu_init (struct vcpu *v);int viridian_domain_init (struct domain *d);
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 9/11 viridian: add implementation of synthetic interrupt MSRs
In reply to Paul DurrantThis patch introduces an implementation of the SCONTROL, SVERSION, SIEFP, SIMP, EOM and SINT0-15 SynIC MSRs. No message source is added and, as such, nothing will yet generate a synthetic interrupt. A subsequent patch will add an implementation of synthetic timers which will need the infrastructure added by this patch to deliver expiry messages to the guest.
NOTE: A 'synic' option is added to the toolstack viridian enlightenments
enumeration but is deliberately not documented as enabling these SynIC registers without a message source is only useful for debugging.
---
Cc: Ian Jackson <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v8:
- Squash in https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg01332.html
v7:
- Fix out label indentation
v6:
- Address further comments from Jan
v4:
- Address comments from Jan
v3:
- Add the 'SintPollingModeAvailable' bit in CPUID leaf 3
--- tools/libxl/libxl.h | 6 + tools/libxl/libxl_dom.c | 3 + tools/libxl/libxl_types.idl | 1 + xen/arch/x86/hvm/viridian/synic.c | 241 ++++++++++++++++++++++++- xen/arch/x86/hvm/viridian/viridian.c | 19 ++ xen/arch/x86/hvm/vlapic.c | 20 +- xen/include/asm-x86/hvm/hvm.h | 3 + xen/include/asm-x86/hvm/viridian.h | 26 +++ xen/include/public/arch-x86/hvm/save.h | 2 + xen/include/public/hvm/params.h | 7 +- 10 files changed, 323 insertions(+), 5 deletions(-)
tools/libxl/libxl.h +6 -0
@@ -318,6 +318,12 @@*/ #define LIBXL_HAVE_VIRIDIAN_CRASH_CTL 1 +/* +* LIBXL_HAVE_VIRIDIAN_SYNIC indicates that the 'synic' value +* is present in the viridian enlightenment enumeration. +*/ +#define LIBXL_HAVE_VIRIDIAN_SYNIC 1 +/* * LIBXL_HAVE_BUILDINFO_HVM_ACPI_LAPTOP_SLATE indicates that * libxl_domain_build_info has the u.hvm.acpi_laptop_slate field.
tools/libxl/libxl_dom.c +3 -0
@@ -317,6 +317,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,if (libxl_bitmap_test (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_CRASH_CTL )) mask |= HVMPV_crash_ctl; +if (libxl_bitmap_test (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_SYNIC )) + mask |= HVMPV_synic; +if (mask !=0 && xc_hvm_param_set(CTX->xch, domid,
tools/libxl/libxl_types.idl +1 -0
@@ -235,6 +235,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [ (4, "hcall_remote_tlb_flush"), (5, "apic_assist"), (6, "crash_ctl"), + (7, "synic"), ]) libxl_hdtype = Enumeration("hdtype", [
xen/arch/x86/hvm/viridian/synic.c +239 -2
318 lines not on this page · raw message
xen/arch/x86/hvm/viridian/viridian.c +19 -0
@@ -89,6 +89,7 @@ typedef union _HV_CRASH_CTL_REG_CONTENTS/* Viridian CPUID leaf 3, Hypervisor Feature Indication */ #define CPUID3D_CRASH_MSRS (1 << 10) +#define CPUID3D_SINT_POLLING (1 << 17)/* Viridian CPUID leaf 4: Implementation Recommendations. */ #define CPUID4A_HCALL_REMOTE_TLB_FLUSH (1 << 2)
@@ -178,6 +179,8 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf, mask.AccessPartitionReferenceCounter =1 ;if (viridian_feature_mask (d) & HVMPV_reference_tsc ) mask.AccessPartitionReferenceTsc =1 ; +if (viridian_feature_mask (d) & HVMPV_synic ) + mask.AccessSynicRegs =1 ; u.mask = mask;
@@ -186,6 +189,8 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,if (viridian_feature_mask (d) & HVMPV_crash_ctl ) res->d =CPUID3D_CRASH_MSRS ; +if (viridian_feature_mask (d) & HVMPV_synic ) + res->d |=CPUID3D_SINT_POLLING ;break ; }
@@ -306,8 +311,16 @@ int guest_wrmsr_viridian(struct vcpu *v, uint32_t idx, uint64_t val)case HV_X64_MSR_ICR :case HV_X64_MSR_TPR :case HV_X64_MSR_VP_ASSIST_PAGE : +case HV_X64_MSR_SCONTROL : +case HV_X64_MSR_SVERSION : +case HV_X64_MSR_SIEFP : +case HV_X64_MSR_SIMP : +case HV_X64_MSR_EOM : + case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:return viridian_synic_wrmsr (v, idx, val); +case HV_X64_MSR_TSC_FREQUENCY : +case HV_X64_MSR_APIC_FREQUENCY :case HV_X64_MSR_REFERENCE_TSC :return viridian_time_wrmsr (v, idx, val);
@@ -378,6 +391,12 @@ int guest_rdmsr_viridian(const struct vcpu *v, uint32_t idx, uint64_t *val)case HV_X64_MSR_ICR :case HV_X64_MSR_TPR :case HV_X64_MSR_VP_ASSIST_PAGE : +case HV_X64_MSR_SCONTROL : +case HV_X64_MSR_SVERSION : +case HV_X64_MSR_SIEFP : +case HV_X64_MSR_SIMP : +case HV_X64_MSR_EOM : + case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:return viridian_synic_rdmsr (v, idx, val);case HV_X64_MSR_TSC_FREQUENCY :
xen/arch/x86/hvm/vlapic.c +18 -2
@@ -461,10 +461,15 @@ void vlapic_EOI_set(struct vlapic *vlapic)void vlapic_handle_EOI (struct vlapic *vlapic,u8 vector) { -struct domain *d =vlapic_domain (vlapic); +struct vcpu *v =vlapic_vcpu (vlapic); +struct domain *d = v->domain; + +/* All synic SINTx vectors are edge triggered */ if (vlapic_test_vector (vector, &vlapic->regs->data[APIC_TMR ]) )vioapic_update_EOI (d, vector); +else if (has_viridian_synic (d) ) +viridian_synic_ack_sint (v, vector);hvm_dpci_msi_eoi (d, vector); }
@@ -1301,6 +1306,13 @@ int vlapic_has_pending_irq(struct vcpu *v)if ( !vlapic_enabled (vlapic) )return -1 ; +/* +* Poll the viridian message queues before checking the IRR since +* a synthetic interrupt may be asserted during the poll. +*/ +if (has_viridian_synic (v->domain) ) +viridian_synic_poll (v); + irr =vlapic_find_highest_irr (vlapic);if ( irr ==-1 )return -1 ;
@@ -1360,8 +1372,12 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool_t force_ack) } done: -vlapic_set_vector (vector, &vlapic->regs->data[APIC_ISR ]); +if ( !has_viridian_synic (v->domain) || + !viridian_synic_is_auto_eoi_sint (v, vector) ) +vlapic_set_vector (vector, &vlapic->regs->data[APIC_ISR ]); +vlapic_clear_irr (vector, vlapic); +return 1 ; }
xen/include/asm-x86/hvm/hvm.h +3 -0
@@ -472,6 +472,9 @@ static inline bool hvm_get_guest_bndcfgs(struct vcpu *v, u64 *val)#define has_viridian_apic_assist (d) \ (is_viridian_domain(d) && (viridian_feature_mask(d) & HVMPV_apic_assist)) +#define has_viridian_synic (d) \ + (is_viridian_domain(d) && (viridian_feature_mask(d) & HVMPV_synic)) +static inline void hvm_inject_exception (unsigned int vector,unsigned int type,unsigned int insn_len,int error_code)
xen/include/asm-x86/hvm/viridian.h +26 -0
@@ -26,10 +26,31 @@ struct viridian_pagevoid *ptr; }; +union viridian_sint_msr +{ +uint64_t raw; +struct + { +uint64_t vector:8 ; +uint64_t reserved_preserved1:8 ; +uint64_t mask:1 ; +uint64_t auto_eoi:1 ; +uint64_t polling:1 ; +uint64_t reserved_preserved2:45 ; + }; +}; +struct viridian_vcpu {struct viridian_page vp_assist;bool apic_assist_pending; +bool polled; +unsigned int msg_pending; +uint64_t scontrol; +uint64_t siefp; +struct viridian_page simp; +union viridian_sint_msr sint[16 ]; +uint8_t vector_to_sintx[256 ];uint64_t crash_param[5 ]; };
@@ -90,6 +111,11 @@ void viridian_apic_assist_set(const struct vcpu *v);bool viridian_apic_assist_completed (const struct vcpu *v);void viridian_apic_assist_clear (const struct vcpu *v); +void viridian_synic_poll (const struct vcpu *v); +bool viridian_synic_is_auto_eoi_sint (const struct vcpu *v, +unsigned int vector); +void viridian_synic_ack_sint (const struct vcpu *v,unsigned int vector); +#endif /* __ASM_X86_HVM_VIRIDIAN_H__ */ /*
xen/include/public/arch-x86/hvm/save.h +2 -0
@@ -602,6 +602,8 @@ struct hvm_viridian_vcpu_context {uint64_t vp_assist_msr;uint8_t apic_assist_pending;uint8_t _pad[7 ]; +uint64_t simp_msr; +uint64_t sint_msr[16 ]; }; DECLARE_HVM_SAVE_TYPE(VIRIDIAN_VCPU, 17, struct hvm_viridian_vcpu_context);
xen/include/public/hvm/params.h +6 -1
@@ -146,6 +146,10 @@#define _HVMPV_crash_ctl 6#define HVMPV_crash_ctl (1 << _HVMPV_crash_ctl) +/* Enable SYNIC MSRs */ +#define _HVMPV_synic 7 +#define HVMPV_synic (1 << _HVMPV_synic) +#define HVMPV_feature_mask \ (HVMPV_base_freq | \ HVMPV_no_freq | \
@@ -153,7 +157,8 @@ HVMPV_reference_tsc | \ HVMPV_hcall_remote_tlb_flush | \ HVMPV_apic_assist | \ - HVMPV_crash_ctl) + HVMPV_crash_ctl | \ + HVMPV_synic)#endif
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 10/11 viridian: add implementation of synthetic timers
In reply to Paul DurrantThis patch introduces an implementation of the STIMER0-15_CONFIG/COUNT MSRs and hence a the first SynIC message source.
The new (and documented) 'stimer' viridian enlightenment group may be specified to enable this feature.
While in the neighbourhood, this patch adds a missing check for an attempt to write the time reference count MSR, which should result in an exception (but not be reported as an unimplemented MSR).
NOTE: It is necessary for correct operation that timer expiration and
message delivery time-stamping use the same time source as the guest. The specification is ambiguous but testing with a Windows 10 1803 guest has shown that using the partition reference counter as a source whilst the guest is using RDTSC and the reference tsc page does not work correctly. Therefore the time_now() function is used. This implements the algorithm for acquiring partition reference time that is documented in the specifiction.
---
Cc: Ian Jackson <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v8:
- Squash in https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg01333.html
v7:
- Make sure missed count cannot be zero if expiration < now
v6:
- Stop using the reference tsc page in time_now()
- Address further comments from Jan
v5:
- Fix time_now() to read TSC as the guest would see it
v4:
- Address comments from Jan
v3:
- Re-worked missed ticks calculation
--- docs/man/xl.cfg.5.pod.in | 12 +- tools/libxl/libxl.h | 6 + tools/libxl/libxl_dom.c | 4 + tools/libxl/libxl_types.idl | 1 + xen/arch/x86/hvm/viridian/private.h | 9 +- xen/arch/x86/hvm/viridian/synic.c | 55 +++- xen/arch/x86/hvm/viridian/time.c | 386 ++++++++++++++++++++++++- xen/arch/x86/hvm/viridian/viridian.c | 5 + xen/include/asm-x86/hvm/viridian.h | 32 +- xen/include/public/arch-x86/hvm/save.h | 2 + xen/include/public/hvm/params.h | 7 +- 11 files changed, 506 insertions(+), 13 deletions(-)
docs/man/xl.cfg.5.pod.in +10 -2
@@ -2167,11 +2167,19 @@ This group incorporates the crash control MSRs. These enlightenments allow Windows to write crash information such that it can be logged by Xen. +=item B<stimer> + +This set incorporates the SynIC and synthetic timer MSRs. Windows will +use synthetic timers in preference to emulated HPET for a source of +ticks and hence enabling this group will ensure that ticks will be +consistent with use of an enlightened time source (B<time_ref_count> or +B<reference_tsc>). + =item B<defaults> This is a special value that enables the default set of groups, which -is currently the B<base>, B<freq>, B<time_ref_count>, B<apic_assist> -and B<crash_ctl> groups. +is currently the B<base>, B<freq>, B<time_ref_count>, B<apic_assist>, +B<crash_ctl> and B<stimer> groups. =item B<all>
tools/libxl/libxl.h +6 -0
@@ -324,6 +324,12 @@*/ #define LIBXL_HAVE_VIRIDIAN_SYNIC 1 +/* +* LIBXL_HAVE_VIRIDIAN_STIMER indicates that the 'stimer' value +* is present in the viridian enlightenment enumeration. +*/ +#define LIBXL_HAVE_VIRIDIAN_STIMER 1 +/* * LIBXL_HAVE_BUILDINFO_HVM_ACPI_LAPTOP_SLATE indicates that * libxl_domain_build_info has the u.hvm.acpi_laptop_slate field.
tools/libxl/libxl_dom.c +4 -0
@@ -269,6 +269,7 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,libxl_bitmap_set (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_TIME_REF_COUNT );libxl_bitmap_set (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_APIC_ASSIST );libxl_bitmap_set (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_CRASH_CTL ); +libxl_bitmap_set (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_STIMER ); }libxl_for_each_set_bit (v, info->u.hvm.viridian_enable) {
@@ -320,6 +321,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,if (libxl_bitmap_test (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_SYNIC )) mask |= HVMPV_synic; +if (libxl_bitmap_test (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_STIMER )) + mask |= HVMPV_time_ref_count | HVMPV_synic | HVMPV_stimer; +if (mask !=0 && xc_hvm_param_set(CTX->xch, domid,
tools/libxl/libxl_types.idl +1 -0
@@ -236,6 +236,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [ (5, "apic_assist"), (6, "crash_ctl"), (7, "synic"), + (8, "stimer"), ]) libxl_hdtype = Enumeration("hdtype", [
xen/arch/x86/hvm/viridian/private.h +8 -1
@@ -74,6 +74,11 @@int viridian_synic_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val);int viridian_synic_rdmsr (const struct vcpu *v,uint32_t idx,uint64_t *val); +bool viridian_synic_deliver_timer_msg (struct vcpu *v,unsigned int sintx, +unsigned int index, +uint64_t expiration, +uint64_t delivery); +int viridian_synic_vcpu_init (const struct vcpu *v);int viridian_synic_domain_init (const struct domain *d);
@@ -93,7 +98,9 @@ void viridian_synic_load_domain_ctxt(int viridian_time_wrmsr (struct vcpu *v,uint32_t idx,uint64_t val);int viridian_time_rdmsr (const struct vcpu *v,uint32_t idx,uint64_t *val); -int viridian_time_vcpu_init (const struct vcpu *v); +void viridian_time_poll_timers (struct vcpu *v); + +int viridian_time_vcpu_init (struct vcpu *v);int viridian_time_domain_init (const struct domain *d);void viridian_time_vcpu_deinit (const struct vcpu *v);
xen/arch/x86/hvm/viridian/synic.c +53 -2
62 lines not on this page · raw message
xen/arch/x86/hvm/viridian/time.c +380 -6
472 lines not on this page · raw message
xen/arch/x86/hvm/viridian/viridian.c +5 -0
@@ -181,6 +181,8 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf, mask.AccessPartitionReferenceTsc =1 ;if (viridian_feature_mask (d) & HVMPV_synic ) mask.AccessSynicRegs =1 ; +if (viridian_feature_mask (d) & HVMPV_stimer ) + mask.AccessSyntheticTimerRegs =1 ; u.mask = mask;
@@ -322,6 +324,8 @@ int guest_wrmsr_viridian(struct vcpu *v, uint32_t idx, uint64_t val)case HV_X64_MSR_TSC_FREQUENCY :case HV_X64_MSR_APIC_FREQUENCY :case HV_X64_MSR_REFERENCE_TSC : +case HV_X64_MSR_TIME_REF_COUNT : + case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:return viridian_time_wrmsr (v, idx, val);case HV_X64_MSR_CRASH_P0 :
@@ -403,6 +407,7 @@ int guest_rdmsr_viridian(const struct vcpu *v, uint32_t idx, uint64_t *val)case HV_X64_MSR_APIC_FREQUENCY :case HV_X64_MSR_REFERENCE_TSC :case HV_X64_MSR_TIME_REF_COUNT : + case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:return viridian_time_rdmsr (v, idx, val);case HV_X64_MSR_CRASH_P0 :
xen/include/asm-x86/hvm/viridian.h +31 -1
@@ -40,6 +40,32 @@ union viridian_sint_msr }; }; +union viridian_stimer_config_msr +{ +uint64_t raw; +struct + { +uint64_t enabled:1 ; +uint64_t periodic:1 ; +uint64_t lazy:1 ; +uint64_t auto_enable:1 ; +uint64_t vector:8 ; +uint64_t direct_mode:1 ; +uint64_t reserved_zero1:3 ; +uint64_t sintx:4 ; +uint64_t reserved_zero2:44 ; + }; +}; + +struct viridian_stimer { +struct vcpu *v; +struct timer timer; +union viridian_stimer_config_msr config; +uint64_t count; +uint64_t expiration; +bool started; +}; +struct viridian_vcpu {struct viridian_page vp_assist;
@@ -51,6 +77,9 @@ struct viridian_vcpustruct viridian_page simp;union viridian_sint_msr sint[16 ];uint8_t vector_to_sintx[256 ]; +struct viridian_stimer stimer[4 ]; +unsigned int stimer_enabled; +unsigned int stimer_pending;uint64_t crash_param[5 ]; };
@@ -87,6 +116,7 @@ struct viridian_domainunion viridian_page_msr hypercall_gpa;struct viridian_time_ref_count time_ref_count;struct viridian_page reference_tsc; +bool reference_tsc_valid; }; void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
@@ -111,7 +141,7 @@ void viridian_apic_assist_set(const struct vcpu *v);bool viridian_apic_assist_completed (const struct vcpu *v);void viridian_apic_assist_clear (const struct vcpu *v); -void viridian_synic_poll (const struct vcpu *v); +void viridian_synic_poll (struct vcpu *v);bool viridian_synic_is_auto_eoi_sint (const struct vcpu *v,unsigned int vector);void viridian_synic_ack_sint (const struct vcpu *v,unsigned int vector);
xen/include/public/arch-x86/hvm/save.h +2 -0
@@ -604,6 +604,8 @@ struct hvm_viridian_vcpu_context {uint8_t _pad[7 ];uint64_t simp_msr;uint64_t sint_msr[16 ]; +uint64_t stimer_config_msr[4 ]; +uint64_t stimer_count_msr[4 ]; }; DECLARE_HVM_SAVE_TYPE(VIRIDIAN_VCPU, 17, struct hvm_viridian_vcpu_context);
xen/include/public/hvm/params.h +6 -1
@@ -150,6 +150,10 @@#define _HVMPV_synic 7#define HVMPV_synic (1 << _HVMPV_synic) +/* Enable STIMER MSRs */ +#define _HVMPV_stimer 8 +#define HVMPV_stimer (1 << _HVMPV_stimer) +#define HVMPV_feature_mask \ (HVMPV_base_freq | \ HVMPV_no_freq | \
@@ -158,7 +162,8 @@ HVMPV_hcall_remote_tlb_flush | \ HVMPV_apic_assist | \ HVMPV_crash_ctl | \ - HVMPV_synic) + HVMPV_synic | \ + HVMPV_stimer)#endif
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 18.03.19 at 12:20, <[email protected]> wrote:
xen/arch/x86/hvm/viridian/time.c +2
-0@@ -72,11 +77,14 @@ static void update_reference_tsc(struct domain *d, bool initialize)* ticks per 100ns shifted left by 64. */ p->TscScale = ((10000ul <<32 ) / d->arch.tsc_khz) <<32 ; +smp_wmb (); +
xen/arch/x86/hvm/viridian/time.c static void update_reference_tsc(struct domain *d, bool initialize) · line 87
Quoted 10 lines
xen/arch/x86/hvm/viridian/time.c +5
-4+ seq = p->TscSequence +1 ; +if ( seq ==0xFFFFFFFF || seq ==0 )/* Avoid both 'invalid' values */ + seq =1 ; - p->TscSequence++; -if ( p->TscSequence ==0xFFFFFFFF || - p->TscSequence ==0 )/* Avoid both 'invalid' values */ - p->TscSequence =1 ; + p->TscSequence = seq; + vd->reference_tsc_valid = true;
Strictly speaking, don't you need another smp_wmb() between these two lines?
Quoted 49 lines
xen/arch/x86/hvm/viridian/time.c +49
-0+static void start_stimer (struct viridian_stimer *vs) +{ +const struct vcpu *v = vs->v; +struct viridian_vcpu *vv = v->arch.hvm.viridian; +unsigned int stimerx = vs - &vv->stimer[0 ]; +int64_t now =time_now (v->domain); +int64_t expiration; +s_time_t timeout; + +if ( !test_and_set_bit (stimerx, &vv->stimer_enabled) ) +printk (XENLOG_G_INFO "%pv: VIRIDIAN STIMER%u: enabled\n" , v, + stimerx); + +if ( vs->config.periodic ) + { +/* +* The specification says that if the timer is lazy then we +* skip over any missed expirations so we can treat this case +* as the same as if the timer is currently stopped, i.e. we +* just schedule expiration to be 'count' ticks from now. +*/ +if ( !vs->started || vs->config.lazy ) + expiration = now + vs->count; +else + { +unsigned int missed =0 ; + +/* +* The timer is already started, so we're re-scheduling. +* Hence advance the timer expiration by one tick. +*/ + expiration = vs->expiration + vs->count; + +/* Now check to see if any expirations have been missed */ +if ( expiration - now <=0 ) + missed = ((now - expiration) / vs->count) +1 ; + +/* +* The specification says that if the timer is not lazy then +* a non-zero missed count should be used to reduce the period +* of the timer until it catches up, unless the count has +* reached a 'significant number', in which case the timer +* should be treated as lazy. Unfortunately the specification +* does not state what that number is so the choice of number +* here is a pure guess. +*/ +if ( missed >3 ) + expiration = now + vs->count; +else if ( missed )
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 263
Quoted 10 lines
xen/arch/x86/hvm/viridian/time.c +10
-0+ expiration = now + (vs->count / missed); + } + } +else + { + expiration = vs->count; +if ( expiration - now <=0 ) + { + vs->expiration = expiration; +stimer_expire (vs);
Aren't you introducing a risk for races by calling the timer function directly from here? start_timer(), after all, gets called from quite a few places.
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 273
Quoted 10 lines
xen/arch/x86/hvm/viridian/time.c +10
-0+return ; + } + } +ASSERT (expiration - now >0 ); + + vs->expiration = expiration; + timeout = (expiration - now) *100ull ; + + vs->started = true; +migrate_timer (&vs->timer,smp_processor_id ());
Why is this smp_processor_id() when viridian_time_vcpu_init() uses v->processor? How relevant is it in the first place to trace the pCPU the vCPU runs on for the timer?
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 22 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 14:24
To: Paul Durrant <[email protected]>
Cc: Julien Grall <[email protected]>; Andrew Cooper <[email protected]>; Roger Pau Monne <[email protected]>; Wei Liu <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Stefano Stabellini <[email protected]>; xen-devel <xen- [email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Tim (Xen.org) <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation of synthetic timersQuoted 11 lines
Quoted 1 line
Quoted 1 line · On 18.03.19 at 12:20, <[email protected]> wrote:
xen/arch/x86/hvm/viridian/time.c +5
-0@@ -72,11 +77,14 @@ static void update_reference_tsc(struct domain *d, bool initialize)* ticks per 100ns shifted left by 64. */ p->TscScale = ((10000ul <<32 ) / d->arch.tsc_khz) <<32 ; +smp_wmb (); + + seq = p->TscSequence +1 ; +if ( seq ==0xFFFFFFFF || seq ==0 )/* Avoid both 'invalid' values */ + seq =1 ;
Quoted 10 lines
xen/arch/x86/hvm/viridian/time.c static void update_reference_tsc(struct domain *d, bool initialize) · line 87
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +2
-4- p->TscSequence++; -if ( p->TscSequence ==0xFFFFFFFF || - p->TscSequence ==0 )/* Avoid both 'invalid' values */ - p->TscSequence =1 ; + p->TscSequence = seq; + vd->reference_tsc_valid = true;Strictly speaking, don't you need another
smp_wmb()between these two lines?
Since the data in the page is not used by time_now() I don't think so.
Quoted 55 lines
Quoted 55 lines
xen/arch/x86/hvm/viridian/time.c +55
-0+static void start_stimer (struct viridian_stimer *vs) +{ +const struct vcpu *v = vs->v; +struct viridian_vcpu *vv = v->arch.hvm.viridian; +unsigned int stimerx = vs - &vv->stimer[0 ]; +int64_t now =time_now (v->domain); +int64_t expiration; +s_time_t timeout; + +if ( !test_and_set_bit (stimerx, &vv->stimer_enabled) ) +printk (XENLOG_G_INFO "%pv: VIRIDIAN STIMER%u: enabled\n" , v, + stimerx); + +if ( vs->config.periodic ) + { +/* +* The specification says that if the timer is lazy then we +* skip over any missed expirations so we can treat this case +* as the same as if the timer is currently stopped, i.e. we +* just schedule expiration to be 'count' ticks from now. +*/ +if ( !vs->started || vs->config.lazy ) + expiration = now + vs->count; +else + { +unsigned int missed =0 ; + +/* +* The timer is already started, so we're re-scheduling. +* Hence advance the timer expiration by one tick. +*/ + expiration = vs->expiration + vs->count; + +/* Now check to see if any expirations have been missed */ +if ( expiration - now <=0 ) + missed = ((now - expiration) / vs->count) +1 ; + +/* +* The specification says that if the timer is not lazy then +* a non-zero missed count should be used to reduce the period +* of the timer until it catches up, unless the count has +* reached a 'significant number', in which case the timer +* should be treated as lazy. Unfortunately the specification +* does not state what that number is so the choice of number +* here is a pure guess. +*/ +if ( missed >3 ) + expiration = now + vs->count; +else if ( missed ) + expiration = now + (vs->count / missed); + } + } +else + { + expiration = vs->count;
Quoted 9 lines
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 263
Quoted 4 lines
xen/arch/x86/hvm/viridian/time.c +4
-0+if ( expiration - now <=0 ) + { + vs->expiration = expiration; +stimer_expire (vs);Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.
In practice I don't think there should be any problematic race, but I'll check again.
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +6
-0+return ; + } + } +ASSERT (expiration - now >0 ); + + vs->expiration = expiration;
Quoted 9 lines
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 273
Quoted 4 lines
xen/arch/x86/hvm/viridian/time.c +4
-0+ timeout = (expiration - now) *100ull ; + + vs->started = true; +migrate_timer (&vs->timer,smp_processor_id ());Why is this
smp_processor_id()whenviridian_time_vcpu_init()usesv->processor? How relevant is it in the first place to trace the pCPU the vCPU runs on for the timer?
I was just following suit with other timer code. It seems to be the norm to migrate to the current pCPU just prior to starting a timer.
Paul
Quoted 2 lines
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 26 lines
Quoted 15 lines
Quoted 1 line · On 18.03.19 at 15:37, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 14:24Quoted 11 lines
Quoted 1 line
Quoted 1 line · On 18.03.19 at 12:20, <[email protected]> wrote:
xen/arch/x86/hvm/viridian/time.c +5
-0@@ -72,11 +77,14 @@ static void update_reference_tsc(struct domain *d, bool initialize)* ticks per 100ns shifted left by 64. */ p->TscScale = ((10000ul <<32 ) / d->arch.tsc_khz) <<32 ; +smp_wmb (); + + seq = p->TscSequence +1 ; +if ( seq ==0xFFFFFFFF || seq ==0 )/* Avoid both 'invalid' values */ + seq =1 ;Quoted 10 lines
xen/arch/x86/hvm/viridian/time.c static void update_reference_tsc(struct domain *d, bool initialize) · line 87
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +2
-4- p->TscSequence++; -if ( p->TscSequence ==0xFFFFFFFF || - p->TscSequence ==0 )/* Avoid both 'invalid' values */ - p->TscSequence =1 ; + p->TscSequence = seq; + vd->reference_tsc_valid = true;Strictly speaking, don't you need another
smp_wmb()between these two lines?
Quoted 1 line
Since the data in the page is not used by
time_now()I don't think so.
Oh, have I been remembering an old version of the patch, where there was a consumer of p->TscSequence?
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +6
-0+return ; + } + } +ASSERT (expiration - now >0 ); + + vs->expiration = expiration;Quoted 8 lines
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 273
Quoted 4 lines
xen/arch/x86/hvm/viridian/time.c +4
-0+ timeout = (expiration - now) *100ull ; + + vs->started = true; +migrate_timer (&vs->timer,smp_processor_id ());Why is this
smp_processor_id()whenviridian_time_vcpu_init()usesv->processor? How relevant is it in the first place to trace the pCPU the vCPU runs on for the timer?
Quoted 2 lines
I was just following suit with other timer code. It seems to be the norm to migrate to the current pCPU just prior to starting a timer.
But wouldn't v->processor then be more visibly correct (besides likely being cheaper to get at), as to the correlation to the vCPU in question? I can't actually see why "migrate to the current pCPU" would be the norm; I could only see this as an implication from that other code you looked at simply acting on the current vCPU.
Then again I'm having trouble spotting why it would be important for the timer to run on the same CPU the vCPU runs one. By the time the timer fires, the vCPU may have gone elsewhere.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 37 lines
-----Original Message-----
From: Xen-devel [mailto:[email protected]] On Behalf Of Jan Beulich
Sent: 18 March 2019 15:21
To: Paul Durrant <[email protected]>
Cc: Stefano Stabellini <[email protected]>; Wei Liu <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Andrew Cooper <[email protected]>; Tim (Xen.org) <[email protected]>; George Dunlap <[email protected]>; Julien Grall <[email protected]>; xen-devel <xen- [email protected]>; Ian Jackson <[email protected]>; Roger Pau Monne <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation of synthetic timersQuoted 26 lines
Quoted 25 lines
Quoted 1 line · On 18.03.19 at 15:37, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 14:24Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 18.03.19 at 12:20, <[email protected]> wrote:
xen/arch/x86/hvm/viridian/time.c +2
-0@@ -72,11 +77,14 @@ static void update_reference_tsc(struct domain *d, bool initialize)* ticks per 100ns shifted left by 64. */ p->TscScale = ((10000ul <<32 ) / d->arch.tsc_khz) <<32 ; +smp_wmb (); +xen/arch/x86/hvm/viridian/time.c static void update_reference_tsc(struct domain *d, bool initialize) · line 87
Quoted 10 lines
xen/arch/x86/hvm/viridian/time.c +5
-4+ seq = p->TscSequence +1 ; +if ( seq ==0xFFFFFFFF || seq ==0 )/* Avoid both 'invalid' values */ + seq =1 ; - p->TscSequence++; -if ( p->TscSequence ==0xFFFFFFFF || - p->TscSequence ==0 )/* Avoid both 'invalid' values */ - p->TscSequence =1 ; + p->TscSequence = seq; + vd->reference_tsc_valid = true;Strictly speaking, don't you need another
smp_wmb()between these two lines?
Quoted 4 lines
Quoted 1 line
Since the data in the page is not used by
time_now()I don't think so.Oh, have I been remembering an old version of the patch, where there was a consumer of
p->TscSequence?
Yes, it was in a previous version of the patch. The reason reference_tsc_valid was added was so that time_now() no longer needs to check any contents of the guest page.
Quoted 25 lines
Quoted 16 lines
Quoted 15 lines
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 273
Quoted 11 lines
xen/arch/x86/hvm/viridian/time.c +10
-0+return ; + } + } +ASSERT (expiration - now >0 ); + + vs->expiration = expiration; + timeout = (expiration - now) *100ull ; + + vs->started = true; +migrate_timer (&vs->timer,smp_processor_id ());Why is this
smp_processor_id()whenviridian_time_vcpu_init()usesv->processor? How relevant is it in the first place to trace the pCPU the vCPU runs on for the timer?Quoted 2 lines
I was just following suit with other timer code. It seems to be the norm to migrate to the current pCPU just prior to starting a timer.
But wouldn't
v->processorthen be more visibly correct (besides likely being cheaper to get at), as to the correlation to the vCPU in question? I can't actually see why "migrate to the current pCPU" would be the norm; I could only see this as an implication from that other code you looked at simply acting on the current vCPU.
Quoted 4 lines
Then again I'm having trouble spotting why it would be important for the timer to run on the same CPU the vCPU runs one. By the time the timer fires, the vCPU may have gone elsewhere.
I have no problem dropping the migrate call. As I said, I was following prior example e.g. in the VCPUOP_set_singleshot_timer handler and in vcpu_periodic_timer_work(), which do indeed run on current... but then so will start_timer() in the vast majority of invocations (the invocation in viridian_time_vcpu_thaw() being the exception). I'm happy for you to swap it for v->processor on commit unless you want me to send a v9 with the change?
Paul
Quoted 8 lines
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 35 lines
Quoted 30 lines
Quoted 1 line · On 18.03.19 at 16:36, <[email protected]> wrote:
From: Xen-devel [mailto:[email protected]] On Behalf Of Jan Beulich
Sent: 18 March 2019 15:21Quoted 17 lines
Quoted 5 lines
Quoted 1 line · On 18.03.19 at 15:37, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 14:24Quoted 1 line
Quoted 1 line
Quoted 1 line · On 18.03.19 at 12:20, <[email protected]> wrote:
Quoted 11 lines
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 273
Quoted 7 lines
xen/arch/x86/hvm/viridian/time.c +7
-0+ASSERT (expiration - now >0 ); + + vs->expiration = expiration; + timeout = (expiration - now) *100ull ; + + vs->started = true; +migrate_timer (&vs->timer,smp_processor_id ());Why is this
smp_processor_id()whenviridian_time_vcpu_init()usesv->processor? How relevant is it in the first place to trace the pCPU the vCPU runs on for the timer?Quoted 2 lines
I was just following suit with other timer code. It seems to be the norm to migrate to the current pCPU just prior to starting a timer.
But wouldn't
v->processorthen be more visibly correct (besides likely being cheaper to get at), as to the correlation to the vCPU in question? I can't actually see why "migrate to the current pCPU" would be the norm; I could only see this as an implication from that other code you looked at simply acting on the current vCPU.Quoted 4 lines
Then again I'm having trouble spotting why it would be important for the timer to run on the same CPU the vCPU runs one. By the time the timer fires, the vCPU may have gone elsewhere.
Quoted 6 lines
I have no problem dropping the migrate call. As I said, I was following prior example e.g. in the
VCPUOP_set_singleshot_timerhandler and invcpu_periodic_timer_work(), which do indeed run on current... but then so willstart_timer()in the vast majority of invocations (the invocation inviridian_time_vcpu_thaw()being the exception). I'm happy for you to swap it forv->processoron commit unless you want me to send a v9 with the change?
No need to send v9 just for this.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
In reply to Paul DurrantQuoted 1 line
-----Original Message-----
[snip]
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +6
-0+ { + expiration = vs->count; +if ( expiration - now <=0 ) + { + vs->expiration = expiration; +stimer_expire (vs);
Quoted 7 lines
Quoted 4 lines
Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.In practice I don't think there should be any problematic race, but I'll check again.
I think the 'periodic' name might be confusing things... The Xen timers are all single-shot, it's just that start_stimer() is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enter start_stimer().
Paul
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines · On 18.03.19 at 16:46, <[email protected]> wrote:
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 263
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +6
-0+ { + expiration = vs->count; +if ( expiration - now <=0 ) + { + vs->expiration = expiration; +stimer_expire (vs);Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.
Quoted 7 lines
Quoted 1 line
In practice I don't think there should be any problematic race, but I'll check again.
I think the 'periodic' name might be confusing things... The Xen timers are all single-shot, it's just that
start_stimer()is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enterstart_stimer().
One of the callers of the function is the WRMSR handler. Why would it be guaranteed that the timer isn't active when such a WRMSR occurs? Of course, this alone is not enough for there to be a problem, as we're fine as long as the guest can only harm itself. But I don't think it goes without saying that there's no issue here; having looked a 2nd time just now, I think I agree though that there's no risk for our own health here.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 31 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:23
To: Paul Durrant <[email protected]>
Cc: Julien Grall <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Roger Pau Monne <[email protected]>; Wei Liu <[email protected]>; Stefano Stabellini <[email protected]>; xen-devel <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Tim (Xen.org) <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation of synthetic timersQuoted 12 lines
Quoted 12 lines
Quoted 11 lines · On 18.03.19 at 16:46, <[email protected]> wrote:
xen/arch/x86/hvm/viridian/time.c static int64_t time_ref_count(const struct domain *d) · line 263
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +6
-0+ { + expiration = vs->count; +if ( expiration - now <=0 ) + { + vs->expiration = expiration; +stimer_expire (vs);Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.Quoted 7 lines
Quoted 1 line
In practice I don't think there should be any problematic race, but I'll check again.
I think the 'periodic' name might be confusing things... The Xen timers are all single-shot, it's just that
start_stimer()is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enterstart_stimer().
Quoted 3 lines
One of the callers of the function is the WRMSR handler. Why would it be guaranteed that the timer isn't active when such a WRMSR occurs?
It's not guaranteed on entry, but the WRMSR handler always calls stop_stimer() before calling start_stimer() which AFAICT should guarantee the timer is not running when start_stimer() is called.
Paul
Quoted 8 lines
Of course, this alone is not enough for there to be a problem, as we're fine as long as the guest can only harm itself. But I don't think it goes without saying that there's no issue here; having looked a 2nd time just now, I think I agree though that there's no risk for our own health here.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 24 lines
Quoted 24 lines
Quoted 1 line · On 18.03.19 at 17:26, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:23Quoted 19 lines
Quoted 8 lines
Quoted 8 lines · On 18.03.19 at 16:46, <[email protected]> wrote:
Quoted 6 lines
xen/arch/x86/hvm/viridian/time.c +6
-0+ { + expiration = vs->count; +if ( expiration - now <=0 ) + { + vs->expiration = expiration; +stimer_expire (vs);Quoted 5 lines
Quoted 3 lines
Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.In practice I don't think there should be any problematic race, but I'll check again.
I think the 'periodic' name might be confusing things... The Xen timers are all single-shot, it's just that
start_stimer()is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enterstart_stimer().
Quoted 7 lines
Quoted 3 lines
One of the callers of the function is the WRMSR handler. Why would it be guaranteed that the timer isn't active when such a WRMSR occurs?
It's not guaranteed on entry, but the WRMSR handler always calls
stop_stimer()before callingstart_stimer()which AFAICT should guarantee the timer is not running whenstart_stimer()is called.
I've looked only briefly, but the stop_timer() -> deactivate_timer() call chain doesn't look to wait for the timer handler to not be active anymore on another CPU before returning.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 39 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:56
To: Paul Durrant <[email protected]>
Cc: Julien Grall <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Roger Pau Monne <[email protected]>; Wei Liu <[email protected]>; Stefano Stabellini <[email protected]>; xen-devel <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Tim (Xen.org) <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation of synthetic timersQuoted 28 lines
Quoted 27 lines
Quoted 1 line · On 18.03.19 at 17:26, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:23Quoted 12 lines
Quoted 12 lines
Quoted 11 lines · On 18.03.19 at 16:46, <[email protected]> wrote:
Review
Quoted 6 lines
(quoted diff) +6
-0+ { + expiration = vs->count; + if ( expiration - now <= 0 ) + { + vs->expiration = expiration; + stimer_expire(vs);Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.Quoted 7 lines
Quoted 1 line
In practice I don't think there should be any problematic race, but I'll check again.
I think the 'periodic' name might be confusing things... The Xen timers are all single-shot, it's just that
start_stimer()is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enterstart_stimer().One of the callers of the function is the WRMSR handler. Why would it be guaranteed that the timer isn't active when such a WRMSR occurs?
Quoted 7 lines
Quoted 3 lines
It's not guaranteed on entry, but the WRMSR handler always calls
stop_stimer()before callingstart_stimer()which AFAICT should guarantee the timer is not running whenstart_stimer()is called.I've looked only briefly, but the
stop_timer()->deactivate_timer()call chain doesn't look to wait for the timer handler to not be active anymore on another CPU before returning.
Oh, it looked to me like the locking would ensure the timer was deactivated and would not run... but I guess I may have misunderstood the code. Still even if both occurrences make it past the test of config.enabled all they'll do is both set the pending bit, as the prior version of the patch did. Although I guess there's now a possibility that, for one occurrence, the poll could occur before config.enabled is cleared and thus the timer may be rescheduled and immediately expire again. I'm not sure whether this is really a problem or not.
Paul
Quoted 3 lines
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 50 lines
Quoted 6 lines
Quoted 1 line · On 18.03.19 at 18:06, <[email protected]> wrote:
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:56
To: Paul Durrant <[email protected]>
Cc: Julien Grall <[email protected]>; Andrew Cooper<[email protected]>; George Dunlap
Quoted 1 line
<[email protected]>; Ian Jackson <[email protected]>; Roger Pau
Monne
Quoted 1 line
<[email protected]>; Wei Liu <[email protected]>; Stefano Stabellini
Quoted 1 line
xen-devel <[email protected]>; Konrad Rzeszutek Wilk
<[email protected]>; Tim
Quoted 2 lines
(Xen.org) <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation ofsynthetic timers
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 2 lines · On 18.03.19 at 17:26, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:23Quoted 8 lines
Quoted 8 lines
Quoted 8 lines · On 18.03.19 at 16:46, <[email protected]> wrote:
Quoted 6 lines
(quoted diff) +6
-0+ { + expiration = vs->count; + if ( expiration - now <= 0 ) + { + vs->expiration = expiration; + stimer_expire(vs);Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.In practice I don't think there should be any problematic race, but I'll
check again.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I think the 'periodic' name might be confusing things... The Xen timers
are
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
all single-shot, it's just that
start_stimer()is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enterstart_stimer().One of the callers of the function is the WRMSR handler. Why would it be guaranteed that the timer isn't active when such a WRMSR occurs?
It's not guaranteed on entry, but the WRMSR handler always calls
stop_stimer()before callingstart_stimer()which AFAICT should guaranteethe
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
timer is not running when
start_stimer()is called.I've looked only briefly, but the
stop_timer()->deactivate_timer()call chain doesn't look to wait for the timer handler to not be active anymore on another CPU before returning.Oh, it looked to me like the locking would ensure the timer was deactivated and would not run... but I guess I may have misunderstood the code.
The lock gets dropped around the call to the handler (in execute_timer()).
Quoted 6 lines
Still
even if both occurrences make it past the test of config.enabled all they'll do is both set the pending bit, as the prior version of the patch did. Although I guess there's now a possibility that, for one occurrence, the poll could occur before config.enabled is cleared and thus the timer may be rescheduled and immediately expire again.
So perhaps config.enabled should get cleared before setting the bit in stimer_pending? Would seem to also be better for this to happen before vcpu_kick(), wouldn't it? (Moving the two lines up would again be easy enough to do while committing, so long as you agree.)
Quoted 1 line
I'm not sure whether this is really a problem or not.
Neither am I.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v8 10/11] viridian: add implementation of synthetic timers
Quoted 79 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 19 March 2019 08:03
To: Paul Durrant <[email protected]>
Cc: Julien Grall <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Roger Pau Monne <[email protected]>; Wei Liu <[email protected]>; Stefano Stabellini <[email protected]>; xen-devel <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Tim (Xen.org) <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation of synthetic timersQuoted 50 lines
Quoted 6 lines
Quoted 1 line · On 18.03.19 at 18:06, <[email protected]> wrote:
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:56
To: Paul Durrant <[email protected]>
Cc: Julien Grall <[email protected]>; Andrew Cooper<[email protected]>; George Dunlap
Quoted 1 line
<[email protected]>; Ian Jackson <[email protected]>; Roger Pau
Monne
Quoted 1 line
<[email protected]>; Wei Liu <[email protected]>; Stefano Stabellini
Quoted 1 line
xen-devel <[email protected]>; Konrad Rzeszutek Wilk
<[email protected]>; Tim
Quoted 2 lines
(Xen.org) <[email protected]>
Subject: Re: [Xen-devel] [PATCH v8 10/11] viridian: add implementation ofsynthetic timers
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 2 lines · On 18.03.19 at 17:26, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: 18 March 2019 16:23Quoted 8 lines
Quoted 8 lines
Quoted 8 lines · On 18.03.19 at 16:46, <[email protected]> wrote:
Quoted 6 lines
(quoted diff) +6
-0+ { + expiration = vs->count; + if ( expiration - now <= 0 ) + { + vs->expiration = expiration; + stimer_expire(vs);Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Aren't you introducing a risk for races by calling the timer function directly from here?
start_timer(), after all, gets called from quite a few places.In practice I don't think there should be any problematic race, but I'll
check again.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I think the 'periodic' name might be confusing things... The Xen timers
are
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
all single-shot, it's just that
start_stimer()is re-called after a successful poll if the viridian timer is configured to be periodic. So I don't think there is case where the underlying Xen timer could actually be running when we enterstart_stimer().One of the callers of the function is the WRMSR handler. Why would it be guaranteed that the timer isn't active when such a WRMSR occurs?
It's not guaranteed on entry, but the WRMSR handler always calls
stop_stimer()before callingstart_stimer()which AFAICT should guaranteethe
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
timer is not running when
start_stimer()is called.I've looked only briefly, but the
stop_timer()->deactivate_timer()call chain doesn't look to wait for the timer handler to not be active anymore on another CPU before returning.Oh, it looked to me like the locking would ensure the timer was deactivated and would not run... but I guess I may have misunderstood the code.
The lock gets dropped around the call to the handler (in
execute_timer()).Quoted 6 lines
Still
even if both occurrences make it past the test of config.enabled all they'll do is both set the pending bit, as the prior version of the patch did. Although I guess there's now a possibility that, for one occurrence, the poll could occur before config.enabled is cleared and thus the timer may be rescheduled and immediately expire again.
Quoted 8 lines
So perhaps config.enabled should get cleared before setting the bit in
stimer_pending? Would seem to also be better for this to happen beforevcpu_kick(), wouldn't it? (Moving the two lines up would again be easy enough to do while committing, so long as you agree.)Quoted 1 line
I'm not sure whether this is really a problem or not.
Neither am I.
Ok. To err on the safe side it is probably best to only access the timer config from current context and have timer expiration simply set the pending bit and kick the vcpu. I'll send a v9.
Paul
Quoted 3 lines
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v8 11/11 viridian: add implementation of the HvSendSyntheticClusterIpi hypercall
In reply to Paul DurrantThis patch adds an implementation of the hypercall as documented in the specification [1], section 10.5.2. This enlightenment, as with others, is advertised by CPUID leaf 0x40000004 and is under control of a new 'hcall_ipi' option in libxl.
If used, this enlightenment should mean the guest only takes a single VMEXIT to issue IPIs to multiple vCPUs rather than the multiple VMEXITs that would result from using the emulated local APIC.
---
Cc: Ian Jackson <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
v4:
- Address comments from Jan
v3:
- New in v3
--- docs/man/xl.cfg.5.pod.in | 6 +++ tools/libxl/libxl.h | 6 +++ tools/libxl/libxl_dom.c | 3 ++ tools/libxl/libxl_types.idl | 1 + xen/arch/x86/hvm/viridian/viridian.c | 63 ++++++++++++++++++++++++++++ xen/include/public/hvm/params.h | 7 +++- 6 files changed, 85 insertions(+), 1 deletion(-)
docs/man/xl.cfg.5.pod.in +6 -0
@@ -2175,6 +2175,12 @@ ticks and hence enabling this group will ensure that ticks will be consistent with use of an enlightened time source (B<time_ref_count> or B<reference_tsc>). +=item B<hcall_ipi> + +This set incorporates use of a hypercall for interprocessor interrupts. +This enlightenment may improve performance of Windows guests with multiple +virtual CPUs. + =item B<defaults> This is a special value that enables the default set of groups, which
tools/libxl/libxl.h +6 -0
@@ -330,6 +330,12 @@*/ #define LIBXL_HAVE_VIRIDIAN_STIMER 1 +/* +* LIBXL_HAVE_VIRIDIAN_HCALL_IPI indicates that the 'hcall_ipi' value +* is present in the viridian enlightenment enumeration. +*/ +#define LIBXL_HAVE_VIRIDIAN_HCALL_IPI 1 +/* * LIBXL_HAVE_BUILDINFO_HVM_ACPI_LAPTOP_SLATE indicates that * libxl_domain_build_info has the u.hvm.acpi_laptop_slate field.
tools/libxl/libxl_dom.c +3 -0
@@ -324,6 +324,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,if (libxl_bitmap_test (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_STIMER )) mask |= HVMPV_time_ref_count | HVMPV_synic | HVMPV_stimer; +if (libxl_bitmap_test (&enlightenments,LIBXL_VIRIDIAN_ENLIGHTENMENT_HCALL_IPI )) + mask |= HVMPV_hcall_ipi; +if (mask !=0 && xc_hvm_param_set(CTX->xch, domid,
tools/libxl/libxl_types.idl +1 -0
@@ -237,6 +237,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [ (6, "crash_ctl"), (7, "synic"), (8, "stimer"), + (9, "hcall_ipi"), ]) libxl_hdtype = Enumeration("hdtype", [
xen/arch/x86/hvm/viridian/viridian.c +63 -0
87 lines not on this page · raw message
xen/include/public/hvm/params.h +6 -1
@@ -154,6 +154,10 @@#define _HVMPV_stimer 8#define HVMPV_stimer (1 << _HVMPV_stimer) +/* Use Synthetic Cluster IPI Hypercall */ +#define _HVMPV_hcall_ipi 9 +#define HVMPV_hcall_ipi (1 << _HVMPV_hcall_ipi) +#define HVMPV_feature_mask \ (HVMPV_base_freq | \ HVMPV_no_freq | \
@@ -163,7 +167,8 @@ HVMPV_apic_assist | \ HVMPV_crash_ctl | \ HVMPV_synic | \ - HVMPV_stimer) + HVMPV_stimer | \ + HVMPV_hcall_ipi)#endif
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel