x86: Make MAX_ALTP2M configurable
PATCH v2 1/7 x86/p2m: Add braces for better code clarity
In reply to Petr BenešFrom: Petr Beneš <[email protected]>
No functional change.
--- xen/arch/x86/mm/p2m.c | 4 ++++ 1 file changed, 4 insertions(+)
xen/arch/x86/mm/p2m.c +4 -0
@@ -106,6 +106,7 @@ void p2m_change_entry_type_global(struct domain *d,unsigned int i;for ( i =0 ; i <MAX_ALTP2M ; i++ ) + {if ( d->arch.altp2m_eptp[i] !=mfn_x (INVALID_MFN ) ) {struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -114,6 +115,7 @@ void p2m_change_entry_type_global(struct domain *d,change_entry_type_global (altp2m, ot, nt);p2m_unlock (altp2m); } + } }p2m_unlock (hostp2m);
@@ -139,6 +141,7 @@ void p2m_memory_type_changed(struct domain *d)unsigned int i;for ( i =0 ; i <MAX_ALTP2M ; i++ ) + {if ( d->arch.altp2m_eptp[i] !=mfn_x (INVALID_MFN ) ) {struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -147,6 +150,7 @@ void p2m_memory_type_changed(struct domain *d)_memory_type_changed (altp2m);p2m_unlock (altp2m); } + } }p2m_unlock (hostp2m);
Signature · 1 line
2.34.1
v2 cover x86: Make MAX_ALTP2M configurable
From: Petr Beneš <[email protected]>
This series introduces the ability to configure the maximum number of altp2m tables during domain creation. Previously, the limits were hardcoded to a maximum of 10. This change allows for greater flexibility in environments that require more or fewer altp2m views.
Adjustments include:
- "Prepare" commits with style changes.
- Adding a new max_altp2m parameter to xl.
- Adding a new max_altp2m parameter to the OCaml bindings.
- Adding a new max_altp2m parameter to the xl.cfg manual.
- Adding a new max_altp2m field into the xen_domctl_createdomain, which, after sanity checks, is stored in newly introduced max_altp2m field of struct domain - leaving room for other architectures to implement the altp2m feature.
- Replacing MAX_ALTP2M macro occurrences with domain->max_altp2m.
- Finally, adjusting the initial allocation of pages in hap_enable from 256 to 1024 pages to accommodate potentially larger max_altp2m values (i.e., maximum of 512).
This enhancement is particularly relevant for users leveraging Xen's features for virtual machine introspection.
Petr Beneš (7):
x86/p2m: Add braces for better code clarity
tools/xl: Add max_altp2m parameter
docs/man: Add max_altp2m parameter to the xl.cfg manual
x86: Make the maximum number of altp2m views configurable tools/libxl: Activate the max_altp2m feature tools/ocaml: Add max_altp2m parameter
x86/hap: Increase the number of initial mempool_size to 1024 pages
docs/man/xl.cfg.5.pod.in | 14 +++++ tools/golang/xenlight/helpers.gen.go | 2 + tools/golang/xenlight/types.gen.go | 1 + tools/include/libxl.h | 8 +++ tools/libs/light/libxl_create.c | 9 ++++ tools/libs/light/libxl_types.idl | 1 + tools/ocaml/libs/xc/xenctrl.ml | 1 + tools/ocaml/libs/xc/xenctrl.mli | 1 + tools/ocaml/libs/xc/xenctrl_stubs.c | 17 +++--- .../paging-mempool/test-paging-mempool.c | 2 +- tools/xl/xl_parse.c | 4 ++ xen/arch/x86/domain.c | 6 +++ xen/arch/x86/hvm/hvm.c | 8 ++- xen/arch/x86/hvm/vmx/vmx.c | 2 +- xen/arch/x86/include/asm/domain.h | 7 ++- xen/arch/x86/include/asm/p2m.h | 4 +- xen/arch/x86/mm/altp2m.c | 27 +++++++++- xen/arch/x86/mm/hap/hap.c | 8 +-- xen/arch/x86/mm/mem_access.c | 14 ++--- xen/arch/x86/mm/mem_sharing.c | 2 +- xen/arch/x86/mm/p2m-ept.c | 6 +-- xen/arch/x86/mm/p2m.c | 54 ++++++++++--------- xen/common/domain.c | 7 +++ xen/include/public/domctl.h | 3 +- xen/include/xen/sched.h | 2 + 25 files changed, 151 insertions(+), 59 deletions(-)
--
2.34.1
PATCH v2 4/7 x86: Make the maximum number of altp2m views configurable
From: Petr Beneš <[email protected]>
This commit introduces the ability to configure the maximum number of altp2m tables during domain creation. Previously, the limits were hardcoded to a maximum of 10. This change allows for greater flexibility in environments that require more or fewer altp2m views.
The maximum configurable limit for max_altp2m on x86 is now set to MAX_EPTP (512). This cap is linked to the architectural limit of the EPTP-switching VMFUNC, which supports up to 512 entries. Despite there being no inherent need for limiting max_altp2m in scenarios not utilizing VMFUNC, decoupling these components would necessitate substantial code changes.
---
Changed since v1:
* Added xen_domctl_createdomain::max_altp2m field to xen/include/public/domctl.h
* Bumped the XEN_DOMCTL_INTERFACE_VERSION
* More elaborate commit message
xen/arch/x86/domain.c | 6 ++++ xen/arch/x86/hvm/hvm.c | 8 ++++- xen/arch/x86/hvm/vmx/vmx.c | 2 +- xen/arch/x86/include/asm/domain.h | 7 ++--- xen/arch/x86/include/asm/p2m.h | 4 +-- xen/arch/x86/mm/altp2m.c | 27 +++++++++++++++-- xen/arch/x86/mm/hap/hap.c | 6 ++-- xen/arch/x86/mm/mem_access.c | 14 ++++----- xen/arch/x86/mm/mem_sharing.c | 2 +- xen/arch/x86/mm/p2m-ept.c | 6 ++-- xen/arch/x86/mm/p2m.c | 50 +++++++++++++++---------------- xen/common/domain.c | 7 +++++ xen/include/public/domctl.h | 3 +- xen/include/xen/sched.h | 2 ++ 14 files changed, 94 insertions(+), 50 deletions(-)
xen/arch/x86/domain.c +6 -0
@@ -685,6 +685,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)return -EINVAL ; } +if ( config->max_altp2m >MAX_EPTP ) + { +dprintk (XENLOG_INFO ,"max_altp2m must be <= %u\n" ,MAX_EPTP ); +return -EINVAL ; + } +if ( config->vmtrace_size ) {unsigned int size = config->vmtrace_size;
xen/arch/x86/hvm/hvm.c +7 -1
@@ -4633,6 +4633,12 @@ static int do_altp2m_op( goto out; } +if ( d->max_altp2m ==0 ) + { + rc = -EINVAL ; + goto out; + } +if ( (rc =xsm_hvm_altp2mhvm_op (XSM_OTHER , d, mode, a.cmd)) ) goto out;
@@ -5222,7 +5228,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t p2midx)if ( !hvm_is_singlestep_supported () )return ; -if ( p2midx >=MAX_ALTP2M ) +if ( p2midx >= v->domain->max_altp2m )return ; v->arch.hvm.single_step = true;
xen/arch/x86/hvm/vmx/vmx.c +1 -1
@@ -4888,7 +4888,7 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs) {unsigned int i; -for ( i =0 ; i <MAX_ALTP2M ; ++i ) +for ( i =0 ; i < currd->max_altp2m; ++i ) {if ( currd->arch.altp2m_eptp[i] ==mfn_x (INVALID_MFN ) )continue ;
xen/arch/x86/include/asm/domain.h +3 -4
@@ -258,11 +258,10 @@ struct paging_vcpu {struct shadow_vcpu shadow; }; +#define INVALID_ALTP2M 0xffff +#define MAX_EPTP ((unsigned int)(PAGE_SIZE / sizeof(uint64_t)))#define MAX_NESTEDP2M 10 -#define MAX_ALTP2M 10/* arbitrary */ -#define INVALID_ALTP2M 0xffff -#define MAX_EPTP (PAGE_SIZE / sizeof(uint64_t))struct p2m_domain ;struct time_scale {int shift;
@@ -353,7 +352,7 @@ struct arch_domain/* altp2m: allow multiple copies of host p2m */ bool altp2m_active; -struct p2m_domain *altp2m_p2m[MAX_ALTP2M ]; +struct p2m_domain **altp2m_p2m;mm_lock_t altp2m_list_lock;uint64_t *altp2m_eptp;uint64_t *altp2m_visible_eptp;
xen/arch/x86/include/asm/p2m.h +2 -2
@@ -881,7 +881,7 @@ static inline struct p2m_domain *p2m_get_altp2m(struct vcpu *v)if ( index ==INVALID_ALTP2M )return NULL ; -BUG_ON (index >=MAX_ALTP2M ); +BUG_ON (index >= v->domain->max_altp2m);return v->domain->arch.altp2m_p2m[index]; }
@@ -891,7 +891,7 @@ static inline bool p2m_set_altp2m(struct vcpu *v, unsigned int idx) {struct p2m_domain *orig; -BUG_ON (idx >=MAX_ALTP2M ); +BUG_ON (idx >= v->domain->max_altp2m);if ( idx ==vcpu_altp2m (v).p2midx )return false;
xen/arch/x86/mm/altp2m.c +25 -2
@@ -13,6 +13,12 @@void altp2m_vcpu_initialise (struct vcpu *v) { +struct domain *d = v->domain; + +/* Skip initialisation if no altp2m will be used. */ +if ( d->max_altp2m ==0 ) +return ; +if ( v != current )vcpu_pause (v);
@@ -28,8 +34,13 @@ altp2m_vcpu_initialise(struct vcpu *v)void altp2m_vcpu_destroy (struct vcpu *v) { +struct domain *d = v->domain;struct p2m_domain *p2m; +/* Skip destruction if no altp2m was used. */ +if ( d->max_altp2m ==0 ) +return ; +if ( v != current )vcpu_pause (v);
@@ -120,7 +131,13 @@ int p2m_init_altp2m(struct domain *d)struct p2m_domain *hostp2m =p2m_get_hostp2m (d);mm_lock_init (&d->arch.altp2m_list_lock); -for ( i =0 ; i <MAX_ALTP2M ; i++ ) + + if ( (d->arch.altp2m_p2m = xzalloc_array(struct p2m_domain *, d->max_altp2m)) == NULL ) + { +return -ENOMEM ; + } + +for ( i =0 ; i < d->max_altp2m; i++ ) { d->arch.altp2m_p2m[i] = p2m =p2m_init_one (d);if ( p2m ==NULL )
@@ -141,7 +158,10 @@ void p2m_teardown_altp2m(struct domain *d)unsigned int i;struct p2m_domain *p2m; -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +if ( !d->arch.altp2m_p2m ) +return ; + +for ( i =0 ; i < d->max_altp2m; i++ ) {if ( !d->arch.altp2m_p2m[i] )continue ;
@@ -149,6 +169,9 @@ void p2m_teardown_altp2m(struct domain *d) d->arch.altp2m_p2m[i] =NULL ;p2m_free_one (p2m); } + +xfree (d->arch.altp2m_p2m); + d->arch.altp2m_p2m =NULL ; }/*
xen/arch/x86/mm/hap/hap.c +3 -3
@@ -515,7 +515,7 @@ int hap_enable(struct domain *d, u32 mode) d->arch.altp2m_visible_eptp[i] =mfn_x (INVALID_MFN ); } -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ ) { rv =p2m_alloc_table (d->arch.altp2m_p2m[i]);if ( rv !=0 )
@@ -538,7 +538,7 @@ void hap_final_teardown(struct domain *d)unsigned int i;if (hvm_altp2m_supported () ) -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ )p2m_teardown (d->arch.altp2m_p2m[i], true,NULL );/* Destroy nestedp2m's first */
@@ -590,7 +590,7 @@ void hap_teardown(struct domain *d, bool *preempted)FREE_XENHEAP_PAGE (d->arch.altp2m_eptp);FREE_XENHEAP_PAGE (d->arch.altp2m_visible_eptp); -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ ) {p2m_teardown (d->arch.altp2m_p2m[i], false, preempted);if ( preempted && *preempted )
xen/arch/x86/mm/mem_access.c +7 -7
@@ -347,12 +347,12 @@ long p2m_set_mem_access(struct domain *d, gfn_t gfn, uint32_t nr,/* altp2m view 0 is treated as the hostp2m */ if ( altp2m_idx ) { -if ( altp2m_idx >=min (ARRAY_SIZE (d->arch.altp2m_p2m),MAX_EPTP ) || +if ( altp2m_idx >=min (d->max_altp2m,MAX_EPTP ) || d->arch.altp2m_eptp[array_index_nospec (altp2m_idx,MAX_EPTP )] ==mfn_x (INVALID_MFN ) )return -EINVAL ; - ap2m =array_access_nospec (d->arch.altp2m_p2m, altp2m_idx); + ap2m = d->arch.altp2m_p2m[array_index_nospec (altp2m_idx, d->max_altp2m)]; }if ( !xenmem_access_to_p2m_access (p2m, access, &a) )
@@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,/* altp2m view 0 is treated as the hostp2m */ if ( altp2m_idx ) { -if ( altp2m_idx >=min (ARRAY_SIZE (d->arch.altp2m_p2m),MAX_EPTP ) || +if ( altp2m_idx >=min (d->max_altp2m,MAX_EPTP ) || d->arch.altp2m_eptp[array_index_nospec (altp2m_idx,MAX_EPTP )] ==mfn_x (INVALID_MFN ) )return -EINVAL ; - ap2m =array_access_nospec (d->arch.altp2m_p2m, altp2m_idx); + ap2m = d->arch.altp2m_p2m[array_index_nospec (altp2m_idx, d->max_altp2m)]; }p2m_lock (p2m);
@@ -466,12 +466,12 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, xenmem_access_t *access, }else if ( altp2m_idx )/* altp2m view 0 is treated as the hostp2m */ { -if ( altp2m_idx >=min (ARRAY_SIZE (d->arch.altp2m_p2m),MAX_EPTP ) || +if ( altp2m_idx >=min (d->max_altp2m,MAX_EPTP ) || d->arch.altp2m_eptp[array_index_nospec (altp2m_idx,MAX_EPTP )] ==mfn_x (INVALID_MFN ) )return -EINVAL ; - p2m =array_access_nospec (d->arch.altp2m_p2m, altp2m_idx); + p2m = d->arch.altp2m_p2m[array_index_nospec (altp2m_idx, d->max_altp2m)]; }return _p2m_get_mem_access (p2m, gfn, access);
@@ -486,7 +486,7 @@ void arch_p2m_set_access_required(struct domain *d, bool access_required)if (altp2m_active (d) ) {unsigned int i; -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ ) {struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
xen/arch/x86/mm/mem_sharing.c +1 -1
@@ -912,7 +912,7 @@ static int nominate_page(struct domain *d, gfn_t gfn,altp2m_list_lock (d); -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ ) { ap2m = d->arch.altp2m_p2m[i];if ( !ap2m )
xen/arch/x86/mm/p2m-ept.c +3 -3
@@ -1293,7 +1293,7 @@ static void ept_set_ad_sync(struct domain *d, bool value) {unsigned int i; -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ ) {struct p2m_domain *p2m;
@@ -1500,7 +1500,7 @@ void setup_ept_dump(void)void p2m_init_altp2m_ept (struct domain *d,unsigned int i) { -struct p2m_domain *p2m =array_access_nospec (d->arch.altp2m_p2m, i); +struct p2m_domain *p2m = d->arch.altp2m_p2m[array_index_nospec (i, d->max_altp2m)];struct p2m_domain *hostp2m =p2m_get_hostp2m (d);struct ept_data *ept;
@@ -1519,7 +1519,7 @@ unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)altp2m_list_lock (d); -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +for ( i =0 ; i < d->max_altp2m; i++ ) {if ( d->arch.altp2m_eptp[i] ==mfn_x (INVALID_MFN ) )continue ;
xen/arch/x86/mm/p2m.c +25 -25
179 lines not on this page · raw message
xen/common/domain.c +7 -0
@@ -568,6 +568,12 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config) } } +if ( config->max_altp2m && !hvm_altp2m_supported () ) + { +dprintk (XENLOG_INFO ,"altp2m requested but not available\n" ); +return -EINVAL ; + } +if ( config->vmtrace_size && !vmtrace_available ) {dprintk (XENLOG_INFO ,"vmtrace requested but not available\n" );
@@ -610,6 +616,7 @@ struct domain *domain_create(domid_t domid,if ( config ) { d->options = config->flags; + d->max_altp2m = config->max_altp2m; d->vmtrace_size = config->vmtrace_size; }
xen/include/public/domctl.h +2 -1
@@ -21,7 +21,7 @@#include "hvm/save.h" #include "memory.h" -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016 +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017/* * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -77,6 +77,7 @@ struct xen_domctl_createdomain {*/ uint32_t max_vcpus;uint32_t max_evtchn_port; +uint32_t max_altp2m;int32_t max_grant_frames;int32_t max_maptrack_frames;
xen/include/xen/sched.h +2 -0
@@ -602,6 +602,8 @@ struct domain unsigned int guest_request_sync : 1; } monitor; +unsigned int max_altp2m;/* Maximum number of altp2m tables */ +unsigned int vmtrace_size;/* Buffer size in bytes, or 0 to disable. */ #ifdef CONFIG_ARGO
--
2.34.1
PATCH v2 5/7 tools/libxl: Activate the max_altp2m feature
In reply to Petr BenešFrom: Petr Beneš <[email protected]>
This commit activates the previously introduced max_altp2m parameter, establishing the connection between libxl and Xen.
---
Changed since v1:
* This is a new commit in the series
tools/libs/light/libxl_create.c | 1 + 1 file changed, 1 insertion(+)
tools/libs/light/libxl_create.c +1 -0
@@ -653,6 +653,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config, .ssidref = info->ssidref, .max_vcpus = b_info->max_vcpus, .max_evtchn_port = b_info->event_channels, + .max_altp2m = b_info->max_altp2m, .max_grant_frames = b_info->max_grant_frames, .max_maptrack_frames = b_info->max_maptrack_frames, .grant_opts = XEN_DOMCTL_GRANT_version(b_info->max_grant_version),
--
2.34.1
PATCH v2 6/7 tools/ocaml: Add max_altp2m parameter
In reply to Petr BenešFrom: Petr Beneš <[email protected]>
Allow developers using the OCaml bindings to set the max_altp2m parameter.
---
Changed since v1:
* Moved this commit AFTER Xen changes (where xen_domctl_createdomain::max_altp2m field is introduced) to avoid breaking the build
tools/ocaml/libs/xc/xenctrl.ml | 1 + tools/ocaml/libs/xc/xenctrl.mli | 1 + tools/ocaml/libs/xc/xenctrl_stubs.c | 17 ++++++++++------- 3 files changed, 12 insertions(+), 7 deletions(-)
tools/ocaml/libs/xc/xenctrl.ml +1 -0
@@ -82,6 +82,7 @@ type domctl_create_config = iommu_opts: domain_create_iommu_opts list; max_vcpus: int; max_evtchn_port: int; + max_altp2m: int; max_grant_frames: int; max_maptrack_frames: int; max_grant_version: int;
tools/ocaml/libs/xc/xenctrl.mli +1 -0
@@ -74,6 +74,7 @@ type domctl_create_config = { iommu_opts: domain_create_iommu_opts list; max_vcpus: int; max_evtchn_port: int; + max_altp2m: int; max_grant_frames: int; max_maptrack_frames: int; max_grant_version: int;
tools/ocaml/libs/xc/xenctrl_stubs.c +10 -7
@@ -207,12 +207,13 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co#define VAL_IOMMU_OPTS Field(config, 3)#define VAL_MAX_VCPUS Field(config, 4)#define VAL_MAX_EVTCHN_PORT Field(config, 5) -#define VAL_MAX_GRANT_FRAMES Field(config, 6) -#define VAL_MAX_MAPTRACK_FRAMES Field(config, 7) -#define VAL_MAX_GRANT_VERSION Field(config, 8) -#define VAL_VMTRACE_BUF_KB Field(config, 9) -#define VAL_CPUPOOL_ID Field(config, 10) -#define VAL_ARCH Field(config, 11) +#define VAL_MAX_ALTP2M Field(config, 6) +#define VAL_MAX_GRANT_FRAMES Field(config, 7) +#define VAL_MAX_MAPTRACK_FRAMES Field(config, 8) +#define VAL_MAX_GRANT_VERSION Field(config, 9) +#define VAL_VMTRACE_BUF_KB Field(config, 10) +#define VAL_CPUPOOL_ID Field(config, 11) +#define VAL_ARCH Field(config, 12)uint32_t domid =Int_val (wanted_domid);uint64_t vmtrace_size =Int32_val (VAL_VMTRACE_BUF_KB );
@@ -226,6 +227,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co .ssidref = Int32_val(VAL_SSIDREF), .max_vcpus = Int_val(VAL_MAX_VCPUS), .max_evtchn_port = Int_val(VAL_MAX_EVTCHN_PORT), + .max_altp2m = Int_val(VAL_MAX_ALTP2M), .max_grant_frames = Int_val(VAL_MAX_GRANT_FRAMES), .max_maptrack_frames = Int_val(VAL_MAX_MAPTRACK_FRAMES), .grant_opts =
@@ -257,7 +259,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co#if defined(__i386__) || defined(__x86_64__)/* Quick & dirty check for ABI changes. */ -BUILD_BUG_ON (sizeof (cfg) !=64 ); +BUILD_BUG_ON (sizeof (cfg) !=68 );/* Mnemonics for the named fields inside xen_x86_arch_domainconfig */ #define VAL_EMUL_FLAGS Field(arch_domconfig, 0)
@@ -291,6 +293,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co#undef VAL_MAX_GRANT_VERSION#undef VAL_MAX_MAPTRACK_FRAMES#undef VAL_MAX_GRANT_FRAMES +#undef VAL_MAX_ALTP2M#undef VAL_MAX_EVTCHN_PORT#undef VAL_MAX_VCPUS#undef VAL_IOMMU_OPTS
--
2.34.1
PATCH v2 3/7 docs/man: Add max_altp2m parameter to the xl.cfg manual
In reply to Petr BenešFrom: Petr Beneš <[email protected]>
Update manual pages to include detailed information about the max_altp2m configuration parameter.
--- docs/man/xl.cfg.5.pod.in | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
docs/man/xl.cfg.5.pod.in +14 -0
@@ -2039,6 +2039,20 @@ a single guest HVM domain. B<This option is deprecated, use the option B<Note>: While the option "altp2mhvm" is deprecated, legacy applications for x86 systems will continue to work using it. +=item B<max_altp2m=NUMBER> + +Specifies the maximum number of alternate-p2m views available to the guest. +This setting is crucial in domain introspection scenarios that require +multiple physical-to-machine (p2m) memory mappings to be established +simultaneously. + +Enabling multiple p2m views may increase memory usage. It is advisable to +review and adjust the B<shadow_memory> setting as necessary to accommodate +the additional memory requirements. + +B<Note>: This option is ignored if B<altp2m> is disabled. The default value +is 10. + =item B<nestedhvm=BOOLEAN> Enable or disables guest access to hardware virtualisation features,
Signature · 1 line
2.34.1
PATCH v2 7/7 x86/hap: Increase the number of initial mempool_size to 1024 pages
In reply to Petr BenešFrom: Petr Beneš <[email protected]>
This change anticipates scenarios where max_altp2m is set to its maximum supported value (i.e., 512), ensuring sufficient memory is allocated upfront to accommodate all altp2m tables without initialization failure.
The necessity for this increase arises from the current mechanism where altp2m tables are allocated at initialization, requiring one page from the mempool for each altp2m view.
--- tools/tests/paging-mempool/test-paging-mempool.c | 2 +- xen/arch/x86/mm/hap/hap.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/tests/paging-mempool/test-paging-mempool.c +1 -1
@@ -35,7 +35,7 @@ static struct xen_domctl_createdomain create = { static uint64_t default_mempool_size_bytes =#if defined(__x86_64__) || defined(__i386__) -256 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ +1024 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ #elif defined (__arm__) || defined(__aarch64__)16 <<12 ;#endif
xen/arch/x86/mm/hap/hap.c +1 -1
@@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)if ( old_pages ==0 ) {paging_lock (d); - rv =hap_set_allocation (d,256 ,NULL ); + rv =hap_set_allocation (d,1024 ,NULL );if ( rv !=0 ) {hap_set_allocation (d,0 ,NULL );
Signature · 1 line
2.34.1
PATCH v2 2/7 tools/xl: Add max_altp2m parameter
In reply to Petr BenešFrom: Petr Beneš <[email protected]>
Introduce a new max_altp2m parameter to control the maximum number of altp2m views a domain can use. By default, if max_altp2m is unspecified and altp2m is enabled, the value is set to 10, reflecting the legacy behavior.
This change is preparatory; it establishes the groundwork for the feature but does not activate it.
---
Changed since v1:
* Removed the change of xen/include/public/domctl.h (moved into future commit)
* Removed corresponding assignment of xen_domctl_createdomain::max_altp2m in libxl_create.c (moved into future commit)
* Clarify in commit message that this change is preparatory
tools/golang/xenlight/helpers.gen.go | 2 ++ tools/golang/xenlight/types.gen.go | 1 + tools/include/libxl.h | 8 ++++++++ tools/libs/light/libxl_create.c | 8 ++++++++ tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_parse.c | 4 ++++ 6 files changed, 24 insertions(+)
tools/golang/xenlight/helpers.gen.go +2 -0
@@ -1158,6 +1158,7 @@ if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil { return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err) } x.Altp2M = Altp2MMode(xc.altp2m) +x.MaxAltp2M = uint32(xc.max_altp2m) x.VmtraceBufKb = int(xc.vmtrace_buf_kb) if err := x.Vpmu.fromC(&xc.vpmu);err != nil { return fmt.Errorf("converting field Vpmu: %v", err)
@@ -1674,6 +1675,7 @@ if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil { return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err) } xc.altp2m = C.libxl_altp2m_mode(x.Altp2M) +xc.max_altp2m = C.uint32_t(x.MaxAltp2M) xc.vmtrace_buf_kb = C.int(x.VmtraceBufKb) if err := x.Vpmu.toC(&xc.vpmu); err != nil { return fmt.Errorf("converting field Vpmu: %v", err)
tools/golang/xenlight/types.gen.go +1 -0
@@ -602,6 +602,7 @@ ArchX86 struct { MsrRelaxed Defbool } Altp2M Altp2MMode +MaxAltp2M uint32 VmtraceBufKb int Vpmu Defbool }
tools/include/libxl.h +8 -0
@@ -1239,6 +1239,14 @@ typedef struct libxl__ctx libxl_ctx;*/ #define LIBXL_HAVE_ALTP2M 1 +/* +* LIBXL_HAVE_MAX_ALTP2M +* If this is defined, then libxl supports setting the maximum number of +* alternate p2m tables. +*/ +#define LIBXL_HAVE_MAX_ALTP2M 1 +#define LIBXL_MAX_ALTP2M_DEFAULT (~(uint32_t)0) +/* * LIBXL_HAVE_REMUS * If this is defined, then libxl supports remus.
tools/libs/light/libxl_create.c +8 -0
@@ -483,6 +483,14 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,return -ERROR_INVAL ; } +if (b_info->max_altp2m ==LIBXL_MAX_ALTP2M_DEFAULT ) { +if ((libxl_defbool_val (b_info->u.hvm.altp2m) || + b_info->altp2m !=LIBXL_ALTP2M_MODE_DISABLED )) + b_info->max_altp2m =10 ; +else + b_info->max_altp2m =0 ; + } +/* Assume that providing a bootloader user implies enabling restrict. */ libxl_defbool_setdefault (&b_info->bootloader_restrict, !!b_info->bootloader_user);
tools/libs/light/libxl_types.idl +1 -0
@@ -728,6 +728,7 @@ libxl_domain_build_info = Struct("domain_build_info",[ # Alternate p2m is not bound to any architecture or guest type, as it is # supported by x86 HVM and ARM support is planned. ("altp2m", libxl_altp2m_mode), + ("max_altp2m", uint32, {'init_val': 'LIBXL_MAX_ALTP2M_DEFAULT'}), # Size of preallocated vmtrace trace buffers (in KBYTES). # Use zero value to disable this feature.
tools/xl/xl_parse.c +4 -0
@@ -2061,6 +2061,10 @@ void parse_config_data(const char *config_source, } } +if (!xlu_cfg_get_long (config,"max_altp2m" , &l,1 )) { + b_info->max_altp2m = l; + } +if (!xlu_cfg_get_long (config,"vmtrace_buf_kb" , &l,1 ) && l) { b_info->vmtrace_buf_kb = l; }
--
2.34.1
Re: [PATCH v2 1/7] x86/p2m: Add braces for better code clarity
In reply to Petr BenešQuoted 5 lines · Petr Beneš
From: Petr Beneš <[email protected]>
No functional change.
Signed-off-by Petr Beneš <[email protected]>
Where did Stefano's R-b go?
Jan
Re: [PATCH v2 1/7] x86/p2m: Add braces for better code clarity
Quoted 11 lines · Jan Beulich
Quoted 5 lines · Petr Beneš
From: Petr Beneš <[email protected]>
No functional change.
Signed-off-by Petr Beneš <[email protected]>Where did Stefano's R-b go?
Jan
Oh no, I missed that one. Should I do v3?
Re: [PATCH v2 1/7] x86/p2m: Add braces for better code clarity
Quoted 5 lines · Petr Beneš
Quoted 4 lines · Jan Beulich
Quoted 2 lines · Petr Beneš
From: Petr Beneš <[email protected]>
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
No functional change.
Signed-off-by Petr Beneš <[email protected]>Where did Stefano's R-b go?
Oh no, I missed that one. Should I do v3?
Not just for this, I'd say. Just don't forget it again, if the patch needs re-posting.
Jan
Re: [PATCH v2 4/7] x86: Make the maximum number of altp2m views configurable
In reply to Petr BenešQuoted 7 lines · Petr Beneš
From: Petr Beneš <[email protected]>
This commit introduces the ability to configure the maximum number of altp2m tables during domain creation. Previously, the limits were hardcoded to a maximum of 10. This change allows for greater flexibility in environments that require more or fewer altp2m views.
Quoted 5 lines
The maximum configurable limit for
max_altp2mon x86 is now set toMAX_EPTP(512). This cap is linked to the architectural limit of the EPTP-switching VMFUNC, which supports up to 512 entries. Despite there being no inherent need for limitingmax_altp2min scenarios not utilizing VMFUNC, decoupling these components would necessitate substantial code changes.
While I don't mind this connection, ...
xen/arch/x86/domain.c int arch_sanitise_domain_config(struct xen_domctl_createdomain *config) · line 692
Quoted 11 lines
xen/arch/x86/domain.c +5
-0@@ -685,6 +685,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)return -EINVAL ; } +if ( config->max_altp2m >MAX_EPTP ) + { +dprintk (XENLOG_INFO ,"max_altp2m must be <= %u\n" ,MAX_EPTP ); +return -EINVAL ; + }
... using MAX_EPTP here feels like a layering violation to me. Imo there want to be separate constants, tied together with a suitably placed BUILD_BUG_ON().
Furthermore comparisons like this (there are further ones elsewhere) suggest there is a (continued) naming issue: A max_ or MAX_ prefix ought to name a "maximum valid value", not "number of permitted values". This is not a request to alter MAX_EPTP, but one to make sure the new struct fields really have matching names and purposes.
xen/arch/x86/include/asm/domain.h struct paging_vcpu { · line 262
Quoted 8 lines
xen/arch/x86/include/asm/domain.h +2
-0@@ -258,11 +258,10 @@ struct paging_vcpu {struct shadow_vcpu shadow; }; +#define INVALID_ALTP2M 0xffff +#define MAX_EPTP ((unsigned int)(PAGE_SIZE / sizeof(uint64_t)))
Aiui you add this cast because of the various min() uses. However, besides wanting to avoid such casts (or in fact any, whenever possible), I don't see why you need to retain all those min(): You bound d->max_altp2m against MAX_EPTP during domain creation anyway.
xen/arch/x86/mm/altp2m.c line 20
Quoted 11 lines
xen/arch/x86/mm/altp2m.c +5
-0@@ -13,6 +13,12 @@void altp2m_vcpu_initialise (struct vcpu *v) { +struct domain *d = v->domain; + +/* Skip initialisation if no altp2m will be used. */ +if ( d->max_altp2m ==0 ) +return ;
While I'm fine with this comment, ...
xen/arch/x86/mm/altp2m.c altp2m_vcpu_initialise(struct vcpu *v) · line 42
Quoted 10 lines
xen/arch/x86/mm/altp2m.c +4
-0@@ -28,8 +34,13 @@ altp2m_vcpu_initialise(struct vcpu *v)void altp2m_vcpu_destroy (struct vcpu *v) { +struct domain *d = v->domain;struct p2m_domain *p2m; +/* Skip destruction if no altp2m was used. */ +if ( d->max_altp2m ==0 ) +return ;
... this one doesn't look quite right: Even if altp2m-s were initialized, none may have been used (yet).
xen/arch/x86/mm/altp2m.c int p2m_init_altp2m(struct domain *d) · line 138
Quoted 10 lines
xen/arch/x86/mm/altp2m.c +5
-1@@ -120,7 +131,13 @@ int p2m_init_altp2m(struct domain *d)struct p2m_domain *hostp2m =p2m_get_hostp2m (d);mm_lock_init (&d->arch.altp2m_list_lock); -for ( i =0 ; i <MAX_ALTP2M ; i++ ) + + if ( (d->arch.altp2m_p2m = xzalloc_array(struct p2m_domain *, d->max_altp2m)) == NULL ) + { +return -ENOMEM ; + }
Nit: Overlong line and no need for the braces.
xen/arch/x86/mm/altp2m.c int p2m_init_altp2m(struct domain *d) · line 142
Quoted 3 lines
xen/arch/x86/mm/altp2m.c +1
-0+for ( i =0 ; i < d->max_altp2m; i++ ) { d->arch.altp2m_p2m[i] = p2m =p2m_init_one (d);
This loop, btw, also demonstrates that "maximum" isn't true here. The domain gets all of them allocated in one go.
Quoted 13 lines
xen/arch/x86/mm/altp2m.c +4
-1@@ -141,7 +158,10 @@ void p2m_teardown_altp2m(struct domain *d)unsigned int i;struct p2m_domain *p2m; -for ( i =0 ; i <MAX_ALTP2M ; i++ ) +if ( !d->arch.altp2m_p2m ) +return ; + +for ( i =0 ; i < d->max_altp2m; i++ ) {if ( !d->arch.altp2m_p2m[i] )continue ;
xen/arch/x86/mm/altp2m.c void p2m_teardown_altp2m(struct domain *d) · line 175
Quoted 7 lines
xen/arch/x86/mm/altp2m.c +3
-0@@ -149,6 +169,9 @@ void p2m_teardown_altp2m(struct domain *d) d->arch.altp2m_p2m[i] =NULL ;p2m_free_one (p2m); } + +xfree (d->arch.altp2m_p2m); + d->arch.altp2m_p2m =NULL ; }
Please don't (wrongly) open-code XFREE().
xen/arch/x86/mm/altp2m.c int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx, · line 2093
Quoted 6 lines
xen/arch/x86/mm/altp2m.c +1
-1@@ -2090,13 +2090,13 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,mfn_t mfn;int rc = -EINVAL ; - if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) || + if ( idx >= min(d->max_altp2m, MAX_EPTP) ||
Nit: Please take the opportunity and remove the excess blank.
xen/arch/x86/mm/altp2m.c int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx, · line 2094
Quoted 1 line
xen/arch/x86/mm/altp2m.c +0
-0d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
This use of MAX_EPTP also needs replacing, to avoid speculatively overrunning the allocated array. At least one more instance elsewhere.
xen/arch/x86/mm/altp2m.c int p2m_set_suppress_ve_multi(struct domain *d, · line 2583
Quoted 12 lines
xen/arch/x86/mm/altp2m.c +2
-2@@ -2575,12 +2575,12 @@ int p2m_set_suppress_ve_multi(struct domain *d,if ( sve->view >0 ) { -if ( sve->view >=min (ARRAY_SIZE (d->arch.altp2m_p2m),MAX_EPTP ) || +if ( sve->view >=min (d->max_altp2m,MAX_EPTP ) || d->arch.altp2m_eptp[array_index_nospec (sve->view,MAX_EPTP )] ==mfn_x (INVALID_MFN ) )return -EINVAL ; - p2m = ap2m =array_access_nospec (d->arch.altp2m_p2m, sve->view); + p2m = ap2m = d->arch.altp2m_p2m[array_index_nospec (sve->view, d->max_altp2m)];
Nit: Overlong line (more elsewhere).
xen/common/domain.c static int sanitise_domain_config(struct xen_domctl_createdomain *config) · line 571
Quoted 7 lines
xen/common/domain.c +1
-0@@ -568,6 +568,12 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config) } } +if ( config->max_altp2m && !hvm_altp2m_supported () )
This looks like it'll break the build on non-x86.
Quoted 12 lines
xen/include/public/domctl.h +1
-1@@ -21,7 +21,7 @@#include "hvm/save.h" #include "memory.h" -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000016 +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017/* * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
xen/include/public/domctl.h struct xen_domctl_createdomain { · line 82
Quoted 6 lines
xen/include/public/domctl.h +1
-0@@ -77,6 +77,7 @@ struct xen_domctl_createdomain {*/ uint32_t max_vcpus;uint32_t max_evtchn_port; +uint32_t max_altp2m;int32_t max_grant_frames;int32_t max_maptrack_frames;
Both this and ...
xen/include/xen/sched.h struct domain · line 607
Quoted 9 lines
xen/include/xen/sched.h +2
-0@@ -602,6 +602,8 @@ struct domain unsigned int guest_request_sync : 1; } monitor; +unsigned int max_altp2m;/* Maximum number of altp2m tables */ +unsigned int vmtrace_size;/* Buffer size in bytes, or 0 to disable. */
... this suggest you're confident other architectures will also want to support altp2m. Yet nothing like that is said in the description. In the absence of that common code should not require touching (much).
Jan
Re: [PATCH v2 7/7] x86/hap: Increase the number of initial mempool_size to 1024 pages
In reply to Petr BenešQuoted 5 lines · Petr Beneš
From: Petr Beneš <[email protected]>
This change anticipates scenarios where
max_altp2mis set to its maximum supported value (i.e., 512), ensuring sufficient memory is allocated upfront to accommodate all altp2m tables without initialization failure.
And guests with fewer or even no altp2m-s still need the same bump? You know the number of altp2m-s upon domain creation, so why bump by any more than what's strictly needed for that?
Quoted 3 lines
The necessity for this increase arises from the current mechanism where altp2m tables are allocated at initialization, requiring one page from the mempool for each altp2m view.
So that's the p2m_alloc_table() out of hap_enable()? If you're permitting up to 512 altp2m-s, I think it needs considering to not waste up to 2Mb without knowing how many of the altp2m-s are actually going to be used. How complicate on-demand allocation would be I can't tell though, I have to admit.
tools/tests/paging-mempool/test-paging-mempool.c static struct xen_domctl_createdomain create = { · line 38
Quoted 8 lines
tools/tests/paging-mempool/test-paging-mempool.c +1
-1@@ -35,7 +35,7 @@ static struct xen_domctl_createdomain create = { static uint64_t default_mempool_size_bytes =#if defined(__x86_64__) || defined(__i386__) -256 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ +1024 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */
I also can't derive from the description why we'd need to go from 256 to 1024 here and ...
xen/arch/x86/mm/hap/hap.c int hap_enable(struct domain *d, u32 mode) · line 471
Quoted 8 lines
xen/arch/x86/mm/hap/hap.c +1
-1@@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)if ( old_pages ==0 ) {paging_lock (d); - rv =hap_set_allocation (d,256 ,NULL ); + rv =hap_set_allocation (d,1024 ,NULL );
... here. You talk of (up to) 512 pages there only.
Also isn't there at least one more place where the tool stack (libxl I think) would need changing, where Dom0 ballooning needs are calculated? And/or doesn't the pool size have a default calculation in the tool stack, too?
Jan
Re: [PATCH v2 7/7] x86/hap: Increase the number of initial mempool_size to 1024 pages
Quoted 11 lines · Jan Beulich
Quoted 5 lines · Petr Beneš
From: Petr Beneš <[email protected]>
This change anticipates scenarios where
max_altp2mis set to its maximum supported value (i.e., 512), ensuring sufficient memory is allocated upfront to accommodate all altp2m tables without initialization failure.And guests with fewer or even no altp2m-s still need the same bump? You know the number of altp2m-s upon domain creation, so why bump by any more than what's strictly needed for that?
I have to admit I've considered computing the value which goes to hap_set_allocation
by simply adding 256 + max_altp2m, but that felt so arbitrary - the 256 value itself
feels arbitrary, as I haven't found any reasoning for it anywhere.
I have also tried to make code changes to make the initial allocation size configurable
via libxl (possibly reusing the shadow_memkb) - which seemed to me like the "correct"
solution, but those changes were more complicated than I had anticipated and I would
definitely not make it till the 4.19 deadline.
Question is, what to do now? Should I change it to 256 + max_altp2m?
Quoted 7 lines
Quoted 3 lines
The necessity for this increase arises from the current mechanism where altp2m tables are allocated at initialization, requiring one page from the mempool for each altp2m view.
So that's the
p2m_alloc_table()out ofhap_enable()? If you're permitting up to 512 altp2m-s, I think it needs considering to not waste up to 2Mb without knowing how many of the altp2m-s are actually going to be used.
Yes and I ultimately agree.
Quoted 2 lines
How complicate on-demand allocation would be I can't tell though, I have to admit.
That's also a fix I've been trying to work on - to make whole altp2m allocations on-demand. Unfortunately, I didn't make it in time.
Quoted 11 lines
tools/tests/paging-mempool/test-paging-mempool.c static struct xen_domctl_createdomain create = { · line 38
Quoted 8 lines
tools/tests/paging-mempool/test-paging-mempool.c +1
-1@@ -35,7 +35,7 @@ static struct xen_domctl_createdomain create = { static uint64_t default_mempool_size_bytes =#if defined(__x86_64__) || defined(__i386__) -256 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ +1024 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ I also can't derive from the description why we'd need to go from 256 to 1024 here and ...
It's explained in the code few lines below:
/*
* Check that the domain has the expected default allocation size. This
* will fail if the logic in Xen is altered without an equivalent
* adjustment here.
*/
I have verified that the default_mempool_size_bytes must reflect the number of pages in the initial hap_set_allocation() call.
Is it something I should include in the commit message, too?
Quoted 11 lines
xen/arch/x86/mm/hap/hap.c int hap_enable(struct domain *d, u32 mode) · line 471
Quoted 8 lines
xen/arch/x86/mm/hap/hap.c +1
-1@@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)if ( old_pages ==0 ) {paging_lock (d); - rv =hap_set_allocation (d,256 ,NULL ); + rv =hap_set_allocation (d,1024 ,NULL );... here. You talk of (up to) 512 pages there only.
Quoted 4 lines
Also isn't there at least one more place where the tool stack (libxl I think) would need changing, where Dom0 ballooning needs are calculated? And/or doesn't the pool size have a default calculation in the tool stack, too?
I have found places in libxl where the mempool_size is calculated, but that mempool
size is then set AFTER the domain is created via xc_set_paging_mempool_size.
In my opinion it doesn't necessarily require change, since it's expected by the user
to manually set it via shadow_memkb. The only current problem is (which this commit is trying to fix) that setting shadow_memkb doesn't help when max_altp2m > (256 - 1 + vcpus + MAX_NESTEDP2M), since the initial mempool size is hardcoded.
I didn't find any other places that would require reflecting the current "256" value.
P.
Re: [PATCH v2 4/7] x86: Make the maximum number of altp2m views configurable
In reply to Jan BeulichQuoted 16 lines · Jan Beulich
xen/arch/x86/domain.c int arch_sanitise_domain_config(struct xen_domctl_createdomain *config) · line 692
Quoted 12 lines
xen/arch/x86/domain.c +5
-0@@ -685,6 +685,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)return -EINVAL ; } +if ( config->max_altp2m >MAX_EPTP ) + { +dprintk (XENLOG_INFO ,"max_altp2m must be <= %u\n" ,MAX_EPTP ); +return -EINVAL ; + }... using
MAX_EPTPhere feels like a layering violation to me. Imo there want to be separate constants, tied together with a suitably placedBUILD_BUG_ON().
Quoted 5 lines
Furthermore comparisons like this (there are further ones elsewhere) suggest there is a (continued) naming issue: A
max_orMAX_prefix ought to name a "maximum valid value", not "number of permitted values". This is not a request to alterMAX_EPTP, but one to make sure the new struct fields really have matching names and purposes.
Do you have any proposals? I was considering nr_altp2m. Another question is what it should be named in xl.cfg - also nr_altp2m? I was too hesitant to name it like that, since there aren't any nr_* fields currently.
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/include/asm/domain.h +0
-0@@ -258,11 +258,10 @@ struct paging_vcpu {struct shadow_vcpu shadow; };
Quoted 7 lines
xen/arch/x86/include/asm/domain.h struct paging_vcpu { · line 262
Quoted 2 lines
xen/arch/x86/include/asm/domain.h +2
-0+#define INVALID_ALTP2M 0xffff +#define MAX_EPTP ((unsigned int)(PAGE_SIZE / sizeof(uint64_t)))Aiui you add this cast because of the various
min()uses. However, besides wanting to avoid such casts (or in fact any, whenever possible), I don't see why you need to retain all thosemin(): You boundd->max_altp2magainstMAX_EPTPduring domain creation anyway.
Fair. I considered removing the min()s altogether. I left them in place because I was too paranoid.
Quoted 8 lines
Quoted 8 lines
xen/arch/x86/include/asm/domain.h +1
-0@@ -28,8 +34,13 @@ altp2m_vcpu_initialise(struct vcpu *v)void altp2m_vcpu_destroy (struct vcpu *v) { +struct domain *d = v->domain;struct p2m_domain *p2m;
Quoted 6 lines
xen/arch/x86/include/asm/domain.h altp2m_vcpu_initialise(struct vcpu *v) · line 42
Quoted 3 lines
xen/arch/x86/include/asm/domain.h +3
-0+/* Skip destruction if no altp2m was used. */ +if ( d->max_altp2m ==0 ) +return ;... this one doesn't look quite right: Even if altp2m-s were initialized, none may have been used (yet).
Fair. Bad choice of words.
Quoted 7 lines
Quoted 7 lines
xen/include/xen/sched.h +0
-0@@ -602,6 +602,8 @@ struct domain unsigned int guest_request_sync : 1; } monitor;
Quoted 7 lines
xen/include/xen/sched.h struct domain · line 607
Quoted 3 lines
xen/include/xen/sched.h +2
-0+unsigned int max_altp2m;/* Maximum number of altp2m tables */ +unsigned int vmtrace_size;/* Buffer size in bytes, or 0 to disable. */ ... this suggest you're confident other architectures will also want to support altp2m. Yet nothing like that is said in the description. In the absence of that common code should not require touching (much).
Depends on the definition of "want". altp2m patches for arm/aarch64 were sent to Xen some 6 years ago but were unfortunately rejected. I would very much welcome them.
I have added the field to domain instead of arch_domain simply because it is not an arch-bound feature - similarly to vmtrace below, which also doesn't have an equivalent implementation on other archs than x86 (yet).
As far as other comments/nits go - understood.
P.
Re: [PATCH v2 4/7] x86: Make the maximum number of altp2m views configurable
Quoted 22 lines · Petr Beneš
Quoted 15 lines · Jan Beulich
xen/arch/x86/domain.c int arch_sanitise_domain_config(struct xen_domctl_createdomain *config) · line 692
Quoted 11 lines
xen/arch/x86/domain.c +5
-0@@ -685,6 +685,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)return -EINVAL ; } +if ( config->max_altp2m >MAX_EPTP ) + { +dprintk (XENLOG_INFO ,"max_altp2m must be <= %u\n" ,MAX_EPTP ); +return -EINVAL ; + }... using
MAX_EPTPhere feels like a layering violation to me. Imo there want to be separate constants, tied together with a suitably placedBUILD_BUG_ON().Quoted 5 lines
Furthermore comparisons like this (there are further ones elsewhere) suggest there is a (continued) naming issue: A
max_orMAX_prefix ought to name a "maximum valid value", not "number of permitted values". This is not a request to alterMAX_EPTP, but one to make sure the new struct fields really have matching names and purposes.
Quoted 4 lines
Do you have any proposals? I was considering
nr_altp2m. Another question is what it should be named in xl.cfg - alsonr_altp2m? I was too hesitant to name it like that, since there aren't any nr_* fields currently.
Internally nr_ or num_ are going to be fine. For xl whether either of those would be, or maybe altp2ms= (along the lines of e.g. vcpus=), or altp2m_count, or yet something else I simply don't know. That'll be the maintainers there to help with.
Jan
Re: [PATCH v2 7/7] x86/hap: Increase the number of initial mempool_size to 1024 pages
In reply to Petr BenešQuoted 27 lines · Petr Beneš
Quoted 11 lines · Jan Beulich
Quoted 5 lines · Petr Beneš
From: Petr Beneš <[email protected]>
This change anticipates scenarios where
max_altp2mis set to its maximum supported value (i.e., 512), ensuring sufficient memory is allocated upfront to accommodate all altp2m tables without initialization failure.And guests with fewer or even no altp2m-s still need the same bump? You know the number of altp2m-s upon domain creation, so why bump by any more than what's strictly needed for that?
I have to admit I've considered computing the value which goes to
hap_set_allocation
by simply adding 256 +max_altp2m, but that felt so arbitrary - the 256 value itself
feels arbitrary, as I haven't found any reasoning for it anywhere.I have also tried to make code changes to make the initial allocation size configurable
via libxl (possibly reusing theshadow_memkb) - which seemed to me like the "correct"
solution, but those changes were more complicated than I had anticipated and I would
definitely not make it till the 4.19 deadline.
Quoted 1 line
Question is, what to do now? Should I change it to 256 +
max_altp2m?
Counter question: Is accounting for just the root page table really enough? Meaning to say: I'm not convinced that minimum would really be appropriate for altp2m use even before your changes. You growing the number of root page tables _always_ required just makes things worse even without considering how (many) altp2m-s are then going to be used. Such an issue, if I'm right with this, would imo want addressing up front, in a separate patch.
You may also want to take a look at the shadow side of things, even if for altp2m shadow mode may not be relevant. Things like sh_min_allocation() and shadow_min_acceptable_pages() may well need proper counterparts in HAP now.
Quoted 20 lines
Quoted 11 lines
tools/tests/paging-mempool/test-paging-mempool.c static struct xen_domctl_createdomain create = { · line 38
Quoted 8 lines
tools/tests/paging-mempool/test-paging-mempool.c +1
-1@@ -35,7 +35,7 @@ static struct xen_domctl_createdomain create = { static uint64_t default_mempool_size_bytes =#if defined(__x86_64__) || defined(__i386__) -256 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ +1024 <<12 ;/* Only x86 HAP for now. x86 Shadow needs more work. */ I also can't derive from the description why we'd need to go from 256 to 1024 here and ...
It's explained in the code few lines below:
/*
* Check that the domain has the expected default allocation size. This
* will fail if the logic in Xen is altered without an equivalent
* adjustment here.
*/
Quoted 4 lines
I have verified that the
default_mempool_size_bytesmust reflect the number of pages in the initialhap_set_allocation()call.Is it something I should include in the commit message, too?
Well, yes and no. My question wasn't about the "why", but ...
Quoted 10 lines
Quoted 10 lines
xen/arch/x86/mm/hap/hap.c int hap_enable(struct domain *d, u32 mode) · line 471
Quoted 8 lines
xen/arch/x86/mm/hap/hap.c +1
-1@@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)if ( old_pages ==0 ) {paging_lock (d); - rv =hap_set_allocation (d,256 ,NULL ); + rv =hap_set_allocation (d,1024 ,NULL );... here. You talk of (up to) 512 pages there only.
... about the "by how many".
Quoted 9 lines
Quoted 4 lines
Also isn't there at least one more place where the tool stack (libxl I think) would need changing, where Dom0 ballooning needs are calculated? And/or doesn't the pool size have a default calculation in the tool stack, too?
I have found places in libxl where the
mempool_sizeis calculated, but that mempool
size is then set AFTER the domain is created viaxc_set_paging_mempool_size.
Quoted 6 lines
In my opinion it doesn't necessarily require change, since it's expected by the user
to manually set it viashadow_memkb. The only current problem is (which this commit is trying to fix) that settingshadow_memkbdoesn't help whenmax_altp2m> (256 - 1 + vcpus +MAX_NESTEDP2M), since the initial mempool size is hardcoded.
Wait - are you saying the guest config value isn't respected in certain cases? That would be another thing wanting to be fixed separately, up front.
Jan
Re: [PATCH v2 7/7] x86/hap: Increase the number of initial mempool_size to 1024 pages
Quoted 31 lines · Jan Beulich
Quoted 27 lines · Petr Beneš
Quoted 11 lines · Jan Beulich
Quoted 5 lines · Petr Beneš
From: Petr Beneš <[email protected]>
This change anticipates scenarios where
max_altp2mis set to its maximum supported value (i.e., 512), ensuring sufficient memory is allocated upfront to accommodate all altp2m tables without initialization failure.And guests with fewer or even no altp2m-s still need the same bump? You know the number of altp2m-s upon domain creation, so why bump by any more than what's strictly needed for that?
I have to admit I've considered computing the value which goes to
hap_set_allocation
by simply adding 256 +max_altp2m, but that felt so arbitrary - the 256 value itself
feels arbitrary, as I haven't found any reasoning for it anywhere.I have also tried to make code changes to make the initial allocation size configurable
via libxl (possibly reusing theshadow_memkb) - which seemed to me like the "correct"
solution, but those changes were more complicated than I had anticipated and I would
definitely not make it till the 4.19 deadline.Quoted 1 line
Question is, what to do now? Should I change it to 256 +
max_altp2m?
Quoted 7 lines
Counter question: Is accounting for just the root page table really enough? Meaning to say: I'm not convinced that minimum would really be appropriate for altp2m use even before your changes. You growing the number of root page tables _always_ required just makes things worse even without considering how (many) altp2m-s are then going to be used. Such an issue, if I'm right with this, would imo want addressing up front, in a separate patch.
It is enough - at least based on my experiments. I'll try to explain it below.
Quoted 16 lines
Quoted 9 lines
Quoted 4 lines
Also isn't there at least one more place where the tool stack (libxl I think) would need changing, where Dom0 ballooning needs are calculated? And/or doesn't the pool size have a default calculation in the tool stack, too?
I have found places in libxl where the
mempool_sizeis calculated, but that mempool
size is then set AFTER the domain is created viaxc_set_paging_mempool_size.Quoted 6 lines
In my opinion it doesn't necessarily require change, since it's expected by the user
to manually set it viashadow_memkb. The only current problem is (which this commit is trying to fix) that settingshadow_memkbdoesn't help whenmax_altp2m> (256 - 1 + vcpus +MAX_NESTEDP2M), since the initial mempool size is hardcoded.
Quoted 3 lines
Wait - are you saying the guest config value isn't respected in certain cases? That would be another thing wanting to be fixed separately, up front.
The xc_set_paging_mempool_size is still called within domain_create. So the value of shadow_memkb is respected before any of the guest code is run. My point was that shadow_memkb isn't respected here:
Quoted 8 lines
Quoted 8 lines
xen/arch/x86/mm/hap/hap.c int hap_enable(struct domain *d, u32 mode) · line 471
Quoted 8 lines
xen/arch/x86/mm/hap/hap.c +1
-1@@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)if ( old_pages ==0 ) {paging_lock (d); - rv =hap_set_allocation (d,256 ,NULL ); + rv =hap_set_allocation (d,1024 ,NULL );
This code (+ the root altp2ms allocation) is executed before the libxl sends the shadow_memkb.
In another words, the sequence is following:
libxl:
------
do_domain_create
initiate_domain_create
libxl__domain_make
xc_domain_create // MAX_ALTP2M is passed here
// and hap_enable is called
dcs->bl.callback = domcreate_bootloader_donedomcreate_bootloader_done
libxl__domain_build
libxl__build_pre
libxl__arch_domain_create
libxl__domain_set_paging_mempool_size
xc_set_paging_mempool_size(shadow_mem)
xen (xc_domain_create cont.):
-----------------------------domain_create
arch_domain_create
hvm_domain_initialise
paging_enable
hap_enable
// note that we shadow_mem (from config) hasn't been
// provided yet
hap_set_allocation(d, 1024, NULL);
p2m_alloc_table(p2m_get_hostp2m(d));
p2m_alloc_table(d->arch.nested_p2m[i..MAX_NESTEDP2M]);
p2m_alloc_table(d->arch.altp2m_p2m[i..MAX_ALTP2M]);(I hope the email will preserve the spacing...)
Based on this, I would argue that shadow_memkb should be also part of xc_domain_create/xen_domctl_createdomain. Which is why I've said in previous email:
Quoted 7 lines
Quoted 7 lines
I have also tried to make code changes to make the initial allocation size configurable
via libxl (possibly reusing theshadow_memkb) - which seemed to me like the "correct"
solution, but those changes were more complicated than I had anticipated and I would
definitely not make it till the 4.19 deadline.
P.