libxl memkb & pt defaulting
v5 cover libxl memkb & pt defaulting
I still want to sort out the shadow/iommu memory and pci passthrough situation in 4.13, to avoid promulgating a wrong libxl API in a stable release.
The only change since v4 is to change the default value name for the "passthrough" config setting to "default", apropos of the discussion. That is also the only patch without an appropriate ack/review.
This is also available here:
https://xenbits.xen.org/gitweb/?p=people/iwj/xen.git;a=summary
wip.libxl-memkb-ptcfg.v5
Ian Jackson (10):
a 01 libxl: Offer API versions 0x040700 and 0x040800 r 02 xl: Pass libxl_domain_config to freemem(), instead of b_info r 03 libxl: libxl__domain_config_setdefault: New function r 04 libxl: libxl_domain_need_memory: Make it take a domain_config r 05 libxl: Move shadow_memkb and iommu_memkb defaulting into libxl a 06 libxl: Remove/deprecate libxl_get_required_*_memory from the API a 07 libxl: create: setdefault: Make libxl_physinfo info[1] a 08 libxl: create: setdefault: Move physinfo into config_setdefault a 09 libxl: Move domain_create_info_setdefault earlier
* 10 libxl/xl: Overhaul passthrough setting logic
a=acked; r=reviewed; *=patch changed
docs/man/xl.cfg.5.pod.in | 6 ++ tools/libxl/libxl.h | 24 +++++- tools/libxl/libxl_arch.h | 6 ++ tools/libxl/libxl_arm.c | 24 ++++++ tools/libxl/libxl_create.c | 172 +++++++++++++++++++++++++++++++++---------- tools/libxl/libxl_dm.c | 7 +- tools/libxl/libxl_dom.c | 7 +- tools/libxl/libxl_internal.h | 13 +++- tools/libxl/libxl_mem.c | 69 +++++++++++++---- tools/libxl/libxl_types.idl | 7 +- tools/libxl/libxl_utils.c | 15 ---- tools/libxl/libxl_utils.h | 2 +- tools/libxl/libxl_x86.c | 41 +++++++++++ tools/xl/xl_parse.c | 82 ++------------------- tools/xl/xl_vmcontrol.c | 6 +- 15 files changed, 325 insertions(+), 156 deletions(-)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 1/10 libxl: Offer API versions 0x040700 and 0x040800
According to git log -G:
0x040700 was introduced in 304400459ef0 (aka 4.7.0-rc1~481)
"tools/libxl: rename remus device to checkpoint device"
0x040800 was introduced in 57f8b13c7240 (aka 4.8.0-rc1~437)
"libxl: memory size in kb requires 64 bit variable"
It is surprising that no-one noticed this.
Anyway, in the meantime, we should fix it. Backporting this is probably a good idea: it won't change the behaviour for existing callers but it will avoid errors for some older correct uses.
---
v2: Adjusted commit message slightly.
--- tools/libxl/libxl.h | 1 + 1 file changed, 1 insertion(+)
tools/libxl/libxl.h +1 -0
@@ -655,6 +655,7 @@ typedef struct libxl__ctx libxl_ctx;#ifdef LIBXL_API_VERSION #if LIBXL_API_VERSION !=0x040200 &&LIBXL_API_VERSION !=0x040300 && \LIBXL_API_VERSION !=0x040400 &&LIBXL_API_VERSION !=0x040500 && \ +LIBXL_API_VERSION !=0x040700 &&LIBXL_API_VERSION !=0x040800 && \LIBXL_API_VERSION !=0x041300 #error Unknown LIBXL_API_VERSION#endif
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 2/10 xl: Pass libxl_domain_config to freemem(), instead of b_info
In reply to Ian JacksonWe are going to change the libxl API in a moment and this change will make it simpler.
--- tools/xl/xl_vmcontrol.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/xl/xl_vmcontrol.c +3 -3
@@ -314,7 +314,7 @@ static int domain_wait_event(uint32_t domid, libxl_event **event_r)* Returns true in case there is already, or we manage to free it, enough * memory, but also if autoballoon is false. */ -static bool freemem (uint32_t domid,libxl_domain_build_info *b_info) +static bool freemem (uint32_t domid,libxl_domain_config *d_config) {int rc, retries =3 ;uint64_t need_memkb, free_memkb;
@@ -322,7 +322,7 @@ static bool freemem(uint32_t domid, libxl_domain_build_info *b_info)if (!autoballoon)return true; - rc =libxl_domain_need_memory (ctx, b_info, &need_memkb); + rc =libxl_domain_need_memory (ctx, &d_config->b_info, &need_memkb);if (rc <0 )return false;
@@ -879,7 +879,7 @@ start: goto error_out;if (domid_soft_reset ==INVALID_DOMID ) { -if (!freemem (domid, &d_config.b_info)) { +if (!freemem (domid, &d_config)) {fprintf (stderr,"failed to free memory for the domain\n" ); ret =ERROR_FAIL ; goto error_out;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 3/10 libxl: libxl__domain_config_setdefault: New function
In reply to Ian JacksonBreak out this into a new function. We are going to want to call it from a new call site.
Unfortunately not all of the defaults can be moved into the new function without changing the order in which things are done. That does not seem wise at this stage of the release. The effect is that additional calls to libxl__domain_config_setdefault (which are going to be introduced) do not quite set everything. But they will do what is needed. After Xen 4.13 is done, we should move those settings into the right order.
No functional change.
---
v2: Add missing error check
--- tools/libxl/libxl_create.c | 41 +++++++++++++++++++++++++++++------------ tools/libxl/libxl_internal.h | 3 +++ 2 files changed, 32 insertions(+), 12 deletions(-)
tools/libxl/libxl_create.c +29 -12
@@ -862,22 +862,14 @@ static void domcreate_destruction_cb(libxl__egc *egc,libxl__domain_destroy_state *dds, int rc); -static void initiate_domain_create (libxl__egc *egc, -libxl__domain_create_state *dcs) +int libxl__domain_config_setdefault (libxl__gc *gc, +libxl_domain_config *d_config, +uint32_t domid) { -STATE_AO_GC (dcs->ao);libxl_ctx *ctx =libxl__gc_owner (gc); -uint32_t domid; -int i, ret; +int ret;bool pod_enabled = false; -/* convenience aliases */ -libxl_domain_config *const d_config = dcs->guest_config; -const int restore_fd = dcs->restore_fd; - - domid = dcs->domid_soft_reset; -libxl__domain_build_state_init (&dcs->build_state); -if (d_config->c_info.ssid_label) {char *s = d_config->c_info.ssid_label; ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
@@ -1008,6 +1000,28 @@ static void initiate_domain_create(libxl__egc *egc, goto error_out; } + ret =0 ; + error_out: +return ret; +} + +static void initiate_domain_create (libxl__egc *egc, +libxl__domain_create_state *dcs) +{ +STATE_AO_GC (dcs->ao); +uint32_t domid; +int i, ret; + +/* convenience aliases */ +libxl_domain_config *const d_config = dcs->guest_config; +const int restore_fd = dcs->restore_fd; + + domid = dcs->domid_soft_reset; +libxl__domain_build_state_init (&dcs->build_state); + + ret =libxl__domain_config_setdefault (gc,d_config,domid); +if (ret) goto error_out; + ret =libxl__domain_make (gc, d_config, &dcs->build_state, &domid);if (ret) {LOGD (ERROR , domid,"cannot make domain: %d" , ret);
@@ -1019,6 +1033,9 @@ static void initiate_domain_create(libxl__egc *egc, dcs->guest_domid = domid; dcs->sdss.dm.guest_domid =0 ;/* means we haven't spawned */ +/* post-4.13 todo: move these next bits of defaulting to +* libxl__domain_config_setdefault */ +/* * Set the dm version quite early so that libxl doesn't have to pass the * build info around just to know if the domain has a device model or not.
tools/libxl/libxl_internal.h +3 -0
@@ -1440,6 +1440,9 @@ _hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,* All libxl API functions are expected to have arranged for this * to be called before using any values within these structures. */ +_hidden int libxl__domain_config_setdefault(libxl__gc *gc, +libxl_domain_config *d_config, +uint32_t domid/* logging only */ ); _hidden int libxl__domain_create_info_setdefault(libxl__gc *gc,libxl_domain_create_info *c_info); _hidden int libxl__domain_build_info_setdefault(libxl__gc *gc,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 4/10 libxl: libxl_domain_need_memory: Make it take a domain_config
In reply to Ian JacksonThis should calculate the extra memory needed for shadow and iommu, the defaults for which depend on values in c_info. So we need this to have the complete domain config available.
And the defaults should actually be updated and stored. So make it non-const.
We provide the usual kind of compatibility function for callers expecting 4.12 and earlier. This function becomes responsible for the clone-and-modify of the b_info.
No overall functional change for external libxl callers which use the API version system to request a particular API version.
Other external libxl callers will need to update their calling code, and will then find that the new version of this function fills in most of the defaults in d_config. Because libxl__domain_config_setdefault doesn't quite do all of the defaults, that's only partial. For present purposes that doesn't matter because none of the missing settings are used by the memory calculations. It does mean we need to document in the API spec that the defaulting is only partial.
This lack of functional change is despite the fact that numa_place_domain now no longer calls libxl__domain_build_info_setdefault (via libxl_domain_need_memory). That is OK because it's idempotent and numa_place_domain's one call site is libxl__build_pre which is called from libxl__domain_build which is called from domcreate_bootloader_done, well after the defaults are set by initiate_domain_create.
---
v2: Drop now-erroneous GC_FREE as well as the corresponding GC_INIT.
--- tools/libxl/libxl.h | 23 +++++++++++++++- tools/libxl/libxl_dom.c | 7 +++-- tools/libxl/libxl_internal.h | 4 +++ tools/libxl/libxl_mem.c | 65 +++++++++++++++++++++++++++++++++++--------- tools/xl/xl_vmcontrol.c | 2 +- 5 files changed, 83 insertions(+), 18 deletions(-)
tools/libxl/libxl.h +22 -1
@@ -1245,6 +1245,20 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);*/ #define LIBXL_HAVE_FN_USING_QMP_ASYNC 1 +/* +* LIBXL_HAVE_DOMAIN_NEED_MEMORY_CONFIG +* +* If this is set, libxl_domain_need_memory takes a +* libxl_domain_config* (non-const) and uint32_t domid_for_logging +* (instead of a const libxl_domain_build_info*). +* +* If this is set, there is no need to call +* libxl_get_required_shadow_memory and instead the caller should +* simply leave shadow_memkb set to LIBXL_MEMKB_DEFAULT and allow +* libxl to fill in a suitable default in the usual way. +*/ +#define LIBXL_HAVE_DOMAIN_NEED_MEMORY_CONFIG +typedef char **libxl_string_list ;void libxl_string_list_dispose (libxl_string_list *sl);int libxl_string_list_length (const libxl_string_list *sl);
@@ -1723,8 +1737,13 @@ int libxl_get_memory_target_0x040700(libxl_ctx *ctx, uint32_t domid,*/ /* how much free memory in the system a domain needs to be built */ int libxl_domain_need_memory (libxl_ctx *ctx, -const libxl_domain_build_info *b_info_in, +libxl_domain_config *config +/* ^ will be partially defaulted */ , +uint32_t domid_for_logging/* INVALID_DOMID ok */ ,uint64_t *need_memkb); +int libxl_domain_need_memory_0x041200 (libxl_ctx *ctx, +const libxl_domain_build_info *b_info_in, +uint64_t *need_memkb);int libxl_domain_need_memory_0x040700 (libxl_ctx *ctx,const libxl_domain_build_info *b_info_in,uint32_t *need_memkb)
@@ -1754,6 +1773,8 @@ int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs);#define libxl_get_memory_target libxl_get_memory_target_0x040700#define libxl_domain_need_memory libxl_domain_need_memory_0x040700#define libxl_get_free_memory libxl_get_free_memory_0x040700 +#elif defined(LIBXL_API_VERSION ) &&LIBXL_API_VERSION <0x041300 +#define libxl_domain_need_memory libxl_domain_need_memory_0x041200#endif int libxl_vncviewer_exec (libxl_ctx *ctx,uint32_t domid,int autopass);
tools/libxl/libxl_dom.c +4 -3
@@ -140,8 +140,9 @@ static int numa_cmpf(const libxl__numa_candidate *c1,/* The actual automatic NUMA placement routine */ static int numa_place_domain (libxl__gc *gc,uint32_t domid, -libxl_domain_build_info *info) +libxl_domain_config *d_config) { +libxl_domain_build_info *info = &d_config->b_info;int found;libxl__numa_candidate candidate;libxl_bitmap cpumap, cpupool_nodemap, *map;
@@ -195,7 +196,7 @@ static int numa_place_domain(libxl__gc *gc, uint32_t domid, } } - rc =libxl_domain_need_memory (CTX , info, &memkb); + rc =libxl__domain_need_memory_calculate (gc, info, &memkb);if (rc) goto out;if (libxl_node_bitmap_alloc (CTX , &cpupool_nodemap,0 )) {
@@ -432,7 +433,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,if (rc)return rc; - rc =numa_place_domain (gc, domid, info); + rc =numa_place_domain (gc, domid, d_config);if (rc) {libxl_bitmap_dispose (&cpumap_soft);return rc;
tools/libxl/libxl_internal.h +4 -0
@@ -1450,6 +1450,10 @@ _hidden int libxl__domain_build_info_setdefault(libxl__gc *gc, _hidden void libxl__rdm_setdefault(libxl__gc *gc,libxl_domain_build_info *b_info); +_hidden int libxl__domain_need_memory_calculate(libxl__gc *gc, +libxl_domain_build_info *b_info, +uint64_t *need_memkb); + _hidden const char *libxl__device_nic_devname(libxl__gc *gc, uint32_t domid, uint32_t devid,
tools/libxl/libxl_mem.c +52 -13
89 lines not on this page · raw message
tools/xl/xl_vmcontrol.c +1 -1
@@ -322,7 +322,7 @@ static bool freemem(uint32_t domid, libxl_domain_config *d_config)if (!autoballoon)return true; - rc =libxl_domain_need_memory (ctx, &d_config->b_info, &need_memkb); + rc =libxl_domain_need_memory (ctx, d_config, domid, &need_memkb);if (rc <0 )return false;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 5/10 libxl: Move shadow_memkb and iommu_memkb defaulting into libxl
In reply to Ian JacksonDefaulting is supposed to be done by libxl. So these calculations should be here in libxl. libxl__domain_config_setdefault has all the necessary information including the values of max_memkb and max_vcpus.
The overall functional effect depends on the caller:
For xl, no change. The code moves from xl to libxl.
For callers who set one or both shadow_memkb and iommu_memkb (whether from libxl_get_required_shadow_memory or otherwise) before calling libxl_domain_need_memory (any version): the new code will leave their setting(s) unchanged.
For callers who do not call libxl_domain_need_memory at all, and who fail to set one of these memory values: now they are both are properly set. The shadow and iommu memory to be properly accounted for as intended.
For callers which call libxl_domain_need_memory and request the current API (4.13) or which track libxl, the default values are also now right and everything works as intended.
For callers which call libxl_domain_need_memory, and request an old pre-4.13 libxl API, and which leave one of these memkb settings unset, we take special measures to preserve the old behaviour.
This means that they don't get the additional iommu memory and are at risk of the domain running out of memory as a result of f89f555827a6 "remove late (on-demand) construction of IOMMU page tables". But this is no worse than the state just after f89f555827a6, which already broke such callers in that way. This is perhaps justifiable because of the API stability warning next to libxl_domain_need_memory.
An alternative would be to drop the special-casing of these callers. That would cause a discrepancy between libxl_domain_need_memory andlibxl_domain_create: the former would not include the iommu memory and the latter would. That seems worse, but it's debateable.
---
v4: Provide a fallback default for iommu_memkb too, for old callers.
v2: Replace _Bool with bool
Fix logic sense in ok_to_default_memkb_in_create
--- tools/libxl/libxl_create.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ tools/libxl/libxl_internal.h | 3 +++ tools/libxl/libxl_mem.c | 4 ++++ tools/xl/xl_parse.c | 15 ++------------- 4 files changed, 53 insertions(+), 13 deletions(-)
tools/libxl/libxl_create.c +44 -0
@@ -247,9 +247,14 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,libxl__arch_domain_build_info_setdefault (gc, b_info);libxl_defbool_setdefault (&b_info->dm_restrict, false); +if (b_info->iommu_memkb ==LIBXL_MEMKB_DEFAULT ) +/* Normally defaulted in libxl__domain_create_info_setdefault */ + b_info->iommu_memkb =0 ; +switch (b_info->type) {case LIBXL_DOMAIN_TYPE_HVM :if (b_info->shadow_memkb ==LIBXL_MEMKB_DEFAULT ) +/* Normally defaulted in libxl__domain_create_info_setdefault */ b_info->shadow_memkb =0 ;if (b_info->u.hvm.mmio_hole_memkb ==LIBXL_MEMKB_DEFAULT ) b_info->u.hvm.mmio_hole_memkb =0 ;
@@ -395,6 +400,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,if (b_info->video_memkb ==LIBXL_MEMKB_DEFAULT ) b_info->video_memkb =0 ;if (b_info->shadow_memkb ==LIBXL_MEMKB_DEFAULT ) +/* Normally defaulted in libxl__domain_create_info_setdefault */ b_info->shadow_memkb =0 ;if (b_info->u.pv.slack_memkb ==LIBXL_MEMKB_DEFAULT ) b_info->u.pv.slack_memkb =0 ;
@@ -862,6 +868,30 @@ static void domcreate_destruction_cb(libxl__egc *egc,libxl__domain_destroy_state *dds, int rc); +static bool ok_to_default_memkb_in_create (libxl__gc *gc) +{ +/* +* This is a fudge. We are trying to find whether the caller +* calls the old version of libxl_domain_need_memory. If they do +* then, because it only gets the b_info, and because it can't +* update the b_info (because it's const), it will base its +* calculations on defaulting shadow_memkb and iommu_memkb to 0 +* In that case we probably shouldn't default them differently +* during libxl_domain_create. +* +* The result is that the behaviour with old callers is the same +* as in 4.13: no additional memory is allocated for shadow and +* iommu (unless the caller set shadow_memkb, eg from a call to +* libxl_get_required_shadow_memory). +*/ +return !CTX ->libxl_domain_need_memory_0x041200_called || +CTX ->libxl_domain_need_memory_called; +/* +* Treat mixed callers as new callers. Presumably they know what +* they are doing. +*/ +} +int libxl__domain_config_setdefault (libxl__gc *gc,libxl_domain_config *d_config,uint32_t domid)
@@ -974,6 +1004,20 @@ int libxl__domain_config_setdefault(libxl__gc *gc, goto error_out; } +if (d_config->b_info.shadow_memkb ==LIBXL_MEMKB_DEFAULT + &&ok_to_default_memkb_in_create (gc)) + d_config->b_info.shadow_memkb = +libxl_get_required_shadow_memory (d_config->b_info.max_memkb, + d_config->b_info.max_vcpus); + +/* No IOMMU reservation is needed if passthrough mode is not 'sync_pt' */ +if (d_config->b_info.iommu_memkb ==LIBXL_MEMKB_DEFAULT + &&ok_to_default_memkb_in_create (gc)) + d_config->b_info.iommu_memkb = + (d_config->c_info.passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) + ?libxl_get_required_iommu_memory (d_config->b_info.max_memkb) + :0 ; + ret =libxl__domain_build_info_setdefault (gc, &d_config->b_info);if (ret) {LOGD (ERROR , domid,"Unable to set domain build info defaults" );
tools/libxl/libxl_internal.h +3 -0
@@ -637,6 +637,9 @@ struct libxl__ctx {LIBXL_LIST_ENTRY (libxl_ctx) sigchld_users_entry;libxl_version_info version_info; + +bool libxl_domain_need_memory_0x041200_called, + libxl_domain_need_memory_called; };/*
tools/libxl/libxl_mem.c +4 -0
@@ -484,6 +484,8 @@ int libxl_domain_need_memory(libxl_ctx *ctx,GC_INIT (ctx);int rc; + ctx->libxl_domain_need_memory_called =1 ; + rc =libxl__domain_config_setdefault (gc, d_config, domid_for_logging);
@@ -507,6 +509,8 @@ int libxl_domain_need_memory_0x041200(libxl_ctx *ctx,GC_INIT (ctx);int rc; + ctx->libxl_domain_need_memory_0x041200_called =1 ; +libxl_domain_build_info b_info[1 ];libxl_domain_build_info_init (b_info);libxl_domain_build_info_copy (ctx, b_info, b_info_in);
tools/xl/xl_parse.c +2 -13
@@ -1572,19 +1572,8 @@ void parse_config_data(const char *config_source,exit (-ERROR_FAIL ); } -/* libxl_get_required_shadow_memory() and -* libxl_get_required_iommu_memory() must be called after final values -* (default or specified) for vcpus and memory are set, because the -* calculations depend on those values. */ - b_info->shadow_memkb = !xlu_cfg_get_long (config,"shadow_memory" , &l,0 ) - ? l *1024 - :libxl_get_required_shadow_memory (b_info->max_memkb, - b_info->max_vcpus); - -/* No IOMMU reservation is needed if passthrough mode is not 'sync_pt' */ - b_info->iommu_memkb = (c_info->passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) - ?libxl_get_required_iommu_memory (b_info->max_memkb) - :0 ; +if (!xlu_cfg_get_long (config,"shadow_memory" , &l,0 )) + b_info->shadow_memkb = l *1024 ;xlu_cfg_get_defbool (config,"nomigrate" , &b_info->disable_migrate,0 );
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 6/10 libxl: Remove/deprecate libxl_get_required_*_memory from the API
In reply to Ian JacksonThese are now redundant because shadow_memkb and iommu_memkb are now defaulted automatically by libxl_domain_need_memory and libxl_domain_create etc. Callers should not now call these; instead, they should just let libxl take care of it.
libxl_get_required_shadow_memory was introduced in f89f555827a6
"remove late (on-demand) construction of IOMMU page tables"
We can freely remove it because it was never in any release.
libxl_get_required_shadow_memory has been in libxl approximately forever. It should probably not have survived the creation of libxl_domain_create, but it seems the API awkwardnesses we see in recent commits prevented this. So we have to keep it. It remains functional but we can deprecate it. Hopefully we can get rid of it completely before we find the need to change the calculation to use additional information which its arguments do not currently supply.
--- tools/libxl/libxl_create.c | 17 ++++++++++++++++- tools/libxl/libxl_utils.c | 15 --------------- tools/libxl/libxl_utils.h | 2 +- 3 files changed, 17 insertions(+), 17 deletions(-)
tools/libxl/libxl_create.c +16 -1
@@ -892,6 +892,21 @@ static bool ok_to_default_memkb_in_create(libxl__gc *gc)*/ } +static unsigned long libxl__get_required_iommu_memory (unsigned long maxmem_kb) +{ +unsigned long iommu_pages =0 , mem_pages = maxmem_kb /4 ; +unsigned int level; + +/* Assume a 4 level page table with 512 entries per level */ +for (level =0 ; level <4 ; level++) + { + mem_pages =DIV_ROUNDUP (mem_pages,512 ); + iommu_pages += mem_pages; + } + +return iommu_pages *4 ; +} +int libxl__domain_config_setdefault (libxl__gc *gc,libxl_domain_config *d_config,uint32_t domid)
@@ -1015,7 +1030,7 @@ int libxl__domain_config_setdefault(libxl__gc *gc, && ok_to_default_memkb_in_create(gc)) d_config->b_info.iommu_memkb = (d_config->c_info.passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) - ?libxl_get_required_iommu_memory (d_config->b_info.max_memkb) + ?libxl__get_required_iommu_memory (d_config->b_info.max_memkb) :0 ; ret =libxl__domain_build_info_setdefault (gc, &d_config->b_info);
tools/libxl/libxl_utils.c +0 -15
@@ -48,21 +48,6 @@ unsigned long libxl_get_required_shadow_memory(unsigned long maxmem_kb, unsignedreturn 4 * (256 * smp_cpus +2 * (maxmem_kb /1024 )); } -unsigned long libxl_get_required_iommu_memory (unsigned long maxmem_kb) -{ -unsigned long iommu_pages =0 , mem_pages = maxmem_kb /4 ; -unsigned int level; - -/* Assume a 4 level page table with 512 entries per level */ -for (level =0 ; level <4 ; level++) - { - mem_pages =DIV_ROUNDUP (mem_pages,512 ); - iommu_pages += mem_pages; - } - -return iommu_pages *4 ; -} -char *libxl_domid_to_name (libxl_ctx *ctx,uint32_t domid) {unsigned int len;
tools/libxl/libxl_utils.h +1 -1
@@ -24,7 +24,7 @@ constchar *libxl_basename (const char *name);/* returns string from strdup */ unsigned long libxl_get_required_shadow_memory (unsigned long maxmem_kb,unsigned int smp_cpus); -unsigned long libxl_get_required_iommu_memory (unsigned long maxmem_kb); +/* deprecated; see LIBXL_HAVE_DOMAIN_NEED_MEMORY_CONFIG in libxl.h */ int libxl_name_to_domid (libxl_ctx *ctx,const char *name,uint32_t *domid);int libxl_domain_qualifier_to_domid (libxl_ctx *ctx,const char *name,uint32_t *domid);char *libxl_domid_to_name (libxl_ctx *ctx,uint32_t domid);
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 7/10 libxl: create: setdefault: Make libxl_physinfo info[1]
In reply to Ian JacksonNo functional change. This will let us make it into a pointer without textual change other than to the definition.
While we are here, fix some style errors (missing { }).
---
v2: New patch in this version of the series.
--- tools/libxl/libxl_create.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
tools/libxl/libxl_create.c +8 -8
@@ -30,10 +30,10 @@int libxl__domain_create_info_setdefault (libxl__gc *gc,libxl_domain_create_info *c_info) { -libxl_physinfo info; +libxl_physinfo info[1 ];int rc; - rc =libxl_get_physinfo (CTX , &info); + rc =libxl_get_physinfo (CTX , info);if (rc)return rc;
@@ -45,11 +45,11 @@ int libxl__domain_create_info_setdefault(libxl__gc *gc,libxl__arch_domain_create_info_setdefault (gc, c_info);if (c_info->type !=LIBXL_DOMAIN_TYPE_PV ) { -if (info.cap_hap) +if (info->cap_hap) {libxl_defbool_setdefault (&c_info->hap, true); -else if (info.cap_shadow) + }else if (info->cap_shadow) {libxl_defbool_setdefault (&c_info->hap, false); -else { + }else {LOG (ERROR ,"neither hap nor shadow paging available" );return ERROR_INVAL ; }
@@ -63,12 +63,12 @@ int libxl__domain_create_info_setdefault(libxl__gc *gc,if (!c_info->ssidref) c_info->ssidref =SECINITSID_DOMU ; -if (info.cap_hvm_directio && +if (info->cap_hvm_directio && (c_info->passthrough ==LIBXL_PASSTHROUGH_UNKNOWN )) { c_info->passthrough = ((c_info->type ==LIBXL_DOMAIN_TYPE_PV ) || - !info.cap_iommu_hap_pt_share) ? + !info->cap_iommu_hap_pt_share) ?LIBXL_PASSTHROUGH_SYNC_PT :LIBXL_PASSTHROUGH_SHARE_PT ; - }else if (!info.cap_hvm_directio) { + }else if (!info->cap_hvm_directio) { c_info->passthrough =LIBXL_PASSTHROUGH_DISABLED ; }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 8/10 libxl: create: setdefault: Move physinfo into config_setdefault
In reply to Ian JacksonNo functional change. This will let us refer to it in code we are about to add to this function.
---
v2: New patch in this version of the series.
--- tools/libxl/libxl_create.c | 17 ++++++++--------- tools/libxl/libxl_dm.c | 7 ++++++- tools/libxl/libxl_internal.h | 3 ++- 3 files changed, 16 insertions(+), 11 deletions(-)
tools/libxl/libxl_create.c +8 -9
@@ -28,15 +28,9 @@#include <xen-xsm/flask/flask.h> int libxl__domain_create_info_setdefault (libxl__gc *gc, -libxl_domain_create_info *c_info) +libxl_domain_create_info *c_info, +const libxl_physinfo *info) { -libxl_physinfo info[1 ]; -int rc; - - rc =libxl_get_physinfo (CTX , info); -if (rc) -return rc; -if (!c_info->type) {LOG (ERROR ,"domain type unspecified" );return ERROR_INVAL ;
@@ -915,6 +909,10 @@ int libxl__domain_config_setdefault(libxl__gc *gc,int ret;bool pod_enabled = false; +libxl_physinfo physinfo; + ret =libxl_get_physinfo (CTX , &physinfo); +if (ret) goto error_out; +if (d_config->c_info.ssid_label) {char *s = d_config->c_info.ssid_label; ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
@@ -1013,7 +1011,8 @@ int libxl__domain_config_setdefault(libxl__gc *gc, goto error_out; } - ret =libxl__domain_create_info_setdefault (gc, &d_config->c_info); + ret =libxl__domain_create_info_setdefault (gc, &d_config->c_info, + &physinfo);if (ret) {LOGD (ERROR , domid,"Unable to set domain create info defaults" ); goto error_out;
tools/libxl/libxl_dm.c +6 -1
@@ -2167,7 +2167,12 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss) dm_config->c_info.run_hotplug_scripts = guest_config->c_info.run_hotplug_scripts; - ret =libxl__domain_create_info_setdefault (gc, &dm_config->c_info); +libxl_physinfo physinfo; + ret =libxl_get_physinfo (CTX , &physinfo); +if (ret) goto out; + + ret =libxl__domain_create_info_setdefault (gc, &dm_config->c_info, + &physinfo);if (ret) goto out; ret =libxl__domain_build_info_setdefault (gc, &dm_config->b_info);if (ret) goto out;
tools/libxl/libxl_internal.h +2 -1
@@ -1447,7 +1447,8 @@ _hidden int libxl__domain_config_setdefault(libxl__gc *gc,libxl_domain_config *d_config, uint32_t domid /* logging only */); _hidden int libxl__domain_create_info_setdefault(libxl__gc *gc, -libxl_domain_create_info *c_info); +libxl_domain_create_info *c_info, +const libxl_physinfo *info); _hidden int libxl__domain_build_info_setdefault(libxl__gc *gc,libxl_domain_build_info *b_info); _hidden void libxl__rdm_setdefault(libxl__gc *gc,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 9/10 libxl: Move domain_create_info_setdefault earlier
In reply to Ian JacksonWe need this before we start to figure out the passthrough mode.
I have checked that nothing in libxl__domain_create_info_setdefault nor the two implementations of ..._arch_... accesses anything else, other than (i) the domain type (which this function is responsible for setting and nothing before it looks at) (ii) c_info->ssidref (which is defaulted by flask code near the top of libxl__domain_config_setdefault and not accessed afterwards).
So no functional change.
---
v3: New patch in this version of the series.
--- tools/libxl/libxl_create.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tools/libxl/libxl_create.c +7 -7
@@ -972,6 +972,13 @@ int libxl__domain_config_setdefault(libxl__gc *gc, goto error_out; } + ret =libxl__domain_create_info_setdefault (gc, &d_config->c_info, + &physinfo); +if (ret) { +LOGD (ERROR , domid,"Unable to set domain create info defaults" ); + goto error_out; + } +/* If target_memkb is smaller than max_memkb, the subsequent call * to libxc when building HVM domain will enable PoD mode. */
@@ -1011,13 +1018,6 @@ int libxl__domain_config_setdefault(libxl__gc *gc, goto error_out; } - ret =libxl__domain_create_info_setdefault (gc, &d_config->c_info, - &physinfo); -if (ret) { -LOGD (ERROR , domid,"Unable to set domain create info defaults" ); - goto error_out; - } -if (d_config->b_info.shadow_memkb ==LIBXL_MEMKB_DEFAULT &&ok_to_default_memkb_in_create (gc)) d_config->b_info.shadow_memkb =
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 10/10 libxl/xl: Overhaul passthrough setting logic
In reply to Ian JacksonLIBXL_PASSTHROUGH_UNKNOWN (aka "ENABLED" in an earlier uncommitted version of this code) is doing double duty. We actually need all of the following to be specifiable:
* "default": enable PT iff we have devices to
pass through specified in the initial config file.
* "enabled" (and fail if the platform doesn't support it).
* "disabled" (and reject future PT hotplug).
* "share_pt"/"sync_pt": enable PT and set a specific PT mode.
Defaulting and error checking should be done in libxl. So, we make several changes here.
We introduce "enabled", and rename "unknown" to "default".
We move all of the error checking and defaulting code from xl into libxl. Now, libxl__domain_config_setdefault has all of the necessary information to get this right. So we can do it all there. Choosing the specific mode is arch-specific.
We can also arrange to have only one place each which calculates
(i) whether passthrough needs to be enabled because pt devices were specified (ii) whether pt_share can be used (for each arch).
xl now only has to parse the enum in the same way as it parses all other enums.
This change fixes a regression from earlier 4.13-pre: until recent changes, passthrough was only enabled by default if passthrough devices were specified. We restore this behaviour.
---
v5: Change "unknown" to "default". In the manpage too.
v4: Fix trailing whitespace
No longer change "unknown" to "unspecified".
v3: Drop paragraph about masking another osstest regression,
as that's now fixed.
Drop redundant "ERROR:" in two log messages.
Add a comment about the way "enabled" gets changed to a specific value. Split passthrough mode defaulting into arch specific functions. On ARM, always choose (and insist on) share_pt.
Reject share_pt for non-HAP guests.
Reject passthrough for PVH guests.
Actually document "unspecified" option in xl.cfg(5)
Rename "unknown" to "unspecified"
v2: New patch in this version of the series.
--- docs/man/xl.cfg.5.pod.in | 6 ++++ tools/libxl/libxl_arch.h | 6 ++++ tools/libxl/libxl_arm.c | 24 ++++++++++++++++ tools/libxl/libxl_create.c | 43 ++++++++++++++++++++--------- tools/libxl/libxl_types.idl | 7 +++-- tools/libxl/libxl_x86.c | 41 +++++++++++++++++++++++++++ tools/xl/xl_parse.c | 67 ++++----------------------------------------- 7 files changed, 116 insertions(+), 78 deletions(-)
docs/man/xl.cfg.5.pod.in +6 -0
@@ -660,6 +660,12 @@ in preference. However, the availability of this option is hardware specific. If B<xl info> reports B<virt_caps> containing B<iommu_hap_pt_share> then this option may be used. +=item B<default> + +The default, which chooses between B<disabled> and B<enabled> +according to whether passthrough devices are enabled in the config +file. + =back =back
tools/libxl/libxl_arch.h +6 -0
@@ -73,6 +73,12 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, libxl_domain_build_info *b_info);_hidden +int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc, +uint32_t domid, +libxl_domain_config *d_config, +const libxl_physinfo *physinfo); + +_hidden int libxl__arch_extra_memory(libxl__gc *gc,const libxl_domain_build_info *info,uint64_t *out);
tools/libxl/libxl_arm.c +24 -0
@@ -1191,6 +1191,30 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,libxl_domain_build_info_init_type (b_info,LIBXL_DOMAIN_TYPE_PVH ); } +int libxl__arch_passthrough_mode_setdefault (libxl__gc *gc, +uint32_t domid, +libxl_domain_config *d_config, +const libxl_physinfo *physinfo) +{ +int rc; +libxl_domain_create_info *const c_info = &d_config->c_info; + +if (c_info->passthrough ==LIBXL_PASSTHROUGH_ENABLED ) { + c_info->passthrough =LIBXL_PASSTHROUGH_SHARE_PT ; + } + +if (c_info->passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) { +LOGD (ERROR , domid, +"passthrough=\"sync_pt\" not supported on ARM\n" ); + rc =ERROR_INVAL ; + goto out; + } + + rc =0 ; + out: +return rc; +} +/* * Local variables: * mode: C
tools/libxl/libxl_create.c +30 -13
67 lines not on this page · raw message
tools/libxl/libxl_types.idl +4 -3
@@ -264,10 +264,11 @@ libxl_vkb_backend = Enumeration("vkb_backend", [ ]) libxl_passthrough = Enumeration("passthrough", [ - (0, "unknown"), + (0, "default"), (1, "disabled"), - (2, "sync_pt"), - (3, "share_pt"), + (2, "enabled"), # becomes {sync,share}_pt once defaults are evaluated + (3, "sync_pt"), + (4, "share_pt"), ]) #
tools/libxl/libxl_x86.c +41 -0
47 lines not on this page · raw message
tools/xl/xl_parse.c +5 -62
87 lines not on this page · raw message
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic
Hi Ian,
Quoted 16 lines · Ian Jackson
tools/libxl/libxl_arm.c +8
-0@@ -1191,6 +1191,30 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,libxl_domain_build_info_init_type (b_info,LIBXL_DOMAIN_TYPE_PVH ); } +int libxl__arch_passthrough_mode_setdefault (libxl__gc *gc, +uint32_t domid, +libxl_domain_config *d_config, +const libxl_physinfo *physinfo) +{ +int rc; +libxl_domain_create_info *const c_info = &d_config->c_info; +
tools/libxl/libxl_arm.c void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, · line 1211
Quoted 10 lines
tools/libxl/libxl_arm.c +10
-0+if (c_info->passthrough ==LIBXL_PASSTHROUGH_ENABLED ) { + c_info->passthrough =LIBXL_PASSTHROUGH_SHARE_PT ; + } + +if (c_info->passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) { +LOGD (ERROR , domid, +"passthrough=\"sync_pt\" not supported on ARM\n" ); + rc =ERROR_INVAL ; + goto out; + }
Would it make sense to use a switch case or whitelist here? So we don't end up to slip through if a new type is added and unsupported on Arm.
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic
Julien Grall writes ("Re: [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic"):
Quoted 1 line · Ian Jackson
...
Quoted 9 lines
tools/libxl/libxl_arm.c void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, · line 1211
Quoted 6 lines
tools/libxl/libxl_arm.c +6
-0+if (c_info->passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) { +LOGD (ERROR , domid, +"passthrough=\"sync_pt\" not supported on ARM\n" ); + rc =ERROR_INVAL ; + goto out; + }Would it make sense to use a switch case or whitelist here? So we don't end up to slip through if a new type is added and unsupported on Arm.
I wouldn't mind changing this to
c_info->passthrough != LIBXL_PASSTHROUGH_SHARE_PT
Although it's a shame you're raising this now. This code has been like this since v3. Would you mind if we did this as a followup patch ?
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic
Hi Ian,
Quoted 10 lines · Ian Jackson
Julien Grall writes ("Re: [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic"):
Quoted 1 line · Ian Jackson
...
Quoted 7 lines
Quoted 6 lines
tools/libxl/libxl_arm.c +6
-0+if (c_info->passthrough ==LIBXL_PASSTHROUGH_SYNC_PT ) { +LOGD (ERROR , domid, +"passthrough=\"sync_pt\" not supported on ARM\n" ); + rc =ERROR_INVAL ; + goto out; + }
Quoted 5 lines
Quoted 2 lines
Would it make sense to use a switch case or whitelist here? So we don't end up to slip through if a new type is added and unsupported on Arm.
I wouldn't mind changing this to
c_info->passthrough != LIBXL_PASSTHROUGH_SHARE_PT
I think it would need to be
c_info->passthrough != LIBXL_PASSTHROUGH_SHARE_PT && c_info->passthrough != LIBXL_PASSTHROUGH_DISABLED.
Quoted 4 lines
Although it's a shame you're raising this now. This code has been like this since v3. Would you mind if we did this as a followup patch ?
Sorry I haven't been really good at reviewing this week :(.
I am happy if this is done in a follow-up.
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic
Julien Grall writes ("Re: [XEN PATCH for-4.13 v5 10/10] libxl/xl: Overhaul passthrough setting logic"):
Quoted 8 lines · Ian Jackson
Quoted 2 lines
I wouldn't mind changing this to
c_info->passthrough != LIBXL_PASSTHROUGH_SHARE_PTI think it would need to be
c_info->passthrough!=LIBXL_PASSTHROUGH_SHARE_PT&&c_info->passthrough!=LIBXL_PASSTHROUGH_DISABLED.
Err, indeed. I'll do this or a switch().
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel