Skip to contentSkip to conversation

libxl: choose a sane default for HAP

PATCH Superseded Roger Pau Monne 5 participants 12 messages Sep 6, 2019 – Sep 10, 2019

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover libxl: choose a sane default for HAP

Hello,

First patch is a preparatory change to also make use of the physcaps on ARM, second patch introduces a new physcap (HAP) in order for the toolstack to decide whether to use HAP if the user hasn't made a selection.

Thanks, Roger.

Roger Pau Monne (2):
sysctl: report existing physcaps on ARM
sysctl/libxl: choose a sane default for HAP

 tools/libxl/libxl.c         |  1 +
 tools/libxl/libxl.h         |  7 +++++++
 tools/libxl/libxl_create.c  |  8 +++++++-
 tools/libxl/libxl_types.idl |  1 +
 tools/xl/xl_info.c          |  5 +++--
 xen/arch/arm/sysctl.c       |  5 ++++-
 xen/arch/x86/sysctl.c       |  4 ++--
 xen/common/sysctl.c         |  2 ++
 xen/include/public/sysctl.h | 10 +++++++---
 9 files changed, 34 insertions(+), 9 deletions(-)
Signature · 7 lines
2.22.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v3 1/2 sysctl: report existing physcaps on ARM

Current physcaps in XEN_SYSCTL_physinfo are only used by x86, albeit the capabilities themselves are not x86 specific.

This patch adds support for also reporting the current capabilities on ARM hardware. Note that on ARM PHYSCAP_hvm is always reported, and setting PHYSCAP_directio has been moved to common code since the same logic to set it is used by x86 and ARM.

---
Changes since v1:
- New in this version.

---
 xen/arch/arm/sysctl.c       | 5 ++++-
 xen/arch/x86/sysctl.c       | 2 --
 xen/common/sysctl.c         | 2 ++
 xen/include/public/sysctl.h | 6 +++---
 4 files changed, 9 insertions(+), 6 deletions(-)
xen/arch/arm/sysctl.c +4 -1
index fbfdb44eff..92ac99c928 100644
@@ -12,7 +12,10 @@
 #include <xen/hypercall.h>
 #include <public/sysctl.h>
 
-void arch_do_physinfo(struct xen_sysctl_physinfo *pi) { }
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm;
+}
 
 long arch_do_sysctl(struct xen_sysctl *sysctl,
                     XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
xen/arch/x86/sysctl.c +0 -2
index c50d910a1c..7ec6174e6b 100644
@@ -163,8 +163,6 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm;
     if ( IS_ENABLED(CONFIG_PV) )
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_pv;
-    if ( iommu_enabled )
-        pi->capabilities |= XEN_SYSCTL_PHYSCAP_directio;
 }
 
 long arch_do_sysctl(
xen/common/sysctl.c +2 -0
index fcf2d2fd7c..92b4ea0d21 100644
@@ -267,6 +267,8 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
         pi->cpu_khz = cpu_khz;
         pi->max_mfn = get_upper_mfn_bound();
         arch_do_physinfo(pi);
+        if ( iommu_enabled )
+            pi->capabilities |= XEN_SYSCTL_PHYSCAP_directio;
 
         if ( copy_to_guest(u_sysctl, op, 1) )
             ret = -EFAULT;
xen/include/public/sysctl.h +3 -3
index 91c48dcae0..36b3f8c429 100644
@@ -81,13 +81,13 @@ struct xen_sysctl_tbuf_op {
  * Get physical information about the host machine
  */
 /* XEN_SYSCTL_physinfo */
- /* (x86) The platform supports HVM guests. */
+ /* The platform supports HVM guests. */
 #define _XEN_SYSCTL_PHYSCAP_hvm          0
 #define XEN_SYSCTL_PHYSCAP_hvm           (1u<<_XEN_SYSCTL_PHYSCAP_hvm)
- /* (x86) The platform supports PV guests. */
+ /* The platform supports PV guests. */
 #define _XEN_SYSCTL_PHYSCAP_pv           1
 #define XEN_SYSCTL_PHYSCAP_pv            (1u<<_XEN_SYSCTL_PHYSCAP_pv)
- /* (x86) The platform supports direct access to I/O devices with IOMMU. */
+ /* The platform supports direct access to I/O devices with IOMMU. */
 #define _XEN_SYSCTL_PHYSCAP_directio     2
 #define XEN_SYSCTL_PHYSCAP_directio  (1u<<_XEN_SYSCTL_PHYSCAP_directio)
 struct xen_sysctl_physinfo {
Signature · 7 lines
2.22.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v3 1/2
#2 lore raw

Re: [Xen-devel] [PATCH v3 1/2] sysctl: report existing physcaps on ARM

Hi,

Title NIT: s/ARM/Arm/

Quoted 5 lines · Roger Pau Monne

Current physcaps in XEN_SYSCTL_physinfo are only used by x86, albeit the capabilities themselves are not x86 specific.

This patch adds support for also reporting the current capabilities on ARM hardware. Note that on ARM PHYSCAP_hvm is always reported, and

Ditto.

Quoted 2 lines

setting PHYSCAP_directio has been moved to common code since the same logic to set it is used by x86 and ARM.

Ditto.

Quoted 4 lines

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH v3 2/2 sysctl/libxl: choose a sane default for HAP

In reply to Roger Pau Monne

Current libxl code will always enable Hardware Assisted Paging (HAP), expecting that the hypervisor will fallback to shadow if HAP is not available. With the changes to the domain builder that's not the case any longer, and the hypervisor will raise an error if HAP is not available instead of silently falling back to shadow.

In order to keep the previous functionality report whether HAP is available or not in XEN_SYSCTL_physinfo, so that the toolstack can select a sane default if there's no explicit user selection of whether HAP should be used.

Note that on ARM hardware HAP capability is always reported since it's a required feature in order to run Xen.

Fixes d0c0ba7d3de ('x86/hvm/domain: remove the 'hap_enabled' flag')

---
Cc: Paul Durrant <[email protected]>
---
Changes since v2:
- Add a LIBXL_HAVE_PHYSINFO_CAP_HAP for compatibility.

Changes since v1:
- Also report HAP capability for ARM.
- Print hap capability in xl info.

---
 tools/libxl/libxl.c         | 1 +
 tools/libxl/libxl.h         | 7 +++++++
 tools/libxl/libxl_create.c  | 8 +++++++-
 tools/libxl/libxl_types.idl | 1 +
 tools/xl/xl_info.c          | 5 +++--
 xen/arch/arm/sysctl.c       | 2 +-
 xen/arch/x86/sysctl.c       | 2 ++
 xen/include/public/sysctl.h | 4 ++++
 8 files changed, 26 insertions(+), 4 deletions(-)
tools/libxl/libxl.c +1 -0
index ec71574e99..5c0fcf320e 100644
@@ -399,6 +399,7 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo)
     physinfo->cap_pv = !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_pv);
     physinfo->cap_hvm_directio =
         !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_directio);
+    physinfo->cap_hap = !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_hap);
 
     GC_FREE;
     return 0;
tools/libxl/libxl.h +7 -0
index 9bacfb97f0..3ff67792a7 100644
@@ -394,6 +394,13 @@
  */
 #define LIBXL_HAVE_EXTENDED_VKB 1
 
+/*
+ * LIBXL_HAVE_PHYSINFO_CAP_HAP indicates that libxl_physinfo has a cap_hap
+ * field that indicates whether the hardware supports Hardware Assisted
+ * Paging.
+ */
+#define LIBXL_HAVE_PHYSINFO_CAP_HAP 1
+
 /*
  * libxl ABI compatibility
  *
tools/libxl/libxl_create.c +7 -1
index 03ce166f4f..6a556dea8f 100644
@@ -38,7 +38,13 @@ int libxl__domain_create_info_setdefault(libxl__gc *gc,
     libxl__arch_domain_create_info_setdefault(gc, c_info);
 
     if (c_info->type != LIBXL_DOMAIN_TYPE_PV) {
-        libxl_defbool_setdefault(&c_info->hap, true);
+        libxl_physinfo info;
+        int rc = libxl_get_physinfo(CTX, &info);
+
+        if (rc)
+            return rc;
+
+        libxl_defbool_setdefault(&c_info->hap, info.cap_hap);
         libxl_defbool_setdefault(&c_info->oos, true);
     }
 
tools/libxl/libxl_types.idl +1 -0
index b61399ce36..9e1f8515d3 100644
@@ -1025,6 +1025,7 @@ libxl_physinfo = Struct("physinfo", [
     ("cap_hvm", bool),
     ("cap_pv", bool),
     ("cap_hvm_directio", bool), # No longer HVM specific
+    ("cap_hap", bool),
     ], dir=DIR_OUT)
 
 libxl_connectorinfo = Struct("connectorinfo", [
tools/xl/xl_info.c +3 -2
index 46d9c9f712..aa6724bc7f 100644
@@ -210,11 +210,12 @@ static void output_physinfo(void)
          info.hw_cap[4], info.hw_cap[5], info.hw_cap[6], info.hw_cap[7]
         );
 
-    maybe_printf("virt_caps              :%s%s%s%s\n",
+    maybe_printf("virt_caps              :%s%s%s%s%s\n",
          info.cap_pv ? " pv" : "",
          info.cap_hvm ? " hvm" : "",
          info.cap_hvm && info.cap_hvm_directio ? " hvm_directio" : "",
-         info.cap_pv && info.cap_hvm_directio ? " pv_directio" : ""
+         info.cap_pv && info.cap_hvm_directio ? " pv_directio" : "",
+         info.cap_hap ? " hap" : ""
         );
 
     vinfo = libxl_get_version_info(ctx);
xen/arch/arm/sysctl.c +1 -1
index 92ac99c928..f87944e847 100644
@@ -14,7 +14,7 @@
 
 void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
 {
-    pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm;
+    pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm | XEN_SYSCTL_PHYSCAP_hap;
 }
 
 long arch_do_sysctl(struct xen_sysctl *sysctl,
xen/arch/x86/sysctl.c +2 -0
index 7ec6174e6b..5777a05ffc 100644
@@ -163,6 +163,8 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_hvm;
     if ( IS_ENABLED(CONFIG_PV) )
         pi->capabilities |= XEN_SYSCTL_PHYSCAP_pv;
+    if ( hvm_hap_supported() )
+        pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap;
 }
 
 long arch_do_sysctl(
xen/include/public/sysctl.h +4 -0
index 36b3f8c429..e02d7ce4c6 100644
@@ -90,6 +90,10 @@ struct xen_sysctl_tbuf_op {
  /* The platform supports direct access to I/O devices with IOMMU. */
 #define _XEN_SYSCTL_PHYSCAP_directio     2
 #define XEN_SYSCTL_PHYSCAP_directio  (1u<<_XEN_SYSCTL_PHYSCAP_directio)
+/* The platform supports Hardware Assisted Paging. */
+#define _XEN_SYSCTL_PHYSCAP_hap          3
+#define XEN_SYSCTL_PHYSCAP_hap           (1u<<_XEN_SYSCTL_PHYSCAP_hap)
+
 struct xen_sysctl_physinfo {
     uint32_t threads_per_core;
     uint32_t cores_per_socket;
Signature · 7 lines
2.22.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

Re: [Xen-devel] [PATCH v3 2/2] sysctl/libxl: choose a sane default for HAP

Quoted 3 lines · Roger Pau Monne

Current libxl code will always enable Hardware Assisted Paging (HAP), expecting that the hypervisor will fallback to shadow if HAP is not available. With the changes to the domain builder that's not the case

"domain builder" is usually libxenguest.  I'd suggest using DOMCTL_createdomain specifically to make it obvious that it was a hypervisor side change.

otherwise, everything LGTM.  This should probably be adjusted on commit.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Acked-by on v3 2/2
#5 lore raw

Re: [Xen-devel] [PATCH v3 2/2] sysctl/libxl: choose a sane default for HAP

In reply to Roger Pau Monne

Hi,

Quoted 6 lines · Roger Pau Monne

Current libxl code will always enable Hardware Assisted Paging (HAP), expecting that the hypervisor will fallback to shadow if HAP is not available. With the changes to the domain builder that's not the case any longer, and the hypervisor will raise an error if HAP is not available instead of silently falling back to shadow.

Quoted 6 lines

In order to keep the previous functionality report whether HAP is available or not in XEN_SYSCTL_physinfo, so that the toolstack can select a sane default if there's no explicit user selection of whether HAP should be used.

Note that on ARM hardware HAP capability is always reported since it's

NIT: s/ARM/Arm/

Quoted 6 lines

a required feature in order to run Xen.

Fixes d0c0ba7d3de ('x86/hvm/domain: remove the 'hap_enabled' flag')

For the Arm bits:

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#6 lore raw

PATCH 1/1 fixup sysctl/libxl: choose a sane default for HAP

In reply to Roger Pau Monne

This delta wants folding to keep the Ocaml stubs in line.

---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Christian Lindig <[email protected]>
CC: Rob Hoes <[email protected]>

---
 tools/ocaml/libs/xc/xenctrl.ml      | 1 +
 tools/ocaml/libs/xc/xenctrl.mli     | 1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c | 3 ++-
 xen/include/public/sysctl.h         | 2 +-
 4 files changed, 5 insertions(+), 2 deletions(-)
tools/ocaml/libs/xc/xenctrl.ml +1 -0
index cd7e95a6fa..5da7c96cee 100644
@@ -101,6 +101,7 @@ type physinfo_cap_flag =
 	| CAP_HVM
 	| CAP_PV
 	| CAP_DirectIO
+	| CAP_HAP
 
 type physinfo =
 {
tools/ocaml/libs/xc/xenctrl.mli +1 -0
index 0bd06ed920..b4e76b827e 100644
@@ -86,6 +86,7 @@ type physinfo_cap_flag =
   | CAP_HVM
   | CAP_PV
   | CAP_DirectIO
+  | CAP_HAP
 type physinfo = {
   threads_per_core : int;
   cores_per_socket : int;
tools/ocaml/libs/xc/xenctrl_stubs.c +2 -1
index ce8dbba437..10b306f9f8 100644
@@ -690,7 +690,8 @@ CAMLprim value stub_xc_physinfo(value xch)
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_hvm      != (1u <<  0));
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_pv       != (1u <<  1));
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_directio != (1u <<  2));
-	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_MAX      != XEN_SYSCTL_PHYSCAP_directio);
+	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_hap      != (1u <<  3));
+	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_MAX      != XEN_SYSCTL_PHYSCAP_hap);
 	cap_list = c_bitmap_to_ocaml_list(c_physinfo.capabilities);
 
 	physinfo = caml_alloc_tuple(10);
xen/include/public/sysctl.h +1 -1
index c325ae496c..d4b455619c 100644
@@ -95,7 +95,7 @@ struct xen_sysctl_tbuf_op {
 #define XEN_SYSCTL_PHYSCAP_hap           (1u<<_XEN_SYSCTL_PHYSCAP_hap)
 
 /* Max XEN_SYSCTL_PHYSCAP_* constant.  Used for ABI checking. */
-#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_directio
+#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_hap
 
 struct xen_sysctl_physinfo {
     uint32_t threads_per_core;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#8 lore raw

Re: [Xen-devel] [PATCH] fixup sysctl/libxl: choose a sane default for HAP

In reply to Andrew Cooper

tools/ocaml/libs/xc/xenctrl_stubs.c CAMLprim value stub_xc_physinfo(value xch) · line 697

Quoted 12 lines · Andrew Cooper
tools/ocaml/libs/xc/xenctrl_stubs.c +2 -1
@@ -690,7 +690,8 @@ CAMLprim value stub_xc_physinfo(value xch)
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_hvm      != (1u <<  0));
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_pv       != (1u <<  1));
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_directio != (1u <<  2));
-	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_MAX      != XEN_SYSCTL_PHYSCAP_directio);
+	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_hap      != (1u <<  3));
+	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_MAX      != XEN_SYSCTL_PHYSCAP_hap);
 	cap_list = c_bitmap_to_ocaml_list(c_physinfo.capabilities);
 
 	physinfo = caml_alloc_tuple(10);

I'm afraid in particular this hunk doesn't apply anymore, and I'm not confident that simply dropping it is the right answer. I'd therefore like to ask for a proper re-base, or for someone else to apply the result of folding these additions into the base patch.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#9 lore raw

Re: [Xen-devel] [PATCH] fixup sysctl/libxl: choose a sane default for HAP

Quoted 12 lines · Jan Beulich
Quoted 11 lines · Andrew Cooper
tools/ocaml/libs/xc/xenctrl_stubs.c +2 -1
@@ -690,7 +690,8 @@ CAMLprim value stub_xc_physinfo(value xch)
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_hvm      != (1u <<  0));
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_pv       != (1u <<  1));
 	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_directio != (1u <<  2));
-	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_MAX      != XEN_SYSCTL_PHYSCAP_directio);
+	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_hap      != (1u <<  3));
+	BUILD_BUG_ON(XEN_SYSCTL_PHYSCAP_MAX      != XEN_SYSCTL_PHYSCAP_hap);
 	cap_list = c_bitmap_to_ocaml_list(c_physinfo.capabilities);
 
Quoted 6 lines

tools/ocaml/libs/xc/xenctrl_stubs.c CAMLprim value stub_xc_physinfo(value xch) · line 697

Quoted 1 line
tools/ocaml/libs/xc/xenctrl_stubs.c +0 -0
 	physinfo = caml_alloc_tuple(10);

I'm afraid in particular this hunk doesn't apply anymore, and I'm not confident that simply dropping it is the right answer. I'd therefore like to ask for a proper re-base, or for someone else to apply the result of folding these additions into the base patch.

Roger is currently working on a rebase.  There is also a logical error which requires the addition of CAP_SHADOW.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Acked-by on v3 2/2
#10 lore raw

Re: [Xen-devel] [PATCH v3 2/2] sysctl/libxl: choose a sane default for HAP

In reply to Roger Pau Monne

Roger Pau Monne writes ("[PATCH v3 2/2] sysctl/libxl: choose a sane default for HAP"):

Quoted 11 lines

Current libxl code will always enable Hardware Assisted Paging (HAP), expecting that the hypervisor will fallback to shadow if HAP is not available. With the changes to the domain builder that's not the case any longer, and the hypervisor will raise an error if HAP is not available instead of silently falling back to shadow.

In order to keep the previous functionality report whether HAP is available or not in XEN_SYSCTL_physinfo, so that the toolstack can select a sane default if there's no explicit user selection of whether HAP should be used.

Quoted 7 lines

Note that on ARM hardware HAP capability is always reported since it's a required feature in order to run Xen.

Fixes d0c0ba7d3de ('x86/hvm/domain: remove the 'hap_enabled' flag')

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#11 lore raw
In reply to Roger Pau Monne

Hi,

Quoted 11 lines · Roger Pau Monne

Hello,

First patch is a preparatory change to also make use of the physcaps on ARM, second patch introduces a new physcap (HAP) in order for the toolstack to decide whether to use HAP if the user hasn't made a selection.

Thanks, Roger.

Roger Pau Monne (2):
sysctl: report existing physcaps on ARM

I have committed this patch.

Quoted 1 line

sysctl/libxl: choose a sane default for HAP

This one requires an Ack from the tools maintainers.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel