Skip to contentSkip to conversation

expose host-phys-bits to guest

PATCH In review Gerd Hoffmann 2 participants 8 messages Sep 8, 2022 – Sep 9, 2022 KVM

lore
Share & tools
lore
#0 lore raw

v2 cover expose host-phys-bits to guest

When the guest (firmware specifically) knows how big the address space actually is it can be used better.

Some more background:
https://bugzilla.redhat.com/show_bug.cgi?id=2084533

This is a RfC series exposes the information via cpuid.

v2:
- change fvm hint name.
- better commit message.

take care,
Gerd

Gerd Hoffmann (2):
[temporary] reserve bit KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID [RfC] expose host-phys-bits to guest

 include/standard-headers/asm-x86/kvm_para.h | 3 ++-
 target/i386/cpu.h                           | 3 ---
 hw/i386/microvm.c                           | 7 ++++++-
 target/i386/cpu.c                           | 3 +--
 target/i386/host-cpu.c                      | 5 ++++-
 target/i386/kvm/kvm.c                       | 1 +
 6 files changed, 14 insertions(+), 8 deletions(-)
Signature · 1 line
2.37.3
#1 lore raw

PATCH v2 1/2 reserve bit KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID

The KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID bit hints to the guest that the size of the physical address space as advertised by CPUID leaf 0x80000008 is actually valid and can be used.

Unfortunately this is not the case today with qemu. Default behavior is to advertise 40 address bits (which I think comes from the very first x64 opteron processors). There are lots of intel desktop processors around which support less than that (36 or 39 depending on age), and when trying to use the full 40 bit address space on those things go south quickly.

This renders the physical address size information effectively useless for guests. This patch paves the way to fix that by adding a hint for the guest so it knows whenever the physical address size is usable or not.

The plan for qemu is to set the bit when the physical address size is valid. That is the case when qemu is started with the host-phys-bits=on option set for the cpu. Eventually qemu can also flip the default for that option from off to on, unfortunately that isn't easy for backward compatibility reasons.

The plan for the firmware is to check that bit and when it is set just query and use the available physical address space. When the bit is not set be conservative and try not exceed 36 bits (aka 64G) address space. The latter is what the firmware does today unconditionally.

[ Temporary qemu patch for RfC patch and testing. This change must
actually be done in the linux kernel, then picked up by qemu via header file sync ].

---
 include/standard-headers/asm-x86/kvm_para.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/standard-headers/asm-x86/kvm_para.h +2 -1
index f0235e58a1d3..962dabcfdb68 100644
@@ -37,7 +37,8 @@
 #define KVM_FEATURE_HC_MAP_GPA_RANGE	16
 #define KVM_FEATURE_MIGRATION_CONTROL	17
 
-#define KVM_HINTS_REALTIME      0
+#define KVM_HINTS_REALTIME                      0
+#define KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID  1
 
 /* The last 8 bits are used to indicate how to interpret the flags field
  * in pvclock structure. If no bits are set, all flags are ignored.
Signature · 1 line
2.37.3
#2 lore raw

RFC PATCH v2 2/2 expose host-phys-bits to guest

In reply to Gerd Hoffmann

Move "host-phys-bits" property from cpu->host_phys_bits to cpu->env.features[FEAT_KVM_HINTS] (KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID bit).

This has the effect that the guest can see whenever host-phys-bits is turned on or not and act accordingly.

---
 target/i386/cpu.h      | 3 ---
 hw/i386/microvm.c      | 7 ++++++-
 target/i386/cpu.c      | 3 +--
 target/i386/host-cpu.c | 5 ++++-
 target/i386/kvm/kvm.c  | 1 +
 5 files changed, 12 insertions(+), 7 deletions(-)
target/i386/cpu.h +0 -3
index 82004b65b944..b9c6d3d9cac6 100644
@@ -1898,9 +1898,6 @@ struct ArchCPU {
     /* if true fill the top bits of the MTRR_PHYSMASKn variable range */
     bool fill_mtrr_mask;
 
-    /* if true override the phys_bits value with a value read from the host */
-    bool host_phys_bits;
-
     /* if set, limit maximum value for phys_bits when host_phys_bits is true */
     uint8_t host_phys_bits_limit;
 
hw/i386/microvm.c +6 -1
index 52cafa003d8a..316bbc8ef946 100644
@@ -54,6 +54,8 @@
 #include "kvm/kvm_i386.h"
 #include "hw/xen/start_info.h"
 
+#include "standard-headers/asm-x86/kvm_para.h"
+
 #define MICROVM_QBOOT_FILENAME "qboot.rom"
 #define MICROVM_BIOS_FILENAME  "bios-microvm.bin"
 
@@ -424,7 +426,10 @@ static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev,
 {
     X86CPU *cpu = X86_CPU(dev);
 
-    cpu->host_phys_bits = true; /* need reliable phys-bits */
+    /* need reliable phys-bits */
+    cpu->env.features[FEAT_KVM_HINTS] |=
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID);
+
     x86_cpu_pre_plug(hotplug_dev, dev, errp);
 }
 
target/i386/cpu.c +1 -2
index 1db1278a599b..d60f4498a3c3 100644
@@ -778,7 +778,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     [FEAT_KVM_HINTS] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
-            "kvm-hint-dedicated", NULL, NULL, NULL,
+            "kvm-hint-dedicated", "host-phys-bits", NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
@@ -7016,7 +7016,6 @@ static Property x86_cpu_properties[] = {
     DEFINE_PROP_BOOL("x-force-features", X86CPU, force_features, false),
     DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true),
     DEFINE_PROP_UINT32("phys-bits", X86CPU, phys_bits, 0),
-    DEFINE_PROP_BOOL("host-phys-bits", X86CPU, host_phys_bits, false),
     DEFINE_PROP_UINT8("host-phys-bits-limit", X86CPU, host_phys_bits_limit, 0),
     DEFINE_PROP_BOOL("fill-mtrr-mask", X86CPU, fill_mtrr_mask, true),
     DEFINE_PROP_UINT32("level-func7", X86CPU, env.cpuid_level_func7,
target/i386/host-cpu.c +4 -1
index 10f8aba86e53..a1d6b3ac962e 100644
@@ -13,6 +13,8 @@
 #include "qapi/error.h"
 #include "sysemu/sysemu.h"
 
+#include "standard-headers/asm-x86/kvm_para.h"
+
 /* Note: Only safe for use on x86(-64) hosts */
 static uint32_t host_cpu_phys_bits(void)
 {
@@ -68,7 +70,8 @@ static uint32_t host_cpu_adjust_phys_bits(X86CPU *cpu)
         warned = true;
     }
 
-    if (cpu->host_phys_bits) {
+    if (cpu->env.features[FEAT_KVM_HINTS] &
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID)) {
         /* The user asked for us to use the host physical bits */
         phys_bits = host_phys_bits;
         if (cpu->host_phys_bits_limit &&
target/i386/kvm/kvm.c +1 -0
index a1fd1f53791d..3335c57b21b2 100644
@@ -459,6 +459,7 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         }
     } else if (function == KVM_CPUID_FEATURES && reg == R_EDX) {
         ret |= 1U << KVM_HINTS_REALTIME;
+        ret |= 1U << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID;
     }
 
     return ret;
Signature · 1 line
2.37.3
#3 lore raw
Quoted 43 lines · Gerd Hoffmann

Move "host-phys-bits" property from cpu->host_phys_bits to cpu->env.features[FEAT_KVM_HINTS] (KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID bit).

This has the effect that the guest can see whenever host-phys-bits is turned on or not and act accordingly.

---
 target/i386/cpu.h      | 3 ---
 hw/i386/microvm.c      | 7 ++++++-
 target/i386/cpu.c      | 3 +--
 target/i386/host-cpu.c | 5 ++++-
 target/i386/kvm/kvm.c  | 1 +
 5 files changed, 12 insertions(+), 7 deletions(-)
target/i386/cpu.h +0 -3
index 82004b65b944..b9c6d3d9cac6 100644
@@ -1898,9 +1898,6 @@ struct ArchCPU {
     /* if true fill the top bits of the MTRR_PHYSMASKn variable range */
     bool fill_mtrr_mask;
 
-    /* if true override the phys_bits value with a value read from the host */
-    bool host_phys_bits;
-
     /* if set, limit maximum value for phys_bits when host_phys_bits is true */
     uint8_t host_phys_bits_limit;
 
hw/i386/microvm.c +2 -0
index 52cafa003d8a..316bbc8ef946 100644
@@ -54,6 +54,8 @@
 #include "kvm/kvm_i386.h"
 #include "hw/xen/start_info.h"
 
+#include "standard-headers/asm-x86/kvm_para.h"
+
 #define MICROVM_QBOOT_FILENAME "qboot.rom"
 #define MICROVM_BIOS_FILENAME  "bios-microvm.bin"
 

hw/i386/microvm.c static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 432

Quoted 8 lines
hw/i386/microvm.c +4 -1
@@ -424,7 +426,10 @@ static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev,
 {
     X86CPU *cpu = X86_CPU(dev);
 
-    cpu->host_phys_bits = true; /* need reliable phys-bits */
+    /* need reliable phys-bits */
+    cpu->env.features[FEAT_KVM_HINTS] |=
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID);
+

Do we need compat machinery for this?

Quoted 38 lines
hw/i386/microvm.c +0 -0
     x86_cpu_pre_plug(hotplug_dev, dev, errp);
 }
 
target/i386/cpu.c +1 -2
index 1db1278a599b..d60f4498a3c3 100644
@@ -778,7 +778,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     [FEAT_KVM_HINTS] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
-            "kvm-hint-dedicated", NULL, NULL, NULL,
+            "kvm-hint-dedicated", "host-phys-bits", NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
@@ -7016,7 +7016,6 @@ static Property x86_cpu_properties[] = {
     DEFINE_PROP_BOOL("x-force-features", X86CPU, force_features, false),
     DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true),
     DEFINE_PROP_UINT32("phys-bits", X86CPU, phys_bits, 0),
-    DEFINE_PROP_BOOL("host-phys-bits", X86CPU, host_phys_bits, false),
     DEFINE_PROP_UINT8("host-phys-bits-limit", X86CPU, host_phys_bits_limit, 0),
     DEFINE_PROP_BOOL("fill-mtrr-mask", X86CPU, fill_mtrr_mask, true),
     DEFINE_PROP_UINT32("level-func7", X86CPU, env.cpuid_level_func7,
target/i386/host-cpu.c +2 -0
index 10f8aba86e53..a1d6b3ac962e 100644
@@ -13,6 +13,8 @@
 #include "qapi/error.h"
 #include "sysemu/sysemu.h"
 
+#include "standard-headers/asm-x86/kvm_para.h"
+
 /* Note: Only safe for use on x86(-64) hosts */
 static uint32_t host_cpu_phys_bits(void)
 {

target/i386/host-cpu.c static uint32_t host_cpu_adjust_phys_bits(X86CPU *cpu) · line 77

Quoted 9 lines
target/i386/host-cpu.c +2 -1
@@ -68,7 +70,8 @@ static uint32_t host_cpu_adjust_phys_bits(X86CPU *cpu)
         warned = true;
     }
 
-    if (cpu->host_phys_bits) {
+    if (cpu->env.features[FEAT_KVM_HINTS] &
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID)) {
         /* The user asked for us to use the host physical bits */
         phys_bits = host_phys_bits;
         if (cpu->host_phys_bits_limit &&

I think we still want to key this one off host_phys_bits so it works for e.g. hyperv emulation too.

Quoted 14 lines
target/i386/kvm/kvm.c +1 -0
index a1fd1f53791d..3335c57b21b2 100644
@@ -459,6 +459,7 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         }
     } else if (function == KVM_CPUID_FEATURES && reg == R_EDX) {
         ret |= 1U << KVM_HINTS_REALTIME;
+        ret |= 1U << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID;
     }
 
     return ret;
Signature · 1 line
2.37.3
#4 lore raw

Hi,

Quoted 11 lines

hw/i386/microvm.c static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 432

Quoted 9 lines
hw/i386/microvm.c +4 -1
@@ -424,7 +426,10 @@ static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev,
 {
     X86CPU *cpu = X86_CPU(dev);
 
-    cpu->host_phys_bits = true; /* need reliable phys-bits */
+    /* need reliable phys-bits */
+    cpu->env.features[FEAT_KVM_HINTS] |=
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID);
+

Do we need compat machinery for this?

Don't think so, microvm has no versioned machine types anyway.

Quoted 18 lines
Quoted 7 lines
target/i386/cpu.c +0 -1
 
     [FEAT_KVM_HINTS] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
-            "kvm-hint-dedicated", NULL, NULL, NULL,

target/i386/cpu.c

Quoted 10 lines
target/i386/cpu.c +3 -2
+            "kvm-hint-dedicated", "host-phys-bits", NULL, NULL,
 
-    DEFINE_PROP_BOOL("host-phys-bits", X86CPU, host_phys_bits, false),
 
-    if (cpu->host_phys_bits) {
+    if (cpu->env.features[FEAT_KVM_HINTS] &
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID)) {
         /* The user asked for us to use the host physical bits */
         phys_bits = host_phys_bits;
         if (cpu->host_phys_bits_limit &&
Quoted 2 lines

I think we still want to key this one off host_phys_bits so it works for e.g. hyperv emulation too.

I think that should be the case. The chunks above change the host-phys-bits option from setting cpu->host_phys_bits to setting the FEAT_KVM_HINTS bit. That should also happen with hyperv emulation enabled, and the bit should also be visible to the guest then, just at another location (base 0x40000100 instead of 0x40000000).

take care,
Gerd

#5 lore raw
Quoted 37 lines · Gerd Hoffmann

Hi,

Quoted 11 lines

hw/i386/microvm.c static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 432

Quoted 9 lines
hw/i386/microvm.c +4 -1
@@ -424,7 +426,10 @@ static void microvm_device_pre_plug_cb(HotplugHandler *hotplug_dev,
 {
     X86CPU *cpu = X86_CPU(dev);
 
-    cpu->host_phys_bits = true; /* need reliable phys-bits */
+    /* need reliable phys-bits */
+    cpu->env.features[FEAT_KVM_HINTS] |=
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID);
+

Do we need compat machinery for this?

Don't think so, microvm has no versioned machine types anyway.

Quoted 18 lines
Quoted 7 lines
target/i386/cpu.c +0 -1
 
     [FEAT_KVM_HINTS] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
-            "kvm-hint-dedicated", NULL, NULL, NULL,

target/i386/cpu.c

Quoted 10 lines
target/i386/cpu.c +3 -2
+            "kvm-hint-dedicated", "host-phys-bits", NULL, NULL,
 
-    DEFINE_PROP_BOOL("host-phys-bits", X86CPU, host_phys_bits, false),
 
-    if (cpu->host_phys_bits) {
+    if (cpu->env.features[FEAT_KVM_HINTS] &
+        (1 << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID)) {
         /* The user asked for us to use the host physical bits */
         phys_bits = host_phys_bits;
         if (cpu->host_phys_bits_limit &&
Quoted 2 lines

I think we still want to key this one off host_phys_bits so it works for e.g. hyperv emulation too.

Quoted 8 lines

I think that should be the case. The chunks above change the host-phys-bits option from setting cpu->host_phys_bits to setting the FEAT_KVM_HINTS bit. That should also happen with hyperv emulation enabled, and the bit should also be visible to the guest then, just at another location (base 0x40000100 instead of 0x40000000).

take care,
Gerd

You are right, I forgot. Hmm, ok. What about !cpu->expose_kvm ?

We have

    if (!kvm_enabled() || !cpu->expose_kvm) {
        env->features[FEAT_KVM] = 0;
    }   

This is quick grep, I didn't check whether this is called after the point where you currently use it, but
it frankly seems fragile to pass a generic user specified flag inside a cpuid where everyone pokes at it.

Signature · 1 line
MST
#6 lore raw

Hi,

Quoted 17 lines
Quoted 11 lines
Quoted 2 lines

I think we still want to key this one off host_phys_bits so it works for e.g. hyperv emulation too.

I think that should be the case. The chunks above change the host-phys-bits option from setting cpu->host_phys_bits to setting the FEAT_KVM_HINTS bit. That should also happen with hyperv emulation enabled, and the bit should also be visible to the guest then, just at another location (base 0x40000100 instead of 0x40000000).

take care,
Gerd

You are right, I forgot. Hmm, ok. What about !cpu->expose_kvm ?

We have

Quoted 8 lines
    if (!kvm_enabled() || !cpu->expose_kvm) {
        env->features[FEAT_KVM] = 0;
    }   

This is quick grep, I didn't check whether this is called after the point where you currently use it, but
it frankly seems fragile to pass a generic user specified flag inside a cpuid where everyone pokes at it.

I tried to avoid keeping the state of the host_phys_bits option at multiple places. Maybe that wasn't a good idea after all. How about doing this instead:

target/i386/cpu.c +5 -0
index 1db1278a599b..279fde095d7c 100644
@@ -6219,6 +6219,11 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
         env->features[FEAT_KVM] = 0;
     }
 
+    if (kvm_enabled() && cpu->host_phys_bits) {
+        env->features[FEAT_KVM_HINTS] |=
+            (1U << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID);
+    }
+
     x86_cpu_enable_xsave_components(cpu);
 
     /* CPUID[EAX=7,ECX=0].EBX always increased level automatically: */
target/i386/kvm/kvm.c +1 -0
index a1fd1f53791d..3335c57b21b2 100644
@@ -459,6 +459,7 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         }
     } else if (function == KVM_CPUID_FEATURES && reg == R_EDX) {
         ret |= 1U << KVM_HINTS_REALTIME;
+        ret |= 1U << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID;
     }
 
     return ret;
#7 lore raw
Quoted 53 lines · Gerd Hoffmann

Hi,

Quoted 17 lines
Quoted 11 lines
Quoted 2 lines

I think we still want to key this one off host_phys_bits so it works for e.g. hyperv emulation too.

I think that should be the case. The chunks above change the host-phys-bits option from setting cpu->host_phys_bits to setting the FEAT_KVM_HINTS bit. That should also happen with hyperv emulation enabled, and the bit should also be visible to the guest then, just at another location (base 0x40000100 instead of 0x40000000).

take care,
Gerd

You are right, I forgot. Hmm, ok. What about !cpu->expose_kvm ?

We have

Quoted 8 lines
    if (!kvm_enabled() || !cpu->expose_kvm) {
        env->features[FEAT_KVM] = 0;
    }   

This is quick grep, I didn't check whether this is called after the point where you currently use it, but
it frankly seems fragile to pass a generic user specified flag inside a cpuid where everyone pokes at it.

I tried to avoid keeping the state of the host_phys_bits option at multiple places. Maybe that wasn't a good idea after all. How about doing this instead:

target/i386/cpu.c +5 -0
index 1db1278a599b..279fde095d7c 100644
@@ -6219,6 +6219,11 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
         env->features[FEAT_KVM] = 0;
     }
 
+    if (kvm_enabled() && cpu->host_phys_bits) {
+        env->features[FEAT_KVM_HINTS] |=
+            (1U << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID);
+    }
+
     x86_cpu_enable_xsave_components(cpu);
 
     /* CPUID[EAX=7,ECX=0].EBX always increased level automatically: */
target/i386/kvm/kvm.c +0 -0
index a1fd1f53791d..3335c57b21b2 100644

target/i386/kvm/kvm.c uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function, · line 465

Quoted 7 lines
target/i386/kvm/kvm.c +1 -0
@@ -459,6 +459,7 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         }
     } else if (function == KVM_CPUID_FEATURES && reg == R_EDX) {
         ret |= 1U << KVM_HINTS_REALTIME;
+        ret |= 1U << KVM_HINTS_PHYS_ADDRESS_SIZE_DATA_VALID;
     }
 
     return ret;

/me nods.
That seems much more straight-forward.

Signature · 1 line
MST