Skip to contentSkip to conversation

Enable x2apic by default on KVM

PATCH Applied Eduardo Habkost 5 participants 12 messages Feb 19, 2014 – Mar 12, 2014

lore
Share & tools
lore
#0 lore raw

cover Enable x2apic by default on KVM

This is the approach I believe was agreed upon during the last QEMU developers conf call (sorry I didn't join, the meeting timezone change confused me).

Some may notice that representing this new behavior using solely default values on CPU properties may be a little difficult. Making it possible to replace x86_cpu_compat_disable_kvm_features() with compat_props entries may be an interesting problem, too.

Eduardo Habkost (3):
target-i386: Make kvm_default_features an array
target-i386: Introduce x86_cpu_compat_disable_kvm_features()
target-i386: Enable x2apic by default on KVM

 hw/i386/pc_piix.c |  7 ++++---
 hw/i386/pc_q35.c  |  2 ++
 target-i386/cpu.c | 20 +++++++++++++++-----
 target-i386/cpu.h |  4 ++--
 4 files changed, 23 insertions(+), 10 deletions(-)
Signature · 1 line
1.8.5.3
#1 lore raw

PATCH 1/3 target-i386: Make kvm_default_features an array

We will later make the KVM-specific code affect other feature words, too.

---
 target-i386/cpu.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
target-i386/cpu.c +13 -4
index 5a530b5..ee9dff1 100644
@@ -358,17 +358,22 @@ typedef struct model_features_t {
     FeatureWord feat_word;
 } model_features_t;
 
-static uint32_t kvm_default_features = (1 << KVM_FEATURE_CLOCKSOURCE) |
+/* KVM-specific features that are automatically added to all CPU models
+ * when KVM is enabled.
+ */
+static uint32_t kvm_default_features[FEATURE_WORDS] = {
+    [FEAT_KVM] = (1 << KVM_FEATURE_CLOCKSOURCE) |
         (1 << KVM_FEATURE_NOP_IO_DELAY) |
         (1 << KVM_FEATURE_CLOCKSOURCE2) |
         (1 << KVM_FEATURE_ASYNC_PF) |
         (1 << KVM_FEATURE_STEAL_TIME) |
         (1 << KVM_FEATURE_PV_EOI) |
-        (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT);
+        (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT),
+};
 
 void disable_kvm_pv_eoi(void)
 {
-    kvm_default_features &= ~(1UL << KVM_FEATURE_PV_EOI);
+    kvm_default_features[FEAT_KVM] &= ~(1UL << KVM_FEATURE_PV_EOI);
 }
 
 void host_cpuid(uint32_t function, uint32_t count,
@@ -1851,8 +1856,12 @@ static void x86_cpu_load_def(X86CPU *cpu, const char *name, Error **errp)
 
     /* Special cases not set in the X86CPUDefinition structs: */
     if (kvm_enabled()) {
-        env->features[FEAT_KVM] |= kvm_default_features;
+        FeatureWord w;
+        for (w = 0; w < FEATURE_WORDS; w++) {
+            env->features[w] |= kvm_default_features[w];
+        }
     }
+
     env->features[FEAT_1_ECX] |= CPUID_EXT_HYPERVISOR;
 
     /* sysenter isn't supported in compatibility mode on AMD,
Signature · 1 line
1.8.5.3
#2 lore raw

Re: [Qemu-devel] [PATCH 1/3] target-i386: Make kvm_default_features an array

Quoted 2 lines · On Wed, 19 Feb 2014 11:58:10 -0300 Eduardo Habkost <[email protected]> wrote:

We will later make the KVM-specific code affect other feature words, too.

patch doesn't apply to current master, is there dependencies on list?

Quoted 50 lines
---
 target-i386/cpu.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
target-i386/cpu.c +13 -4
index 5a530b5..ee9dff1 100644
@@ -358,17 +358,22 @@ typedef struct model_features_t {
     FeatureWord feat_word;
 } model_features_t;
 
-static uint32_t kvm_default_features = (1 << KVM_FEATURE_CLOCKSOURCE) |
+/* KVM-specific features that are automatically added to all CPU models
+ * when KVM is enabled.
+ */
+static uint32_t kvm_default_features[FEATURE_WORDS] = {
+    [FEAT_KVM] = (1 << KVM_FEATURE_CLOCKSOURCE) |
         (1 << KVM_FEATURE_NOP_IO_DELAY) |
         (1 << KVM_FEATURE_CLOCKSOURCE2) |
         (1 << KVM_FEATURE_ASYNC_PF) |
         (1 << KVM_FEATURE_STEAL_TIME) |
         (1 << KVM_FEATURE_PV_EOI) |
-        (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT);
+        (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT),
+};
 
 void disable_kvm_pv_eoi(void)
 {
-    kvm_default_features &= ~(1UL << KVM_FEATURE_PV_EOI);
+    kvm_default_features[FEAT_KVM] &= ~(1UL << KVM_FEATURE_PV_EOI);
 }
 
 void host_cpuid(uint32_t function, uint32_t count,
@@ -1851,8 +1856,12 @@ static void x86_cpu_load_def(X86CPU *cpu, const char *name, Error **errp)
 
     /* Special cases not set in the X86CPUDefinition structs: */
     if (kvm_enabled()) {
-        env->features[FEAT_KVM] |= kvm_default_features;
+        FeatureWord w;
+        for (w = 0; w < FEATURE_WORDS; w++) {
+            env->features[w] |= kvm_default_features[w];
+        }
     }
+
     env->features[FEAT_1_ECX] |= CPUID_EXT_HYPERVISOR;
 
     /* sysenter isn't supported in compatibility mode on AMD,
#3 lore raw

Re: [Qemu-devel] [PATCH 1/3] target-i386: Make kvm_default_features an array

Quoted 6 lines · Igor Mammedov
Quoted 2 lines · On Wed, 19 Feb 2014 11:58:10 -0300 Eduardo Habkost <[email protected]> wrote:

We will later make the KVM-specific code affect other feature words, too.

patch doesn't apply to current master, is there dependencies on list?

It is against the qom-cpu tree. I forgot to add it to Subject, sorry.

Git tree at:
https://github.com/ehabkost/qemu/tree/work/x2apic-kvm-only

Signature · 1 line
Eduardo
#4 lore raw

PATCH 2/3 target-i386: Introduce x86_cpu_compat_disable_kvm_features()

In reply to Eduardo Habkost

Instead of the feature-specific disable_kvm_pv_eoi() function, create a more general function that can be used to disable other feature bits in machine-type compat code.

---
 hw/i386/pc_piix.c | 6 +++---
 target-i386/cpu.c | 4 ++--
 target-i386/cpu.h | 4 ++--
 3 files changed, 7 insertions(+), 7 deletions(-)
hw/i386/pc_piix.c +3 -3
index 1acd2b2..dab0c78 100644
@@ -292,7 +292,7 @@ static void pc_compat_1_3(QEMUMachineInitArgs *args)
 static void pc_compat_1_2(QEMUMachineInitArgs *args)
 {
     pc_compat_1_3(args);
-    disable_kvm_pv_eoi();
+    x86_cpu_compat_disable_kvm_features(FEAT_KVM, KVM_FEATURE_PV_EOI);
 }
 
 static void pc_init_pci_1_7(QEMUMachineInitArgs *args)
@@ -338,7 +338,7 @@ static void pc_init_pci_no_kvmclock(QEMUMachineInitArgs *args)
     has_pci_info = false;
     has_acpi_build = false;
     smbios_type1_defaults = false;
-    disable_kvm_pv_eoi();
+    x86_cpu_compat_disable_kvm_features(FEAT_KVM, KVM_FEATURE_PV_EOI);
     enable_compat_apic_id_mode();
     pc_init1(args, 1, 0);
 }
@@ -351,7 +351,7 @@ static void pc_init_isa(QEMUMachineInitArgs *args)
     if (!args->cpu_model) {
         args->cpu_model = "486";
     }
-    disable_kvm_pv_eoi();
+    x86_cpu_compat_disable_kvm_features(FEAT_KVM, KVM_FEATURE_PV_EOI);
     enable_compat_apic_id_mode();
     pc_init1(args, 0, 1);
 }
target-i386/cpu.c +2 -2
index ee9dff1..d65ca26 100644
@@ -371,9 +371,9 @@ static uint32_t kvm_default_features[FEATURE_WORDS] = {
         (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT),
 };
 
-void disable_kvm_pv_eoi(void)
+void x86_cpu_compat_disable_kvm_features(FeatureWord w, uint32_t features)
 {
-    kvm_default_features[FEAT_KVM] &= ~(1UL << KVM_FEATURE_PV_EOI);
+    kvm_default_features[w] &= ~features;
 }
 
 void host_cpuid(uint32_t function, uint32_t count,
target-i386/cpu.h +2 -2
index 9989216..d6eb97d 100644
@@ -1257,11 +1257,11 @@ void do_smm_enter(X86CPU *cpu);
 
 void cpu_report_tpr_access(CPUX86State *env, TPRAccess access);
 
-void disable_kvm_pv_eoi(void);
-
 void x86_cpu_compat_set_features(const char *cpu_model, FeatureWord w,
                                  uint32_t feat_add, uint32_t feat_remove);
 
+void x86_cpu_compat_disable_kvm_features(FeatureWord w, uint32_t features);
+
 
 /* Return name of 32-bit register, from a R_* constant */
 const char *get_register_name_32(unsigned int reg);
Signature · 1 line
1.8.5.3
#5 lore raw

PATCH 3/3 target-i386: Enable x2apic by default on KVM

In reply to Eduardo Habkost

When on KVM mode, enable x2apic by default on all CPU models.

Normally we try to keep the CPU model definitions as close as the real CPUs as possible, but x2apic can be emulated by KVM without host CPU support for x2apic, and it improves performance by reducing APIC access overhead. x2apic emulation is available on KVM since 2009 (Linux 2.6.32-rc1), there's no reason for not enabling x2apic by default when running KVM.

---
 hw/i386/pc_piix.c | 1 +
 hw/i386/pc_q35.c  | 2 ++
 target-i386/cpu.c | 1 +
 3 files changed, 4 insertions(+)
hw/i386/pc_piix.c +1 -0
index dab0c78..cf12873 100644
@@ -260,6 +260,7 @@ static void pc_compat_1_7(QEMUMachineInitArgs *args)
 {
     smbios_type1_defaults = false;
     gigabyte_align = false;
+    x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
 }
 
 static void pc_compat_1_6(QEMUMachineInitArgs *args)
hw/i386/pc_q35.c +2 -0
index a7f6260..71b05f1 100644
@@ -244,6 +244,8 @@ static void pc_compat_1_7(QEMUMachineInitArgs *args)
 {
     smbios_type1_defaults = false;
     gigabyte_align = false;
+    x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
+
 }
 
 static void pc_compat_1_6(QEMUMachineInitArgs *args)
target-i386/cpu.c +1 -0
index d65ca26..089328e 100644
@@ -369,6 +369,7 @@ static uint32_t kvm_default_features[FEATURE_WORDS] = {
         (1 << KVM_FEATURE_STEAL_TIME) |
         (1 << KVM_FEATURE_PV_EOI) |
         (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT),
+    [FEAT_1_ECX] = CPUID_EXT_X2APIC,
 };
 
 void x86_cpu_compat_disable_kvm_features(FeatureWord w, uint32_t features)
Signature · 1 line
1.8.5.3
#6 lore raw

Re: [Qemu-devel] [PATCH 3/3] target-i386: Enable x2apic by default on KVM

Am 19.02.2014 15:58, schrieb Eduardo Habkost:

Quoted 33 lines

When on KVM mode, enable x2apic by default on all CPU models.

Normally we try to keep the CPU model definitions as close as the real CPUs as possible, but x2apic can be emulated by KVM without host CPU support for x2apic, and it improves performance by reducing APIC access overhead. x2apic emulation is available on KVM since 2009 (Linux 2.6.32-rc1), there's no reason for not enabling x2apic by default when running KVM.

---
 hw/i386/pc_piix.c | 1 +
 hw/i386/pc_q35.c  | 2 ++
 target-i386/cpu.c | 1 +
 3 files changed, 4 insertions(+)
hw/i386/pc_piix.c +1 -0
index dab0c78..cf12873 100644
@@ -260,6 +260,7 @@ static void pc_compat_1_7(QEMUMachineInitArgs *args)
 {
     smbios_type1_defaults = false;
     gigabyte_align = false;
+    x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
 }
 
 static void pc_compat_1_6(QEMUMachineInitArgs *args)
hw/i386/pc_q35.c +0 -0
index a7f6260..71b05f1 100644

hw/i386/pc_q35.c static void pc_compat_1_7(QEMUMachineInitArgs *args) · line 251

Quoted 8 lines
hw/i386/pc_q35.c +2 -0
@@ -244,6 +244,8 @@ static void pc_compat_1_7(QEMUMachineInitArgs *args)
 {
     smbios_type1_defaults = false;
     gigabyte_align = false;
+    x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
+
 }
 
 static void pc_compat_1_6(QEMUMachineInitArgs *args)

[snip]

Needed rebase:

hw/i386/pc_piix.c +0 -0
diff --cc hw/i386/pc_piix.c
index 5011c3a,ebd442e..0000000
@@@ -266,7 -266,7 +266,8 @@@ static void pc_compat_1_7(QEMUMachineIn
  {
      smbios_type1_defaults = false;
      gigabyte_align = false;
 +    option_rom_has_mr = true;
+     x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
  }

static void pc_compat_1_6(QEMUMachineInitArgs *args)

hw/i386/pc_q35.c +0 -0
diff --cc hw/i386/pc_q35.c
index 4b0456a,71b05f1..0000000
@@@ -244,7 -244,8 +244,8 @@@ static void pc_compat_1_7(QEMUMachineIn
  {
      smbios_type1_defaults = false;
      gigabyte_align = false;
 +    option_rom_has_mr = true;
+     x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
 -
  }

static void pc_compat_1_6(QEMUMachineInitArgs *args)

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
author Reviewed-by
#7 lore raw

Re: [Qemu-devel] [PATCH 3/3] target-i386: Enable x2apic by default on KVM

On Wed, Mar 12, 2014 at 03:36:02PM +0100, Andreas Färber wrote:
[...]

Quoted 22 lines

[snip]

Needed rebase:

hw/i386/pc_piix.c +1 -0
diff --cc hw/i386/pc_piix.c
index 5011c3a,ebd442e..0000000
@@@ -266,7 -266,7 +266,8 @@@ static void pc_compat_1_7(QEMUMachineIn
  {
      smbios_type1_defaults = false;
      gigabyte_align = false;
 +    option_rom_has_mr = true;
+     x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
  }
 
  static void pc_compat_1_6(QEMUMachineInitArgs *args)
hw/i386/pc_q35.c +0 -0
diff --cc hw/i386/pc_q35.c
index 4b0456a,71b05f1..0000000

hw/i386/pc_q35.c

Quoted 9 lines
hw/i386/pc_q35.c +1 -0
@@@ -244,7 -244,8 +244,8 @@@ static void pc_compat_1_7(QEMUMachineIn
  {
      smbios_type1_defaults = false;
      gigabyte_align = false;
 +    option_rom_has_mr = true;
+     x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
 -
  }
 
  static void pc_compat_1_6(QEMUMachineInitArgs *args)

(I am guessing there's no need to resubmit. That's right?)

Signature · 1 line
Eduardo
#8 lore raw

Re: [Qemu-devel] [PATCH 3/3] target-i386: Enable x2apic by default on KVM

Am 12.03.2014 16:42, schrieb Eduardo Habkost:

Quoted 32 lines · Andreas Färber

[...]

Quoted 30 lines

[snip]

Needed rebase:

hw/i386/pc_piix.c +1 -0
diff --cc hw/i386/pc_piix.c
index 5011c3a,ebd442e..0000000
@@@ -266,7 -266,7 +266,8 @@@ static void pc_compat_1_7(QEMUMachineIn
  {
      smbios_type1_defaults = false;
      gigabyte_align = false;
 +    option_rom_has_mr = true;
+     x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
  }
 
  static void pc_compat_1_6(QEMUMachineInitArgs *args)
hw/i386/pc_q35.c +1 -0
diff --cc hw/i386/pc_q35.c
index 4b0456a,71b05f1..0000000
@@@ -244,7 -244,8 +244,8 @@@ static void pc_compat_1_7(QEMUMachineIn
  {
      smbios_type1_defaults = false;
      gigabyte_align = false;
 +    option_rom_has_mr = true;
+     x86_cpu_compat_disable_kvm_features(FEAT_1_ECX, CPUID_EXT_X2APIC);
 -
  }
Quoted 5 lines
Quoted 1 line

static void pc_compat_1_6(QEMUMachineInitArgs *args)

(I am guessing there's no need to resubmit. That's right?)

Right, just FYI that there's changes happening on my branch and that things are still being prepared and retested for submission.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#9 lore raw
In reply to Eduardo Habkost

Il 19/02/2014 15:58, Eduardo Habkost ha scritto:

Quoted 2 lines

This is the approach I believe was agreed upon during the last QEMU developers conf call (sorry I didn't join, the meeting timezone change confused me).

Yes, that's correct.

Quoted 4 lines

Some may notice that representing this new behavior using solely default values on CPU properties may be a little difficult. Making it possible to replace x86_cpu_compat_disable_kvm_features() with compat_props entries may be an interesting problem, too.

Very clean patches!

Paolo

Acked-by on the v1 series
#10 lore raw
In reply to Eduardo Habkost
Quoted 2 lines · Eduardo Habkost

This is the approach I believe was agreed upon during the last QEMU developers conf call (sorry I didn't join, the meeting timezone change confused me).

Yes.

Quoted 4 lines

Some may notice that representing this new behavior using solely default values on CPU properties may be a little difficult. Making it possible to replace x86_cpu_compat_disable_kvm_features() with compat_props entries may be an interesting problem, too.

Quoted 14 lines

Eduardo Habkost (3):
target-i386: Make kvm_default_features an array
target-i386: Introduce x86_cpu_compat_disable_kvm_features()
target-i386: Enable x2apic by default on KVM

 hw/i386/pc_piix.c |  7 ++++---
 hw/i386/pc_q35.c  |  2 ++
 target-i386/cpu.c | 20 +++++++++++++++-----
 target-i386/cpu.h |  4 ++--
 4 files changed, 23 insertions(+), 10 deletions(-)
Signature · 1 line
1.8.5.3
#11 lore raw

Am 19.02.2014 20:13, schrieb Michael S. Tsirkin:

Quoted 5 lines · Eduardo Habkost
Quoted 2 lines

This is the approach I believe was agreed upon during the last QEMU developers conf call (sorry I didn't join, the meeting timezone change confused me).

Yes.

Matches my understanding as well.

Quoted 6 lines
Quoted 4 lines

Some may notice that representing this new behavior using solely default values on CPU properties may be a little difficult. Making it possible to replace x86_cpu_compat_disable_kvm_features() with compat_props entries may be an interesting problem, too.

Paolo clarified that he is okay with me going ahead with the series, so let's get the behavior changed first and then we can later worry about refactoring the compat infrastructure.

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

One question for 3/3 though: Is it "running KVM", "running on KVM", "running in KVM" or "running with KVM"? I was unsure, so left it as is.

Regards,
Andreas

Quoted 4 lines
Quoted 4 lines

Eduardo Habkost (3):
target-i386: Make kvm_default_features an array
target-i386: Introduce x86_cpu_compat_disable_kvm_features()
target-i386: Enable x2apic by default on KVM

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg