Skip to contentSkip to conversation

KVM: qemu patches for few KVM features I developed

PATCH Superseded Maxim Levitsky 2 participants 9 messages Sep 14, 2021 – Oct 14, 2021 KVM

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover KVM: qemu patches for few KVM features I developed

These patches implement the qemu side logic to support the KVM features I developed recently.

First two patches are for features that are already accepted upstream, and I already posted them on the qemu mailing list once.

And the 3rd patch is for nested TSC scaling on SVM
which isn't yet accepted in KVM but can already be added to qemu since it is conditional on KVM supporting it, and ABI wise it only relies on SVM spec.

Best regards,
Maxim Levitsky

Maxim Levitsky (3):
KVM: use KVM_{GET|SET}_SREGS2 when supported.
gdbstub: implement NOIRQ support for single step on KVM
KVM: SVM: add migration support for nested TSC scaling

 accel/kvm/kvm-all.c   |  30 +++++++++++
 gdbstub.c             |  60 +++++++++++++++++----
 include/sysemu/kvm.h  |  17 ++++++
 target/i386/cpu.c     |   5 ++
 target/i386/cpu.h     |   7 +++
 target/i386/kvm/kvm.c | 122 +++++++++++++++++++++++++++++++++++++++++-
 target/i386/machine.c |  53 ++++++++++++++++++
 7 files changed, 282 insertions(+), 12 deletions(-)
Signature · 1 line
2.26.3
#1 lore raw

PATCH 1/3 KVM: use KVM_{GET|SET}_SREGS2 when supported.

This allows to make PDPTRs part of the migration stream and thus not reload them after migration which is against X86 spec.

---
 accel/kvm/kvm-all.c   |   5 ++
 include/sysemu/kvm.h  |   4 ++
 target/i386/cpu.h     |   3 ++
 target/i386/kvm/kvm.c | 107 +++++++++++++++++++++++++++++++++++++++++-
 target/i386/machine.c |  30 ++++++++++++
 5 files changed, 147 insertions(+), 2 deletions(-)
accel/kvm/kvm-all.c +5 -0
index 0125c17edb..6b187e9c96 100644
@@ -163,6 +163,7 @@ bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
 bool kvm_gsi_direct_mapping;
 bool kvm_allowed;
+bool kvm_sregs2;
 bool kvm_readonly_mem_allowed;
 bool kvm_vm_attributes_allowed;
 bool kvm_direct_msi_allowed;
@@ -2554,6 +2555,10 @@ static int kvm_init(MachineState *ms)
     kvm_ioeventfd_any_length_allowed =
         (kvm_check_extension(s, KVM_CAP_IOEVENTFD_ANY_LENGTH) > 0);
 
+
+    kvm_sregs2 =
+        (kvm_check_extension(s, KVM_CAP_SREGS2) > 0);
+
     kvm_state = s;
 
     ret = kvm_arch_init(ms, s);
include/sysemu/kvm.h +4 -0
index a1ab1ee12d..b3d4538c55 100644
@@ -32,6 +32,7 @@
 #ifdef CONFIG_KVM_IS_POSSIBLE
 
 extern bool kvm_allowed;
+extern bool kvm_sregs2;
 extern bool kvm_kernel_irqchip;
 extern bool kvm_split_irqchip;
 extern bool kvm_async_interrupts_allowed;
@@ -139,6 +140,9 @@ extern bool kvm_msi_use_devid;
  */
 #define kvm_gsi_direct_mapping() (kvm_gsi_direct_mapping)
 
+
+#define kvm_supports_sregs2() (kvm_sregs2)
+
 /**
  * kvm_readonly_mem_enabled:
  *
target/i386/cpu.h +3 -0
index 71ae3141c3..9adae12426 100644
@@ -1436,6 +1436,9 @@ typedef struct CPUX86State {
     SegmentCache idt; /* only base and limit are used */
 
     target_ulong cr[5]; /* NOTE: cr1 is unused */
+
+    bool pdptrs_valid;
+    uint64_t pdptrs[4];
     int32_t a20_mask;
 
     BNDReg bnd_regs[4];
target/i386/kvm/kvm.c +105 -2

131 lines not on this page · raw message

target/i386/machine.c +30 -0
index b0943118d1..154666e7c0 100644
@@ -1415,6 +1415,35 @@ static const VMStateDescription vmstate_msr_tsx_ctrl = {
     }
 };
 
+static bool pdptrs_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+    return env->pdptrs_valid;
+}
+
+static int pdptrs_post_load(void *opaque, int version_id)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+    env->pdptrs_valid = true;
+    return 0;
+}
+
+
+static const VMStateDescription vmstate_pdptrs = {
+    .name = "cpu/pdptrs",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = pdptrs_needed,
+    .post_load = pdptrs_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64_ARRAY(env.pdptrs, X86CPU, 4),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
 const VMStateDescription vmstate_x86_cpu = {
     .name = "cpu",
     .version_id = 12,
@@ -1551,6 +1580,7 @@ const VMStateDescription vmstate_x86_cpu = {
         &vmstate_nested_state,
 #endif
         &vmstate_msr_tsx_ctrl,
+        &vmstate_pdptrs,
         NULL
     }
 };
Signature · 1 line
2.26.3
#2 lore raw

PATCH 2/3 gdbstub: implement NOIRQ support for single step on KVM

In reply to Maxim Levitsky
---
 accel/kvm/kvm-all.c  | 25 ++++++++++++++++++
 gdbstub.c            | 60 ++++++++++++++++++++++++++++++++++++--------
 include/sysemu/kvm.h | 13 ++++++++++
 3 files changed, 88 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +25 -0
index 6b187e9c96..e141260796 100644
@@ -169,6 +169,8 @@ bool kvm_vm_attributes_allowed;
 bool kvm_direct_msi_allowed;
 bool kvm_ioeventfd_any_length_allowed;
 bool kvm_msi_use_devid;
+bool kvm_has_guest_debug;
+int kvm_sstep_flags;
 static bool kvm_immediate_exit;
 static hwaddr kvm_max_slot_size = ~0;
 
@@ -2559,6 +2561,25 @@ static int kvm_init(MachineState *ms)
     kvm_sregs2 =
         (kvm_check_extension(s, KVM_CAP_SREGS2) > 0);
 
+    kvm_has_guest_debug =
+        (kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG) > 0);
+
+    kvm_sstep_flags = 0;
+
+    if (kvm_has_guest_debug) {
+        /* Assume that single stepping is supported */
+        kvm_sstep_flags = SSTEP_ENABLE;
+
+        int guest_debug_flags =
+            kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG2);
+
+        if (guest_debug_flags > 0) {
+            if (guest_debug_flags & KVM_GUESTDBG_BLOCKIRQ) {
+                kvm_sstep_flags |= SSTEP_NOIRQ;
+            }
+        }
+    }
+
     kvm_state = s;
 
     ret = kvm_arch_init(ms, s);
@@ -3188,6 +3209,10 @@ int kvm_update_guest_debug(CPUState *cpu, unsigned long reinject_trap)
 
     if (cpu->singlestep_enabled) {
         data.dbg.control |= KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP;
+
+        if (cpu->singlestep_enabled & SSTEP_NOIRQ) {
+            data.dbg.control |= KVM_GUESTDBG_BLOCKIRQ;
+        }
     }
     kvm_arch_update_guest_debug(cpu, &data.dbg);
 
gdbstub.c +50 -10

109 lines not on this page · raw message

include/sysemu/kvm.h +13 -0
index b3d4538c55..9c0665363d 100644
@@ -47,6 +47,8 @@ extern bool kvm_readonly_mem_allowed;
 extern bool kvm_direct_msi_allowed;
 extern bool kvm_ioeventfd_any_length_allowed;
 extern bool kvm_msi_use_devid;
+extern bool kvm_has_guest_debug;
+extern int kvm_sstep_flags;
 
 #define kvm_enabled()           (kvm_allowed)
 /**
@@ -171,6 +173,17 @@ extern bool kvm_msi_use_devid;
  */
 #define kvm_msi_devid_required() (kvm_msi_use_devid)
 
+/*
+ * Does KVM support guest debugging
+ */
+#define kvm_supports_guest_debug() (kvm_has_guest_debug)
+
+/*
+ * kvm_supported_sstep_flags
+ * Returns: SSTEP_* flags that KVM supports for guest debug
+ */
+#define kvm_get_supported_sstep_flags() (kvm_sstep_flags)
+
 #else
 
 #define kvm_enabled()           (0)
Signature · 1 line
2.26.3
#3 lore raw

Re: [PATCH 2/3] gdbstub: implement NOIRQ support for single step on KVM

Quoted 92 lines · Maxim Levitsky
---
 accel/kvm/kvm-all.c  | 25 ++++++++++++++++++
 gdbstub.c            | 60 ++++++++++++++++++++++++++++++++++++--------
 include/sysemu/kvm.h | 13 ++++++++++
 3 files changed, 88 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +25 -0
index 6b187e9c96..e141260796 100644
@@ -169,6 +169,8 @@ bool kvm_vm_attributes_allowed;
 bool kvm_direct_msi_allowed;
 bool kvm_ioeventfd_any_length_allowed;
 bool kvm_msi_use_devid;
+bool kvm_has_guest_debug;
+int kvm_sstep_flags;
 static bool kvm_immediate_exit;
 static hwaddr kvm_max_slot_size = ~0;
 
@@ -2559,6 +2561,25 @@ static int kvm_init(MachineState *ms)
     kvm_sregs2 =
         (kvm_check_extension(s, KVM_CAP_SREGS2) > 0);
 
+    kvm_has_guest_debug =
+        (kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG) > 0);
+
+    kvm_sstep_flags = 0;
+
+    if (kvm_has_guest_debug) {
+        /* Assume that single stepping is supported */
+        kvm_sstep_flags = SSTEP_ENABLE;
+
+        int guest_debug_flags =
+            kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG2);
+
+        if (guest_debug_flags > 0) {
+            if (guest_debug_flags & KVM_GUESTDBG_BLOCKIRQ) {
+                kvm_sstep_flags |= SSTEP_NOIRQ;
+            }
+        }
+    }
+
     kvm_state = s;
 
     ret = kvm_arch_init(ms, s);
@@ -3188,6 +3209,10 @@ int kvm_update_guest_debug(CPUState *cpu, unsigned long reinject_trap)
 
     if (cpu->singlestep_enabled) {
         data.dbg.control |= KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP;
+
+        if (cpu->singlestep_enabled & SSTEP_NOIRQ) {
+            data.dbg.control |= KVM_GUESTDBG_BLOCKIRQ;
+        }
     }
     kvm_arch_update_guest_debug(cpu, &data.dbg);
 
gdbstub.c +4 -6
index 5d8e6ae3cd..48bb803bae 100644
@@ -368,12 +368,11 @@ typedef struct GDBState {
     gdb_syscall_complete_cb current_syscall_cb;
     GString *str_buf;
     GByteArray *mem_buf;
+    int sstep_flags;
+    int supported_sstep_flags;
 } GDBState;
 
-/* By default use no IRQs and no timers while single stepping so as to
- * make single stepping like an ICE HW step.
- */
-static int sstep_flags = SSTEP_ENABLE|SSTEP_NOIRQ|SSTEP_NOTIMER;
+static GDBState gdbserver_state;
 
 /* Retrieves flags for single step mode. */
 static int get_sstep_flags(void)
@@ -385,11 +384,10 @@ static int get_sstep_flags(void)
     if (replay_mode != REPLAY_MODE_NONE) {
         return SSTEP_ENABLE;
     } else {
-        return sstep_flags;
+        return gdbserver_state.sstep_flags;
     }
 }
 
-static GDBState gdbserver_state;
 
 static void init_gdbserver_state(void)
 {
@@ -399,6 +397,23 @@ static void init_gdbserver_state(void)
     gdbserver_state.str_buf = g_string_new(NULL);

gdbstub.c static void init_gdbserver_state(void) · line 407

Quoted 10 lines
gdbstub.c +8 -0
     gdbserver_state.mem_buf = g_byte_array_sized_new(MAX_PACKET_LENGTH);
     gdbserver_state.last_packet = g_byte_array_sized_new(MAX_PACKET_LENGTH + 4);
+
+
+    if (kvm_enabled()) {
+        gdbserver_state.supported_sstep_flags = kvm_get_supported_sstep_flags();
+    } else {
+        gdbserver_state.supported_sstep_flags =
+            SSTEP_ENABLE | SSTEP_NOIRQ | SSTEP_NOTIMER;
+    }

This fails to build:

o -c ../../gdbstub.c
../../gdbstub.c: In function ‘init_gdbserver_state’:
../../gdbstub.c:403:49: error: implicit declaration of function ‘kvm_get_supported_sstep_flags’ [-Werror=implicit-function-declaration]

  403 |         gdbserver_state.supported_sstep_flags = kvm_get_supported_sstep_flags();
      |                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../gdbstub.c:403:49: error: nested extern declaration of ‘kvm_get_supported_sstep_flags’ [-Werror=nested-externs]
../../gdbstub.c: In function ‘gdbserver_start’:
../../gdbstub.c:3531:27: error: implicit declaration of function ‘kvm_supports_guest_debug’; did you mean ‘kvm_update_guest_debug’? [-Werror=implicit-function-declaration]
 3531 |     if (kvm_enabled() && !kvm_supports_guest_debug()) {
      |                           ^~~~~~~~~~~~~~~~~~~~~~~~
      |                           kvm_update_guest_debug

../../gdbstub.c:3531:27: error: nested extern declaration of ‘kvm_supports_guest_debug’ [-Werror=nested-externs]
cc1: all warnings being treated as errors

In fact looking back I can see I mentioned this last time:

Subject: Re: [PATCH 2/2] gdbstub: implement NOIRQ support for single step on
KVM, when kvm's KVM_GUESTDBG_BLOCKIRQ debug flag is supported.
Date: Mon, 19 Apr 2021 17:29:25 +0100
In-reply-to: <[email protected]>
Message-ID: <[email protected]>

Please in future could you include a revision number for your spin and mention bellow the --- what changes have been made since the last posting.

Signature · 1 line
Alex Bennée
#4 lore raw

Re: [PATCH 2/3] gdbstub: implement NOIRQ support for single step on KVM

Quoted 129 lines · Alex Bennée
Quoted 92 lines · Maxim Levitsky
---
 accel/kvm/kvm-all.c  | 25 ++++++++++++++++++
 gdbstub.c            | 60 ++++++++++++++++++++++++++++++++++++--------
 include/sysemu/kvm.h | 13 ++++++++++
 3 files changed, 88 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +25 -0
index 6b187e9c96..e141260796 100644
@@ -169,6 +169,8 @@ bool kvm_vm_attributes_allowed;
 bool kvm_direct_msi_allowed;
 bool kvm_ioeventfd_any_length_allowed;
 bool kvm_msi_use_devid;
+bool kvm_has_guest_debug;
+int kvm_sstep_flags;
 static bool kvm_immediate_exit;
 static hwaddr kvm_max_slot_size = ~0;
 
@@ -2559,6 +2561,25 @@ static int kvm_init(MachineState *ms)
     kvm_sregs2 =
         (kvm_check_extension(s, KVM_CAP_SREGS2) > 0);
 
+    kvm_has_guest_debug =
+        (kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG) > 0);
+
+    kvm_sstep_flags = 0;
+
+    if (kvm_has_guest_debug) {
+        /* Assume that single stepping is supported */
+        kvm_sstep_flags = SSTEP_ENABLE;
+
+        int guest_debug_flags =
+            kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG2);
+
+        if (guest_debug_flags > 0) {
+            if (guest_debug_flags & KVM_GUESTDBG_BLOCKIRQ) {
+                kvm_sstep_flags |= SSTEP_NOIRQ;
+            }
+        }
+    }
+
     kvm_state = s;
 
     ret = kvm_arch_init(ms, s);
@@ -3188,6 +3209,10 @@ int kvm_update_guest_debug(CPUState *cpu, unsigned long reinject_trap)
 
     if (cpu->singlestep_enabled) {
         data.dbg.control |= KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP;
+
+        if (cpu->singlestep_enabled & SSTEP_NOIRQ) {
+            data.dbg.control |= KVM_GUESTDBG_BLOCKIRQ;
+        }
     }
     kvm_arch_update_guest_debug(cpu, &data.dbg);
 
gdbstub.c +4 -6
index 5d8e6ae3cd..48bb803bae 100644
@@ -368,12 +368,11 @@ typedef struct GDBState {
     gdb_syscall_complete_cb current_syscall_cb;
     GString *str_buf;
     GByteArray *mem_buf;
+    int sstep_flags;
+    int supported_sstep_flags;
 } GDBState;
 
-/* By default use no IRQs and no timers while single stepping so as to
- * make single stepping like an ICE HW step.
- */
-static int sstep_flags = SSTEP_ENABLE|SSTEP_NOIRQ|SSTEP_NOTIMER;
+static GDBState gdbserver_state;
 
 /* Retrieves flags for single step mode. */
 static int get_sstep_flags(void)
@@ -385,11 +384,10 @@ static int get_sstep_flags(void)
     if (replay_mode != REPLAY_MODE_NONE) {
         return SSTEP_ENABLE;
     } else {
-        return sstep_flags;
+        return gdbserver_state.sstep_flags;
     }
 }
 
-static GDBState gdbserver_state;
 
 static void init_gdbserver_state(void)
 {
@@ -399,6 +397,23 @@ static void init_gdbserver_state(void)
     gdbserver_state.str_buf = g_string_new(NULL);

gdbstub.c static void init_gdbserver_state(void) · line 407

Quoted 10 lines
gdbstub.c +8 -0
     gdbserver_state.mem_buf = g_byte_array_sized_new(MAX_PACKET_LENGTH);
     gdbserver_state.last_packet = g_byte_array_sized_new(MAX_PACKET_LENGTH + 4);
+
+
+    if (kvm_enabled()) {
+        gdbserver_state.supported_sstep_flags = kvm_get_supported_sstep_flags();
+    } else {
+        gdbserver_state.supported_sstep_flags =
+            SSTEP_ENABLE | SSTEP_NOIRQ | SSTEP_NOTIMER;
+    }

This fails to build:

o -c ../../gdbstub.c
../../gdbstub.c: In function ‘init_gdbserver_state’:
../../gdbstub.c:403:49: error: implicit declaration of function ‘kvm_get_supported_sstep_flags’ [-Werror=implicit-function-declaration]

  403 |         gdbserver_state.supported_sstep_flags = kvm_get_supported_sstep_flags();
      |                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../gdbstub.c:403:49: error: nested extern declaration of ‘kvm_get_supported_sstep_flags’ [-Werror=nested-externs]
../../gdbstub.c: In function ‘gdbserver_start’:
../../gdbstub.c:3531:27: error: implicit declaration of function ‘kvm_supports_guest_debug’; did you mean ‘kvm_update_guest_debug’? [-Werror=implicit-function-declaration]
 3531 |     if (kvm_enabled() && !kvm_supports_guest_debug()) {
      |                           ^~~~~~~~~~~~~~~~~~~~~~~~
      |                           kvm_update_guest_debug

../../gdbstub.c:3531:27: error: nested extern declaration of ‘kvm_supports_guest_debug’ [-Werror=nested-externs]
cc1: all warnings being treated as errors

In fact looking back I can see I mentioned this last time:

Subject: Re: [PATCH 2/2] gdbstub: implement NOIRQ support for single step on
KVM, when kvm's KVM_GUESTDBG_BLOCKIRQ debug flag is supported.
Date: Mon, 19 Apr 2021 17:29:25 +0100
In-reply-to: <[email protected]>
Message-ID: <[email protected]>

Quoted 3 lines

Please in future could you include a revision number for your spin and mention bellow the --- what changes have been made since the last posting.

You mean it fails to build without KVM? I swear I tested build with TTG only after you mentioned this (or as it seems I only tried to).

Could you give me the ./configure parameters you used?

Sorry for this!
Best regards,
Maxim Levitsky

Quoted 2 lines
#5 lore raw

Re: [PATCH 2/3] gdbstub: implement NOIRQ support for single step on KVM

Quoted 132 lines · Maxim Levitsky
Quoted 131 lines · Alex Bennée
Quoted 92 lines · Maxim Levitsky
---
 accel/kvm/kvm-all.c  | 25 ++++++++++++++++++
 gdbstub.c            | 60 ++++++++++++++++++++++++++++++++++++--------
 include/sysemu/kvm.h | 13 ++++++++++
 3 files changed, 88 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +25 -0
index 6b187e9c96..e141260796 100644
@@ -169,6 +169,8 @@ bool kvm_vm_attributes_allowed;
 bool kvm_direct_msi_allowed;
 bool kvm_ioeventfd_any_length_allowed;
 bool kvm_msi_use_devid;
+bool kvm_has_guest_debug;
+int kvm_sstep_flags;
 static bool kvm_immediate_exit;
 static hwaddr kvm_max_slot_size = ~0;
 
@@ -2559,6 +2561,25 @@ static int kvm_init(MachineState *ms)
     kvm_sregs2 =
         (kvm_check_extension(s, KVM_CAP_SREGS2) > 0);
 
+    kvm_has_guest_debug =
+        (kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG) > 0);
+
+    kvm_sstep_flags = 0;
+
+    if (kvm_has_guest_debug) {
+        /* Assume that single stepping is supported */
+        kvm_sstep_flags = SSTEP_ENABLE;
+
+        int guest_debug_flags =
+            kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG2);
+
+        if (guest_debug_flags > 0) {
+            if (guest_debug_flags & KVM_GUESTDBG_BLOCKIRQ) {
+                kvm_sstep_flags |= SSTEP_NOIRQ;
+            }
+        }
+    }
+
     kvm_state = s;
 
     ret = kvm_arch_init(ms, s);
@@ -3188,6 +3209,10 @@ int kvm_update_guest_debug(CPUState *cpu, unsigned long reinject_trap)
 
     if (cpu->singlestep_enabled) {
         data.dbg.control |= KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP;
+
+        if (cpu->singlestep_enabled & SSTEP_NOIRQ) {
+            data.dbg.control |= KVM_GUESTDBG_BLOCKIRQ;
+        }
     }
     kvm_arch_update_guest_debug(cpu, &data.dbg);
 
gdbstub.c +4 -6
index 5d8e6ae3cd..48bb803bae 100644
@@ -368,12 +368,11 @@ typedef struct GDBState {
     gdb_syscall_complete_cb current_syscall_cb;
     GString *str_buf;
     GByteArray *mem_buf;
+    int sstep_flags;
+    int supported_sstep_flags;
 } GDBState;
 
-/* By default use no IRQs and no timers while single stepping so as to
- * make single stepping like an ICE HW step.
- */
-static int sstep_flags = SSTEP_ENABLE|SSTEP_NOIRQ|SSTEP_NOTIMER;
+static GDBState gdbserver_state;
 
 /* Retrieves flags for single step mode. */
 static int get_sstep_flags(void)
@@ -385,11 +384,10 @@ static int get_sstep_flags(void)
     if (replay_mode != REPLAY_MODE_NONE) {
         return SSTEP_ENABLE;
     } else {
-        return sstep_flags;
+        return gdbserver_state.sstep_flags;
     }
 }
 
-static GDBState gdbserver_state;
 
 static void init_gdbserver_state(void)
 {
@@ -399,6 +397,23 @@ static void init_gdbserver_state(void)
     gdbserver_state.str_buf = g_string_new(NULL);

gdbstub.c static void init_gdbserver_state(void) · line 407

Quoted 10 lines
gdbstub.c +8 -0
     gdbserver_state.mem_buf = g_byte_array_sized_new(MAX_PACKET_LENGTH);
     gdbserver_state.last_packet = g_byte_array_sized_new(MAX_PACKET_LENGTH + 4);
+
+
+    if (kvm_enabled()) {
+        gdbserver_state.supported_sstep_flags = kvm_get_supported_sstep_flags();
+    } else {
+        gdbserver_state.supported_sstep_flags =
+            SSTEP_ENABLE | SSTEP_NOIRQ | SSTEP_NOTIMER;
+    }

This fails to build:

o -c ../../gdbstub.c
../../gdbstub.c: In function ‘init_gdbserver_state’:
../../gdbstub.c:403:49: error: implicit declaration of function ‘kvm_get_supported_sstep_flags’ [-Werror=implicit-function-declaration]

  403 |         gdbserver_state.supported_sstep_flags = kvm_get_supported_sstep_flags();
      |                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

../../gdbstub.c:403:49: error: nested extern declaration of ‘kvm_get_supported_sstep_flags’ [-Werror=nested-externs] ../../gdbstub.c: In function ‘gdbserver_start’:
../../gdbstub.c:3531:27: error: implicit declaration of function
‘kvm_supports_guest_debug’; did you mean ‘kvm_update_guest_debug’? [-Werror=implicit-function-declaration]

 3531 |     if (kvm_enabled() && !kvm_supports_guest_debug()) {
      |                           ^~~~~~~~~~~~~~~~~~~~~~~~
      |                           kvm_update_guest_debug

../../gdbstub.c:3531:27: error: nested extern declaration of ‘kvm_supports_guest_debug’ [-Werror=nested-externs]
cc1: all warnings being treated as errors

In fact looking back I can see I mentioned this last time:

Subject: Re: [PATCH 2/2] gdbstub: implement NOIRQ support for single step on
KVM, when kvm's KVM_GUESTDBG_BLOCKIRQ debug flag is supported.
Date: Mon, 19 Apr 2021 17:29:25 +0100
In-reply-to: <[email protected]>
Message-ID: <[email protected]>

Quoted 8 lines
Quoted 3 lines

Please in future could you include a revision number for your spin and mention bellow the --- what changes have been made since the last posting.

You mean it fails to build without KVM? I swear I tested build with TTG only after you mentioned this (or as it seems I only tried to).

Could you give me the ./configure parameters you used?

That's with the standard parameters on an x86 host (which will disable KVM and be TCG only for ARM). I suspect you could get the same results with --disable-kvm on a native ARM system but I haven't tested that.

Quoted 7 lines

Sorry for this!
Best regards,
Maxim Levitsky

Quoted 2 lines
Signature · 1 line
Alex Bennée
#6 lore raw

PATCH 3/3 KVM: SVM: add migration support for nested TSC scaling

In reply to Maxim Levitsky
---
 target/i386/cpu.c     |  5 +++++
 target/i386/cpu.h     |  4 ++++
 target/i386/kvm/kvm.c | 15 +++++++++++++++
 target/i386/machine.c | 23 +++++++++++++++++++++++
 4 files changed, 47 insertions(+)
target/i386/cpu.c +5 -0
index 6b029f1bdf..0870b53509 100644
@@ -5770,6 +5770,11 @@ static void x86_cpu_reset(DeviceState *dev)
     if (kvm_enabled()) {
         kvm_arch_reset_vcpu(cpu);
     }
+
+    if (env->features[FEAT_SVM] & CPUID_SVM_TSCSCALE) {
+        env->amd_tsc_scale_msr =  MSR_AMD64_TSC_RATIO_DEFAULT;
+    }
+
 #endif
 }
 
target/i386/cpu.h +4 -0
index 9adae12426..b9e1a3b7db 100644
@@ -491,6 +491,9 @@ typedef enum X86Seg {
 #define MSR_GSBASE                      0xc0000101
 #define MSR_KERNELGSBASE                0xc0000102
 #define MSR_TSC_AUX                     0xc0000103
+#define MSR_AMD64_TSC_RATIO             0xc0000104
+
+#define MSR_AMD64_TSC_RATIO_DEFAULT     0x100000000ULL
 
 #define MSR_VM_HSAVE_PA                 0xc0010117
 
@@ -1522,6 +1525,7 @@ typedef struct CPUX86State {
     uint32_t tsx_ctrl;
 
     uint64_t spec_ctrl;
+    uint64_t amd_tsc_scale_msr;
     uint64_t virt_ssbd;
 
     /* End of state preserved by INIT (dummy marker).  */
target/i386/kvm/kvm.c +15 -0
index 841b3b98f7..bd53a55148 100644
@@ -106,6 +106,7 @@ static bool has_msr_hv_reenlightenment;
 static bool has_msr_xss;
 static bool has_msr_umwait;
 static bool has_msr_spec_ctrl;
+static bool has_tsc_scale_msr;
 static bool has_msr_tsx_ctrl;
 static bool has_msr_virt_ssbd;
 static bool has_msr_smi_count;
@@ -2157,6 +2158,9 @@ static int kvm_get_supported_msrs(KVMState *s)
             case MSR_IA32_SPEC_CTRL:
                 has_msr_spec_ctrl = true;
                 break;
+            case MSR_AMD64_TSC_RATIO:
+                has_tsc_scale_msr = true;
+                break;
             case MSR_IA32_TSX_CTRL:
                 has_msr_tsx_ctrl = true;
                 break;
@@ -2968,6 +2972,10 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
     if (has_msr_spec_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, env->spec_ctrl);
     }
+    if (has_tsc_scale_msr) {
+        kvm_msr_entry_add(cpu, MSR_AMD64_TSC_RATIO, env->amd_tsc_scale_msr);
+    }
+
     if (has_msr_tsx_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_TSX_CTRL, env->tsx_ctrl);
     }
@@ -3409,6 +3417,10 @@ static int kvm_get_msrs(X86CPU *cpu)
     if (has_msr_spec_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, 0);
     }
+    if (has_tsc_scale_msr) {
+        kvm_msr_entry_add(cpu, MSR_AMD64_TSC_RATIO, 0);
+    }
+
     if (has_msr_tsx_ctrl) {
         kvm_msr_entry_add(cpu, MSR_IA32_TSX_CTRL, 0);
     }
@@ -3813,6 +3825,9 @@ static int kvm_get_msrs(X86CPU *cpu)
         case MSR_IA32_SPEC_CTRL:
             env->spec_ctrl = msrs[i].data;
             break;
+        case MSR_AMD64_TSC_RATIO:
+            env->amd_tsc_scale_msr = msrs[i].data;
+            break;
         case MSR_IA32_TSX_CTRL:
             env->tsx_ctrl = msrs[i].data;
             break;
target/i386/machine.c +23 -0
index 154666e7c0..39c8faf0ce 100644
@@ -1280,6 +1280,28 @@ static const VMStateDescription vmstate_spec_ctrl = {
     }
 };
 
+
+static bool amd_tsc_scale_msr_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    return env->amd_tsc_scale_msr &&
+           env->amd_tsc_scale_msr != MSR_AMD64_TSC_RATIO_DEFAULT;
+}
+
+static const VMStateDescription amd_tsc_scale_msr_ctrl = {
+    .name = "cpu/amd_tsc_scale_msr",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = amd_tsc_scale_msr_needed,
+    .fields = (VMStateField[]){
+        VMSTATE_UINT64(env.amd_tsc_scale_msr, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
 static bool intel_pt_enable_needed(void *opaque)
 {
     X86CPU *cpu = opaque;
@@ -1568,6 +1590,7 @@ const VMStateDescription vmstate_x86_cpu = {
         &vmstate_pkru,
         &vmstate_pkrs,
         &vmstate_spec_ctrl,
+        &amd_tsc_scale_msr_ctrl,
         &vmstate_mcg_ext_ctl,
         &vmstate_msr_intel_pt,
         &vmstate_msr_virt_ssbd,
Signature · 1 line
2.26.3
#7 lore raw
In reply to Maxim Levitsky
Quoted 19 lines · Maxim Levitsky

These patches implement the qemu side logic to support the KVM features I developed recently.

First two patches are for features that are already accepted upstream, and I already posted them on the qemu mailing list once.

And the 3rd patch is for nested TSC scaling on SVM
which isn't yet accepted in KVM but can already be added to qemu since it is conditional on KVM supporting it, and ABI wise it only relies on SVM spec.

Best regards,
Maxim Levitsky

Maxim Levitsky (3):
KVM: use KVM_{GET|SET}_SREGS2 when supported.
gdbstub: implement NOIRQ support for single step on KVM
KVM: SVM: add migration support for nested TSC scaling

Quoted 13 lines
 accel/kvm/kvm-all.c   |  30 +++++++++++
 gdbstub.c             |  60 +++++++++++++++++----
 include/sysemu/kvm.h  |  17 ++++++
 target/i386/cpu.c     |   5 ++
 target/i386/cpu.h     |   7 +++
 target/i386/kvm/kvm.c | 122 +++++++++++++++++++++++++++++++++++++++++-
 target/i386/machine.c |  53 ++++++++++++++++++
 7 files changed, 282 insertions(+), 12 deletions(-)
Signature · 1 line
2.26.3

Very polite ping on these patches.

Best regards,
Maxim Levitsky

#8 lore raw
In reply to Maxim Levitsky
Quoted 19 lines · Maxim Levitsky

These patches implement the qemu side logic to support the KVM features I developed recently.

First two patches are for features that are already accepted upstream, and I already posted them on the qemu mailing list once.

And the 3rd patch is for nested TSC scaling on SVM
which isn't yet accepted in KVM but can already be added to qemu since it is conditional on KVM supporting it, and ABI wise it only relies on SVM spec.

Best regards,
Maxim Levitsky

Maxim Levitsky (3):
KVM: use KVM_{GET|SET}_SREGS2 when supported.
gdbstub: implement NOIRQ support for single step on KVM
KVM: SVM: add migration support for nested TSC scaling

Quoted 13 lines
 accel/kvm/kvm-all.c   |  30 +++++++++++
 gdbstub.c             |  60 +++++++++++++++++----
 include/sysemu/kvm.h  |  17 ++++++
 target/i386/cpu.c     |   5 ++
 target/i386/cpu.h     |   7 +++
 target/i386/kvm/kvm.c | 122 +++++++++++++++++++++++++++++++++++++++++-
 target/i386/machine.c |  53 ++++++++++++++++++
 7 files changed, 282 insertions(+), 12 deletions(-)
Signature · 1 line
2.26.3

Kind ping on these patches.

Best regards,
Maxim Levitsky