Skip to contentSkip to conversation

Prepare for more qemu-kvm merging

PATCH Superseded Jan Kiszka 4 participants 42 messages Jan 3, 2011 – Jan 4, 2011 KVM

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

v2 cover Prepare for more qemu-kvm merging

All previously sent patches against current uq combined, some smaller tweaks applied, and 4 new patches added. Major news is support for using up to 16M BIOSes and optional code dump for cpu_dump_state. Those features are already present qemu-kvm but were reworked to provide cleaner upstream versions.

Based on this series, I've an experimental tree here where I eliminated another 1500 LOC from qemu-kvm code. Specifically, that tree sets an end to duplicate KVM and VCPU initialization functions, KVMState copies, and redundant state saving/loading functions. Will be rolled out after some more review and testing.

Jan Kiszka (17):
kvm: Fix coding style violations
kvm: Drop return value of kvm_cpu_exec
kvm: Stop on all fatal exit reasons
kvm: Improve reporting of fatal errors
x86: Optionally dump code bytes on cpu_dump_state
kvm: x86: Align kvm_arch_put_registers code with comment
kvm: x86: Prepare kvm_get_mp_state for in-kernel irqchip
kvm: x86: Remove redundant mp_state initialization
kvm: x86: Fix xcr0 reset mismerge
kvm: x86: Refactor msr_star/hsave_pa setup and checks
kvm: x86: Reset paravirtual MSRs
kvm: x86: Drop MCE MSRs write back restrictions
kvm: Eliminate KVMState arguments
kvm: x86: Introduce kvmclock device to save/restore its state
kvm: Drop smp_cpus argument from init functions
kvm: x86: Rework identity map and TSS setup for larger BIOS sizes
kvm: Drop dependencies on very old capabilities

 cpu-all.h             |    2 +
 cpu-defs.h            |    2 -
 kvm-all.c             |  404 +++++++++++++++++++++++--------------------------
 kvm-stub.c            |    8 +-
 kvm.h                 |   19 +--
 target-i386/cpu.h     |    2 +
 target-i386/cpuid.c   |   14 +-
 target-i386/helper.c  |   21 +++
 target-i386/kvm.c     |  331 ++++++++++++++++++++++++----------------
 target-i386/kvm_x86.h |    3 +
 target-ppc/kvm.c      |   16 ++-
 target-s390x/kvm.c    |    8 +-
 vl.c                  |    2 +-
 13 files changed, 448 insertions(+), 384 deletions(-)
author
#1 lore raw

PATCH v2 1/17 kvm: Fix coding style violations

From: Jan Kiszka <[email protected]>

No functional changes.

---
 kvm-all.c |  139 ++++++++++++++++++++++++++++++++++--------------------------
 1 files changed, 79 insertions(+), 60 deletions(-)
kvm-all.c +79 -60
index 35fc73c..2538283 100644
@@ -82,10 +82,12 @@ static KVMSlot *kvm_alloc_slot(KVMState *s)
 
     for (i = 0; i < ARRAY_SIZE(s->slots); i++) {
         /* KVM private memory slots */
-        if (i >= 8 && i < 12)
+        if (i >= 8 && i < 12) {
             continue;
-        if (s->slots[i].memory_size == 0)
+        }
+        if (s->slots[i].memory_size == 0) {
             return &s->slots[i];
+        }
     }
 
     fprintf(stderr, "%s: no free slot available\n", __func__);
@@ -220,9 +222,10 @@ int kvm_init_vcpu(CPUState *env)
     }
 
 #ifdef KVM_CAP_COALESCED_MMIO
-    if (s->coalesced_mmio && !s->coalesced_mmio_ring)
-        s->coalesced_mmio_ring = (void *) env->kvm_run +
-		s->coalesced_mmio * PAGE_SIZE;
+    if (s->coalesced_mmio && !s->coalesced_mmio_ring) {
+        s->coalesced_mmio_ring =
+            (void *)env->kvm_run + s->coalesced_mmio * PAGE_SIZE;
+    }
 #endif
 
     ret = kvm_arch_init_vcpu(env);
@@ -269,16 +272,14 @@ static int kvm_dirty_pages_log_change(target_phys_addr_t phys_addr,
 
 int kvm_log_start(target_phys_addr_t phys_addr, ram_addr_t size)
 {
-        return kvm_dirty_pages_log_change(phys_addr, size,
-                                          KVM_MEM_LOG_DIRTY_PAGES,
-                                          KVM_MEM_LOG_DIRTY_PAGES);
+    return kvm_dirty_pages_log_change(phys_addr, size, KVM_MEM_LOG_DIRTY_PAGES,
+                                      KVM_MEM_LOG_DIRTY_PAGES);
 }
 
 int kvm_log_stop(target_phys_addr_t phys_addr, ram_addr_t size)
 {
-        return kvm_dirty_pages_log_change(phys_addr, size,
-                                          0,
-                                          KVM_MEM_LOG_DIRTY_PAGES);
+    return kvm_dirty_pages_log_change(phys_addr, size, 0,
+                                      KVM_MEM_LOG_DIRTY_PAGES);
 }
 
 static int kvm_set_migration_log(int enable)
@@ -350,7 +351,7 @@ static int kvm_get_dirty_pages_log_range(unsigned long start_addr,
  * @end_addr: end of logged region.
  */
 static int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
-					  target_phys_addr_t end_addr)
+                                          target_phys_addr_t end_addr)
 {
     KVMState *s = kvm_state;
     unsigned long size, allocated_size = 0;
@@ -441,9 +442,8 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
     return ret;
 }
 
-static void kvm_set_phys_mem(target_phys_addr_t start_addr,
-			     ram_addr_t size,
-			     ram_addr_t phys_offset)
+static void kvm_set_phys_mem(target_phys_addr_t start_addr, ram_addr_t size,
+                             ram_addr_t phys_offset)
 {
     KVMState *s = kvm_state;
     ram_addr_t flags = phys_offset & ~TARGET_PAGE_MASK;
@@ -550,13 +550,13 @@ static void kvm_set_phys_mem(target_phys_addr_t start_addr,
     }
 
     /* in case the KVM bug workaround already "consumed" the new slot */
-    if (!size)
+    if (!size) {
         return;
-
+    }
     /* KVM does not need to know about this memory */
-    if (flags >= IO_MEM_UNASSIGNED)
+    if (flags >= IO_MEM_UNASSIGNED) {
         return;
-
+    }
     mem = kvm_alloc_slot(s);
     mem->memory_size = size;
     mem->start_addr = start_addr;
@@ -572,30 +572,29 @@ static void kvm_set_phys_mem(target_phys_addr_t start_addr,
 }
 
 static void kvm_client_set_memory(struct CPUPhysMemoryClient *client,
-				  target_phys_addr_t start_addr,
-				  ram_addr_t size,
-				  ram_addr_t phys_offset)
+                                  target_phys_addr_t start_addr,
+                                  ram_addr_t size, ram_addr_t phys_offset)
 {
-	kvm_set_phys_mem(start_addr, size, phys_offset);
+    kvm_set_phys_mem(start_addr, size, phys_offset);
 }
 
 static int kvm_client_sync_dirty_bitmap(struct CPUPhysMemoryClient *client,
-					target_phys_addr_t start_addr,
-					target_phys_addr_t end_addr)
+                                        target_phys_addr_t start_addr,
+                                        target_phys_addr_t end_addr)
 {
-	return kvm_physical_sync_dirty_bitmap(start_addr, end_addr);
+    return kvm_physical_sync_dirty_bitmap(start_addr, end_addr);
 }
 
 static int kvm_client_migration_log(struct CPUPhysMemoryClient *client,
-				    int enable)
+                                    int enable)
 {
-	return kvm_set_migration_log(enable);
+    return kvm_set_migration_log(enable);
 }
 
 static CPUPhysMemoryClient kvm_cpu_phys_memory_client = {
-	.set_memory = kvm_client_set_memory,
-	.sync_dirty_bitmap = kvm_client_sync_dirty_bitmap,
-	.migration_log = kvm_client_migration_log,
+    .set_memory = kvm_client_set_memory,
+    .sync_dirty_bitmap = kvm_client_sync_dirty_bitmap,
+    .migration_log = kvm_client_migration_log,
 };
 
 int kvm_init(int smp_cpus)
@@ -612,9 +611,9 @@ int kvm_init(int smp_cpus)
 #ifdef KVM_CAP_SET_GUEST_DEBUG
     QTAILQ_INIT(&s->kvm_sw_breakpoints);
 #endif
-    for (i = 0; i < ARRAY_SIZE(s->slots); i++)
+    for (i = 0; i < ARRAY_SIZE(s->slots); i++) {
         s->slots[i].slot = i;
-
+    }
     s->vmfd = -1;
     s->fd = qemu_open("/dev/kvm", O_RDWR);
     if (s->fd == -1) {
@@ -625,8 +624,9 @@ int kvm_init(int smp_cpus)
 
     ret = kvm_ioctl(s, KVM_GET_API_VERSION, 0);
     if (ret < KVM_API_VERSION) {
-        if (ret > 0)
+        if (ret > 0) {
             ret = -EINVAL;
+        }
         fprintf(stderr, "kvm version too old\n");
         goto err;
     }
@@ -711,8 +711,9 @@ int kvm_init(int smp_cpus)
 #endif
 
     ret = kvm_arch_init(s, smp_cpus);
-    if (ret < 0)
+    if (ret < 0) {
         goto err;
+    }
 
     kvm_state = s;
     cpu_register_phys_memory_client(&kvm_cpu_phys_memory_client);
@@ -721,10 +722,12 @@ int kvm_init(int smp_cpus)
 
 err:
     if (s) {
-        if (s->vmfd != -1)
+        if (s->vmfd != -1) {
             close(s->vmfd);
-        if (s->fd != -1)
+        }
+        if (s->fd != -1) {
             close(s->fd);
+        }
     }
     qemu_free(s);
 
@@ -788,8 +791,9 @@ static void kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
     cpu_dump_state(env, stderr, fprintf, 0);
     if (run->internal.suberror == KVM_INTERNAL_ERROR_EMULATION) {
         fprintf(stderr, "emulation failure\n");
-        if (!kvm_arch_stop_on_emulation_error(env))
-		return;
+        if (!kvm_arch_stop_on_emulation_error(env)) {
+            return;
+        }
     }
     /* FIXME: Should trigger a qmp message to let management know
      * something went wrong.
@@ -829,8 +833,9 @@ static void do_kvm_cpu_synchronize_state(void *_env)
 
 void kvm_cpu_synchronize_state(CPUState *env)
 {
-    if (!env->kvm_vcpu_dirty)
+    if (!env->kvm_vcpu_dirty) {
         run_on_cpu(env, do_kvm_cpu_synchronize_state, env);
+    }
 }
 
 void kvm_cpu_synchronize_post_reset(CPUState *env)
@@ -970,9 +975,9 @@ int kvm_ioctl(KVMState *s, int type, ...)
     va_end(ap);
 
     ret = ioctl(s->fd, type, arg);
-    if (ret == -1)
+    if (ret == -1) {
         ret = -errno;
-
+    }
     return ret;
 }
 
@@ -987,9 +992,9 @@ int kvm_vm_ioctl(KVMState *s, int type, ...)
     va_end(ap);
 
     ret = ioctl(s->vmfd, type, arg);
-    if (ret == -1)
+    if (ret == -1) {
         ret = -errno;
-
+    }
     return ret;
 }
 
@@ -1004,9 +1009,9 @@ int kvm_vcpu_ioctl(CPUState *env, int type, ...)
     va_end(ap);
 
     ret = ioctl(env->kvm_fd, type, arg);
-    if (ret == -1)
+    if (ret == -1) {
         ret = -errno;
-
+    }
     return ret;
 }
 
@@ -1067,8 +1072,9 @@ struct kvm_sw_breakpoint *kvm_find_sw_breakpoint(CPUState *env,
     struct kvm_sw_breakpoint *bp;
 
     QTAILQ_FOREACH(bp, &env->kvm_state->kvm_sw_breakpoints, entry) {
-        if (bp->pc == pc)
+        if (bp->pc == pc) {
             return bp;
+        }
     }
     return NULL;
 }
@@ -1123,8 +1129,9 @@ int kvm_insert_breakpoint(CPUState *current_env, target_ulong addr,
         }
 
         bp = qemu_malloc(sizeof(struct kvm_sw_breakpoint));
-        if (!bp)
+        if (!bp) {
             return -ENOMEM;
+        }
 
         bp->pc = addr;
         bp->use_count = 1;
@@ -1138,14 +1145,16 @@ int kvm_insert_breakpoint(CPUState *current_env, target_ulong addr,
                           bp, entry);
     } else {
         err = kvm_arch_insert_hw_breakpoint(addr, len, type);
-        if (err)
+        if (err) {
             return err;
+        }
     }
 
     for (env = first_cpu; env != NULL; env = env->next_cpu) {
         err = kvm_update_guest_debug(env, 0);
-        if (err)
+        if (err) {
             return err;
+        }
     }
     return 0;
 }
@@ -1159,8 +1168,9 @@ int kvm_remove_breakpoint(CPUState *current_env, target_ulong addr,
 
     if (type == GDB_BREAKPOINT_SW) {
         bp = kvm_find_sw_breakpoint(current_env, addr);
-        if (!bp)
+        if (!bp) {
             return -ENOENT;
+        }
 
         if (bp->use_count > 1) {
             bp->use_count--;
@@ -1168,21 +1178,24 @@ int kvm_remove_breakpoint(CPUState *current_env, target_ulong addr,
         }
 
         err = kvm_arch_remove_sw_breakpoint(current_env, bp);
-        if (err)
+        if (err) {
             return err;
+        }
 
         QTAILQ_REMOVE(&current_env->kvm_state->kvm_sw_breakpoints, bp, entry);
         qemu_free(bp);
     } else {
         err = kvm_arch_remove_hw_breakpoint(addr, len, type);
-        if (err)
+        if (err) {
             return err;
+        }
     }
 
     for (env = first_cpu; env != NULL; env = env->next_cpu) {
         err = kvm_update_guest_debug(env, 0);
-        if (err)
+        if (err) {
             return err;
+        }
     }
     return 0;
 }
@@ -1197,15 +1210,17 @@ void kvm_remove_all_breakpoints(CPUState *current_env)
         if (kvm_arch_remove_sw_breakpoint(current_env, bp) != 0) {
             /* Try harder to find a CPU that currently sees the breakpoint. */
             for (env = first_cpu; env != NULL; env = env->next_cpu) {
-                if (kvm_arch_remove_sw_breakpoint(env, bp) == 0)
+                if (kvm_arch_remove_sw_breakpoint(env, bp) == 0) {
                     break;
+                }
             }
         }
     }
     kvm_arch_remove_all_hw_breakpoints();
 
-    for (env = first_cpu; env != NULL; env = env->next_cpu)
+    for (env = first_cpu; env != NULL; env = env->next_cpu) {
         kvm_update_guest_debug(env, 0);
+    }
 }
 
 #else /* !KVM_CAP_SET_GUEST_DEBUG */
@@ -1237,8 +1252,9 @@ int kvm_set_signal_mask(CPUState *env, const sigset_t *sigset)
     struct kvm_signal_mask *sigmask;
     int r;
 
-    if (!sigset)
+    if (!sigset) {
         return kvm_vcpu_ioctl(env, KVM_SET_SIGNAL_MASK, NULL);
+    }
 
     sigmask = qemu_malloc(sizeof(*sigmask) + sizeof(*sigset));
 
@@ -1293,13 +1309,16 @@ int kvm_set_ioeventfd_pio_word(int fd, uint16_t addr, uint16_t val, bool assign)
         .fd = fd,
     };
     int r;
-    if (!kvm_enabled())
+    if (!kvm_enabled()) {
         return -ENOSYS;
-    if (!assign)
+    }
+    if (!assign) {
         kick.flags |= KVM_IOEVENTFD_FLAG_DEASSIGN;
+    }
     r = kvm_vm_ioctl(kvm_state, KVM_IOEVENTFD, &kick);
-    if (r < 0)
+    if (r < 0) {
         return r;
+    }
     return 0;
 #else
     return -ENOSYS;
Signature · 1 line
1.7.1
author
#2 lore raw

PATCH v2 2/17 kvm: Drop return value of kvm_cpu_exec

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

It is not used, it is not needed, so let's remove it.

---
 kvm-all.c  |    6 ++----
 kvm-stub.c |    4 ++--
 kvm.h      |    2 +-
 3 files changed, 5 insertions(+), 7 deletions(-)
kvm-all.c +2 -4
index 2538283..7518f2c 100644
@@ -850,7 +850,7 @@ void kvm_cpu_synchronize_post_init(CPUState *env)
     env->kvm_vcpu_dirty = 0;
 }
 
-int kvm_cpu_exec(CPUState *env)
+void kvm_cpu_exec(CPUState *env)
 {
     struct kvm_run *run = env->kvm_run;
     int ret;
@@ -943,7 +943,7 @@ int kvm_cpu_exec(CPUState *env)
 #ifdef KVM_CAP_SET_GUEST_DEBUG
             if (kvm_arch_debug(&run->debug.arch)) {
                 env->exception_index = EXCP_DEBUG;
-                return 0;
+                return;
             }
             /* re-enter, this exception was guest-internal */
             ret = 1;
@@ -960,8 +960,6 @@ int kvm_cpu_exec(CPUState *env)
         env->exit_request = 0;
         env->exception_index = EXCP_INTERRUPT;
     }
-
-    return ret;
 }
 
 int kvm_ioctl(KVMState *s, int type, ...)
kvm-stub.c +2 -2
index 5384a4b..352c6a6 100644
@@ -79,9 +79,9 @@ void kvm_cpu_synchronize_post_init(CPUState *env)
 {
 }
 
-int kvm_cpu_exec(CPUState *env)
+void kvm_cpu_exec(CPUState *env)
 {
-    abort ();
+    abort();
 }
 
 int kvm_has_sync_mmu(void)
kvm.h +1 -1
index 60a9b42..51ad56f 100644
@@ -46,7 +46,7 @@ int kvm_has_xcrs(void);
 #ifdef NEED_CPU_H
 int kvm_init_vcpu(CPUState *env);
 
-int kvm_cpu_exec(CPUState *env);
+void kvm_cpu_exec(CPUState *env);
 
 #if !defined(CONFIG_USER_ONLY)
 int kvm_log_start(target_phys_addr_t phys_addr, ram_addr_t size);
Signature · 1 line
1.7.1
author
#3 lore raw

PATCH v2 3/17 kvm: Stop on all fatal exit reasons

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Ensure that we stop the guest whenever we face a fatal or unknown exit reason. If we stop, we also have to enforce a cpu loop exit.

---
 kvm-all.c         |   15 +++++++++++----
 target-i386/kvm.c |    4 ++++
 target-ppc/kvm.c  |    4 ++++
 3 files changed, 19 insertions(+), 4 deletions(-)
kvm-all.c +11 -4
index 7518f2c..a46a3b6 100644
@@ -774,7 +774,7 @@ static int kvm_handle_io(uint16_t port, void *data, int direction, int size,
 }
 
 #ifdef KVM_CAP_INTERNAL_ERROR_DATA
-static void kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
+static int kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
 {
 
     if (kvm_check_extension(kvm_state, KVM_CAP_INTERNAL_ERROR_DATA)) {
@@ -792,13 +792,13 @@ static void kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
     if (run->internal.suberror == KVM_INTERNAL_ERROR_EMULATION) {
         fprintf(stderr, "emulation failure\n");
         if (!kvm_arch_stop_on_emulation_error(env)) {
-            return;
+            return 0;
         }
     }
     /* FIXME: Should trigger a qmp message to let management know
      * something went wrong.
      */
-    vm_stop(0);
+    return -1;
 }
 #endif
 
@@ -926,16 +926,19 @@ void kvm_cpu_exec(CPUState *env)
             break;
         case KVM_EXIT_UNKNOWN:
             DPRINTF("kvm_exit_unknown\n");
+            ret = -1;
             break;
         case KVM_EXIT_FAIL_ENTRY:
             DPRINTF("kvm_exit_fail_entry\n");
+            ret = -1;
             break;
         case KVM_EXIT_EXCEPTION:
             DPRINTF("kvm_exit_exception\n");
+            ret = -1;
             break;
 #ifdef KVM_CAP_INTERNAL_ERROR_DATA
         case KVM_EXIT_INTERNAL_ERROR:
-            kvm_handle_internal_error(env, run);
+            ret = kvm_handle_internal_error(env, run);
             break;
 #endif
         case KVM_EXIT_DEBUG:
@@ -956,6 +959,10 @@ void kvm_cpu_exec(CPUState *env)
         }
     } while (ret > 0);
 
+    if (ret < 0) {
+        vm_stop(0);
+        env->exit_request = 1;
+    }
     if (env->exit_request) {
         env->exit_request = 0;
         env->exception_index = EXCP_INTERRUPT;
target-i386/kvm.c +4 -0
index fda07d2..2431a1f 100644
@@ -1534,6 +1534,10 @@ int kvm_arch_handle_exit(CPUState *env, struct kvm_run *run)
         DPRINTF("handle_hlt\n");
         ret = kvm_handle_halt(env);
         break;
+    default:
+        fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
+        ret = -1;
+        break;
     }
 
     return ret;
target-ppc/kvm.c +4 -0
index 5caa07c..849b404 100644
@@ -307,6 +307,10 @@ int kvm_arch_handle_exit(CPUState *env, struct kvm_run *run)
         dprintf("handle halt\n");
         ret = kvmppc_handle_halt(env);
         break;
+    default:
+        fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
+        ret = -1;
+        break;
     }
 
     return ret;
Signature · 1 line
1.7.1
author
#4 lore raw

PATCH v2 4/17 kvm: Improve reporting of fatal errors

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Report KVM_EXIT_UNKNOWN, KVM_EXIT_FAIL_ENTRY, and KVM_EXIT_EXCEPTION with more details to stderr. The latter two are so far x86-only, so move them into the arch-specific handler. Integrate the Intel real mode warning on KVM_EXIT_FAIL_ENTRY that qemu-kvm carries, but actually restrict it to Intel CPUs. Moreover, always dump the CPU state in case we fail.

---
 kvm-all.c           |   22 ++++++++--------------
 target-i386/cpu.h   |    2 ++
 target-i386/cpuid.c |    5 ++---
 target-i386/kvm.c   |   33 +++++++++++++++++++++++++++++++++
 4 files changed, 45 insertions(+), 17 deletions(-)
kvm-all.c +8 -14
index a46a3b6..ad1d0a8 100644
@@ -776,22 +776,22 @@ static int kvm_handle_io(uint16_t port, void *data, int direction, int size,
 #ifdef KVM_CAP_INTERNAL_ERROR_DATA
 static int kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
 {
-
+    fprintf(stderr, "KVM internal error.");
     if (kvm_check_extension(kvm_state, KVM_CAP_INTERNAL_ERROR_DATA)) {
         int i;
 
-        fprintf(stderr, "KVM internal error. Suberror: %d\n",
-                run->internal.suberror);
-
+        fprintf(stderr, " Suberror: %d\n", run->internal.suberror);
         for (i = 0; i < run->internal.ndata; ++i) {
             fprintf(stderr, "extra data[%d]: %"PRIx64"\n",
                     i, (uint64_t)run->internal.data[i]);
         }
+    } else {
+        fprintf(stderr, "\n");
     }
-    cpu_dump_state(env, stderr, fprintf, 0);
     if (run->internal.suberror == KVM_INTERNAL_ERROR_EMULATION) {
         fprintf(stderr, "emulation failure\n");
         if (!kvm_arch_stop_on_emulation_error(env)) {
+            cpu_dump_state(env, stderr, fprintf, 0);
             return 0;
         }
     }
@@ -925,15 +925,8 @@ void kvm_cpu_exec(CPUState *env)
             ret = 1;
             break;
         case KVM_EXIT_UNKNOWN:
-            DPRINTF("kvm_exit_unknown\n");
-            ret = -1;
-            break;
-        case KVM_EXIT_FAIL_ENTRY:
-            DPRINTF("kvm_exit_fail_entry\n");
-            ret = -1;
-            break;
-        case KVM_EXIT_EXCEPTION:
-            DPRINTF("kvm_exit_exception\n");
+            fprintf(stderr, "KVM: unknown exit, hardware reason %" PRIx64 "\n",
+                    (uint64_t)run->hw.hardware_exit_reason);
             ret = -1;
             break;
 #ifdef KVM_CAP_INTERNAL_ERROR_DATA
@@ -960,6 +953,7 @@ void kvm_cpu_exec(CPUState *env)
     } while (ret > 0);
 
     if (ret < 0) {
+        cpu_dump_state(env, stderr, fprintf, 0);
         vm_stop(0);
         env->exit_request = 1;
     }
target-i386/cpu.h +2 -0
index dddcd74..a457423 100644
@@ -874,6 +874,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
                    uint32_t *ecx, uint32_t *edx);
 int cpu_x86_register (CPUX86State *env, const char *cpu_model);
 void cpu_clear_apic_feature(CPUX86State *env);
+void host_cpuid(uint32_t function, uint32_t count,
+                uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 
 /* helper.c */
 int cpu_x86_handle_mmu_fault(CPUX86State *env, target_ulong addr,
target-i386/cpuid.c +2 -3
index 165045e..5382a28 100644
@@ -103,9 +103,8 @@ typedef struct model_features_t {
 int check_cpuid = 0;
 int enforce_cpuid = 0;
 
-static void host_cpuid(uint32_t function, uint32_t count,
-                       uint32_t *eax, uint32_t *ebx,
-                       uint32_t *ecx, uint32_t *edx)
+void host_cpuid(uint32_t function, uint32_t count,
+                uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx)
 {
 #if defined(CONFIG_KVM)
     uint32_t vec[4];
target-i386/kvm.c +33 -0
index 2431a1f..d4f253e 100644
@@ -1525,8 +1525,19 @@ static int kvm_handle_halt(CPUState *env)
     return 1;
 }
 
+static bool host_supports_vmx(void)
+{
+    uint32_t ecx, unused;
+
+    host_cpuid(1, 0, &unused, &unused, &ecx, &unused);
+    return ecx & CPUID_EXT_VMX;
+}
+
+#define VMX_INVALID_GUEST_STATE 0x80000021
+
 int kvm_arch_handle_exit(CPUState *env, struct kvm_run *run)
 {
+    uint64_t code;
     int ret = 0;
 
     switch (run->exit_reason) {
@@ -1534,6 +1545,28 @@ int kvm_arch_handle_exit(CPUState *env, struct kvm_run *run)
         DPRINTF("handle_hlt\n");
         ret = kvm_handle_halt(env);
         break;
+    case KVM_EXIT_FAIL_ENTRY:
+        code = run->fail_entry.hardware_entry_failure_reason;
+        fprintf(stderr, "KVM: entry failed, hardware error 0x%" PRIx64 "\n",
+                code);
+        if (host_supports_vmx() && code == VMX_INVALID_GUEST_STATE) {
+            fprintf(stderr,
+                    "\nIf you're runnning a guest on an Intel machine without "
+                        "unrestricted mode\n"
+                    "support, the failure can be most likely due to the guest "
+                        "entering an invalid\n"
+                    "state for Intel VT. For example, the guest maybe running "
+                        "in big real mode\n"
+                    "which is not supported on less recent Intel processors."
+                        "\n\n");
+        }
+        ret = -1;
+        break;
+    case KVM_EXIT_EXCEPTION:
+        fprintf(stderr, "KVM: exception %d exit (error code 0x%x)\n",
+                run->ex.exception, run->ex.error_code);
+        ret = -1;
+        break;
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
Signature · 1 line
1.7.1
author
#5 lore raw

PATCH v2 5/17 x86: Optionally dump code bytes on cpu_dump_state

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Introduce the cpu_dump_state flag CPU_DUMP_CODE and implement it for x86. This writes out the code bytes around the current instruction pointer. Make use of this feature in KVM to help debugging fatal vm exits.

---
 cpu-all.h            |    2 ++
 kvm-all.c            |    4 ++--
 target-i386/helper.c |   21 +++++++++++++++++++++
 3 files changed, 25 insertions(+), 2 deletions(-)
cpu-all.h +2 -0
index 4ce4e83..ffbd6a4 100644
@@ -765,6 +765,8 @@ int page_check_range(target_ulong start, target_ulong len, int flags);
 CPUState *cpu_copy(CPUState *env);
 CPUState *qemu_get_cpu(int cpu);
 
+#define CPU_DUMP_CODE 0x00010000
+
 void cpu_dump_state(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
                     int flags);
 void cpu_dump_statistics(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
kvm-all.c +2 -2
index ad1d0a8..ef2ca3b 100644
@@ -791,7 +791,7 @@ static int kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
     if (run->internal.suberror == KVM_INTERNAL_ERROR_EMULATION) {
         fprintf(stderr, "emulation failure\n");
         if (!kvm_arch_stop_on_emulation_error(env)) {
-            cpu_dump_state(env, stderr, fprintf, 0);
+            cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
             return 0;
         }
     }
@@ -953,7 +953,7 @@ void kvm_cpu_exec(CPUState *env)
     } while (ret > 0);
 
     if (ret < 0) {
-        cpu_dump_state(env, stderr, fprintf, 0);
+        cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
         vm_stop(0);
         env->exit_request = 1;
     }
target-i386/helper.c +21 -0
index adf9542..1ab42c3 100644
@@ -249,6 +249,9 @@ done:
     cpu_fprintf(f, "\n");
 }
 
+#define DUMP_CODE_BYTES_TOTAL    50
+#define DUMP_CODE_BYTES_BACKWARD 20
+
 void cpu_dump_state(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
                     int flags)
 {
@@ -434,6 +437,24 @@ void cpu_dump_state(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
                 cpu_fprintf(f, " ");
         }
     }
+    if (flags & CPU_DUMP_CODE) {
+        target_phys_addr_t base = env->segs[R_CS].base + env->eip;
+        target_phys_addr_t offs = MIN(env->eip, DUMP_CODE_BYTES_BACKWARD);
+        uint8_t code;
+        char codestr[3];
+
+        cpu_fprintf(f, "Code=");
+        for (i = 0; i < DUMP_CODE_BYTES_TOTAL; i++) {
+            if (cpu_memory_rw_debug(env, base - offs + i, &code, 1, 0) == 0) {
+                snprintf(codestr, sizeof(codestr), "%02x", code);
+            } else {
+                snprintf(codestr, sizeof(codestr), "??");
+            }
+            cpu_fprintf(f, "%s%s%s%s", i > 0 ? " ": "",
+                        i == offs ? "<" : "", codestr, i == offs ? ">" : "");
+        }
+        cpu_fprintf(f, "\n");
+    }
 }
 
 /***********************************************************/
Signature · 1 line
1.7.1
author
#6 lore raw

PATCH v3 5/17 x86: Optionally dump code bytes on cpu_dump_state

From: Jan Kiszka <[email protected]>

Introduce the cpu_dump_state flag CPU_DUMP_CODE and implement it for x86. This writes out the code bytes around the current instruction pointer. Make use of this feature in KVM to help debugging fatal vm exits.

---

Argh, the broken qemu-user build led me to check my own patch.

Changes in v3:
- fix qemu-user build by using proper address type

 cpu-all.h            |    2 ++
 kvm-all.c            |    4 ++--
 target-i386/helper.c |   21 +++++++++++++++++++++
 3 files changed, 25 insertions(+), 2 deletions(-)
cpu-all.h +2 -0
index 4ce4e83..ffbd6a4 100644
@@ -765,6 +765,8 @@ int page_check_range(target_ulong start, target_ulong len, int flags);
 CPUState *cpu_copy(CPUState *env);
 CPUState *qemu_get_cpu(int cpu);
 
+#define CPU_DUMP_CODE 0x00010000
+
 void cpu_dump_state(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
                     int flags);
 void cpu_dump_statistics(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
kvm-all.c +2 -2
index ad1d0a8..ef2ca3b 100644
@@ -791,7 +791,7 @@ static int kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
     if (run->internal.suberror == KVM_INTERNAL_ERROR_EMULATION) {
         fprintf(stderr, "emulation failure\n");
         if (!kvm_arch_stop_on_emulation_error(env)) {
-            cpu_dump_state(env, stderr, fprintf, 0);
+            cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
             return 0;
         }
     }
@@ -953,7 +953,7 @@ void kvm_cpu_exec(CPUState *env)
     } while (ret > 0);
 
     if (ret < 0) {
-        cpu_dump_state(env, stderr, fprintf, 0);
+        cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE);
         vm_stop(0);
         env->exit_request = 1;
     }
target-i386/helper.c +21 -0
index adf9542..fa37da3 100644
@@ -249,6 +249,9 @@ done:
     cpu_fprintf(f, "\n");
 }
 
+#define DUMP_CODE_BYTES_TOTAL    50
+#define DUMP_CODE_BYTES_BACKWARD 20
+
 void cpu_dump_state(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
                     int flags)
 {
@@ -434,6 +437,24 @@ void cpu_dump_state(CPUState *env, FILE *f, fprintf_function cpu_fprintf,
                 cpu_fprintf(f, " ");
         }
     }
+    if (flags & CPU_DUMP_CODE) {
+        target_ulong base = env->segs[R_CS].base + env->eip;
+        target_ulong offs = MIN(env->eip, DUMP_CODE_BYTES_BACKWARD);
+        uint8_t code;
+        char codestr[3];
+
+        cpu_fprintf(f, "Code=");
+        for (i = 0; i < DUMP_CODE_BYTES_TOTAL; i++) {
+            if (cpu_memory_rw_debug(env, base - offs + i, &code, 1, 0) == 0) {
+                snprintf(codestr, sizeof(codestr), "%02x", code);
+            } else {
+                snprintf(codestr, sizeof(codestr), "??");
+            }
+            cpu_fprintf(f, "%s%s%s%s", i > 0 ? " ": "",
+                        i == offs ? "<" : "", codestr, i == offs ? ">" : "");
+        }
+        cpu_fprintf(f, "\n");
+    }
 }
 
 /***********************************************************/
author
#7 lore raw

PATCH v2 6/17 kvm: x86: Align kvm_arch_put_registers code with comment

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

The ordering doesn't matter in this case, but better keep it consistent.

---
 target-i386/kvm.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)
target-i386/kvm.c +3 -3
index d4f253e..684430f 100644
@@ -1388,12 +1388,12 @@ int kvm_arch_put_registers(CPUState *env, int level)
     if (ret < 0) {
         return ret;
     }
-    /* must be last */
-    ret = kvm_guest_debug_workarounds(env);
+    ret = kvm_put_debugregs(env);
     if (ret < 0) {
         return ret;
     }
-    ret = kvm_put_debugregs(env);
+    /* must be last */
+    ret = kvm_guest_debug_workarounds(env);
     if (ret < 0) {
         return ret;
     }
Signature · 1 line
1.7.1
author
#8 lore raw

PATCH v2 7/17 kvm: x86: Prepare kvm_get_mp_state for in-kernel irqchip

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

This code path will not yet be taken as we still lack in-kernel irqchip support. But qemu-kvm can already make use of it and drop its own mp_state access services.

---
 target-i386/kvm.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
target-i386/kvm.c +3 -0
index 684430f..30aa51c 100644
@@ -1198,6 +1198,9 @@ static int kvm_get_mp_state(CPUState *env)
         return ret;
     }
     env->mp_state = mp_state.mp_state;
+    if (kvm_irqchip_in_kernel()) {
+        env->halted = (mp_state.mp_state == KVM_MP_STATE_HALTED);
+    }
     return 0;
 }
 
Signature · 1 line
1.7.1
author
#9 lore raw

PATCH v2 8/17 kvm: x86: Remove redundant mp_state initialization

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

kvm_arch_reset_vcpu initializes mp_state, and that function is invoked right after kvm_arch_init_vcpu.

---
 target-i386/kvm.c |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)
target-i386/kvm.c +0 -2
index 30aa51c..1403327 100644
@@ -321,8 +321,6 @@ int kvm_arch_init_vcpu(CPUState *env)
     uint32_t signature[3];
 #endif
 
-    env->mp_state = KVM_MP_STATE_RUNNABLE;
-
     env->cpuid_features &= kvm_arch_get_supported_cpuid(env, 1, 0, R_EDX);
 
     i = env->cpuid_ext_features & CPUID_EXT_HYPERVISOR;
Signature · 1 line
1.7.1
author
#10 lore raw

PATCH v2 9/17 kvm: x86: Fix xcr0 reset mismerge

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

For unknown reasons, xcr0 reset ended up in kvm_arch_update_guest_debug on upstream merge. Fix this and also remove the misleading comment (1 is THE reset value).

---
 target-i386/kvm.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
target-i386/kvm.c +1 -2
index 1403327..e46b901 100644
@@ -450,6 +450,7 @@ void kvm_arch_reset_vcpu(CPUState *env)
     env->interrupt_injected = -1;
     env->nmi_injected = 0;
     env->nmi_pending = 0;
+    env->xcr0 = 1;
     if (kvm_irqchip_in_kernel()) {
         env->mp_state = cpu_is_bsp(env) ? KVM_MP_STATE_RUNNABLE :
                                           KVM_MP_STATE_UNINITIALIZED;
@@ -1756,8 +1757,6 @@ void kvm_arch_update_guest_debug(CPUState *env, struct kvm_guest_debug *dbg)
                 ((uint32_t)len_code[hw_breakpoint[n].len] << (18 + n*4));
         }
     }
-    /* Legal xcr0 for loading */
-    env->xcr0 = 1;
 }
 #endif /* KVM_CAP_SET_GUEST_DEBUG */
 
Signature · 1 line
1.7.1
author
#11 lore raw

PATCH v2 10/17 kvm: x86: Refactor msr_star/hsave_pa setup and checks

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Simplify kvm_has_msr_star/hsave_pa to booleans and push their one-time initialization into kvm_arch_init. Also handle potential errors of that setup procedure.

---
 target-i386/kvm.c |   47 +++++++++++++++++++----------------------------
 1 files changed, 19 insertions(+), 28 deletions(-)
target-i386/kvm.c +19 -28
index e46b901..d8f26bf 100644
@@ -54,6 +54,8 @@
 #define BUS_MCEERR_AO 5
 #endif
 
+static bool has_msr_star;
+static bool has_msr_hsave_pa;
 static int lm_capable_kernel;
 
 #ifdef KVM_CAP_EXT_CPUID
@@ -459,13 +461,10 @@ void kvm_arch_reset_vcpu(CPUState *env)
     }
 }
 
-int has_msr_star;
-int has_msr_hsave_pa;
-
-static void kvm_supported_msrs(CPUState *env)
+static int kvm_get_supported_msrs(KVMState *s)
 {
     static int kvm_supported_msrs;
-    int ret;
+    int ret = 0;
 
     /* first time */
     if (kvm_supported_msrs == 0) {
@@ -476,9 +475,9 @@ static void kvm_supported_msrs(CPUState *env)
         /* Obtain MSR list from KVM.  These are the MSRs that we must
          * save/restore */
         msr_list.nmsrs = 0;
-        ret = kvm_ioctl(env->kvm_state, KVM_GET_MSR_INDEX_LIST, &msr_list);
+        ret = kvm_ioctl(s, KVM_GET_MSR_INDEX_LIST, &msr_list);
         if (ret < 0 && ret != -E2BIG) {
-            return;
+            return ret;
         }
         /* Old kernel modules had a bug and could write beyond the provided
            memory. Allocate at least a safe amount of 1K. */
@@ -487,17 +486,17 @@ static void kvm_supported_msrs(CPUState *env)
                                               sizeof(msr_list.indices[0])));
 
         kvm_msr_list->nmsrs = msr_list.nmsrs;
-        ret = kvm_ioctl(env->kvm_state, KVM_GET_MSR_INDEX_LIST, kvm_msr_list);
+        ret = kvm_ioctl(s, KVM_GET_MSR_INDEX_LIST, kvm_msr_list);
         if (ret >= 0) {
             int i;
 
             for (i = 0; i < kvm_msr_list->nmsrs; i++) {
                 if (kvm_msr_list->indices[i] == MSR_STAR) {
-                    has_msr_star = 1;
+                    has_msr_star = true;
                     continue;
                 }
                 if (kvm_msr_list->indices[i] == MSR_VM_HSAVE_PA) {
-                    has_msr_hsave_pa = 1;
+                    has_msr_hsave_pa = true;
                     continue;
                 }
             }
@@ -506,19 +505,7 @@ static void kvm_supported_msrs(CPUState *env)
         free(kvm_msr_list);
     }
 
-    return;
-}
-
-static int kvm_has_msr_hsave_pa(CPUState *env)
-{
-    kvm_supported_msrs(env);
-    return has_msr_hsave_pa;
-}
-
-static int kvm_has_msr_star(CPUState *env)
-{
-    kvm_supported_msrs(env);
-    return has_msr_star;
+    return ret;
 }
 
 static int kvm_init_identity_map_page(KVMState *s)
@@ -543,9 +530,13 @@ static int kvm_init_identity_map_page(KVMState *s)
 int kvm_arch_init(KVMState *s, int smp_cpus)
 {
     int ret;
-
     struct utsname utsname;
 
+    ret = kvm_get_supported_msrs(s);
+    if (ret < 0) {
+        return ret;
+    }
+
     uname(&utsname);
     lm_capable_kernel = strcmp(utsname.machine, "x86_64") == 0;
 
@@ -830,10 +821,10 @@ static int kvm_put_msrs(CPUState *env, int level)
     kvm_msr_entry_set(&msrs[n++], MSR_IA32_SYSENTER_CS, env->sysenter_cs);
     kvm_msr_entry_set(&msrs[n++], MSR_IA32_SYSENTER_ESP, env->sysenter_esp);
     kvm_msr_entry_set(&msrs[n++], MSR_IA32_SYSENTER_EIP, env->sysenter_eip);
-    if (kvm_has_msr_star(env)) {
+    if (has_msr_star) {
         kvm_msr_entry_set(&msrs[n++], MSR_STAR, env->star);
     }
-    if (kvm_has_msr_hsave_pa(env)) {
+    if (has_msr_hsave_pa) {
         kvm_msr_entry_set(&msrs[n++], MSR_VM_HSAVE_PA, env->vm_hsave);
     }
 #ifdef TARGET_X86_64
@@ -1076,10 +1067,10 @@ static int kvm_get_msrs(CPUState *env)
     msrs[n++].index = MSR_IA32_SYSENTER_CS;
     msrs[n++].index = MSR_IA32_SYSENTER_ESP;
     msrs[n++].index = MSR_IA32_SYSENTER_EIP;
-    if (kvm_has_msr_star(env)) {
+    if (has_msr_star) {
         msrs[n++].index = MSR_STAR;
     }
-    if (kvm_has_msr_hsave_pa(env)) {
+    if (has_msr_hsave_pa) {
         msrs[n++].index = MSR_VM_HSAVE_PA;
     }
     msrs[n++].index = MSR_IA32_TSC;
Signature · 1 line
1.7.1
author
#12 lore raw

PATCH v2 11/17 kvm: x86: Reset paravirtual MSRs

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Make sure to clear MSR_KVM_SYSTEM_TIME, MSR_KVM_WALL_CLOCK, and MSR_KVM_ASYNC_PF_EN so that a freshly booted guest cannot be disturbed by old values.

CC Glauber Costa <[email protected]>
---
 target-i386/kvm.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
target-i386/kvm.c +10 -0
index d8f26bf..664a4a0 100644
@@ -453,6 +453,9 @@ void kvm_arch_reset_vcpu(CPUState *env)
     env->nmi_injected = 0;
     env->nmi_pending = 0;
     env->xcr0 = 1;
+    env->system_time_msr = 0;
+    env->wall_clock_msr = 0;
+    env->async_pf_en_msr = 0;
     if (kvm_irqchip_in_kernel()) {
         env->mp_state = cpu_is_bsp(env) ? KVM_MP_STATE_RUNNABLE :
                                           KVM_MP_STATE_UNINITIALIZED;
@@ -845,6 +848,13 @@ static int kvm_put_msrs(CPUState *env, int level)
         if (smp_cpus == 1 || env->tsc != 0) {
             kvm_msr_entry_set(&msrs[n++], MSR_IA32_TSC, env->tsc);
         }
+    }
+    /*
+     * The following paravirtual MSRs have side effects on the guest or are
+     * too heavy for normal writeback. Limit them to reset or full state
+     * updates.
+     */
+    if (level >= KVM_PUT_RESET_STATE) {
         kvm_msr_entry_set(&msrs[n++], MSR_KVM_SYSTEM_TIME,
                           env->system_time_msr);
         kvm_msr_entry_set(&msrs[n++], MSR_KVM_WALL_CLOCK, env->wall_clock_msr);
Signature · 1 line
1.7.1
#13 lore raw

Re: [PATCH v2 11/17] kvm: x86: Reset paravirtual MSRs

Quoted 17 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

Make sure to clear MSR_KVM_SYSTEM_TIME, MSR_KVM_WALL_CLOCK, and MSR_KVM_ASYNC_PF_EN so that a freshly booted guest cannot be disturbed by old values.

CC Glauber Costa <[email protected]>
---
 target-i386/kvm.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
target-i386/kvm.c +0 -0
index d8f26bf..664a4a0 100644

target-i386/kvm.c void kvm_arch_reset_vcpu(CPUState *env) · line 458

Quoted 6 lines
target-i386/kvm.c +3 -0
@@ -453,6 +453,9 @@ void kvm_arch_reset_vcpu(CPUState *env)
     env->nmi_injected = 0;
     env->nmi_pending = 0;
     env->xcr0 = 1;
+    env->system_time_msr = 0;
+    env->wall_clock_msr = 0;
+    env->async_pf_en_msr = 0;

Have you seen this happening? I'd expect CPUState to be zeroed out over init. And if it is not, I guess we should...

author
#14 lore raw

Re: [PATCH v2 11/17] kvm: x86: Reset paravirtual MSRs

Quoted 13 lines · Glauber Costa
Quoted 12 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

Make sure to clear MSR_KVM_SYSTEM_TIME, MSR_KVM_WALL_CLOCK, and MSR_KVM_ASYNC_PF_EN so that a freshly booted guest cannot be disturbed by old values.

CC Glauber Costa <[email protected]>
---
 target-i386/kvm.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
Quoted 14 lines

target-i386/kvm.c void kvm_arch_reset_vcpu(CPUState *env) · line 458

Quoted 11 lines
target-i386/kvm.c +3 -0
index d8f26bf..664a4a0 100644
@@ -453,6 +453,9 @@ void kvm_arch_reset_vcpu(CPUState *env)
     env->nmi_injected = 0;
     env->nmi_pending = 0;
     env->xcr0 = 1;
+    env->system_time_msr = 0;
+    env->wall_clock_msr = 0;
+    env->async_pf_en_msr = 0;

Have you seen this happening? I'd expect CPUState to be zeroed out over init. And if it is not, I guess we should...

Ah, true, those three are part of the section that is zeroed. Will drop that hunk on repost.

Guess we should rather move some other variables in that region too and avoid clearing them manually like above...

Jan

#15 lore raw

Re: [PATCH v2 11/17] kvm: x86: Reset paravirtual MSRs

Quoted 29 lines · Jan Kiszka
Quoted 13 lines · Glauber Costa
Quoted 12 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

Make sure to clear MSR_KVM_SYSTEM_TIME, MSR_KVM_WALL_CLOCK, and MSR_KVM_ASYNC_PF_EN so that a freshly booted guest cannot be disturbed by old values.

CC Glauber Costa <[email protected]>
---
 target-i386/kvm.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
Quoted 14 lines

target-i386/kvm.c void kvm_arch_reset_vcpu(CPUState *env) · line 458

Quoted 11 lines
target-i386/kvm.c +3 -0
index d8f26bf..664a4a0 100644
@@ -453,6 +453,9 @@ void kvm_arch_reset_vcpu(CPUState *env)
     env->nmi_injected = 0;
     env->nmi_pending = 0;
     env->xcr0 = 1;
+    env->system_time_msr = 0;
+    env->wall_clock_msr = 0;
+    env->async_pf_en_msr = 0;

Have you seen this happening? I'd expect CPUState to be zeroed out over init. And if it is not, I guess we should...

Quoted 8 lines

Ah, true, those three are part of the section that is zeroed. Will drop that hunk on repost.

Guess we should rather move some other variables in that region too and avoid clearing them manually like above...

Jan

Agreed.

author
#16 lore raw

PATCH v2 12/17 kvm: x86: Drop MCE MSRs write back restrictions

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

There is no need to restrict writing back MCE MSRs to reset or full state updates as setting their values has no side effects.

---
 target-i386/kvm.c |   12 ++++--------
 1 files changed, 4 insertions(+), 8 deletions(-)
target-i386/kvm.c +4 -8
index 664a4a0..ef8f6e6 100644
@@ -866,14 +866,10 @@ static int kvm_put_msrs(CPUState *env, int level)
     if (env->mcg_cap) {
         int i;
 
-        if (level == KVM_PUT_RESET_STATE) {
-            kvm_msr_entry_set(&msrs[n++], MSR_MCG_STATUS, env->mcg_status);
-        } else if (level == KVM_PUT_FULL_STATE) {
-            kvm_msr_entry_set(&msrs[n++], MSR_MCG_STATUS, env->mcg_status);
-            kvm_msr_entry_set(&msrs[n++], MSR_MCG_CTL, env->mcg_ctl);
-            for (i = 0; i < (env->mcg_cap & 0xff) * 4; i++) {
-                kvm_msr_entry_set(&msrs[n++], MSR_MC0_CTL + i, env->mce_banks[i]);
-            }
+        kvm_msr_entry_set(&msrs[n++], MSR_MCG_STATUS, env->mcg_status);
+        kvm_msr_entry_set(&msrs[n++], MSR_MCG_CTL, env->mcg_ctl);
+        for (i = 0; i < (env->mcg_cap & 0xff) * 4; i++) {
+            kvm_msr_entry_set(&msrs[n++], MSR_MC0_CTL + i, env->mce_banks[i]);
         }
     }
 #endif
Signature · 1 line
1.7.1
author
#17 lore raw

PATCH v2 13/17 kvm: Eliminate KVMState arguments

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

QEMU supports only one VM, so there is only one kvm_state per process, and we gain nothing passing a reference to it around. Eliminate any need to refer to it outside of kvm-all.c.

CC Alexander Graf <[email protected]>
---
 cpu-defs.h            |    2 -
 kvm-all.c             |  232 +++++++++++++++++++++----------------------------
 kvm-stub.c            |    2 +-
 kvm.h                 |   15 +--
 target-i386/cpuid.c   |    9 +-
 target-i386/kvm.c     |   77 ++++++++--------
 target-i386/kvm_x86.h |    3 +
 target-ppc/kvm.c      |   12 ++--
 target-s390x/kvm.c    |    8 +--
 9 files changed, 160 insertions(+), 200 deletions(-)
cpu-defs.h +0 -2
index eaed43e..ada6629 100644
@@ -131,7 +131,6 @@ typedef struct icount_decr_u16 {
 #endif
 
 struct kvm_run;
-struct KVMState;
 struct qemu_work_item;
 
 typedef struct CPUBreakpoint {
@@ -208,7 +207,6 @@ typedef struct CPUWatchpoint {
     struct QemuCond *halt_cond;                                         \
     struct qemu_work_item *queued_work_first, *queued_work_last;        \
     const char *cpu_model_str;                                          \
-    struct KVMState *kvm_state;                                         \
     struct kvm_run *kvm_run;                                            \
     int kvm_fd;                                                         \
     int kvm_vcpu_dirty;
kvm-all.c +100 -132

662 lines not on this page · raw message

kvm-stub.c +1 -1
index 352c6a6..3a058ad 100644
@@ -53,7 +53,7 @@ int kvm_uncoalesce_mmio_region(target_phys_addr_t start, ram_addr_t size)
     return -ENOSYS;
 }
 
-int kvm_check_extension(KVMState *s, unsigned int extension)
+int kvm_check_extension(unsigned int extension)
 {
     return 0;
 }
kvm.h +5 -10
index 51ad56f..26ca8c1 100644
@@ -74,12 +74,9 @@ int kvm_irqchip_in_kernel(void);
 
 /* internal API */
 
-struct KVMState;
-typedef struct KVMState KVMState;
+int kvm_ioctl(int type, ...);
 
-int kvm_ioctl(KVMState *s, int type, ...);
-
-int kvm_vm_ioctl(KVMState *s, int type, ...);
+int kvm_vm_ioctl(int type, ...);
 
 int kvm_vcpu_ioctl(CPUState *env, int type, ...);
 
@@ -104,7 +101,7 @@ int kvm_arch_get_registers(CPUState *env);
 
 int kvm_arch_put_registers(CPUState *env, int level);
 
-int kvm_arch_init(KVMState *s, int smp_cpus);
+int kvm_arch_init(int smp_cpus);
 
 int kvm_arch_init_vcpu(CPUState *env);
 
@@ -146,10 +143,8 @@ void kvm_arch_update_guest_debug(CPUState *env, struct kvm_guest_debug *dbg);
 
 bool kvm_arch_stop_on_emulation_error(CPUState *env);
 
-int kvm_check_extension(KVMState *s, unsigned int extension);
+int kvm_check_extension(unsigned int extension);
 
-uint32_t kvm_arch_get_supported_cpuid(CPUState *env, uint32_t function,
-                                      uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUState *env);
 void kvm_cpu_synchronize_post_reset(CPUState *env);
 void kvm_cpu_synchronize_post_init(CPUState *env);
@@ -179,7 +174,7 @@ static inline void cpu_synchronize_post_init(CPUState *env)
 
 
 #if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_ram(KVMState *s, ram_addr_t ram_addr,
+int kvm_physical_memory_addr_from_ram(ram_addr_t ram_addr,
                                       target_phys_addr_t *phys_addr);
 #endif
 
target-i386/cpuid.c +5 -4
index 5382a28..17ab619 100644
@@ -23,6 +23,7 @@
 
 #include "cpu.h"
 #include "kvm.h"
+#include "kvm_x86.h"
 
 #include "qemu-option.h"
 #include "qemu-config.h"
@@ -1138,10 +1139,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             break;
         }
         if (kvm_enabled()) {
-            *eax = kvm_arch_get_supported_cpuid(env, 0xd, count, R_EAX);
-            *ebx = kvm_arch_get_supported_cpuid(env, 0xd, count, R_EBX);
-            *ecx = kvm_arch_get_supported_cpuid(env, 0xd, count, R_ECX);
-            *edx = kvm_arch_get_supported_cpuid(env, 0xd, count, R_EDX);
+            *eax = kvm_x86_get_supported_cpuid(0xd, count, R_EAX);
+            *ebx = kvm_x86_get_supported_cpuid(0xd, count, R_EBX);
+            *ecx = kvm_x86_get_supported_cpuid(0xd, count, R_ECX);
+            *edx = kvm_x86_get_supported_cpuid(0xd, count, R_EDX);
         } else {
             *eax = 0;
             *ebx = 0;
target-i386/kvm.c +38 -39

233 lines not on this page · raw message

target-i386/kvm_x86.h +3 -0
index 9d7b584..304d0cb 100644
@@ -22,4 +22,7 @@ void kvm_inject_x86_mce(CPUState *cenv, int bank, uint64_t status,
                         uint64_t mcg_status, uint64_t addr, uint64_t misc,
                         int flag);
 
+uint32_t kvm_x86_get_supported_cpuid(uint32_t function, uint32_t index,
+                                     int reg);
+
 #endif
target-ppc/kvm.c +6 -6
index 849b404..56d30cc 100644
@@ -56,13 +56,13 @@ static void kvm_kick_env(void *env)
     qemu_cpu_kick(env);
 }
 
-int kvm_arch_init(KVMState *s, int smp_cpus)
+int kvm_arch_init(int smp_cpus)
 {
 #ifdef KVM_CAP_PPC_UNSET_IRQ
-    cap_interrupt_unset = kvm_check_extension(s, KVM_CAP_PPC_UNSET_IRQ);
+    cap_interrupt_unset = kvm_check_extension(KVM_CAP_PPC_UNSET_IRQ);
 #endif
 #ifdef KVM_CAP_PPC_IRQ_LEVEL
-    cap_interrupt_level = kvm_check_extension(s, KVM_CAP_PPC_IRQ_LEVEL);
+    cap_interrupt_level = kvm_check_extension(KVM_CAP_PPC_IRQ_LEVEL);
 #endif
 
     if (!cap_interrupt_level) {
@@ -164,7 +164,7 @@ int kvm_arch_get_registers(CPUState *env)
         env->gpr[i] = regs.gpr[i];
 
 #ifdef KVM_CAP_PPC_SEGSTATE
-    if (kvm_check_extension(env->kvm_state, KVM_CAP_PPC_SEGSTATE)) {
+    if (kvm_check_extension(KVM_CAP_PPC_SEGSTATE)) {
         env->sdr1 = sregs.u.s.sdr1;
 
         /* Sync SLB */
@@ -371,8 +371,8 @@ int kvmppc_get_hypercall(CPUState *env, uint8_t *buf, int buf_len)
 #ifdef KVM_CAP_PPC_GET_PVINFO
     struct kvm_ppc_pvinfo pvinfo;
 
-    if (kvm_check_extension(env->kvm_state, KVM_CAP_PPC_GET_PVINFO) &&
-        !kvm_vm_ioctl(env->kvm_state, KVM_PPC_GET_PVINFO, &pvinfo)) {
+    if (kvm_check_extension(KVM_CAP_PPC_GET_PVINFO) &&
+        !kvm_vm_ioctl(KVM_PPC_GET_PVINFO, &pvinfo)) {
         memcpy(buf, pvinfo.hcall, buf_len);
 
         return 0;
target-s390x/kvm.c +2 -6
index adf4a9e..927a37e 100644
@@ -70,7 +70,7 @@
 #define SCLP_CMDW_READ_SCP_INFO         0x00020001
 #define SCLP_CMDW_READ_SCP_INFO_FORCED  0x00120001
 
-int kvm_arch_init(KVMState *s, int smp_cpus)
+int kvm_arch_init(int smp_cpus)
 {
     return 0;
 }
@@ -186,10 +186,6 @@ static void kvm_s390_interrupt_internal(CPUState *env, int type, uint32_t parm,
     struct kvm_s390_interrupt kvmint;
     int r;
 
-    if (!env->kvm_state) {
-        return;
-    }
-
     env->halted = 0;
     env->exception_index = -1;
 
@@ -198,7 +194,7 @@ static void kvm_s390_interrupt_internal(CPUState *env, int type, uint32_t parm,
     kvmint.parm64 = parm64;
 
     if (vm) {
-        r = kvm_vm_ioctl(env->kvm_state, KVM_S390_INTERRUPT, &kvmint);
+        r = kvm_vm_ioctl(KVM_S390_INTERRUPT, &kvmint);
     } else {
         r = kvm_vcpu_ioctl(env, KVM_S390_INTERRUPT, &kvmint);
     }
Signature · 1 line
1.7.1
Acked-by on v2 13/17
#18 lore raw

Re: [PATCH v2 13/17] kvm: Eliminate KVMState arguments

Quoted 5 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

QEMU supports only one VM, so there is only one kvm_state per process, and we gain nothing passing a reference to it around. Eliminate any need to refer to it outside of kvm-all.c.

Slightly unqemu'ish, but I like the simplicity of it :).

Alex

author
#19 lore raw

PATCH v2 14/17 kvm: x86: Introduce kvmclock device to save/restore its state

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

CC Glauber Costa <[email protected]>
---
 target-i386/kvm.c |   71 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 70 insertions(+), 1 deletions(-)
target-i386/kvm.c +70 -1
index 26f459c..c4b6813 100644
@@ -29,6 +29,7 @@
 #include "hw/apic.h"
 #include "ioport.h"
 #include "kvm_x86.h"
+#include "hw/sysbus.h"
 
 #ifdef CONFIG_KVM_PARA
 #include <linux/kvm_para.h>
@@ -309,6 +310,64 @@ void kvm_inject_x86_mce(CPUState *cenv, int bank, uint64_t status,
 #endif
 }
 
+#ifdef KVM_CAP_ADJUST_CLOCK
+typedef struct KVMClockState {
+    SysBusDevice busdev;
+    uint64_t clock;
+    struct kvm_clock_data data;
+} KVMClockState;
+
+static void kvmclock_pre_save(void *opaque)
+{
+    KVMClockState *s = opaque;
+    struct kvm_clock_data data;
+    int ret;
+
+    ret = kvm_vm_ioctl(KVM_GET_CLOCK, &data);
+    if (ret < 0) {
+        fprintf(stderr, "KVM_GET_CLOCK failed: %s\n", strerror(ret));
+        data.clock = 0;
+    }
+    s->clock = data.clock;
+}
+
+static int kvmclock_post_load(void *opaque, int version_id)
+{
+    KVMClockState *s = opaque;
+    struct kvm_clock_data data;
+
+    data.clock = s->clock;
+    data.flags = 0;
+    return kvm_vm_ioctl(KVM_SET_CLOCK, &data);
+}
+
+static int kvmclock_init(SysBusDevice *dev)
+{
+    return 0;
+}
+
+static const VMStateDescription kvmclock_vmsd= {
+    .name = "kvmclock",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .pre_save = kvmclock_pre_save,
+    .post_load = kvmclock_post_load,
+    .fields = (VMStateField []) {
+        VMSTATE_UINT64(clock, KVMClockState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static SysBusDeviceInfo kvmclock_info = {
+    .qdev.name = "kvmclock",
+    .qdev.size = sizeof(KVMClockState),
+    .qdev.vmsd = &kvmclock_vmsd,
+    .qdev.no_user = 1,
+    .init = kvmclock_init,
+};
+#endif /* KVM_CAP_ADJUST_CLOCK */
+
 int kvm_arch_init_vcpu(CPUState *env)
 {
     struct {
@@ -335,7 +394,6 @@ int kvm_arch_init_vcpu(CPUState *env)
     env->cpuid_svm_features  &= kvm_x86_get_supported_cpuid(0x8000000A,
                                                             0, R_EDX);
 
-
     cpuid_i = 0;
 
 #ifdef CONFIG_KVM_PARA
@@ -442,6 +500,13 @@ int kvm_arch_init_vcpu(CPUState *env)
     }
 #endif
 
+#ifdef KVM_CAP_ADJUST_CLOCK
+    if (cpu_is_bsp(env) &&
+        (env->cpuid_kvm_features & (1ULL << KVM_FEATURE_CLOCKSOURCE))) {
+        sysbus_create_simple("kvmclock", -1, NULL);
+    }
+#endif
+
     return kvm_vcpu_ioctl(env, KVM_SET_CPUID2, &cpuid_data);
 }
 
@@ -534,6 +599,10 @@ int kvm_arch_init(int smp_cpus)
     int ret;
     struct utsname utsname;
 
+#ifdef KVM_CAP_ADJUST_CLOCK
+    sysbus_register_withprop(&kvmclock_info);
+#endif
+
     ret = kvm_get_supported_msrs();
     if (ret < 0) {
         return ret;
Signature · 1 line
1.7.1
#20 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

Quoted 7 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

kvmclock is a per-cpu affair.

target-i386/kvm.c int kvm_arch_init(int smp_cpus) · line 605

Quoted 9 lines
target-i386/kvm.c +4 -0
@@ -534,6 +599,10 @@ int kvm_arch_init(int smp_cpus)
      int ret;
      struct utsname utsname;
 
+#ifdef KVM_CAP_ADJUST_CLOCK
+    sysbus_register_withprop(&kvmclock_info);
+#endif
+

So this doesn't look right. I think we're fine with just migrating the MSRs, like we migrate anything else that has to do with the cpu.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#21 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

kvmclock is a per-cpu affair.

Nope, it's state (the one save/restored here) is per VM.

Quoted 6 lines
Quoted 6 lines
target-i386/kvm.c +0 -0
@@ -534,6 +599,10 @@ int kvm_arch_init(int smp_cpus)
      int ret;
      struct utsname utsname;
 
Quoted 8 lines

target-i386/kvm.c int kvm_arch_init(int smp_cpus) · line 605

Quoted 4 lines
target-i386/kvm.c +4 -0
+#ifdef KVM_CAP_ADJUST_CLOCK
+    sysbus_register_withprop(&kvmclock_info);
+#endif
+

So this doesn't look right. I think we're fine with just migrating the MSRs, like we migrate anything else that has to do with the cpu.

The kvmclock state is not contained in any MSR. It's an independent machine state that can be indirectly obtained via MSR access. Therefore, qemu-kvm currently registers only one vmstate entry per machine, and this patch just turns this into a clean device - because that's what kvmclock is in the end, something like an HPET.

Jan

#22 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

Quoted 29 lines · Jan Kiszka
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

kvmclock is a per-cpu affair.

Nope, it's state (the one save/restored here) is per VM.

Quoted 6 lines
Quoted 6 lines
target-i386/kvm.c +0 -0
@@ -534,6 +599,10 @@ int kvm_arch_init(int smp_cpus)
      int ret;
      struct utsname utsname;
 
Quoted 8 lines

target-i386/kvm.c int kvm_arch_init(int smp_cpus) · line 605

Quoted 4 lines
target-i386/kvm.c +4 -0
+#ifdef KVM_CAP_ADJUST_CLOCK
+    sysbus_register_withprop(&kvmclock_info);
+#endif
+

So this doesn't look right. I think we're fine with just migrating the MSRs, like we migrate anything else that has to do with the cpu.

Quoted 5 lines

The kvmclock state is not contained in any MSR. It's an independent machine state that can be indirectly obtained via MSR access. Therefore, qemu-kvm currently registers only one vmstate entry per machine, and this patch just turns this into a clean device - because that's what kvmclock is in the end, something like an HPET.

Jan is right, the per-cpu information only cares about which piece of memory will be written to.

#23 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

In reply to Jan Kiszka
Quoted 29 lines · Jan Kiszka
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

kvmclock is a per-cpu affair.

Nope, it's state (the one save/restored here) is per VM.

Quoted 6 lines
Quoted 6 lines

@@ -534,6 +599,10 @@ int kvm_arch_init(int smp_cpus)

       int ret;
       struct utsname utsname;
Quoted 8 lines
Quoted 4 lines

+#ifdef KVM_CAP_ADJUST_CLOCK
+ sysbus_register_withprop(&kvmclock_info);
+#endif
+

So this doesn't look right. I think we're fine with just migrating the MSRs, like we migrate anything else that has to do with the cpu.

Quoted 5 lines

The kvmclock state is not contained in any MSR. It's an independent machine state that can be indirectly obtained via MSR access. Therefore, qemu-kvm currently registers only one vmstate entry per machine, and this patch just turns this into a clean device - because that's what kvmclock is in the end, something like an HPET.

Okay, thanks for the correction.

Signature · 1 line
error compiling committee.c: too many arguments to function
#24 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

In reply to Avi Kivity
Quoted 16 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

kvmclock is a per-cpu affair.

Quoted 5 lines
target-i386/kvm.c +0 -0
@@ -534,6 +599,10 @@ int kvm_arch_init(int smp_cpus)
      int ret;
      struct utsname utsname;
 
Quoted 7 lines

target-i386/kvm.c int kvm_arch_init(int smp_cpus) · line 605

Quoted 4 lines
target-i386/kvm.c +4 -0
+#ifdef KVM_CAP_ADJUST_CLOCK
+    sysbus_register_withprop(&kvmclock_info);
+#endif
+

So this doesn't look right. I think we're fine with just migrating the MSRs, like we migrate anything else that has to do with the cpu.

The ioctl jan is handling here is a global one, that adjusts the base offset for the clock over migration. It is okay.

#25 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

In reply to Jan Kiszka
Quoted 9 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

CC Glauber Costa <[email protected]>

Hi Jan.

I've just recently posted a patch (not sure what was made from it), that fixes a bug that you reintroduce here.

The bug is: if we call KVM_GET_CLOCK ioctl in pre_save, this means that this value will change every time we issue savevm, even if the machine is not run in between.

Ideally, we'd like to have two consecutive savevms returning the exact same thing in that situation.

I like the general approach of moving it to sysbus, but please move the ioctl to change state notifiers.

Cheers!

author
#26 lore raw

Re: [PATCH v2 14/17] kvm: x86: Introduce kvmclock device to save/restore its state

Quoted 8 lines · Glauber Costa
Quoted 7 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

If kvmclock is used, which implies the kernel supports it, register a kvmclock device with the sysbus. Its main purpose is to save and restore the kernel state on migration, but this will also allow to visualize it one day.

Quoted 7 lines
Quoted 2 lines
CC Glauber Costa <[email protected]>

Hi Jan.

I've just recently posted a patch (not sure what was made from it), that fixes a bug that you reintroduce here.

Hmm, must have missed it.

Quoted 10 lines

The bug is: if we call KVM_GET_CLOCK ioctl in pre_save, this means that this value will change every time we issue savevm, even if the machine is not run in between.

Ideally, we'd like to have two consecutive savevms returning the exact same thing in that situation.

I like the general approach of moving it to sysbus, but please move the ioctl to change state notifiers.

OK, will look into this.

Jan

author
#27 lore raw

PATCH v2 15/17 kvm: Drop smp_cpus argument from init functions

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

No longer used.

---
 kvm-all.c          |    4 ++--
 kvm-stub.c         |    2 +-
 kvm.h              |    4 ++--
 target-i386/kvm.c  |    2 +-
 target-ppc/kvm.c   |    2 +-
 target-s390x/kvm.c |    2 +-
 vl.c               |    2 +-
 7 files changed, 9 insertions(+), 9 deletions(-)
kvm-all.c +2 -2
index d8820c7..190fcdf 100644
@@ -584,7 +584,7 @@ static CPUPhysMemoryClient kvm_cpu_phys_memory_client = {
     .migration_log = kvm_client_migration_log,
 };
 
-int kvm_init(int smp_cpus)
+int kvm_init(void)
 {
     static const char upgrade_note[] =
         "Please upgrade to at least kernel 2.6.29 or recent kvm-kmod\n"
@@ -687,7 +687,7 @@ int kvm_init(int smp_cpus)
     kvm_state.xcrs = kvm_check_extension(KVM_CAP_XCRS);
 #endif
 
-    ret = kvm_arch_init(smp_cpus);
+    ret = kvm_arch_init();
     if (ret < 0) {
         goto err;
     }
kvm-stub.c +1 -1
index 3a058ad..e00d7df 100644
@@ -58,7 +58,7 @@ int kvm_check_extension(unsigned int extension)
     return 0;
 }
 
-int kvm_init(int smp_cpus)
+int kvm_init(void)
 {
     return -ENOSYS;
 }
kvm.h +2 -2
index 26ca8c1..31d9f21 100644
@@ -34,7 +34,7 @@ struct kvm_run;
 
 /* external API */
 
-int kvm_init(int smp_cpus);
+int kvm_init(void);
 
 int kvm_has_sync_mmu(void);
 int kvm_has_vcpu_events(void);
@@ -101,7 +101,7 @@ int kvm_arch_get_registers(CPUState *env);
 
 int kvm_arch_put_registers(CPUState *env, int level);
 
-int kvm_arch_init(int smp_cpus);
+int kvm_arch_init(void);
 
 int kvm_arch_init_vcpu(CPUState *env);
 
target-i386/kvm.c +1 -1
index c4b6813..6d31dde 100644
@@ -594,7 +594,7 @@ static int kvm_init_identity_map_page(void)
     return 0;
 }
 
-int kvm_arch_init(int smp_cpus)
+int kvm_arch_init(void)
 {
     int ret;
     struct utsname utsname;
target-ppc/kvm.c +1 -1
index 56d30cc..72f2f94 100644
@@ -56,7 +56,7 @@ static void kvm_kick_env(void *env)
     qemu_cpu_kick(env);
 }
 
-int kvm_arch_init(int smp_cpus)
+int kvm_arch_init(void)
 {
 #ifdef KVM_CAP_PPC_UNSET_IRQ
     cap_interrupt_unset = kvm_check_extension(KVM_CAP_PPC_UNSET_IRQ);
target-s390x/kvm.c +1 -1
index 927a37e..4f9075c 100644
@@ -70,7 +70,7 @@
 #define SCLP_CMDW_READ_SCP_INFO         0x00020001
 #define SCLP_CMDW_READ_SCP_INFO_FORCED  0x00120001
 
-int kvm_arch_init(int smp_cpus)
+int kvm_arch_init(void)
 {
     return 0;
 }
vl.c +1 -1
index 768dbf4..abe4af8 100644
@@ -2834,7 +2834,7 @@ int main(int argc, char **argv, char **envp)
     }
 
     if (kvm_allowed) {
-        int ret = kvm_init(smp_cpus);
+        int ret = kvm_init();
         if (ret < 0) {
             if (!kvm_available()) {
                 printf("KVM not supported for this target\n");
Signature · 1 line
1.7.1
author
#28 lore raw

PATCH v2 16/17 kvm: x86: Rework identity map and TSS setup for larger BIOS sizes

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

First of all, we only need this EPT identity and TSS reservation on Intel CPUs. Then, in order to support loading BIOSes > 256K, reorder the code, adjusting the base if the kernel supports moving the identity map. We can drop the check for KVM_CAP_SET_TSS_ADDR as we already depend on much newer features.

---
 target-i386/kvm.c |   91 ++++++++++++++++++++++++----------------------------
 1 files changed, 42 insertions(+), 49 deletions(-)
target-i386/kvm.c +42 -49
index 6d31dde..caca407 100644
@@ -575,23 +575,12 @@ static int kvm_get_supported_msrs(void)
     return ret;
 }
 
-static int kvm_init_identity_map_page(void)
+static bool host_supports_vmx(void)
 {
-#ifdef KVM_CAP_SET_IDENTITY_MAP_ADDR
-    int ret;
-    uint64_t addr = 0xfffbc000;
-
-    if (!kvm_check_extension(KVM_CAP_SET_IDENTITY_MAP_ADDR)) {
-        return 0;
-    }
+    uint32_t ecx, unused;
 
-    ret = kvm_vm_ioctl(KVM_SET_IDENTITY_MAP_ADDR, &addr);
-    if (ret < 0) {
-        fprintf(stderr, "kvm_set_identity_map_addr: %s\n", strerror(ret));
-        return ret;
-    }
-#endif
-    return 0;
+    host_cpuid(1, 0, &unused, &unused, &ecx, &unused);
+    return ecx & CPUID_EXT_VMX;
 }
 
 int kvm_arch_init(void)
@@ -611,35 +600,47 @@ int kvm_arch_init(void)
     uname(&utsname);
     lm_capable_kernel = strcmp(utsname.machine, "x86_64") == 0;
 
-    /* create vm86 tss.  KVM uses vm86 mode to emulate 16-bit code
-     * directly.  In order to use vm86 mode, a TSS is needed.  Since this
-     * must be part of guest physical memory, we need to allocate it.  Older
-     * versions of KVM just assumed that it would be at the end of physical
-     * memory but that doesn't work with more than 4GB of memory.  We simply
-     * refuse to work with those older versions of KVM. */
-    ret = kvm_check_extension(KVM_CAP_SET_TSS_ADDR);
-    if (ret <= 0) {
-        fprintf(stderr, "kvm does not support KVM_CAP_SET_TSS_ADDR\n");
-        return ret;
-    }
+    if (host_supports_vmx()) {
+        /*
+         * On older Intel CPUs, KVM uses vm86 mode to emulate 16-bit code
+         * directly. In order to use vm86 mode, an EPT identity map and a TSS
+         * are needed. Since these must be part of guest physical memory, we
+         * need to allocate them, both by setting their start addresses in the
+         * kernel and by creating a corresponding e820 entry. We need 4 pages
+         * before the BIOS.
+         *
+         * Older KVM versions may not support setting the identity map base.
+         * In that case we need to stick with the default, i.e. a 256K maximum
+         * BIOS size.
+         */
+        uint64_t identity_base = 0xfffbc000;
 
-    /* this address is 3 pages before the bios, and the bios should present
-     * as unavaible memory.  FIXME, need to ensure the e820 map deals with
-     * this?
-     */
-    /*
-     * Tell fw_cfg to notify the BIOS to reserve the range.
-     */
-    if (e820_add_entry(0xfffbc000, 0x4000, E820_RESERVED) < 0) {
-        perror("e820_add_entry() table is full");
-        exit(1);
-    }
-    ret = kvm_vm_ioctl(KVM_SET_TSS_ADDR, 0xfffbd000);
-    if (ret < 0) {
-        return ret;
+#ifdef KVM_CAP_SET_IDENTITY_MAP_ADDR
+        if (kvm_check_extension(KVM_CAP_SET_IDENTITY_MAP_ADDR)) {
+            /* Allows up to 16M BIOSes. */
+            identity_base = 0xfeffc000;
+
+            ret = kvm_vm_ioctl(KVM_SET_IDENTITY_MAP_ADDR, &identity_base);
+            if (ret < 0) {
+                return ret;
+            }
+        }
+#endif
+        /* Set TSS base one page after EPT identity map. */
+        ret = kvm_vm_ioctl(KVM_SET_TSS_ADDR, identity_base + 0x1000);
+        if (ret < 0) {
+            return ret;
+        }
+
+        /* Tell fw_cfg to notify the BIOS to reserve the range. */
+        ret = e820_add_entry(identity_base, 0x4000, E820_RESERVED);
+        if (ret < 0) {
+            fprintf(stderr, "e820_add_entry() table is full\n");
+            return ret;
+        }
     }
 
-    return kvm_init_identity_map_page();
+    return 0;
 }
 
 static void set_v8086_seg(struct kvm_segment *lhs, const SegmentCache *rhs)
@@ -1592,14 +1593,6 @@ static int kvm_handle_halt(CPUState *env)
     return 1;
 }
 
-static bool host_supports_vmx(void)
-{
-    uint32_t ecx, unused;
-
-    host_cpuid(1, 0, &unused, &unused, &ecx, &unused);
-    return ecx & CPUID_EXT_VMX;
-}
-
 #define VMX_INVALID_GUEST_STATE 0x80000021
 
 int kvm_arch_handle_exit(CPUState *env, struct kvm_run *run)
Signature · 1 line
1.7.1
#29 lore raw

Re: [PATCH v2 16/17] kvm: x86: Rework identity map and TSS setup for larger BIOS sizes

Quoted 4 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

First of all, we only need this EPT identity and TSS reservation on Intel CPUs.

kvm-amd will ignore it just fine. I'd like to keep arch differences away from userspace.

Quoted 4 lines

Then, in order to support loading BIOSes> 256K, reorder the code, adjusting the base if the kernel supports moving the identity map. We can drop the check for KVM_CAP_SET_TSS_ADDR as we already depend on much newer features.

There is no ordering on kvm features. Each can come and go as it pleases.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#30 lore raw

Re: [PATCH v2 16/17] kvm: x86: Rework identity map and TSS setup for larger BIOS sizes

Quoted 8 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

First of all, we only need this EPT identity and TSS reservation on Intel CPUs.

kvm-amd will ignore it just fine. I'd like to keep arch differences away from userspace.

And I would prefer to avoid needlessly cluttering the physical guest address space where not needed. Long term, we could even give user space a hint (unless it can test it directly) that this workaround is no longer needed as the host Intel CPU supports true real mode.

Quoted 8 lines
Quoted 5 lines

Then, in order to support loading BIOSes> 256K, reorder the code, adjusting the base if the kernel supports moving the identity map. We can drop the check for KVM_CAP_SET_TSS_ADDR as we already depend on much newer features.

There is no ordering on kvm features. Each can come and go as it pleases.

Well, at least this is not how kvm upstream works so far.

Jan

#31 lore raw

Re: [PATCH v2 16/17] kvm: x86: Rework identity map and TSS setup for larger BIOS sizes

Quoted 7 lines · Jan Kiszka
Quoted 6 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

First of all, we only need this EPT identity and TSS reservation on Intel CPUs.

Quoted 7 lines
Quoted 2 lines

kvm-amd will ignore it just fine. I'd like to keep arch differences away from userspace.

And I would prefer to avoid needlessly cluttering the physical guest address space where not needed. Long term, we could even give user space a hint (unless it can test it directly) that this workaround is no longer needed as the host Intel CPU supports true real mode.

Having different physical address spaces based on the host cpu is bad, even disregarding live migration. If there's a real need, we can do it as an option. I don't see such a need though.

We can definitely add a new KVM_CAP for "tss/ept identity supported but not needed". If emulate_invalid_guest_state is eventually fully implemented and becomes the default, it will even be true across the board.

Quoted 10 lines
Quoted 8 lines
Quoted 5 lines

Then, in order to support loading BIOSes> 256K, reorder the
code, adjusting the base if the kernel supports moving the identity map. We can drop the check for KVM_CAP_SET_TSS_ADDR as we already depend on much newer features.

There is no ordering on kvm features. Each can come and go as it pleases.

Well, at least this is not how kvm upstream works so far.

Let's change it then.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#32 lore raw

Re: [PATCH v2 16/17] kvm: x86: Rework identity map and TSS setup for larger BIOS sizes

Quoted 11 lines · Avi Kivity
Quoted 10 lines · Jan Kiszka
Quoted 8 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

First of all, we only need this EPT identity and TSS reservation on Intel CPUs.

kvm-amd will ignore it just fine. I'd like to keep arch differences away from userspace.

Quoted 8 lines
Quoted 4 lines

And I would prefer to avoid needlessly cluttering the physical guest address space where not needed. Long term, we could even give user space a hint (unless it can test it directly) that this workaround is no longer needed as the host Intel CPU supports true real mode.

Having different physical address spaces based on the host cpu is bad, even disregarding live migration. If there's a real need, we can do it as an option. I don't see such a need though.

I see, migration is a killer argument.

Jan

author
#33 lore raw

PATCH v2 17/17 kvm: Drop dependencies on very old capabilities

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

COALESCED_MMIO, SYNC_MMU, EXT_CPUID, CLOCKSOURCE, NOP_IO_DELAY, PV_MMU - all these caps predate features on which we already depend at build time. Moreover, the check for KVM_CAP_EXT_CPUID is unneeded as we already test & fail is a more recent feature is missing.

---
 kvm-all.c         |   16 ----------------
 target-i386/kvm.c |   22 ----------------------
 2 files changed, 0 insertions(+), 38 deletions(-)
kvm-all.c +0 -16
index 190fcdf..15d5f32 100644
@@ -57,9 +57,7 @@ static struct KVMState {
     int fd;
     int vmfd;
     int coalesced_mmio;
-#ifdef KVM_CAP_COALESCED_MMIO
     struct kvm_coalesced_mmio_ring *coalesced_mmio_ring;
-#endif
     int broken_set_mem_region;
     int migration_log;
     int vcpu_events;
@@ -214,12 +212,10 @@ int kvm_init_vcpu(CPUState *env)
         goto err;
     }
 
-#ifdef KVM_CAP_COALESCED_MMIO
     if (kvm_state.coalesced_mmio && !kvm_state.coalesced_mmio_ring) {
         kvm_state.coalesced_mmio_ring =
             (void *)env->kvm_run + kvm_state.coalesced_mmio * PAGE_SIZE;
     }
-#endif
 
     ret = kvm_arch_init_vcpu(env);
     if (ret == 0) {
@@ -386,7 +382,6 @@ int kvm_coalesce_mmio_region(target_phys_addr_t start, ram_addr_t size)
 {
     int ret = -ENOSYS;
 
-#ifdef KVM_CAP_COALESCED_MMIO
     if (kvm_state.coalesced_mmio) {
         struct kvm_coalesced_mmio_zone zone;
 
@@ -395,7 +390,6 @@ int kvm_coalesce_mmio_region(target_phys_addr_t start, ram_addr_t size)
 
         ret = kvm_vm_ioctl(KVM_REGISTER_COALESCED_MMIO, &zone);
     }
-#endif
 
     return ret;
 }
@@ -404,7 +398,6 @@ int kvm_uncoalesce_mmio_region(target_phys_addr_t start, ram_addr_t size)
 {
     int ret = -ENOSYS;
 
-#ifdef KVM_CAP_COALESCED_MMIO
     if (kvm_state.coalesced_mmio) {
         struct kvm_coalesced_mmio_zone zone;
 
@@ -413,7 +406,6 @@ int kvm_uncoalesce_mmio_region(target_phys_addr_t start, ram_addr_t size)
 
         ret = kvm_vm_ioctl(KVM_UNREGISTER_COALESCED_MMIO, &zone);
     }
-#endif
 
     return ret;
 }
@@ -654,9 +646,7 @@ int kvm_init(void)
         goto err;
     }
 
-#ifdef KVM_CAP_COALESCED_MMIO
     kvm_state.coalesced_mmio = kvm_check_extension(KVM_CAP_COALESCED_MMIO);
-#endif
 
     kvm_state.broken_set_mem_region = 1;
 #ifdef KVM_CAP_JOIN_MEMORY_REGIONS_WORKS
@@ -777,7 +767,6 @@ static int kvm_handle_internal_error(CPUState *env, struct kvm_run *run)
 
 void kvm_flush_coalesced_mmio_buffer(void)
 {
-#ifdef KVM_CAP_COALESCED_MMIO
     if (kvm_state.coalesced_mmio_ring) {
         struct kvm_coalesced_mmio_ring *ring = kvm_state.coalesced_mmio_ring;
         while (ring->first != ring->last) {
@@ -790,7 +779,6 @@ void kvm_flush_coalesced_mmio_buffer(void)
             ring->first = (ring->first + 1) % KVM_COALESCED_MMIO_MAX;
         }
     }
-#endif
 }
 
 static void do_kvm_cpu_synchronize_state(void *_env)
@@ -988,11 +976,7 @@ int kvm_vcpu_ioctl(CPUState *env, int type, ...)
 
 int kvm_has_sync_mmu(void)
 {
-#ifdef KVM_CAP_SYNC_MMU
     return kvm_check_extension(KVM_CAP_SYNC_MMU);
-#else
-    return 0;
-#endif
 }
 
 int kvm_has_vcpu_events(void)
target-i386/kvm.c +0 -22
index caca407..b4a3463 100644
@@ -59,8 +59,6 @@ static bool has_msr_star;
 static bool has_msr_hsave_pa;
 static int lm_capable_kernel;
 
-#ifdef KVM_CAP_EXT_CPUID
-
 static struct kvm_cpuid2 *try_get_cpuid(int max)
 {
     struct kvm_cpuid2 *cpuid;
@@ -94,10 +92,6 @@ uint32_t kvm_x86_get_supported_cpuid(uint32_t function, uint32_t index,
     uint32_t ret = 0;
     uint32_t cpuid_1_edx;
 
-    if (!kvm_check_extension(KVM_CAP_EXT_CPUID)) {
-        return -1U;
-    }
-
     max = 1;
     while ((cpuid = try_get_cpuid(max)) == NULL) {
         max *= 2;
@@ -141,30 +135,14 @@ uint32_t kvm_x86_get_supported_cpuid(uint32_t function, uint32_t index,
     return ret;
 }
 
-#else
-
-uint32_t kvm_x86_get_supported_cpuid(uint32_t function, uint32_t index,
-                                     int reg)
-{
-    return -1U;
-}
-
-#endif
-
 #ifdef CONFIG_KVM_PARA
 struct kvm_para_features {
     int cap;
     int feature;
 } para_features[] = {
-#ifdef KVM_CAP_CLOCKSOURCE
     { KVM_CAP_CLOCKSOURCE, KVM_FEATURE_CLOCKSOURCE },
-#endif
-#ifdef KVM_CAP_NOP_IO_DELAY
     { KVM_CAP_NOP_IO_DELAY, KVM_FEATURE_NOP_IO_DELAY },
-#endif
-#ifdef KVM_CAP_PV_MMU
     { KVM_CAP_PV_MMU, KVM_FEATURE_MMU_OP },
-#endif
 #ifdef KVM_CAP_ASYNC_PF
     { KVM_CAP_ASYNC_PF, KVM_FEATURE_ASYNC_PF },
 #endif
Signature · 1 line
1.7.1
#34 lore raw

Re: [PATCH v2 17/17] kvm: Drop dependencies on very old capabilities

Quoted 6 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

COALESCED_MMIO, SYNC_MMU, EXT_CPUID, CLOCKSOURCE, NOP_IO_DELAY, PV_MMU - all these caps predate features on which we already depend at build time. Moreover, the check for KVM_CAP_EXT_CPUID is unneeded as we already test& fail is a more recent feature is missing.

No. Each test documents a dependency of qemu on a kvm feature. Even though something like SYNC_MMU is unlikely to go away, as long as we depend on it, we require the feature.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#35 lore raw

Re: [PATCH v2 17/17] kvm: Drop dependencies on very old capabilities

Quoted 12 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

COALESCED_MMIO, SYNC_MMU, EXT_CPUID, CLOCKSOURCE, NOP_IO_DELAY, PV_MMU - all these caps predate features on which we already depend at build time. Moreover, the check for KVM_CAP_EXT_CPUID is unneeded as we already test& fail is a more recent feature is missing.

No. Each test documents a dependency of qemu on a kvm feature. Even though something like SYNC_MMU is unlikely to go away, as long as we depend on it, we require the feature.

Then at least move all those KVM_CAPs we need at build time into configure. I really see no value in keeping ugly conditional code around, A) because those paths won't be tested and B) none of the CAPs touched here are to pass away without a replacement that will require user space adaption anyway.

Jan

#36 lore raw

Re: [PATCH v2 17/17] kvm: Drop dependencies on very old capabilities

Quoted 9 lines · Jan Kiszka
Quoted 8 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

COALESCED_MMIO, SYNC_MMU, EXT_CPUID, CLOCKSOURCE, NOP_IO_DELAY, PV_MMU - all these caps predate features on which we already depend at build time. Moreover, the check for KVM_CAP_EXT_CPUID is unneeded as we
already test& fail is a more recent feature is missing.

Quoted 7 lines
Quoted 4 lines

No. Each test documents a dependency of qemu on a kvm feature. Even though something like SYNC_MMU is unlikely to go away, as long as we depend on it, we require the feature.

Then at least move all those KVM_CAPs we need at build time into configure.

Need a run time check as well (build on new kernel, run on old kernel, or run on even newer kernel that lost a feature).

Quoted 4 lines

I really see no value in keeping ugly conditional code around, A) because those paths won't be tested and B) none of the CAPs touched here are to pass away without a replacement that will require user space adaption anyway.

I'm fine with a series of checks during init time with no fallback. I'm not fine with just dropping those away. Reducing code size is great, but not at the cost of undiagnosed runtime failures.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#37 lore raw

Re: [PATCH v2 17/17] kvm: Drop dependencies on very old capabilities

Quoted 22 lines · Avi Kivity
Quoted 10 lines · Jan Kiszka
Quoted 4 lines · Avi Kivity
Quoted 3 lines · Jan Kiszka

From: Jan Kiszka<[email protected]>

COALESCED_MMIO, SYNC_MMU, EXT_CPUID, CLOCKSOURCE, NOP_IO_DELAY,

PV_MMU -

Quoted 4 lines
Quoted 3 lines

all these caps predate features on which we already depend at build time. Moreover, the check for KVM_CAP_EXT_CPUID is unneeded as we
already test& fail is a more recent feature is missing.

Quoted 7 lines
Quoted 4 lines

No. Each test documents a dependency of qemu on a kvm feature. Even though something like SYNC_MMU is unlikely to go away, as long as we depend on it, we require the feature.

Then at least move all those KVM_CAPs we need at build time into configure.

Need a run time check as well (build on new kernel, run on old kernel, or run on even newer kernel that lost a feature).

Quoted 8 lines
Quoted 4 lines

I really see no value in keeping ugly conditional code around, A) because those paths won't be tested and B) none of the CAPs touched here are to pass away without a replacement that will require user space adaption anyway.

I'm fine with a series of checks during init time with no fallback. I'm not fine with just dropping those away. Reducing code size is great, but not at the cost of undiagnosed runtime failures.

My worry was not code size but untested code. And looking at the EXT_CPUID case again, this is what would happen:
kvm_x86_get_supported_cpuid will return -1 if we lack EXT_CPUID, but that value is interpreted by the callers as "all requested features available" - likely not that helpful in all cases. This also affects qemu-kvm.

I will add generic and per-arch check sections at build and runtime for CAPs we don't want to miss.

Jan

author
#38 lore raw

[PATCH 18/17] kvm: Flush coalesced mmio buffer on IO window exits

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

We must flush pending mmio writes if we leave kvm_cpu_exec for an IO window. Otherwise we risk to loose those requests when migrating to a different host during that window.

---
 kvm-all.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
kvm-all.c +2 -2
index 15d5f32..766a3a7 100644
@@ -844,6 +844,8 @@ void kvm_cpu_exec(CPUState *env)
         cpu_single_env = env;
         kvm_arch_post_run(env, run);
 
+        kvm_flush_coalesced_mmio_buffer();
+
         if (ret == -EINTR || ret == -EAGAIN) {
             cpu_exit(env);
             DPRINTF("io window exit\n");
@@ -856,8 +858,6 @@ void kvm_cpu_exec(CPUState *env)
             abort();
         }
 
-        kvm_flush_coalesced_mmio_buffer();
-
         ret = 0; /* exit loop */
         switch (run->exit_reason) {
         case KVM_EXIT_IO:
#39 lore raw
In reply to Jan Kiszka
Quoted 6 lines · Jan Kiszka

All previously sent patches against current uq combined, some smaller tweaks applied, and 4 new patches added. Major news is support for using up to 16M BIOSes and optional code dump for cpu_dump_state. Those features are already present qemu-kvm but were reworked to provide cleaner upstream versions.

Looks reasonable overall.

Quoted 5 lines

Based on this series, I've an experimental tree here where I eliminated another 1500 LOC from qemu-kvm code. Specifically, that tree sets an end to duplicate KVM and VCPU initialization functions, KVMState copies, and redundant state saving/loading functions. Will be rolled out after some more review and testing.

Sounds really frightening... this glue code is a real breeding ground for subtle bugs and merge problems.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#40 lore raw
Quoted 10 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka

All previously sent patches against current uq combined, some smaller tweaks applied, and 4 new patches added. Major news is support for using up to 16M BIOSes and optional code dump for cpu_dump_state. Those features are already present qemu-kvm but were reworked to provide cleaner upstream versions.

Looks reasonable overall.

Quoted 8 lines
Quoted 5 lines

Based on this series, I've an experimental tree here where I eliminated another 1500 LOC from qemu-kvm code. Specifically, that tree sets an end to duplicate KVM and VCPU initialization functions, KVMState copies, and redundant state saving/loading functions. Will be rolled out after some more review and testing.

Sounds really frightening... this glue code is a real breeding ground for subtle bugs and merge problems.

...and that's why we are more than late with unifying it into a single code base.

After that upcoming cleanup round we will have three major remaining to-dos:
- in-kernel irqchip and PIT support with the subtopics
- VAPIC/TPR-patching support
- IRQ routing
- PCI device assignment
- VCPU & IO-thread execution model

While I'm not that concerned about the first two as they basically add code that "just" needs to be refactored, the last one is worrying me. We need to find a smart way to morph one into the other. The code is full of subtle differences and is likely quite performance sensitive.

Jan

#41 lore raw
Quoted 14 lines · Jan Kiszka
Quoted 3 lines

Sounds really frightening... this glue code is a real breeding ground for subtle bugs and merge problems.

...and that's why we are more than late with unifying it into a single code base.

After that upcoming cleanup round we will have three major remaining to-dos:
- in-kernel irqchip and PIT support with the subtopics
- VAPIC/TPR-patching support
- IRQ routing
- PCI device assignment
- VCPU& IO-thread execution model

Quoted 4 lines

While I'm not that concerned about the first two as they basically add code that "just" needs to be refactored, the last one is worrying me. We need to find a smart way to morph one into the other. The code is full of subtle differences and is likely quite performance sensitive.

Agreed on all counts.

Signature · 1 line
error compiling committee.c: too many arguments to function