Skip to contentSkip to conversation

Update halted state from mpstate only in case of inkernel irq chip

PATCH Applied Gleb Natapov 4 participants 17 messages Sep 22, 2009 – Sep 25, 2009 KVM

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Update halted state from mpstate only in case of inkernel irq chip

Otherwise cpu is always unhalted by call to kvm_arch_get_registers()

---
 qemu-kvm.c |   10 ++++++++++
 qemu-kvm.h |    9 +--------
 2 files changed, 11 insertions(+), 8 deletions(-)
qemu-kvm.c +10 -0
index 6511cb6..2a7fe3d 100644
@@ -1605,6 +1605,16 @@ static void on_vcpu(CPUState *env, void (*func)(void *data), void *data)
         qemu_cond_wait(&qemu_work_cond);
 }
 
+void kvm_arch_get_registers(CPUState *env)
+{
+	kvm_arch_save_regs(env);
+	kvm_arch_save_mpstate(env);
+#ifdef KVM_CAP_MP_STATE
+	if (kvm_irqchip_in_kernel(kvm_context))
+		env->halted = (env->mp_state == KVM_MP_STATE_HALTED);
+#endif
+}
+
 static void do_kvm_cpu_synchronize_state(void *_env)
 {
     CPUState *env = _env;
qemu-kvm.h +1 -8
index b2c8c35..4523e25 100644
@@ -1181,14 +1181,7 @@ static inline int kvm_sync_vcpus(void)
 }
 
 #ifndef QEMU_KVM_NO_CPU
-static inline void kvm_arch_get_registers(CPUState *env)
-{
-    kvm_arch_save_regs(env);
-    kvm_arch_save_mpstate(env);
-#ifdef KVM_CAP_MP_STATE
-    env->halted = (env->mp_state == KVM_MP_STATE_HALTED);
-#endif
-}
+void kvm_arch_get_registers(CPUState *env);
 
 static inline void kvm_arch_put_registers(CPUState *env)
 {
Signature · 1 line
1.6.3.3
#1 lore raw

PATCH 1/1 Don't call kvm_cpu_synchronize_state() if there is no irqchip events to process.

---
 qemu-kvm-x86.c |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)
qemu-kvm-x86.c +6 -3
index 32561f0..acb1b91 100644
@@ -1673,9 +1673,12 @@ uint32_t kvm_arch_get_supported_cpuid(CPUState *env, uint32_t function,
 
 void kvm_arch_process_irqchip_events(CPUState *env)
 {
-    kvm_cpu_synchronize_state(env);
-    if (env->interrupt_request & CPU_INTERRUPT_INIT)
+    if (env->interrupt_request & CPU_INTERRUPT_INIT) {
+        kvm_cpu_synchronize_state(env);
         do_cpu_init(env);
-    if (env->interrupt_request & CPU_INTERRUPT_SIPI)
+    }
+    if (env->interrupt_request & CPU_INTERRUPT_SIPI) {
+        kvm_cpu_synchronize_state(env);
         do_cpu_sipi(env);
+    }
 }
Signature · 1 line
1.6.3.3
Applied
#2 lore raw

Re: [PATCH] Don't call kvm_cpu_synchronize_state() if there is no irqchip events to process.

Quoted 2 lines · Gleb Natapov

Applied, thanks.

Signature · 1 line
error compiling committee.c: too many arguments to function
#3 lore raw

PATCH 1/1 Don't call cpu_synchronize_state() in apic_init_reset()

In reply to Gleb Natapov

Each caller of the function already calls it.

---
 hw/apic.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
hw/apic.c +0 -1
index 3a2e128..a9d1fb8 100644
@@ -488,7 +488,6 @@ void apic_init_reset(CPUState *env)
     if (!s)
         return;
 
-    cpu_synchronize_state(env);
     s->tpr = 0;
     s->spurious_vec = 0xff;
     s->log_dest = 0;
Signature · 1 line
1.6.3.3
#4 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 13 lines · Gleb Natapov

Each caller of the function already calls it.

---
  hw/apic.c |    1 -
  1 files changed, 0 insertions(+), 1 deletions(-)
hw/apic.c +0 -0
index 3a2e128..a9d1fb8 100644

hw/apic.c void apic_init_reset(CPUState *env) · line 493

Quoted 7 lines
hw/apic.c +0 -1
@@ -488,7 +488,6 @@ void apic_init_reset(CPUState *env)
      if (!s)
          return;
 
-    cpu_synchronize_state(env);
      s->tpr = 0;
      s->spurious_vec = 0xff;
      s->log_dest = 0;

Still, it's safer to live this in.

Signature · 1 line
error compiling committee.c: too many arguments to function
#5 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 16 lines · Avi Kivity
Quoted 15 lines · Gleb Natapov

Each caller of the function already calls it.

---
  hw/apic.c |    1 -
  1 files changed, 0 insertions(+), 1 deletions(-)
hw/apic.c +0 -0
index 3a2e128..a9d1fb8 100644
@@ -488,7 +488,6 @@ void apic_init_reset(CPUState *env)
      if (!s)
          return;
 
Quoted 8 lines

hw/apic.c void apic_init_reset(CPUState *env) · line 493

Quoted 5 lines
hw/apic.c +0 -1
-    cpu_synchronize_state(env);
      s->tpr = 0;
      s->spurious_vec = 0xff;
      s->log_dest = 0;

Still, it's safer to live this in.

Yet another diff to upstream...

It's really time to stabilize this still a bit fuzzy register sync model, also in qemu-kvm. If you think we need it, let us push it upstream - with a sound explanation, and maybe even with more sync points for the sake of consistency.

Jan

#6 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 28 lines · Jan Kiszka
Quoted 17 lines · Avi Kivity
Quoted 15 lines · Gleb Natapov

Each caller of the function already calls it.

---
   hw/apic.c |    1 -
   1 files changed, 0 insertions(+), 1 deletions(-)
hw/apic.c +0 -0
index 3a2e128..a9d1fb8 100644
@@ -488,7 +488,6 @@ void apic_init_reset(CPUState *env)
       if (!s)
           return;
 
Quoted 9 lines

hw/apic.c void apic_init_reset(CPUState *env) · line 493

Quoted 6 lines
hw/apic.c +0 -1
-    cpu_synchronize_state(env);
       s->tpr = 0;
       s->spurious_vec = 0xff;
       s->log_dest = 0;

Still, it's safer to live this in.

Quoted 8 lines

Yet another diff to upstream...

It's really time to stabilize this still a bit fuzzy register sync model, also in qemu-kvm. If you think we need it, let us push it upstream - with a sound explanation, and maybe even with more sync points for the sake of consistency.

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

Signature · 1 line
error compiling committee.c: too many arguments to function
#7 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 38 lines · Avi Kivity
Quoted 28 lines · Jan Kiszka
Quoted 17 lines · Avi Kivity
Quoted 15 lines · Gleb Natapov

Each caller of the function already calls it.

---
   hw/apic.c |    1 -
   1 files changed, 0 insertions(+), 1 deletions(-)
hw/apic.c +0 -0
index 3a2e128..a9d1fb8 100644
@@ -488,7 +488,6 @@ void apic_init_reset(CPUState *env)
       if (!s)
           return;
 
Quoted 9 lines

hw/apic.c void apic_init_reset(CPUState *env) · line 493

Quoted 6 lines
hw/apic.c +0 -1
-    cpu_synchronize_state(env);
       s->tpr = 0;
       s->spurious_vec = 0xff;
       s->log_dest = 0;

Still, it's safer to live this in.

Quoted 8 lines

Yet another diff to upstream...

It's really time to stabilize this still a bit fuzzy register sync model, also in qemu-kvm. If you think we need it, let us push it upstream - with a sound explanation, and maybe even with more sync points for the sake of consistency.

Quoted 8 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space. But, e.g., I do not see where we do this on CPU reset.

Jan

#8 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 9 lines
Quoted 4 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

Quoted 3 lines

But,
e.g., I do not see where we do this on CPU reset.

That's a bug.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#9 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 14 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 7 lines
Quoted 3 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

Quoted 9 lines

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

Quoted 5 lines
Quoted 2 lines

But,
e.g., I do not see where we do this on CPU reset.

That's a bug.

Only if kvm support cpus without apic. Otherwise CPU is reset by apic_reset() and cpu_synchronize_state() is called there.

--
Gleb.

#10 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 26 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 7 lines
Quoted 3 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

Quoted 8 lines

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

But,
e.g., I do not see where we do this on CPU reset.

That's a bug.

Only if kvm support cpus without apic. Otherwise CPU is reset by apic_reset() and cpu_synchronize_state() is called there.

No, that's not enough if cpu_reset() first fiddles with some registers that may later on be overwritten on cpu_synchronize_state() with the old in-kernel state. At least in theory, haven't checked yet what happens in reality. That's why not synchronizing properly is "expensive" (or broken IOW).

Jan

#11 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 29 lines · Jan Kiszka
Quoted 28 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 7 lines
Quoted 3 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

Quoted 8 lines

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

Quoted 2 lines
Quoted 2 lines

But,
e.g., I do not see where we do this on CPU reset.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

That's a bug.

Only if kvm support cpus without apic. Otherwise CPU is reset by apic_reset() and cpu_synchronize_state() is called there.

No, that's not enough if cpu_reset() first fiddles with some registers that may later on be overwritten on cpu_synchronize_state() with the old in-kernel state. At least in theory, haven't checked yet what happens in

Can't happen. Call chain is apic_reset() -> cpu_reset() and apic_reset() calls cpu_synchronize_state() before calling cpu_reset().

Quoted 5 lines

reality. That's why not synchronizing properly is "expensive" (or broken IOW).

Jan

--
Gleb.

#12 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 32 lines · Gleb Natapov
Quoted 31 lines · Jan Kiszka
Quoted 30 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 7 lines
Quoted 3 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

Quoted 8 lines

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

Quoted 4 lines
Quoted 2 lines

But,
e.g., I do not see where we do this on CPU reset.

That's a bug.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Only if kvm support cpus without apic. Otherwise CPU is reset by apic_reset() and cpu_synchronize_state() is called there.

No, that's not enough if cpu_reset() first fiddles with some registers that may later on be overwritten on cpu_synchronize_state() with the old in-kernel state. At least in theory, haven't checked yet what happens in

Can't happen. Call chain is apic_reset() -> cpu_reset() and apic_reset() calls cpu_synchronize_state() before calling cpu_reset().

And system_reset?

Jan

#13 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 35 lines · Jan Kiszka
Quoted 34 lines · Gleb Natapov
Quoted 33 lines · Jan Kiszka
Quoted 32 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 7 lines
Quoted 3 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

Quoted 8 lines

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

Quoted 4 lines
Quoted 2 lines

But,
e.g., I do not see where we do this on CPU reset.

That's a bug.

Only if kvm support cpus without apic. Otherwise CPU is reset by apic_reset() and cpu_synchronize_state() is called there.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

No, that's not enough if cpu_reset() first fiddles with some registers that may later on be overwritten on cpu_synchronize_state() with the old in-kernel state. At least in theory, haven't checked yet what happens in

Can't happen. Call chain is apic_reset() -> cpu_reset() and apic_reset() calls cpu_synchronize_state() before calling cpu_reset().

And system_reset?

And system_reset calls apic_reset() if cpu has apic, cpu_reset() otherwise. That is why I said that the bug is only for cpus without apic.

--
Gleb.

#14 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

Quoted 39 lines · Gleb Natapov
Quoted 38 lines · Jan Kiszka
Quoted 37 lines · Gleb Natapov
Quoted 36 lines · Jan Kiszka
Quoted 26 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

Functions calling each other in the same subsystem can rely on callers calling cpu_synchronize_state(). Across subsystems, that's another matter, exported functions should try not to rely on implementation details of their callers.

Quoted 7 lines
Quoted 3 lines

(You might argue that the apic is not separate subsystem wrt an x86 cpu, and I'm not sure I have a counterargument)

I do accept this argument. It's just that my feeling is that we are lacking proper review of the required call sites of cpu_sychronize_state and rather put it where some regression popped up (and that only in qemu-kvm...).

Quoted 8 lines

That's life...

Quoted 2 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

But,
e.g., I do not see where we do this on CPU reset.

That's a bug.

Only if kvm support cpus without apic. Otherwise CPU is reset by apic_reset() and cpu_synchronize_state() is called there.

No, that's not enough if cpu_reset() first fiddles with some registers that may later on be overwritten on cpu_synchronize_state() with the old in-kernel state. At least in theory, haven't checked yet what happens in

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Can't happen. Call chain is apic_reset() -> cpu_reset() and apic_reset() calls cpu_synchronize_state() before calling cpu_reset().

And system_reset?

And system_reset calls apic_reset() if cpu has apic, cpu_reset() otherwise. That is why I said that the bug is only for cpus without apic.

Yes, I see now. Still, it's not a good reference for other archs (did anyone already checked the situation on ppc?). Will file a patch.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#15 lore raw

Re: [PATCH] Don't call cpu_synchronize_state() in apic_init_reset()

In reply to Gleb Natapov
Quoted 4 lines · Gleb Natapov
Quoted 4 lines
Quoted 4 lines

The new rule is: Synchronize the states before accessing registers (or in-kernel devices) the first time after a vmexit to user space.

Quoted 9 lines
Quoted 5 lines

No, the rule is: synchronize state before accessing registers. Extra synchronization is cheap, while missing synchronization is very expensive.

So should we stick cpu_synchronize_state() before each register accesses? I think it is reasonable to omit it if all callers do it already.

If the callee is static we can and should avoid it. If the function is exported then we shouldn't rely on callers.

IOW, it's fine to depend on local details (which a reader can easily gain), but better to avoid depending on global details.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Applied
#16 lore raw
In reply to Gleb Natapov
Quoted 2 lines · Gleb Natapov

Otherwise cpu is always unhalted by call to kvm_arch_get_registers()

Applied, thanks.

Signature · 1 line
error compiling committee.c: too many arguments to function