s390x: TOD refactoring + TCG CPU hotplug support

Let's treat this like a separate device. TCG will have to store the actual state/time later on.

Include cpu-qom.h in kvm_s390x.h (due to S390CPU) to compile tod-kvm.c.

Signed-off-by David Hildenbrand <[email protected]>

---

hw/s390x/tod.c new file +14 -0

+static TypeInfo s390_tod_info = {
+    .name = TYPE_S390_TOD,
+    .parent = TYPE_DEVICE,
+    .instance_size = sizeof(S390TODState),
+    .class_init = s390_tod_class_init,
+    .class_size = sizeof(S390TODClass),
+    .abstract = true,
+};
+
+static void register_types(void)
+{
+    type_register_static(&s390_tod_info);
+}
+type_init(register_types);

include/hw/s390x/tod.h new file +18 -0

new file mode 100644
index 0000000000..43ed71600f
@@ -0,0 +1,46 @@
+/*
+ * TOD (Time Of Day) clock
+ *
+ * Copyright 2018 Red Hat, Inc.
+ * Author(s): David Hildenbrand <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef HW_S390_TOD_H
+#define HW_S390_TOD_H
+
+#include "hw/qdev.h"
+
+typedef struct S390TOD {
+    uint8_t high;
+    uint64_t low;

Quoted 7 lines · David Hildenbrand

Let's treat this like a separate device. TCG will have to store the actual state/time later on.

Include cpu-qom.h in kvm_s390x.h (due to S390CPU) to compile tod-kvm.c.

Signed-off-by David Hildenbrand <[email protected]>

---

[...]

hw/s390x/tod.c line 105

Quoted 12 lines

hw/s390x/tod.c new file +12 -0

+static void s390_tod_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+
+    dc->desc = "TOD (Time Of Day) Clock";
+    dc->realize = s390_tod_realize;
+    set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+
+    /* We only have one TOD clock in the system attached to the machine */
+    dc->user_creatable = false;
+    dc->hotpluggable = false;
+}

IIRC you don't need the hotpluggable = false if you already set user_creatable = false.

Quoted 39 lines

hw/s390x/tod.c new file +14 -0

+static TypeInfo s390_tod_info = {
+    .name = TYPE_S390_TOD,
+    .parent = TYPE_DEVICE,
+    .instance_size = sizeof(S390TODState),
+    .class_init = s390_tod_class_init,
+    .class_size = sizeof(S390TODClass),
+    .abstract = true,
+};
+
+static void register_types(void)
+{
+    type_register_static(&s390_tod_info);
+}
+type_init(register_types);

include/hw/s390x/tod.h new file +18 -0

new file mode 100644
index 0000000000..43ed71600f
@@ -0,0 +1,46 @@
+/*
+ * TOD (Time Of Day) clock
+ *
+ * Copyright 2018 Red Hat, Inc.
+ * Author(s): David Hildenbrand <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef HW_S390_TOD_H
+#define HW_S390_TOD_H
+
+#include "hw/qdev.h"
+
+typedef struct S390TOD {
+    uint8_t high;
+    uint64_t low;

include/hw/s390x/tod.h line 28

Quoted 10 lines

include/hw/s390x/tod.h new file +10 -0

+} S390TOD;
+
+#define TYPE_S390_TOD "s390-tod"
+#define S390_TOD(obj) OBJECT_CHECK(S390TODState, (obj), TYPE_S390_TOD)
+#define S390_TOD_CLASS(oc) OBJECT_CLASS_CHECK(S390TODClass, (oc), \
+                                              TYPE_S390_TOD)
+#define S390_TOD_GET_CLASS(obj) OBJECT_GET_CLASS(S390TODClass, (obj), \
+                                                 TYPE_S390_TOD)
+#define TYPE_KVM_S390_TOD "kvm-" TYPE_S390_TOD
+#define TYPE_QEMU_S390_TOD "qemu-" TYPE_S390_TOD

I'd prefer if you could turn that into suffixes instead of prefixes, like it is done in include/hw/s390x/storage-keys.h and storage-attributes.h already.

Apart from that, the patch looks fine to me, so once you've changed that, feel free to add:

Reviewed-by Thomas Huth <[email protected]>

Never set to anything but 0.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/cpu.c         | 1 -
 target/s390x/cpu.h         | 1 -
 target/s390x/misc_helper.c | 4 ++--
 3 files changed, 2 insertions(+), 4 deletions(-)

include/hw/s390x/tod.h typedef struct S390TODClass { · line 60

Quoted 11 lines · David Hildenbrand

include/hw/s390x/tod.h +11 -0

+/* Converts ns to s390's clock format */
+static inline uint64_t time2tod(uint64_t ns)
+{
+    return (ns << 9) / 125;
+}
+
+/* Converts s390's clock format to ns */
+static inline uint64_t tod2time(uint64_t t)
+{
+    return (t * 125) >> 9;
+}

How many significant bits on input here?

Do you in fact want to be using muldiv64?

r~

Quoted 13 lines · Richard Henderson

include/hw/s390x/tod.h typedef struct S390TODClass { · line 60

Quoted 11 lines · David Hildenbrand

include/hw/s390x/tod.h +11 -0

+/* Converts ns to s390's clock format */
+static inline uint64_t time2tod(uint64_t ns)
+{
+    return (ns << 9) / 125;
+}
+
+/* Converts s390's clock format to ns */
+static inline uint64_t tod2time(uint64_t t)
+{
+    return (t * 125) >> 9;
+}

In this patch I'm only moving the code. If we find this is a problem, this should go into a separate patch.

Quoted 1 line

How many significant bits on input here?

Basically all are significant, and as it is a clock, we will reach these bits at one point.

Quoted 1 line

Do you in fact want to be using muldiv64?

Looking at linux:

arch/s390/include/asm/timex.h

They have a lengthy documentation, resulting in (a spli to avoid overflows)

return ((todval >> 9) * 125) + (((todval & 0x1ff) * 125) >> 9);

Maybe we should do the same?

Right now, each CPU has its own TOD. Especially, the TOD will differ based on creation time of a CPU - e.g. when hotplugging a CPU the times will differ quite a lot, resulting in stall warnings in the guest.

Let's use a single TOD by implementing our new TOD device. Prepare it for TOD-clock epoch extension.

Most importantly, whenever we set the TOD, we have to update the CKC timer.

Introduce "tcg_s390x.h" just like "kvm_s390x.h" for tcg specific function declarations that should not go into cpu.h.

hw/s390x/tod.c +0 -0

index c1b3438452..7495d8c057 100644
@@ -30,6 +30,17 @@ void s390_init_tod(void)
     qdev_init_nofail(DEVICE(obj));
 }
 

Quoted 12 lines · David Hildenbrand

Right now, each CPU has its own TOD. Especially, the TOD will differ based on creation time of a CPU - e.g. when hotplugging a CPU the times will differ quite a lot, resulting in stall warnings in the guest.

Let's use a single TOD by implementing our new TOD device. Prepare it for TOD-clock epoch extension.

Most importantly, whenever we set the TOD, we have to update the CKC timer.

Introduce "tcg_s390x.h" just like "kvm_s390x.h" for tcg specific function declarations that should not go into cpu.h.

[...]

Quoted 8 lines

hw/s390x/tod.c +0 -0

index c1b3438452..7495d8c057 100644
@@ -30,6 +30,17 @@ void s390_init_tod(void)
     qdev_init_nofail(DEVICE(obj));
 }
 

hw/s390x/tod.c void s390_init_tod(void) · line 42

Quoted 10 lines

hw/s390x/tod.c +10 -0

+S390TODState *s390_get_tod(void)
+{
+    static S390TODState *ts;
+
+    if (!ts) {
+        ts = S390_TOD(object_resolve_path_type("", TYPE_S390_TOD, NULL));
+    }
+
+    return ts;
+}

Could you please call that function s390_get_todstate or something similar? Otherwise it sounds like it could be used to retrieve the time, and not the state object.

Thomas

Let's stop the timer and delete any pending CKC IRQ before doing anything else.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/misc_helper.c | 6 ++++++
 1 file changed, 6 insertions(+)

target/s390x/misc_helper.c +0 -0

index 6ccbe1fe9a..d5f9f5e1d3 100644

target/s390x/misc_helper.c +0 -0

         return;
     }

Quoted 13 lines · David Hildenbrand

Let's stop the timer and delete any pending CKC IRQ before doing anything else.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/misc_helper.c | 6 ++++++
 1 file changed, 6 insertions(+)

target/s390x/misc_helper.c +0 -0

index 6ccbe1fe9a..d5f9f5e1d3 100644

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 165

Quoted 8 lines

target/s390x/misc_helper.c +5 -0

@@ -158,6 +158,12 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     Error *err = NULL;
     S390TOD tod_base;
 
+    /* stop the timer and remove pending CKC IRQs */
+    timer_del(env->tod_timer);
+    qemu_mutex_lock_iothread();
+    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
+    qemu_mutex_unlock_iothread();

Reviewed-by Thomas Huth <[email protected]>

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 167

Quoted 1 line

target/s390x/misc_helper.c +0 -0

     if (time == -1ULL) {

I wonder whether that check is still adequate? Is there really a way to disable the clock comparator like this? At least I haven't seen it in the PoP.

Quoted 4 lines

target/s390x/misc_helper.c +0 -0

         return;
     }

Quoted 1 line · Thomas Huth

On 20.06.2018 12:08, David Hildenbrand wrote:

Quoted 5 lines

Quoted 1 line

     if (time == -1ULL) {

I wonder whether that check is still adequate? Is there really a way to disable the clock comparator like this? At least I haven't seen it in the PoP.

e.g. 4-61 (Control)

4. When the clock-comparator sign control is zero,
(a) the program can set the clock comparator to all zeros to ensure that an interruption condition is immediately present, and (b) the program can set the clock comparator to a value of all binary ones to ensure that a clock-comparator interrup- tion is never recognized. [...]

Rational from 4-59 (Control):

The clock comparator provides a means of causing an interruption when the TOD-clock value exceeds a value specified by the program.

We can never exceed all binary 1s. So it is really exceeding, not "hitting"

Quoted 25 lines · Thomas Huth

Quoted 13 lines · David Hildenbrand

Let's stop the timer and delete any pending CKC IRQ before doing anything else.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/misc_helper.c | 6 ++++++
 1 file changed, 6 insertions(+)

target/s390x/misc_helper.c +0 -0

index 6ccbe1fe9a..d5f9f5e1d3 100644

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 165

Quoted 8 lines

target/s390x/misc_helper.c +5 -0

@@ -158,6 +158,12 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     Error *err = NULL;
     S390TOD tod_base;
 
+    /* stop the timer and remove pending CKC IRQs */
+    timer_del(env->tod_timer);
+    qemu_mutex_lock_iothread();
+    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
+    qemu_mutex_unlock_iothread();  

Reviewed-by Thomas Huth <[email protected]>

Quoted 5 lines

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 167

Quoted 1 line

target/s390x/misc_helper.c +0 -0

     if (time == -1ULL) {  

I wonder whether that check is still adequate? Is there really a way to disable the clock comparator like this? At least I haven't seen it in the PoP.

e.g. 4-61 (Control)

4. When the clock-comparator sign control is zero,
(a) the program can set the clock comparator to all zeros to ensure that an interruption condition is immediately present, and (b) the program can set the clock comparator to a value of all binary ones to ensure that a clock-comparator interrup- tion is never recognized. [...]

Rational from 4-59 (Control):

The clock comparator provides a means of causing an interruption when the TOD-clock value exceeds a value specified by the program.

We can never exceed all binary 1s. So it is really exceeding, not "hitting"

Quoted 13 lines · David Hildenbrand

Let's stop the timer and delete any pending CKC IRQ before doing anything else.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/misc_helper.c | 6 ++++++
 1 file changed, 6 insertions(+)

target/s390x/misc_helper.c +0 -0

index 6ccbe1fe9a..d5f9f5e1d3 100644

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 165

Quoted 8 lines

target/s390x/misc_helper.c +5 -0

@@ -158,6 +158,12 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     Error *err = NULL;
     S390TOD tod_base;
 
+    /* stop the timer and remove pending CKC IRQs */
+    timer_del(env->tod_timer);
+    qemu_mutex_lock_iothread();
+    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
+    qemu_mutex_unlock_iothread();  

Reviewed-by Thomas Huth <[email protected]>

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 167

Quoted 1 line

target/s390x/misc_helper.c +0 -0

     if (time == -1ULL) {  

I wonder whether that check is still adequate? Is there really a way to disable the clock comparator like this? At least I haven't seen it in the PoP.

Quoted 4 lines

target/s390x/misc_helper.c +0 -0

         return;
     }

Quoted 25 lines · On Thu, 21 Jun 2018 14:09:28 +0200 Thomas Huth <[email protected]> wrote:

Quoted 13 lines · David Hildenbrand

Let's stop the timer and delete any pending CKC IRQ before doing anything else.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/misc_helper.c | 6 ++++++
 1 file changed, 6 insertions(+)

target/s390x/misc_helper.c +0 -0

index 6ccbe1fe9a..d5f9f5e1d3 100644

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 165

Quoted 8 lines

target/s390x/misc_helper.c +5 -0

@@ -158,6 +158,12 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     Error *err = NULL;
     S390TOD tod_base;
 
+    /* stop the timer and remove pending CKC IRQs */
+    timer_del(env->tod_timer);
+    qemu_mutex_lock_iothread();
+    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
+    qemu_mutex_unlock_iothread();  

Reviewed-by Thomas Huth <[email protected]>

Quoted 5 lines

target/s390x/misc_helper.c void HELPER(sckc)(CPUS390XState *env, uint64_t time) · line 167

Quoted 1 line

target/s390x/misc_helper.c +0 -0

     if (time == -1ULL) {  

I wonder whether that check is still adequate? Is there really a way to disable the clock comparator like this? At least I haven't seen it in the PoP.

Me neither. It seems to have been in the code since the beginning...

This allows a guest to change its TOD. We already take care of updating all CKC timers from within S390TODClass.

Use MO_ALIGN to load the operand manually - this will properly trigger a SPECIFICATION exception.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/helper.h      |  1 +
 target/s390x/insn-data.def |  3 +--
 target/s390x/misc_helper.c | 19 +++++++++++++++++++
 target/s390x/translate.c   |  9 +++++++++
 4 files changed, 30 insertions(+), 2 deletions(-)

target/s390x/helper.h +1 -0

index 59cba86a27..97c60ca7bc 100644
@@ -127,6 +127,7 @@ DEF_HELPER_4(diag, void, env, i32, i32, i32)
 DEF_HELPER_3(load_psw, noreturn, env, i64, i64)
 DEF_HELPER_FLAGS_2(spx, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_1(stck, TCG_CALL_NO_RWG_SE, i64, env)
+DEF_HELPER_FLAGS_2(sck, TCG_CALL_NO_RWG, i32, env, i64)
 DEF_HELPER_FLAGS_2(sckc, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_2(sckpf, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_1(stckc, TCG_CALL_NO_RWG, i64, env)

target/s390x/insn-data.def +1 -2

index 157619403d..5c6f33ed9c 100644
@@ -997,8 +997,7 @@
 /* SET ADDRESS SPACE CONTROL FAST */
     C(0xb279, SACF,    S,     Z,   0, a2, 0, 0, sacf, 0)
 /* SET CLOCK */
-    /* ??? Not implemented - is it necessary? */
-    C(0xb204, SCK,     S,     Z,   0, 0, 0, 0, 0, 0)
+    C(0xb204, SCK,     S,     Z,   la2, 0, 0, 0, sck, 0)
 /* SET CLOCK COMPARATOR */
     C(0xb206, SCKC,    S,     Z,   0, m2_64a, 0, 0, sckc, 0)
 /* SET CLOCK PROGRAMMABLE FIELD */

target/s390x/misc_helper.c +5 -0

index d5f9f5e1d3..c9712b9476 100644
@@ -191,6 +191,25 @@ void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque)
     helper_sckc(env, env->ckc);
 }
 
+/* Set Clock */
+uint32_t HELPER(sck)(CPUS390XState *env, uint64_t tod_low)
+{
+    S390TODState *td = s390_get_tod();
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);

Quoted 59 lines · David Hildenbrand

This allows a guest to change its TOD. We already take care of updating all CKC timers from within S390TODClass.

Use MO_ALIGN to load the operand manually - this will properly trigger a SPECIFICATION exception.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/helper.h      |  1 +
 target/s390x/insn-data.def |  3 +--
 target/s390x/misc_helper.c | 19 +++++++++++++++++++
 target/s390x/translate.c   |  9 +++++++++
 4 files changed, 30 insertions(+), 2 deletions(-)

target/s390x/helper.h +1 -0

index 59cba86a27..97c60ca7bc 100644
@@ -127,6 +127,7 @@ DEF_HELPER_4(diag, void, env, i32, i32, i32)
 DEF_HELPER_3(load_psw, noreturn, env, i64, i64)
 DEF_HELPER_FLAGS_2(spx, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_1(stck, TCG_CALL_NO_RWG_SE, i64, env)
+DEF_HELPER_FLAGS_2(sck, TCG_CALL_NO_RWG, i32, env, i64)
 DEF_HELPER_FLAGS_2(sckc, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_2(sckpf, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_1(stckc, TCG_CALL_NO_RWG, i64, env)

target/s390x/insn-data.def +1 -2

index 157619403d..5c6f33ed9c 100644
@@ -997,8 +997,7 @@
 /* SET ADDRESS SPACE CONTROL FAST */
     C(0xb279, SACF,    S,     Z,   0, a2, 0, 0, sacf, 0)
 /* SET CLOCK */
-    /* ??? Not implemented - is it necessary? */
-    C(0xb204, SCK,     S,     Z,   0, 0, 0, 0, 0, 0)
+    C(0xb204, SCK,     S,     Z,   la2, 0, 0, 0, sck, 0)
 /* SET CLOCK COMPARATOR */
     C(0xb206, SCKC,    S,     Z,   0, m2_64a, 0, 0, sckc, 0)
 /* SET CLOCK PROGRAMMABLE FIELD */

target/s390x/misc_helper.c +11 -0

index d5f9f5e1d3..c9712b9476 100644
@@ -191,6 +191,25 @@ void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque)
     helper_sckc(env, env->ckc);
 }
 
+/* Set Clock */
+uint32_t HELPER(sck)(CPUS390XState *env, uint64_t tod_low)
+{
+    S390TODState *td = s390_get_tod();
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    S390TOD tod = {
+        .high = 0,
+        .low = tod_low,
+    };
+    Error *err = NULL;
+

target/s390x/misc_helper.c void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque) · line 208

Quoted 4 lines

target/s390x/misc_helper.c +4 -0

+    qemu_mutex_lock_iothread();
+    tdc->set(td, &tod, &err);
+    qemu_mutex_unlock_iothread();
+    g_assert(!err);

I know it currently can't happen, but still, I think it would be nicer to use CC3 to tell the guest that something went wrong with the clock, instead of abort QEMU here.

Thomas

Quoted 13 lines · Thomas Huth

Quoted 11 lines · David Hildenbrand

target/s390x/misc_helper.c +11 -0

+/* Set Clock */
+uint32_t HELPER(sck)(CPUS390XState *env, uint64_t tod_low)
+{
+    S390TODState *td = s390_get_tod();
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    S390TOD tod = {
+        .high = 0,
+        .low = tod_low,
+    };
+    Error *err = NULL;
+

Quoted 8 lines

target/s390x/misc_helper.c void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque) · line 208

Quoted 4 lines

target/s390x/misc_helper.c +4 -0

+    qemu_mutex_lock_iothread();
+    tdc->set(td, &tod, &err);
+    qemu_mutex_unlock_iothread();
+    g_assert(!err);  

I know it currently can't happen, but still, I think it would be nicer to use CC3 to tell the guest that something went wrong with the clock, instead of abort QEMU here.

Hmm, I thing I should either use error_abort here or do what you suggest.

However, CC=3 means "Clock in not-operational state".

And this implies that also STORE CLOCK and friends will have to fail and that we have to present a machine check. Especially, once we would implement the TOD-clock steering facility, CC=3 would not apply anymore.

So instead of faking something that is not architecturally correct, I think we really should just quit QEMU, as we expect this to never fail.

If the CPU data is migrated after the TOD clock, the CKC timer of a CPU is not rearmed. Let's rearm it when loading the CPU state.

Introduce tcg-stub.c just like kvm-stub.c for tcg specific stubs.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/Makefile.objs |  1 +
 target/s390x/machine.c     |  6 ++++++
 target/s390x/tcg-stub.c    | 20 ++++++++++++++++++++
 3 files changed, 27 insertions(+)
 create mode 100644 target/s390x/tcg-stub.c

run_on_cpu() doesn't seem to work reliably until the CPU has been fully created if the single-threaded TCG main loop is already running.

Therefore, let's use run_on_cpu() for KVM only - KVM requires it due to the initial CPU reset ioctl.

Signed-off-by David Hildenbrand <[email protected]>

---
 target/s390x/cpu.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

target/s390x/cpu.c +0 -1

index 40d6980229..7536dd0f03 100644
@@ -218,11 +218,12 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
 #endif
     s390_cpu_gdb_init(cs);
     qemu_init_vcpu(cs);
-#if !defined(CONFIG_USER_ONLY)