Skip to contentSkip to conversation

qemu: create helper for event notification

PATCH Applied Anthony Liguori 5 participants 30 messages Apr 16, 2009 – Apr 23, 2009 7 messages not in the archive Kbuild

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 35 lines · [email protected] wrote:

Index: trunk/vl.c
===================================================================

vl.c +2 -9
@@ -1183,9 +1183,8 @@ void qemu_mod_timer(QEMUTimer *ts, int64
             qemu_rearm_alarm_timer(alarm_timer);
         }
         /* Interrupt execution to force deadline recalculation.  */
-        if (use_icount && cpu_single_env) {
-            cpu_exit(cpu_single_env);
-        }
+        if (use_icount)
+            qemu_notify_event();
     }
 }
 
@@ -1338,8 +1337,6 @@ static void host_alarm_handler(int host_
                                qemu_get_clock(vm_clock))) ||
         qemu_timer_expired(active_timers[QEMU_TIMER_REALTIME],
                            qemu_get_clock(rt_clock))) {
-        CPUState *env = next_cpu;
-
 #ifdef _WIN32
         struct qemu_alarm_win32 *data = ((struct qemu_alarm_timer*)dwUser)->priv;
         SetEvent(data->host_alarm);
@@ -1349,16 +1346,7 @@ static void host_alarm_handler(int host_
 #endif
         alarm_timer->flags |= ALARM_FLAG_EXPIRED;
 
-        if (env) {
-            /* stop the currently executing cpu because a timer occured */
-            cpu_exit(env);
-#ifdef USE_KQEMU

vl.c static void host_alarm_handler(int host_ · line 1352

Quoted 10 lines
vl.c +1 -6
-            if (env->kqemu_enabled) {
-                kqemu_cpu_interrupt(env);
-            }
-#endif
-        }
-        event_pending = 1;
+        qemu_notify_event();
     }
 }
 

I always thought next_cpu was used here to cover an explicit race condition.

If you're using TCG, and you get a single after running the loop, but before assigning cpu_single_env, then you'll set the interrupt exit request on the old CPU state. You'll eventually exit I guess but you potentially have to run through multiple VCPUs.

I'd feel more comfortable if we preserved the behavior here that we had before.

Signature · 3 lines
Regards,

Anthony Liguori
#3 lore raw
Quoted 6 lines · Anthony Liguori
Quoted 3 lines
     }

}

I always thought next_cpu was used here to cover an explicit race condition.

Quoted 7 lines

If you're using TCG, and you get a single after running the loop, but before assigning cpu_single_env, then you'll set the interrupt exit request on the old CPU state. You'll eventually exit I guess but you potentially have to run through multiple VCPUs.

I'd feel more comfortable if we preserved the behavior here that we had before.

Right. Patch 7 reverts to the old behaviour.

#4 lore raw
Quoted 6 lines · Marcelo Tosatti
Quoted 4 lines · Anthony Liguori
Quoted 4 lines
     }

}

Quoted 9 lines
Quoted 6 lines

I always thought next_cpu was used here to cover an explicit race condition.

If you're using TCG, and you get a single after running the loop, but before assigning cpu_single_env, then you'll set the interrupt exit request on the old CPU state. You'll eventually exit I guess but you potentially have to run through multiple VCPUs.

I'd feel more comfortable if we preserved the behavior here that we had before.

Right. Patch 7 reverts to the old behaviour.

We need to do this series in a way such that we aren't breaking things as we go along.

Regards,

Anthony Liguori

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

[Qemu-devel] Re: [patch 02/11] qemu: mutex/thread/cond wrappers

Quoted 42 lines · [email protected] wrote:

Index: trunk/configure
===================================================================

configure +5 -0
@@ -180,6 +180,7 @@ build_docs="no"
 uname_release=""
 curses="yes"
 aio="yes"
+io_thread="no"
 nptl="yes"
 mixemu="no"
 bluez="yes"
@@ -463,6 +464,8 @@ for opt do
   ;;
   --disable-aio) aio="no"
   ;;
+  --enable-io-thread) io_thread="yes"
+  ;;
   --disable-blobs) blobs="no"
   ;;
   --kerneldir=*) kerneldir="$optarg"
@@ -577,6 +580,7 @@ echo "  --enable-uname-release=R Return 
 echo "  --sparc_cpu=V            Build qemu for Sparc architecture v7, v8, v8plus, v8plusa, v9"
 echo "  --disable-vde            disable support for vde network"
 echo "  --disable-aio            disable AIO support"
+echo "  --enable-io-thread       enable IO thread"
 echo "  --disable-blobs          disable installing provided firmware blobs"
 echo "  --kerneldir=PATH         look for kernel includes in PATH"
 echo ""
@@ -1204,6 +1208,7 @@ echo "uname -r          $uname_release"
 echo "NPTL support      $nptl"
 echo "vde support       $vde"
 echo "AIO support       $aio"
+echo "IO thread         $io_thread"
 echo "Install blobs     $blobs"
 echo "KVM support       $kvm"
 echo "fdt support       $fdt"

configure if test "$aio" = "yes" ; then · line 1514

Quoted 10 lines
configure +4 -0
@@ -1500,6 +1505,10 @@ if test "$aio" = "yes" ; then
   echo "#define CONFIG_AIO 1" >> $config_h
   echo "CONFIG_AIO=yes" >> $config_mak
 fi
+if test "$io_thread" = "yes" ; then
+  echo "CONFIG_IOTHREAD=yes" >> $config_mak
+  echo "#define CONFIG_IOTHREAD 1" >> $config_h
+fi
 if test "$blobs" = "yes" ; then
   echo "INSTALL_BLOBS=yes" >> $config_mak
 fi

You'll need to probe pthread. For instance, 'configure --disable-aio --enable-io-thread' would succeed but result in a broken build. You should probably break out the pthread probing from CONFIG_AIO, then make CONFIG_AIO and CONFIG_IO_THREAD depend on CONFIG_PTHREAD.

Also, we should convert posix-aio-compat.c to qemu_thread but that can wait...

Regards,

Anthony Liguori

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

[Qemu-devel] Re: [patch 04/11] qemu: introduce main_loop_break

Quoted 52 lines · [email protected] wrote:

Use a pipe to signal pending work for the iothread.

Index: trunk/qemu-common.h
===================================================================

qemu-common.h +2 -0
@@ -186,6 +186,8 @@ int cpu_load(QEMUFile *f, void *opaque, 
 /* Force QEMU to stop what it's doing and service IO */
 void qemu_service_io(void);
 
+void main_loop_break(void);
+
 /* Force QEMU to process pending events */
 void qemu_notify_event(void);
 

Index: trunk/vl.c
===================================================================

vl.c +20 -0
@@ -276,6 +276,8 @@ static QEMUTimer *nographic_timer;
 
 uint8_t qemu_uuid[16];
 
+static int io_thread_fd = -1;
+
 /***********************************************************/
 /* x86 ISA bus support */
 
@@ -3632,6 +3634,55 @@ void qemu_notify_event(void)
      }
 }
 
+void main_loop_break(void)
+{
+    uint64_t value = 1;
+    char buffer[8];
+    size_t offset = 0;
+
+    if (io_thread_fd == -1)
+        return;
+
+    memcpy(buffer, &value, sizeof(value));
+
+    while (offset < 8) {
+        ssize_t len;
+
+        len = write(io_thread_fd, buffer + offset, 8 - offset);
+        if (len == -1 && errno == EINTR)
+            continue;
+

vl.c void qemu_notify_event(void) · line 3664

Quoted 10 lines
vl.c +9 -0
+        if (len <= 0)
+            break;
+
+        offset += len;
+    }
+
+    if (offset != 8)
+        fprintf(stderr, "failed to notify io thread\n");
+}
 

I'd like to see a higher level abstraction here. This is trying to be eventfd compatible which is great, but we should have something like:

qemu_event_increment();
qemu_event_read();

It'll simplify this code a lot. We can also switch some of the other existing code to use this too later.

Regards,

Anthony Liguori

A message that is not in the archive (sent off-list or not yet received): [email protected]

#10 lore raw

[Qemu-devel] Re: [patch 05/11] qemu: separate thread for io

Quoted 46 lines · [email protected] wrote:

Introduce a thread to handle host IO events.

Index: trunk/qemu-common.h
===================================================================

qemu-common.h +4 -0
@@ -191,6 +191,10 @@ void main_loop_break(void);
 /* Force QEMU to process pending events */
 void qemu_notify_event(void);
 
+/* Unblock cpu */
+void qemu_cpu_kick(void *env);
+int qemu_cpu_self(void *env);
+
 typedef struct QEMUIOVector {
     struct iovec *iov;
     int niov;

Index: trunk/vl.c
===================================================================

vl.c +8 -0
@@ -146,6 +146,7 @@ int main(int argc, char **argv)
 #include "gdbstub.h"
 #include "qemu-timer.h"
 #include "qemu-char.h"
+#include "qemu-thread.h"
 #include "cache-utils.h"
 #include "block.h"
 #include "dma.h"
@@ -278,6 +279,13 @@ uint8_t qemu_uuid[16];
 
 static int io_thread_fd = -1;
 
+QemuMutex qemu_global_mutex;
+QemuMutex qemu_fair_mutex;
+
+QemuThread io_thread;
+QemuThread cpus_thread;
+QemuCond halt_cond;
+
 /***********************************************************/
 /* x86 ISA bus support */
 

vl.c static void host_alarm_handler(int host_ · line 1360

Quoted 8 lines
vl.c +0 -2
@@ -1347,8 +1355,6 @@ static void host_alarm_handler(int host_
         write(alarm_timer_wfd, &byte, sizeof(byte));
 #endif
         alarm_timer->flags |= ALARM_FLAG_EXPIRED;
-
-        qemu_notify_event();
     }
 }
 

Isn't this unsafe in TCG? If you have chained TBs in a tight loop, removing qemu_notify_event() eliminates the cpu_interrupt call which then means that you'll never break out of that tight TB loop.

Quoted 9 lines
vl.c +1 -0
@@ -2957,6 +2963,7 @@ int qemu_set_fd_handler2(int fd,
         ioh->opaque = opaque;
         ioh->deleted = 0;
     }
+    main_loop_break();
     return 0;
 }
 

vl.c static int ram_load(QEMUFile *f, void *o · line 3335

Quoted 6 lines
vl.c +0 -1
@@ -3324,7 +3331,6 @@ static int ram_load(QEMUFile *f, void *o
 
 void qemu_service_io(void)
 {
-    qemu_notify_event();
 }
 

The same comment from above is applicable here. qemu_service_io() is called from a signal handler. We should s/qemu_service_io/qemu_notify_event/.

vl.c static void host_main_loop_wait(int *tim · line 3733

Quoted 5 lines
vl.c +0 -0
 void main_loop_break(void)
@@ -3733,6 +3731,105 @@ static void host_main_loop_wait(int *tim
 }
 #endif
 

#ifndef CONFIG_IO_THREAD, doesn't main_loop_break need to do cpu_interrupt?

vl.c static void host_main_loop_wait(int *tim · line 3739

Quoted 6 lines
vl.c +5 -0
+static void qemu_wait_io_event(CPUState *env, int timeout)
+{
+    if (timeout)
+        while (!tcg_has_work(env))
+            qemu_cond_timedwait(&halt_cond, &qemu_global_mutex, timeout);
 

What's the point of having a timeout?

Quoted 4 lines
vl.c +4 -0
+
+static void qemu_signal_lock(unsigned int msecs)
+{
+    qemu_mutex_lock(&qemu_fair_mutex);

vl.c static void host_main_loop_wait(int *tim · line 3753

Quoted 10 lines
vl.c +9 -0
+
+    while (qemu_mutex_trylock(&qemu_global_mutex)) {
+        qemu_thread_signal(&cpus_thread, SIGUSR1);
+        if (!qemu_mutex_timedlock(&qemu_global_mutex, msecs))
+            break;
+    }
+    qemu_mutex_unlock(&qemu_fair_mutex);
+}
+
 

Now's a good time to attempt to move a lot of the main loop stuff into a separate file. We can do it after the IO thread series or you can make it part of the IO thread series.

Signature · 3 lines
Regards,

Anthony Liguori
#11 lore raw

[Qemu-devel] Re: [patch 05/11] qemu: separate thread for io

Quoted 1 line · Anthony Liguori

#ifndef CONFIG_IO_THREAD, doesn't main_loop_break need to do cpu_interrupt?

Yes, also about the previous comments. Will be fixed by bisectable series as suggested.

Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +5 -0
+static void qemu_wait_io_event(CPUState *env, int timeout)
+{
+    if (timeout)
+        while (!tcg_has_work(env))
+            qemu_cond_timedwait(&halt_cond, &qemu_global_mutex, timeout);

What's the point of having a timeout?

I don't think there's much other than honouring the behaviour of the non-io-thread version. Now, I believe icount breaks if you don't honour it.

Quoted 10 lines
Quoted 10 lines
(quoted diff) +10 -0
+
+static void qemu_signal_lock(unsigned int msecs)
+{
+    qemu_mutex_lock(&qemu_fair_mutex);
+
+    while (qemu_mutex_trylock(&qemu_global_mutex)) {
+        qemu_thread_signal(&cpus_thread, SIGUSR1);
+        if (!qemu_mutex_timedlock(&qemu_global_mutex, msecs))
+            break;
+    }
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +3 -0
+    qemu_mutex_unlock(&qemu_fair_mutex);
+}
+

Now's a good time to attempt to move a lot of the main loop stuff into a separate file. We can do it after the IO thread series or you can make it part of the IO thread series.

Sure.

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#15 lore raw

[Qemu-devel] Re: [patch 11/11] qemu: basic kvm iothread support

Quoted 9 lines · [email protected] wrote:

Allow the iothread to run while vcpu is in guest mode.

Index: trunk/kvm-all.c
===================================================================

kvm-all.c int kvm_cpu_exec(CPUState *env) · line 455

Quoted 5 lines
kvm-all.c +1 -0
@@ -451,6 +451,7 @@ int kvm_cpu_exec(CPUState *env)
 
     do {
         kvm_arch_pre_run(env, run);
+        kvm_pre_run(env);
 

Avoid introduce kvm_pre_run() et al. Eventually, we want to avoid grabbing the mutex as long as humanly possible so we can support lockless MMIO/PIO. So let's avoid adding abstractions that will just make this harder.

Signature · 3 lines
Regards,

Anthony Liguori
#16 lore raw

cover qemu: introduce iothread (v4)

Addressing earlier comments.

Anthony, most of the KVM interaction (with global mutex) is missing, since i'm not exactly sure what you want to do there.

#17 lore raw

PATCH 1/14 qemu: create helper for event notification

vl.c +21 -22
Index: qemu/vl.c
===================================================================
@@ -1186,9 +1186,8 @@ void qemu_mod_timer(QEMUTimer *ts, int64
             qemu_rearm_alarm_timer(alarm_timer);
         }
         /* Interrupt execution to force deadline recalculation.  */
-        if (use_icount && cpu_single_env) {
-            cpu_exit(cpu_single_env);
-        }
+        if (use_icount)
+            qemu_notify_event();
     }
 }
 
@@ -1363,6 +1362,7 @@ static void host_alarm_handler(int host_
 #endif
         }
         event_pending = 1;
+        qemu_notify_event();
     }
 }
 
@@ -3339,15 +3339,7 @@ static int ram_load(QEMUFile *f, void *o
 
 void qemu_service_io(void)
 {
-    CPUState *env = cpu_single_env;
-    if (env) {
-        cpu_exit(env);
-#ifdef USE_KQEMU
-        if (env->kqemu_enabled) {
-            kqemu_cpu_interrupt(env);
-        }
-#endif
-    }
+    qemu_notify_event();
 }
 
 /***********************************************************/
@@ -3415,15 +3407,12 @@ void qemu_bh_schedule_idle(QEMUBH *bh)
 
 void qemu_bh_schedule(QEMUBH *bh)
 {
-    CPUState *env = cpu_single_env;
     if (bh->scheduled)
         return;
     bh->scheduled = 1;
     bh->idle = 0;
     /* stop the currently executing CPU to execute the BH ASAP */
-    if (env) {
-        cpu_exit(env);
-    }
+    qemu_notify_event();
 }
 
 void qemu_bh_cancel(QEMUBH *bh)
@@ -3634,22 +3623,32 @@ void qemu_system_reset_request(void)
     } else {
         reset_requested = 1;
     }
-    if (cpu_single_env)
-        cpu_exit(cpu_single_env);
+    qemu_notify_event();
 }
 
 void qemu_system_shutdown_request(void)
 {
     shutdown_requested = 1;
-    if (cpu_single_env)
-        cpu_exit(cpu_single_env);
+    qemu_notify_event();
 }
 
 void qemu_system_powerdown_request(void)
 {
     powerdown_requested = 1;
-    if (cpu_single_env)
-        cpu_exit(cpu_single_env);
+    qemu_notify_event();
+}
+
+void qemu_notify_event(void)
+{
+    CPUState *env = cpu_single_env;
+
+    if (env) {
+        cpu_exit(env);
+#ifdef USE_KQEMU
+        if (env->kqemu_enabled)
+            kqemu_cpu_interrupt(env);
+#endif
+     }
 }
 
 #ifdef _WIN32
qemu-common.h +3 -0
Index: qemu/qemu-common.h
===================================================================
@@ -186,6 +186,9 @@ int cpu_load(QEMUFile *f, void *opaque, 
 /* Force QEMU to stop what it's doing and service IO */
 void qemu_service_io(void);
 
+/* Force QEMU to process pending events */
+void qemu_notify_event(void);
+
 typedef struct QEMUIOVector {
     struct iovec *iov;
     int niov;
hw/mac_dbdma.c +1 -3
Index: qemu/hw/mac_dbdma.c
===================================================================
@@ -651,9 +651,7 @@ void DBDMA_register_channel(void *dbdma,
 
 void DBDMA_schedule(void)
 {
-    CPUState *env = cpu_single_env;
-    if (env)
-        cpu_exit(env);
+    qemu_notify_event();
 }
 
 static void
#18 lore raw

PATCH 2/14 qemu: mutex/thread/cond wrappers and configure tweaks

In reply to [email protected]
Makefile +6 -1
Index: qemu/Makefile
===================================================================
@@ -30,7 +30,8 @@ else
 DOCS=
 endif
 
-LIBS+=$(AIOLIBS)
+LIBS+=$(PTHREADLIBS)
+LIBS+=$(CLOCKLIBS)
 
 ifdef CONFIG_SOLARIS
 LIBS+=-lsocket -lnsl -lresolv
@@ -170,6 +171,10 @@ ifdef CONFIG_COCOA
 OBJS+=cocoa.o
 endif
 
+ifdef CONFIG_IOTHREAD
+OBJS+=qemu-thread.o
+endif
+
 ifdef CONFIG_SLIRP
 CPPFLAGS+=-I$(SRC_PATH)/slirp
 SLIRP_OBJS=cksum.o if.o ip_icmp.o ip_input.o ip_output.o \
Makefile.target +4 -2
Index: qemu/Makefile.target
===================================================================
@@ -315,7 +315,8 @@ endif
 
 OBJS= main.o syscall.o strace.o mmap.o signal.o path.o thunk.o \
       elfload.o linuxload.o uaccess.o envlist.o
-LIBS+= $(AIOLIBS)
+LIBS+= $(PTHREADLIBS)
+LIBS+= $(CLOCKLIBS)
 ifdef TARGET_HAS_BFLT
 OBJS+= flatload.o
 endif
@@ -683,7 +684,8 @@ ifdef CONFIG_SLIRP
 CPPFLAGS+=-I$(SRC_PATH)/slirp
 endif
 
-LIBS+=$(AIOLIBS)
+LIBS+=$(PTHREADLIBS)
+LIBS+=$(CLOCKLIBS)
 # specific flags are needed for non soft mmu emulator
 ifdef CONFIG_STATIC
 LDFLAGS+=-static
configure +31 -12

108 lines not on this page · raw message

qemu-thread.c new file +163 -0

163 lines not on this page · raw message

qemu-thread.h new file +40 -0
Index: qemu/qemu-thread.h
===================================================================
@@ -0,0 +1,40 @@
+#ifndef __QEMU_THREAD_H
+#define __QEMU_THREAD_H 1
+#include "semaphore.h"
+#include "pthread.h"
+
+struct QemuMutex {
+    pthread_mutex_t lock;
+};
+
+struct QemuCond {
+    pthread_cond_t cond;
+};
+
+struct QemuThread {
+    pthread_t thread;
+};
+
+typedef struct QemuMutex QemuMutex;
+typedef struct QemuCond QemuCond;
+typedef struct QemuThread QemuThread;
+
+void qemu_mutex_init(QemuMutex *mutex);
+void qemu_mutex_lock(QemuMutex *mutex);
+int qemu_mutex_trylock(QemuMutex *mutex);
+int qemu_mutex_timedlock(QemuMutex *mutex, uint64_t msecs);
+void qemu_mutex_unlock(QemuMutex *mutex);
+
+void qemu_cond_init(QemuCond *cond);
+void qemu_cond_signal(QemuCond *cond);
+void qemu_cond_broadcast(QemuCond *cond);
+void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex);
+int qemu_cond_timedwait(QemuCond *cond, QemuMutex *mutex, uint64_t msecs);
+
+void qemu_thread_create(QemuThread *thread,
+                       void *(*start_routine)(void*),
+                       void *arg);
+void qemu_thread_signal(QemuThread *thread, int sig);
+void qemu_thread_self(QemuThread *thread);
+int qemu_thread_equal(QemuThread *thread1, QemuThread *thread2);
+#endif
#19 lore raw

[Qemu-devel] Re: [patch 02/14] qemu: mutex/thread/cond wrappers and configure tweaks

configure echo "Documentation $build_docs" · line 1319

Quoted 9 lines · [email protected] wrote:
configure +1 -1
@@ -1302,7 +1315,8 @@ echo "Documentation     $build_docs"
 echo "uname -r          $uname_release"
 echo "NPTL support      $nptl"
 echo "vde support       $vde"
-echo "AIO support       $aio"
+echo "IO support       $aio"
 

Oops. Fixed locally, but if you have to resubmit, make sure to get rid of this.

Signature · 3 lines
Regards,

Anthony Liguori
#20 lore raw

[Qemu-devel] svn trunk currently borked

Hi,

one of the last commits borked the qemu user mode:

CC i386-linux-user/i386-dis.o
AR i386-linux-user/libqemu.a
LINK i386-linux-user/qemu-i386
libqemu.a(op_helper.o): In function `handle_even_inj':
/usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1197:
undefined reference to `ldl_phys' /usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1207:
undefined reference to `stl_phys'
libqemu.a(op_helper.o): In function `do_interrupt':
/usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1270:
undefined reference to `ldl_phys'

Regards, Martin

#21 lore raw

Re: [Qemu-devel] svn trunk currently borked

Quoted 4 lines · Martin Mohring

Hi,

one of the last commits borked the qemu user mode:

Quoted 11 lines

CC i386-linux-user/i386-dis.o
 AR i386-linux-user/libqemu.a
 LINK i386-linux-user/qemu-i386
libqemu.a(op_helper.o): In function `handle_even_inj':
/usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1197:
undefined reference to `ldl_phys' /usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1207:
undefined reference to `stl_phys'
libqemu.a(op_helper.o): In function `do_interrupt':
/usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1270:
undefined reference to `ldl_phys'

This is due to commit 7230.

Could you give the attached patch a try? I just checked it compiled.

Laurent

op_helper.c +8 -0
Index: target-i386/op_helper.c
===================================================================
@@ -1191,6 +1191,7 @@
         EIP = next_eip;
 }
 
+#if !defined(CONFIG_USER_ONLY)
 static void handle_even_inj(int intno, int is_int, int error_code,
 		int is_hw, int rm)
 {
@@ -1209,6 +1210,7 @@
 	    stl_phys(env->vm_vmcb + offsetof(struct vmcb, control.event_inj), event_inj);
     }
 }
+#endif
 
 /*
  * Begin execution of an interruption. is_int is TRUE if coming from
@@ -1250,8 +1252,10 @@
         }
     }
     if (env->cr[0] & CR0_PE_MASK) {
+#if !defined(CONFIG_USER_ONLY)
         if (env->hflags & HF_SVMI_MASK)
             handle_even_inj(intno, is_int, error_code, is_hw, 0);
+#endif
 #ifdef TARGET_X86_64
         if (env->hflags & HF_LMA_MASK) {
             do_interrupt64(intno, is_int, error_code, next_eip, is_hw);
@@ -1261,15 +1265,19 @@
             do_interrupt_protected(intno, is_int, error_code, next_eip, is_hw);
         }
     } else {
+#if !defined(CONFIG_USER_ONLY)
         if (env->hflags & HF_SVMI_MASK)
             handle_even_inj(intno, is_int, error_code, is_hw, 1);
+#endif
         do_interrupt_real(intno, is_int, error_code, next_eip);
     }
 
+#if !defined(CONFIG_USER_ONLY)
     if (env->hflags & HF_SVMI_MASK) {
 	    uint32_t event_inj = ldl_phys(env->vm_vmcb + offsetof(struct vmcb, control.event_inj));
 	    stl_phys(env->vm_vmcb + offsetof(struct vmcb, control.event_inj), event_inj & ~SVM_EVTINJ_VALID);
     }
+#endif
 }
 
 /* This should come from sysemu.h - if we could include it here... */
#22 lore raw

Re: [Qemu-devel] svn trunk currently borked

Quoted 7 lines · Laurent Desnogues

On Thu, Apr 23, 2009 at 1:15 PM, Martin Mohring
<[email protected]> wrote:

This is due to commit 7230.

Could you give the attached patch a try? I just checked it compiled.

Applied. Thanks.

Regards,

Anthony Liguori

Quoted 4 lines

Laurent

#23 lore raw

Re: [Qemu-devel] svn trunk currently borked

In reply to Laurent Desnogues

Your fix solves the problem.

tnx.

Quoted 29 lines · Laurent Desnogues
Quoted 4 lines · Martin Mohring

Hi,

one of the last commits borked the qemu user mode:

Quoted 12 lines

CC i386-linux-user/i386-dis.o
AR i386-linux-user/libqemu.a
LINK i386-linux-user/qemu-i386
libqemu.a(op_helper.o): In function `handle_even_inj':
/usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1197:
undefined reference to `ldl_phys' /usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1207:
undefined reference to `stl_phys'
libqemu.a(op_helper.o): In function `do_interrupt':
/usr/src/packages/BUILD/qemu-0.11svn7232/target-i386/op_helper.c:1270:
undefined reference to `ldl_phys'

This is due to commit 7230.

Could you give the attached patch a try? I just checked it compiled.

Laurent

#24 lore raw

PATCH 3/14 qemu: per-arch cpu_has_work

In reply to [email protected]

Blue Swirl: fix Sparc32 breakage

target-alpha/exec.h +6 -1
Index: qemu/target-alpha/exec.h
===================================================================
@@ -48,10 +48,15 @@ static always_inline void regs_to_env(vo
 {
 }
 
+static always_inline int cpu_has_work(CPUState *env)
+{
+    return (env->interrupt_request & CPU_INTERRUPT_HARD);
+}
+
 static always_inline int cpu_halted(CPUState *env) {
     if (!env->halted)
         return 0;
-    if (env->interrupt_request & CPU_INTERRUPT_HARD) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         return 0;
     }
target-i386/exec.h +12 -3
Index: qemu/target-i386/exec.h
===================================================================
@@ -338,14 +338,23 @@ static inline void regs_to_env(void)
 #endif
 }
 
+static inline int cpu_has_work(CPUState *env)
+{
+    int work;
+
+    work = (env->interrupt_request & CPU_INTERRUPT_HARD) &&
+           (env->eflags & IF_MASK);
+    work |= env->interrupt_request & CPU_INTERRUPT_NMI;
+
+    return work;
+}
+
 static inline int cpu_halted(CPUState *env) {
     /* handle exit of HALTED state */
     if (!env->halted)
         return 0;
     /* disable halt condition */
-    if (((env->interrupt_request & CPU_INTERRUPT_HARD) &&
-         (env->eflags & IF_MASK)) ||
-        (env->interrupt_request & CPU_INTERRUPT_NMI)) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         return 0;
     }
cpu-all.h +2 -0
Index: qemu/cpu-all.h
===================================================================
@@ -775,6 +775,8 @@ void cpu_reset_interrupt(CPUState *env, 
 
 void cpu_exit(CPUState *s);
 
+int qemu_cpu_has_work(CPUState *env);
+
 /* Breakpoint/watchpoint flags */
 #define BP_MEM_READ           0x01
 #define BP_MEM_WRITE          0x02
cpu-exec.c +5 -0
Index: qemu/cpu-exec.c
===================================================================
@@ -50,6 +50,11 @@ int tb_invalidated_flag;
 //#define DEBUG_EXEC
 //#define DEBUG_SIGNAL
 
+int qemu_cpu_has_work(CPUState *env)
+{
+    return cpu_has_work(env);
+}
+
 void cpu_loop_exit(void)
 {
     /* NOTE: the register at this point must be saved by hand because
target-arm/exec.h +7 -2
Index: qemu/target-arm/exec.h
===================================================================
@@ -37,14 +37,19 @@ static inline void regs_to_env(void)
 {
 }
 
+static inline int cpu_has_work(CPUState *env)
+{
+    return (env->interrupt_request &
+            (CPU_INTERRUPT_FIQ | CPU_INTERRUPT_HARD | CPU_INTERRUPT_EXITTB));
+}
+
 static inline int cpu_halted(CPUState *env) {
     if (!env->halted)
         return 0;
     /* An interrupt wakes the CPU even if the I and F CPSR bits are
        set.  We use EXITTB to silently wake CPU without causing an
        actual interrupt.  */
-    if (env->interrupt_request &
-        (CPU_INTERRUPT_FIQ | CPU_INTERRUPT_HARD | CPU_INTERRUPT_EXITTB)) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         return 0;
     }
target-cris/exec.h +5 -0
Index: qemu/target-cris/exec.h
===================================================================
@@ -40,6 +40,11 @@ static inline void regs_to_env(void)
 void cpu_cris_flush_flags(CPUCRISState *env, int cc_op);
 void helper_movec(CPUCRISState *env, int reg, uint32_t val);
 
+static inline int cpu_has_work(CPUState *env)
+{
+    return (env->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI));
+}
+
 static inline int cpu_halted(CPUState *env) {
 	if (!env->halted)
 		return 0;
target-m68k/exec.h +6 -1
Index: qemu/target-m68k/exec.h
===================================================================
@@ -41,10 +41,15 @@ static inline void regs_to_env(void)
 #include "softmmu_exec.h"
 #endif
 
+static inline int cpu_has_work(CPUState *env)
+{
+    return (env->interrupt_request & (CPU_INTERRUPT_HARD));
+}
+
 static inline int cpu_halted(CPUState *env) {
     if (!env->halted)
         return 0;
-    if (env->interrupt_request & CPU_INTERRUPT_HARD) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         return 0;
     }
target-mips/exec.h +8 -2
Index: qemu/target-mips/exec.h
===================================================================
@@ -33,12 +33,18 @@ static inline void regs_to_env(void)
 {
 }
 
+static inline int cpu_has_work(CPUState *env)
+{
+    return (env->interrupt_request &
+            (CPU_INTERRUPT_HARD | CPU_INTERRUPT_TIMER));
+}
+
+
 static inline int cpu_halted(CPUState *env)
 {
     if (!env->halted)
         return 0;
-    if (env->interrupt_request &
-        (CPU_INTERRUPT_HARD | CPU_INTERRUPT_TIMER)) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         return 0;
     }
target-ppc/exec.h +7 -1
Index: qemu/target-ppc/exec.h
===================================================================
@@ -44,11 +44,17 @@ static always_inline void regs_to_env (v
 {
 }
 
+static always_inline int cpu_has_work(CPUState *env)
+{
+    return (msr_ee && (env->interrupt_request & CPU_INTERRUPT_HARD));
+}
+
+
 static always_inline int cpu_halted (CPUState *env)
 {
     if (!env->halted)
         return 0;
-    if (msr_ee && (env->interrupt_request & CPU_INTERRUPT_HARD)) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         return 0;
     }
target-sh4/exec.h +6 -1
Index: qemu/target-sh4/exec.h
===================================================================
@@ -28,10 +28,15 @@ register struct CPUSH4State *env asm(ARE
 #include "cpu.h"
 #include "exec-all.h"
 
+static inline int cpu_has_work(CPUState *env)
+{
+    return (env->interrupt_request & CPU_INTERRUPT_HARD);
+}
+
 static inline int cpu_halted(CPUState *env) {
     if (!env->halted)
         return 0;
-    if (env->interrupt_request & CPU_INTERRUPT_HARD) {
+    if (cpu_has_work(env)) {
         env->halted = 0;
         env->intr_at_halt = 1;
         return 0;
target-sparc/exec.h +8 -1
Index: qemu/target-sparc/exec.h
===================================================================
@@ -24,10 +24,17 @@ static inline void regs_to_env(void)
 /* op_helper.c */
 void do_interrupt(CPUState *env);
 
+static inline int cpu_has_work(CPUState *env1)
+{
+    return (env1->interrupt_request & CPU_INTERRUPT_HARD) &&
+           (env1->psret != 0);
+}
+
+
 static inline int cpu_halted(CPUState *env1) {
     if (!env1->halted)
         return 0;
-    if ((env1->interrupt_request & CPU_INTERRUPT_HARD) && (env1->psret != 0)) {
+    if (cpu_has_work(env1)) {
         env1->halted = 0;
         return 0;
     }
#25 lore raw

PATCH 4/14 qemu: explictly rearm alarm timer on main_loop_wait

In reply to [email protected]

- lift notification code out of try_to_rearm_timer/host_alarm_handler
- probably going to remove alarm timers via signals sometime soon anyway

vl.c +10 -26
Index: trunk/vl.c
===================================================================
@@ -1346,9 +1346,6 @@ static void host_alarm_handler(int host_
 #ifdef _WIN32
         struct qemu_alarm_win32 *data = ((struct qemu_alarm_timer*)dwUser)->priv;
         SetEvent(data->host_alarm);
-#else
-        static const char byte = 0;
-        write(alarm_timer_wfd, &byte, sizeof(byte));
 #endif
         alarm_timer->flags |= ALARM_FLAG_EXPIRED;
 
@@ -1638,27 +1635,13 @@ static void unix_stop_timer(struct qemu_
 
 #endif /* !defined(_WIN32) */
 
-static void try_to_rearm_timer(void *opaque)
-{
-    struct qemu_alarm_timer *t = opaque;
-#ifndef _WIN32
-    ssize_t len;
 
-    /* Drain the notify pipe */
-    do {
-        char buffer[512];
-        len = read(alarm_timer_rfd, buffer, sizeof(buffer));
-    } while ((len == -1 && errno == EINTR) || len > 0);
-#endif
+#ifdef _WIN32
 
-    if (t->flags & ALARM_FLAG_EXPIRED) {
-        alarm_timer->flags &= ~ALARM_FLAG_EXPIRED;
-        qemu_rearm_alarm_timer(alarm_timer);
-    }
+static void dummy_event_handler(void *opaque)
+{
 }
 
-#ifdef _WIN32
-
 static int win32_start_timer(struct qemu_alarm_timer *t)
 {
     TIMECAPS tc;
@@ -1699,7 +1682,7 @@ static int win32_start_timer(struct qemu
         return -1;
     }
 
-    qemu_add_wait_object(data->host_alarm, try_to_rearm_timer, t);
+    qemu_add_wait_object(data->host_alarm, dummy_event_handler, t);
 
     return 0;
 }
@@ -1782,11 +1765,6 @@ static int init_timer_alarm(void)
         goto fail;
     }
 
-#ifndef _WIN32
-    qemu_set_fd_handler2(alarm_timer_rfd, NULL,
-                         try_to_rearm_timer, NULL, t);
-#endif
-
     alarm_timer = t;
 
     return 0;
@@ -3778,6 +3756,12 @@ void main_loop_wait(int timeout)
     }
 #endif
 
+    /* rearm timer, if not periodic */
+    if (alarm_timer->flags & ALARM_FLAG_EXPIRED) {
+        alarm_timer->flags &= ~ALARM_FLAG_EXPIRED;
+        qemu_rearm_alarm_timer(alarm_timer);
+    }
+
     /* vm time timers */
     if (vm_running && likely(!(cur_cpu->singlestep_enabled & SSTEP_NOTIMER)))
         qemu_run_timers(&active_timers[QEMU_TIMER_VIRTUAL],
#26 lore raw

PATCH 5/14 qemu: factor out special event notification

In reply to [email protected]

Special events that have no particular event descriptor (either fd for UNIX or HANDLE for Windows) associated with make use of an artificial one.

vl.c +76 -42
Index: trunk/vl.c
===================================================================
@@ -921,15 +921,11 @@ static void qemu_rearm_alarm_timer(struc
 #define MIN_TIMER_REARM_US 250
 
 static struct qemu_alarm_timer *alarm_timer;
-#ifndef _WIN32
-static int alarm_timer_rfd, alarm_timer_wfd;
-#endif
 
 #ifdef _WIN32
 
 struct qemu_alarm_win32 {
     MMRESULT timerId;
-    HANDLE host_alarm;
     unsigned int period;
 } alarm_win32_data = {0, NULL, -1};
 
@@ -1343,10 +1339,6 @@ static void host_alarm_handler(int host_
                            qemu_get_clock(rt_clock))) {
         CPUState *env = next_cpu;
 
-#ifdef _WIN32
-        struct qemu_alarm_win32 *data = ((struct qemu_alarm_timer*)dwUser)->priv;
-        SetEvent(data->host_alarm);
-#endif
         alarm_timer->flags |= ALARM_FLAG_EXPIRED;
 
         if (env) {
@@ -1638,9 +1630,6 @@ static void unix_stop_timer(struct qemu_
 
 #ifdef _WIN32
 
-static void dummy_event_handler(void *opaque)
-{
-}
 
 static int win32_start_timer(struct qemu_alarm_timer *t)
 {
@@ -1648,12 +1637,6 @@ static int win32_start_timer(struct qemu
     struct qemu_alarm_win32 *data = t->priv;
     UINT flags;
 
-    data->host_alarm = CreateEvent(NULL, FALSE, FALSE, NULL);
-    if (!data->host_alarm) {
-        perror("Failed CreateEvent");
-        return -1;
-    }
-
     memset(&tc, 0, sizeof(tc));
     timeGetDevCaps(&tc, sizeof(tc));
 
@@ -1682,8 +1665,6 @@ static int win32_start_timer(struct qemu
         return -1;
     }
 
-    qemu_add_wait_object(data->host_alarm, dummy_event_handler, t);
-
     return 0;
 }
 
@@ -1733,25 +1714,6 @@ static int init_timer_alarm(void)
     struct qemu_alarm_timer *t = NULL;
     int i, err = -1;
 
-#ifndef _WIN32
-    int fds[2];
-
-    err = pipe(fds);
-    if (err == -1)
-        return -errno;
-
-    err = fcntl_setfl(fds[0], O_NONBLOCK);
-    if (err < 0)
-        goto fail;
-
-    err = fcntl_setfl(fds[1], O_NONBLOCK);
-    if (err < 0)
-        goto fail;
-
-    alarm_timer_rfd = fds[0];
-    alarm_timer_wfd = fds[1];
-#endif
-
     for (i = 0; alarm_timers[i].name; i++) {
         t = &alarm_timers[i];
 
@@ -1770,10 +1732,6 @@ static int init_timer_alarm(void)
     return 0;
 
 fail:
-#ifndef _WIN32
-    close(fds[0]);
-    close(fds[1]);
-#endif
     return err;
 }
 
@@ -3629,6 +3587,82 @@ void qemu_notify_event(void)
      }
 }
 
+#ifdef CONFIG_IOTHREAD
+#ifndef _WIN32
+static int io_thread_fd = -1;
+
+static void qemu_event_increment(void)
+{
+    static const char byte = 0;
+
+    if (io_thread_fd == -1)
+        return;
+
+    write(io_thread_fd, &byte, sizeof(byte));
+}
+
+static void qemu_event_read(void *opaque)
+{
+    int fd = (unsigned long)opaque;
+    ssize_t len;
+
+    /* Drain the notify pipe */
+    do {
+        char buffer[512];
+        len = read(fd, buffer, sizeof(buffer));
+    } while ((len == -1 && errno == EINTR) || len > 0);
+}
+
+static int qemu_event_init(void)
+{
+    int err;
+    int fds[2];
+
+    err = pipe(fds);
+    if (err == -1)
+        return -errno;
+
+    err = fcntl_setfl(fds[0], O_NONBLOCK);
+    if (err < 0)
+        goto fail;
+
+    err = fcntl_setfl(fds[1], O_NONBLOCK);
+    if (err < 0)
+        goto fail;
+
+    qemu_set_fd_handler2(fds[0], NULL, qemu_event_read, NULL,
+                         (void *)(unsigned long)fds[0]);
+
+    io_thread_fd = fds[1];
+fail:
+    close(fds[0]);
+    close(fds[1]);
+    return err;
+}
+#else
+HANDLE qemu_event_handle;
+
+static void dummy_event_handler(void *opaque)
+{
+}
+
+static int qemu_event_init(void)
+{
+    qemu_event_handle = CreateEvent(NULL, FALSE, FALSE, NULL);
+    if (!qemu_event_handle) {
+        perror("Failed CreateEvent");
+        return -1;
+    }
+    qemu_add_wait_object(qemu_event_handle, dummy_event_handler, NULL);
+}
+
+static void qemu_event_increment(void)
+{
+    SetEvent(qemu_event_handle);
+}
+#endif
+#endif
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
#27 lore raw

Re: [Qemu-devel] [patch 05/14] qemu: factor out special event notification

Quoted 15 lines · [email protected] wrote:

Special events that have no particular event descriptor (either fd for UNIX or HANDLE for Windows) associated with make use of an artificial one.

Index: trunk/vl.c
===================================================================

vl.c +0 -2
@@ -921,15 +921,11 @@ static void qemu_rearm_alarm_timer(struc
 #define MIN_TIMER_REARM_US 250
 
 static struct qemu_alarm_timer *alarm_timer;
-#ifndef _WIN32
-static int alarm_timer_rfd, alarm_timer_wfd;

vl.c static void qemu_rearm_alarm_timer(struc · line 931

Quoted 10 lines
vl.c +0 -2
-#endif
 
 #ifdef _WIN32
 
 struct qemu_alarm_win32 {
     MMRESULT timerId;
-    HANDLE host_alarm;
     unsigned int period;
 } alarm_win32_data = {0, NULL, -1};
 

There are a number of spots in vl.c that do CloseHandle(data->host_alarm) that aren't removed in this patch. For instance, the error path in win32_start_timer().

Regards,

Anthony Liguori

#28 lore raw

PATCH 6/14 qemu: refactor main_loop

In reply to [email protected]

Break main loop into 3 main functions.

vl.c +151 -129
Index: qemu-iothread-4/vl.c
===================================================================
@@ -266,7 +266,7 @@ struct drive_opt drives_opt[MAX_DRIVES];
 
 static CPUState *cur_cpu;
 static CPUState *next_cpu;
-static int event_pending = 1;
+static int timer_alarm_pending = 1;
 /* Conversion factor from emulated instructions to virtual clock ticks.  */
 static int icount_time_shift;
 /* Arbitrarily pick 1MIPS as the minimum allowable speed.  */
@@ -1350,7 +1350,7 @@ static void host_alarm_handler(int host_
             }
 #endif
         }
-        event_pending = 1;
+        timer_alarm_pending = 1;
         qemu_notify_event();
     }
 }
@@ -3811,153 +3811,175 @@ void main_loop_wait(int timeout)
 
 }
 
-static int main_loop(void)
+static int qemu_cpu_exec(CPUState *env)
 {
-    int ret, timeout;
+    int ret;
 #ifdef CONFIG_PROFILER
     int64_t ti;
 #endif
-    CPUState *env;
 
-    cur_cpu = first_cpu;
-    next_cpu = cur_cpu->next_cpu ?: first_cpu;
-    for(;;) {
-        if (vm_running) {
-
-            for(;;) {
-                /* get next cpu */
-                env = next_cpu;
 #ifdef CONFIG_PROFILER
-                ti = profile_getclock();
+    ti = profile_getclock();
 #endif
-                if (use_icount) {
-                    int64_t count;
-                    int decr;
-                    qemu_icount -= (env->icount_decr.u16.low + env->icount_extra);
-                    env->icount_decr.u16.low = 0;
-                    env->icount_extra = 0;
-                    count = qemu_next_deadline();
-                    count = (count + (1 << icount_time_shift) - 1)
-                            >> icount_time_shift;
-                    qemu_icount += count;
-                    decr = (count > 0xffff) ? 0xffff : count;
-                    count -= decr;
-                    env->icount_decr.u16.low = decr;
-                    env->icount_extra = count;
-                }
-                ret = cpu_exec(env);
+    if (use_icount) {
+        int64_t count;
+        int decr;
+        qemu_icount -= (env->icount_decr.u16.low + env->icount_extra);
+        env->icount_decr.u16.low = 0;
+        env->icount_extra = 0;
+        count = qemu_next_deadline();
+        count = (count + (1 << icount_time_shift) - 1)
+                >> icount_time_shift;
+        qemu_icount += count;
+        decr = (count > 0xffff) ? 0xffff : count;
+        count -= decr;
+        env->icount_decr.u16.low = decr;
+        env->icount_extra = count;
+    }
+    ret = cpu_exec(env);
 #ifdef CONFIG_PROFILER
-                qemu_time += profile_getclock() - ti;
+    qemu_time += profile_getclock() - ti;
 #endif
-                if (use_icount) {
-                    /* Fold pending instructions back into the
-                       instruction counter, and clear the interrupt flag.  */
-                    qemu_icount -= (env->icount_decr.u16.low
-                                    + env->icount_extra);
-                    env->icount_decr.u32 = 0;
-                    env->icount_extra = 0;
-                }
-                next_cpu = env->next_cpu ?: first_cpu;
-                if (event_pending && likely(ret != EXCP_DEBUG)) {
-                    ret = EXCP_INTERRUPT;
-                    event_pending = 0;
-                    break;
-                }
-                if (ret == EXCP_HLT) {
-                    /* Give the next CPU a chance to run.  */
-                    cur_cpu = env;
-                    continue;
-                }
-                if (ret != EXCP_HALTED)
+    if (use_icount) {
+        /* Fold pending instructions back into the
+           instruction counter, and clear the interrupt flag.  */
+        qemu_icount -= (env->icount_decr.u16.low
+                        + env->icount_extra);
+        env->icount_decr.u32 = 0;
+        env->icount_extra = 0;
+    }
+    return ret;
+}
+
+static int cpu_has_work(CPUState *env)
+{
+    if (!env->halted)
+        return 1;
+    if (qemu_cpu_has_work(env))
+        return 1;
+    return 0;
+}
+
+static int tcg_has_work(void)
+{
+    CPUState *env;
+
+    for (env = first_cpu; env != NULL; env = env->next_cpu)
+        if (cpu_has_work(env))
+            return 1;
+    return 0;
+}
+
+static int qemu_calculate_timeout(void)
+{
+    int timeout;
+
+    if (!vm_running)
+        timeout = 5000;
+    else if (tcg_has_work())
+        timeout = 0;
+    else if (!use_icount)
+        timeout = 5000;
+    else {
+     /* XXX: use timeout computed from timers */
+        int64_t add;
+        int64_t delta;
+        /* Advance virtual time to the next event.  */
+        if (use_icount == 1) {
+            /* When not using an adaptive execution frequency
+               we tend to get badly out of sync with real time,
+               so just delay for a reasonable amount of time.  */
+            delta = 0;
+        } else {
+            delta = cpu_get_icount() - cpu_get_clock();
+        }
+        if (delta > 0) {
+            /* If virtual time is ahead of real time then just
+               wait for IO.  */
+            timeout = (delta / 1000000) + 1;
+        } else {
+            /* Wait for either IO to occur or the next
+               timer event.  */
+            add = qemu_next_deadline();
+            /* We advance the timer before checking for IO.
+               Limit the amount we advance so that early IO
+               activity won't get the guest too far ahead.  */
+            if (add > 10000000)
+                add = 10000000;
+            delta += add;
+            add = (add + (1 << icount_time_shift) - 1)
+                  >> icount_time_shift;
+            qemu_icount += add;
+            timeout = delta / 1000000;
+            if (timeout < 0)
+                timeout = 0;
+        }
+    }
+
+    return timeout;
+}
+
+static int vm_can_run(void)
+{
+    if (powerdown_requested)
+        return 0;
+    if (reset_requested)
+        return 0;
+    if (shutdown_requested)
+        return 0;
+    return 1;
+}
+
+static void main_loop(void)
+{
+    int ret = 0;
+#ifdef CONFIG_PROFILER
+    int64_t ti;
+#endif
+
+    for (;;) {
+        do {
+            if (next_cpu == NULL)
+                next_cpu = first_cpu;
+            for (; next_cpu != NULL; next_cpu = next_cpu->next_cpu) {
+                CPUState *env = cur_cpu = next_cpu;
+
+                if (!vm_running)
                     break;
-                /* all CPUs are halted ? */
-                if (env == cur_cpu)
+                if (timer_alarm_pending) {
+                    timer_alarm_pending = 0;
                     break;
-            }
-            cur_cpu = env;
-
-            if (shutdown_requested) {
-                ret = EXCP_INTERRUPT;
-                if (no_shutdown) {
-                    vm_stop(0);
-                    no_shutdown = 0;
                 }
-                else
+                ret = qemu_cpu_exec(env);
+                if (ret == EXCP_DEBUG) {
+                    gdb_set_stop_cpu(env);
                     break;
-            }
-            if (reset_requested) {
-                reset_requested = 0;
-                qemu_system_reset();
-                ret = EXCP_INTERRUPT;
-            }
-            if (powerdown_requested) {
-                powerdown_requested = 0;
-		qemu_system_powerdown();
-                ret = EXCP_INTERRUPT;
-            }
-            if (unlikely(ret == EXCP_DEBUG)) {
-                gdb_set_stop_cpu(cur_cpu);
-                vm_stop(EXCP_DEBUG);
-            }
-            /* If all cpus are halted then wait until the next IRQ */
-            /* XXX: use timeout computed from timers */
-            if (ret == EXCP_HALTED) {
-                if (use_icount) {
-                    int64_t add;
-                    int64_t delta;
-                    /* Advance virtual time to the next event.  */
-                    if (use_icount == 1) {
-                        /* When not using an adaptive execution frequency
-                           we tend to get badly out of sync with real time,
-                           so just delay for a reasonable amount of time.  */
-                        delta = 0;
-                    } else {
-                        delta = cpu_get_icount() - cpu_get_clock();
-                    }
-                    if (delta > 0) {
-                        /* If virtual time is ahead of real time then just
-                           wait for IO.  */
-                        timeout = (delta / 1000000) + 1;
-                    } else {
-                        /* Wait for either IO to occur or the next
-                           timer event.  */
-                        add = qemu_next_deadline();
-                        /* We advance the timer before checking for IO.
-                           Limit the amount we advance so that early IO
-                           activity won't get the guest too far ahead.  */
-                        if (add > 10000000)
-                            add = 10000000;
-                        delta += add;
-                        add = (add + (1 << icount_time_shift) - 1)
-                              >> icount_time_shift;
-                        qemu_icount += add;
-                        timeout = delta / 1000000;
-                        if (timeout < 0)
-                            timeout = 0;
-                    }
-                } else {
-                    timeout = 5000;
                 }
-            } else {
-                timeout = 0;
             }
-        } else {
-            if (shutdown_requested) {
-                ret = EXCP_INTERRUPT;
-                break;
-            }
-            timeout = 5000;
-        }
 #ifdef CONFIG_PROFILER
-        ti = profile_getclock();
+            ti = profile_getclock();
 #endif
-        main_loop_wait(timeout);
+            main_loop_wait(qemu_calculate_timeout());
 #ifdef CONFIG_PROFILER
-        dev_time += profile_getclock() - ti;
+            dev_time += profile_getclock() - ti;
 #endif
+        } while (ret != EXCP_DEBUG && vm_can_run());
+
+        if (ret == EXCP_DEBUG)
+            vm_stop(EXCP_DEBUG);
+
+        if (qemu_shutdown_requested()) {
+            if (no_shutdown) {
+                vm_stop(0);
+                no_shutdown = 0;
+            } else
+                break;
+        }
+        if (qemu_reset_requested())
+            qemu_system_reset();
+        if (qemu_powerdown_requested())
+            qemu_system_powerdown();
     }
-    cpu_disable_ticks();
-    return ret;
 }
 
 static void version(void)
#29 lore raw

PATCH 7/14 qemu: introduce qemu_init_vcpu

In reply to [email protected]
qemu-common.h +6 -0
Index: trunk/qemu-common.h
===================================================================
@@ -189,6 +189,12 @@ void qemu_service_io(void);
 /* Force QEMU to process pending events */
 void qemu_notify_event(void);
 
+#ifdef CONFIG_USER_ONLY
+#define qemu_init_vcpu(env) do { } while (0)
+#else
+void qemu_init_vcpu(void *env);
+#endif
+
 typedef struct QEMUIOVector {
     struct iovec *iov;
     int niov;
target-alpha/translate.c +1 -0
Index: trunk/target-alpha/translate.c
===================================================================
@@ -2505,6 +2505,7 @@ CPUAlphaState * cpu_alpha_init (const ch
     env->ipr[IPR_SISR] = 0;
     env->ipr[IPR_VIRBND] = -1ULL;
 
+    qemu_init_vcpu(env);
     return env;
 }
 
target-arm/helper.c +1 -0
Index: trunk/target-arm/helper.c
===================================================================
@@ -267,6 +267,7 @@ CPUARMState *cpu_arm_init(const char *cp
         gdb_register_coprocessor(env, vfp_gdb_get_reg, vfp_gdb_set_reg,
                                  19, "arm-vfp.xml", 0);
     }
+    qemu_init_vcpu(env);
     return env;
 }
 
target-cris/translate.c +1 -0
Index: trunk/target-cris/translate.c
===================================================================
@@ -3405,6 +3405,7 @@ CPUCRISState *cpu_cris_init (const char 
 
 	cpu_exec_init(env);
 	cpu_reset(env);
+	qemu_init_vcpu(env);
 
 	if (tcg_initialized)
 		return env;
target-i386/helper.c +3 -2
Index: trunk/target-i386/helper.c
===================================================================
@@ -1668,7 +1668,8 @@ CPUX86State *cpu_x86_init(const char *cp
 #ifdef USE_KQEMU
     kqemu_init(env);
 #endif
-    if (kvm_enabled())
-        kvm_init_vcpu(env);
+
+    qemu_init_vcpu(env);
+
     return env;
 }
target-m68k/helper.c +1 -0
Index: trunk/target-m68k/helper.c
===================================================================
@@ -180,6 +180,7 @@ CPUM68KState *cpu_m68k_init(const char *
     }
 
     cpu_reset(env);
+    qemu_init_vcpu(env);
     return env;
 }
 
target-mips/translate.c +1 -0
Index: trunk/target-mips/translate.c
===================================================================
@@ -8551,6 +8551,7 @@ CPUMIPSState *cpu_mips_init (const char 
     env->cpu_model_str = cpu_model;
     mips_tcg_init();
     cpu_reset(env);
+    qemu_init_vcpu(env);
     return env;
 }
 
target-ppc/helper.c +1 -2
Index: trunk/target-ppc/helper.c
===================================================================
@@ -2831,8 +2831,7 @@ CPUPPCState *cpu_ppc_init (const char *c
     cpu_ppc_register_internal(env, def);
     cpu_ppc_reset(env);
 
-    if (kvm_enabled())
-        kvm_init_vcpu(env);
+    qemu_init_vcpu(env);
 
     return env;
 }
target-sh4/translate.c +1 -0
Index: trunk/target-sh4/translate.c
===================================================================
@@ -290,6 +290,7 @@ CPUSH4State *cpu_sh4_init(const char *cp
     cpu_sh4_reset(env);
     cpu_sh4_register(env, def);
     tlb_flush(env, 1);
+    qemu_init_vcpu(env);
     return env;
 }
 
target-sparc/helper.c +1 -0
Index: trunk/target-sparc/helper.c
===================================================================
@@ -723,6 +723,7 @@ CPUSPARCState *cpu_sparc_init(const char
         return NULL;
     }
     cpu_reset(env);
+    qemu_init_vcpu(env);
 
     return env;
 }
vl.c +9 -0
Index: trunk/vl.c
===================================================================
@@ -3663,6 +3663,15 @@ static void qemu_event_increment(void)
 #endif
 #endif
 
+void qemu_init_vcpu(void *_env)
+{
+    CPUState *env = _env;
+
+    if (kvm_enabled())
+        kvm_init_vcpu(env);
+    return;
+}
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
#30 lore raw

PATCH 8/14 qemu: introduce qemu_cpu_kick

In reply to [email protected]

To notify cpu of pending interrupt.

exec.c +11 -0
Index: trunk/exec.c
===================================================================
@@ -1541,6 +1541,17 @@ void cpu_interrupt(CPUState *env, int ma
     old_mask = env->interrupt_request;
     env->interrupt_request |= mask;
 
+#ifndef CONFIG_USER_ONLY
+    /*
+     * If called from iothread context, wake the target cpu in
+     * case its halted.
+     */
+    if (!qemu_cpu_self(env)) {
+        qemu_cpu_kick(env);
+        return;
+    }
+#endif
+
     if (use_icount) {
         env->icount_decr.u16.high = 0xffff;
 #ifndef CONFIG_USER_ONLY
qemu-common.h +4 -0
Index: trunk/qemu-common.h
===================================================================
@@ -189,6 +189,10 @@ void qemu_service_io(void);
 /* Force QEMU to process pending events */
 void qemu_notify_event(void);
 
+/* Unblock cpu */
+void qemu_cpu_kick(void *env);
+int qemu_cpu_self(void *env);
+
 #ifdef CONFIG_USER_ONLY
 #define qemu_init_vcpu(env) do { } while (0)
 #else
vl.c +10 -0
Index: trunk/vl.c
===================================================================
@@ -3672,6 +3672,16 @@ void qemu_init_vcpu(void *_env)
     return;
 }
 
+int qemu_cpu_self(void *env)
+{
+    return 1;
+}
+
+void qemu_cpu_kick(void *env)
+{
+    return;
+}
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
#31 lore raw

PATCH 9/14 qemu: introduce qemu_init_main_loop

In reply to [email protected]

Hook to allow iothread initialization.

vl.c +5 -0
Index: qemu-iothread-4/vl.c
===================================================================
@@ -3682,6 +3682,10 @@ void qemu_cpu_kick(void *env)
     return;
 }
 
+static void qemu_init_main_loop(void)
+{
+}
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
@@ -5106,6 +5110,7 @@ int main(int argc, char **argv, char **e
     if (smp_cpus > 1)
         kqemu_allowed = 0;
 #endif
+    qemu_init_main_loop();
     linux_boot = (kernel_filename != NULL);
     net_boot = (boot_devices_bitmap >> ('n' - 'a')) & 0xF;
 
#32 lore raw

PATCH 10/14 qemu: introduce lock/unlock_iothread

In reply to [email protected]

Hook to allow iothread to drop the global mutex.

vl.c +5 -0
Index: qemu-iothread-4/vl.c
===================================================================
@@ -3686,6 +3686,9 @@ static void qemu_init_main_loop(void)
 {
 }
 
+#define qemu_mutex_lock_iothread() do { } while (0)
+#define qemu_mutex_unlock_iothread() do { } while (0)
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
@@ -3778,7 +3781,9 @@ void main_loop_wait(int timeout)
         slirp_select_fill(&nfds, &rfds, &wfds, &xfds);
     }
 #endif
+    qemu_mutex_unlock_iothread();
     ret = select(nfds + 1, &rfds, &wfds, &xfds, &tv);
+    qemu_mutex_lock_iothread();
     if (ret > 0) {
         IOHandlerRecord **pioh;
 
#33 lore raw

PATCH 11/14 qemu: use debug_requested global instead of cpu_exec return

In reply to [email protected]

Necessary for the next refactoring patch.

vl.c +14 -3
Index: qemu-iothread-4/vl.c
===================================================================
@@ -3504,6 +3504,7 @@ static QEMUResetEntry *first_reset_entry
 static int reset_requested;
 static int shutdown_requested;
 static int powerdown_requested;
+static int debug_requested;
 
 int qemu_shutdown_requested(void)
 {
@@ -3526,6 +3527,13 @@ int qemu_powerdown_requested(void)
     return r;
 }
 
+static int qemu_debug_requested(void)
+{
+    int r = debug_requested;
+    debug_requested = 0;
+    return r;
+}
+
 void qemu_register_reset(QEMUResetHandler *func, void *opaque)
 {
     QEMUResetEntry **pre, *re;
@@ -3955,6 +3963,8 @@ static int vm_can_run(void)
         return 0;
     if (shutdown_requested)
         return 0;
+    if (debug_requested)
+        return 0;
     return 1;
 }
 
@@ -3981,6 +3991,7 @@ static void main_loop(void)
                 ret = qemu_cpu_exec(env);
                 if (ret == EXCP_DEBUG) {
                     gdb_set_stop_cpu(env);
+                    debug_requested = 1;
                     break;
                 }
             }
@@ -3991,11 +4002,11 @@ static void main_loop(void)
 #ifdef CONFIG_PROFILER
             dev_time += profile_getclock() - ti;
 #endif
-        } while (ret != EXCP_DEBUG && vm_can_run());
+        } while (vm_can_run());
 
-        if (ret == EXCP_DEBUG)
-            vm_stop(EXCP_DEBUG);
 
+        if (qemu_debug_requested())
+            vm_stop(EXCP_DEBUG);
         if (qemu_shutdown_requested()) {
             if (no_shutdown) {
                 vm_stop(0);
#34 lore raw

PATCH 12/14 qemu: refactor tcg cpu execution loop

In reply to [email protected]
vl.c +29 -24
Index: qemu-iothread-4/vl.c
===================================================================
@@ -3887,6 +3887,30 @@ static int qemu_cpu_exec(CPUState *env)
     return ret;
 }
 
+static void tcg_cpu_exec(void)
+{
+    int ret;
+
+    if (next_cpu == NULL)
+        next_cpu = first_cpu;
+    for (; next_cpu != NULL; next_cpu = next_cpu->next_cpu) {
+        CPUState *env = cur_cpu = next_cpu;
+
+        if (!vm_running)
+            break;
+        if (timer_alarm_pending) {
+            timer_alarm_pending = 0;
+            break;
+        }
+        ret = qemu_cpu_exec(env);
+        if (ret == EXCP_DEBUG) {
+            gdb_set_stop_cpu(env);
+            debug_requested = 1;
+            break;
+        }
+    }
+}
+
 static int cpu_has_work(CPUState *env)
 {
     if (!env->halted)
@@ -3970,31 +3994,13 @@ static int vm_can_run(void)
 
 static void main_loop(void)
 {
-    int ret = 0;
-#ifdef CONFIG_PROFILER
-    int64_t ti;
-#endif
-
     for (;;) {
-        do {
-            if (next_cpu == NULL)
-                next_cpu = first_cpu;
-            for (; next_cpu != NULL; next_cpu = next_cpu->next_cpu) {
-                CPUState *env = cur_cpu = next_cpu;
 
-                if (!vm_running)
-                    break;
-                if (timer_alarm_pending) {
-                    timer_alarm_pending = 0;
-                    break;
-                }
-                ret = qemu_cpu_exec(env);
-                if (ret == EXCP_DEBUG) {
-                    gdb_set_stop_cpu(env);
-                    debug_requested = 1;
-                    break;
-                }
-            }
+        do {
+#ifdef CONFIG_PROFILER
+            int64_t ti;
+#endif
+            tcg_cpu_exec();
 #ifdef CONFIG_PROFILER
             ti = profile_getclock();
 #endif
@@ -4004,7 +4010,6 @@ static void main_loop(void)
 #endif
         } while (vm_can_run());
 
-
         if (qemu_debug_requested())
             vm_stop(EXCP_DEBUG);
         if (qemu_shutdown_requested()) {
#35 lore raw

PATCH 13/14 qemu: handle stop request in main loop

In reply to [email protected]
vl.c +26 -10
Index: qemu-iothread-4/vl.c
===================================================================
@@ -3483,15 +3483,6 @@ void vm_start(void)
     }
 }
 
-void vm_stop(int reason)
-{
-    if (vm_running) {
-        cpu_disable_ticks();
-        vm_running = 0;
-        vm_state_notify(0, reason);
-    }
-}
-
 /* reset/shutdown handler */
 
 typedef struct QEMUResetEntry {
@@ -3505,6 +3496,7 @@ static int reset_requested;
 static int shutdown_requested;
 static int powerdown_requested;
 static int debug_requested;
+static int vmstop_requested;
 
 int qemu_shutdown_requested(void)
 {
@@ -3534,6 +3526,22 @@ static int qemu_debug_requested(void)
     return r;
 }
 
+static int qemu_vmstop_requested(void)
+{
+    int r = vmstop_requested;
+    vmstop_requested = 0;
+    return r;
+}
+
+static void do_vm_stop(int reason)
+{
+    if (vm_running) {
+        cpu_disable_ticks();
+        vm_running = 0;
+        vm_state_notify(0, reason);
+    }
+}
+
 void qemu_register_reset(QEMUResetHandler *func, void *opaque)
 {
     QEMUResetEntry **pre, *re;
@@ -3697,6 +3705,11 @@ static void qemu_init_main_loop(void)
 #define qemu_mutex_lock_iothread() do { } while (0)
 #define qemu_mutex_unlock_iothread() do { } while (0)
 
+void vm_stop(int reason)
+{
+    do_vm_stop(reason);
+}
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
@@ -3994,8 +4007,9 @@ static int vm_can_run(void)
 
 static void main_loop(void)
 {
-    for (;;) {
+    int r;
 
+    for (;;) {
         do {
 #ifdef CONFIG_PROFILER
             int64_t ti;
@@ -4023,6 +4037,8 @@ static void main_loop(void)
             qemu_system_reset();
         if (qemu_powerdown_requested())
             qemu_system_powerdown();
+        if ((r = qemu_vmstop_requested()))
+            vm_stop(r);
     }
 }
 
#36 lore raw

PATCH 14/14 qemu: introduce iothread

In reply to [email protected]

Fill in the hooks and introduce iothread.

vl.c +385 -22
Index: qemu-iothread-4/vl.c
===================================================================
@@ -1337,19 +1337,19 @@ static void host_alarm_handler(int host_
                                qemu_get_clock(vm_clock))) ||
         qemu_timer_expired(active_timers[QEMU_TIMER_REALTIME],
                            qemu_get_clock(rt_clock))) {
-        CPUState *env = next_cpu;
-
         alarm_timer->flags |= ALARM_FLAG_EXPIRED;
 
-        if (env) {
+#ifndef CONFIG_IOTHREAD
+        if (next_cpu) {
             /* stop the currently executing cpu because a timer occured */
-            cpu_exit(env);
+            cpu_exit(next_cpu);
 #ifdef USE_KQEMU
-            if (env->kqemu_enabled) {
-                kqemu_cpu_interrupt(env);
+            if (next_cpu->kqemu_enabled) {
+                kqemu_cpu_interrupt(next_cpu);
             }
 #endif
         }
+#endif
         timer_alarm_pending = 1;
         qemu_notify_event();
     }
@@ -3473,6 +3473,9 @@ static void vm_state_notify(int running,
     }
 }
 
+static void resume_all_vcpus(void);
+static void pause_all_vcpus(void);
+
 void vm_start(void)
 {
     if (!vm_running) {
@@ -3480,6 +3483,7 @@ void vm_start(void)
         vm_running = 1;
         vm_state_notify(1, 0);
         qemu_rearm_alarm_timer(alarm_timer);
+        resume_all_vcpus();
     }
 }
 
@@ -3538,6 +3542,7 @@ static void do_vm_stop(int reason)
     if (vm_running) {
         cpu_disable_ticks();
         vm_running = 0;
+        pause_all_vcpus();
         vm_state_notify(0, reason);
     }
 }
@@ -3590,18 +3595,13 @@ void qemu_system_powerdown_request(void)
     qemu_notify_event();
 }
 
-void qemu_notify_event(void)
+#ifdef CONFIG_IOTHREAD
+static void qemu_system_vmstop_request(int reason)
 {
-    CPUState *env = cpu_single_env;
-
-    if (env) {
-        cpu_exit(env);
-#ifdef USE_KQEMU
-        if (env->kqemu_enabled)
-            kqemu_cpu_interrupt(env);
-#endif
-     }
+    vmstop_requested = reason;
+    qemu_notify_event();
 }
+#endif
 
 #ifdef CONFIG_IOTHREAD
 #ifndef _WIN32
@@ -3679,6 +3679,17 @@ static void qemu_event_increment(void)
 #endif
 #endif
 
+
+static int cpu_can_run(CPUState *env)
+{
+    if (env->stop)
+        return 0;
+    if (env->stopped)
+        return 0;
+    return 1;
+}
+
+#ifndef CONFIG_IOTHREAD
 void qemu_init_vcpu(void *_env)
 {
     CPUState *env = _env;
@@ -3698,10 +3709,31 @@ void qemu_cpu_kick(void *env)
     return;
 }
 
+static void resume_all_vcpus(void)
+{
+}
+
+static void pause_all_vcpus(void)
+{
+}
+
 static void qemu_init_main_loop(void)
 {
 }
 
+void qemu_notify_event(void)
+{
+    CPUState *env = cpu_single_env;
+
+    if (env) {
+        cpu_exit(env);
+#ifdef USE_KQEMU
+        if (env->kqemu_enabled)
+            kqemu_cpu_interrupt(env);
+#endif
+     }
+}
+
 #define qemu_mutex_lock_iothread() do { } while (0)
 #define qemu_mutex_unlock_iothread() do { } while (0)
 
@@ -3710,6 +3742,315 @@ void vm_stop(int reason)
     do_vm_stop(reason);
 }
 
+#else /* CONFIG_IOTHREAD */
+
+#include "qemu-thread.h"
+
+QemuMutex qemu_global_mutex;
+static QemuMutex qemu_fair_mutex;
+
+static QemuThread io_thread;
+
+static QemuThread *tcg_cpu_thread;
+static QemuCond *tcg_halt_cond;
+
+static int qemu_system_ready;
+/* cpu creation */
+static QemuCond qemu_cpu_cond;
+/* system init */
+static QemuCond qemu_system_cond;
+static QemuCond qemu_pause_cond;
+
+static void block_io_signals(void);
+static void unblock_io_signals(void);
+static int tcg_has_work(void);
+
+static void qemu_init_main_loop(void)
+{
+    qemu_cond_init(&qemu_pause_cond);
+    qemu_mutex_init(&qemu_fair_mutex);
+    qemu_mutex_init(&qemu_global_mutex);
+    qemu_mutex_lock(&qemu_global_mutex);
+
+    qemu_event_init();
+
+    unblock_io_signals();
+    qemu_thread_self(&io_thread);
+}
+
+static void qemu_wait_io_event(CPUState *env)
+{
+    while (!tcg_has_work())
+        qemu_cond_timedwait(env->halt_cond, &qemu_global_mutex, 1000);
+
+    qemu_mutex_unlock(&qemu_global_mutex);
+
+    /*
+     * Users of qemu_global_mutex can be starved, having no chance
+     * to acquire it since this path will get to it first.
+     * So use another lock to provide fairness.
+     */
+    qemu_mutex_lock(&qemu_fair_mutex);
+    qemu_mutex_unlock(&qemu_fair_mutex);
+
+    qemu_mutex_lock(&qemu_global_mutex);
+    if (env->stop) {
+        env->stop = 0;
+        env->stopped = 1;
+        qemu_cond_signal(&qemu_pause_cond);
+    }
+}
+
+static int qemu_cpu_exec(CPUState *env);
+
+static void *kvm_cpu_thread_fn(void *arg)
+{
+    CPUState *env = arg;
+
+    block_io_signals();
+    qemu_thread_self(env->thread);
+
+    /* signal CPU creation */
+    qemu_mutex_lock(&qemu_global_mutex);
+    env->created = 1;
+    qemu_cond_signal(&qemu_cpu_cond);
+
+    /* and wait for machine initialization */
+    while (!qemu_system_ready)
+        qemu_cond_timedwait(&qemu_system_cond, &qemu_global_mutex, 100);
+
+    while (1) {
+        if (cpu_can_run(env))
+            qemu_cpu_exec(env);
+        qemu_wait_io_event(env);
+    }
+
+    return NULL;
+}
+
+static void tcg_cpu_exec(void);
+
+static void *tcg_cpu_thread_fn(void *arg)
+{
+    CPUState *env = arg;
+
+    block_io_signals();
+    qemu_thread_self(env->thread);
+
+    /* signal CPU creation */
+    qemu_mutex_lock(&qemu_global_mutex);
+    for (env = first_cpu; env != NULL; env = env->next_cpu)
+        env->created = 1;
+    qemu_cond_signal(&qemu_cpu_cond);
+
+    /* and wait for machine initialization */
+    while (!qemu_system_ready)
+        qemu_cond_timedwait(&qemu_system_cond, &qemu_global_mutex, 100);
+
+    while (1) {
+        tcg_cpu_exec();
+        qemu_wait_io_event(cur_cpu);
+    }
+
+    return NULL;
+}
+
+void qemu_cpu_kick(void *_env)
+{
+    CPUState *env = _env;
+    qemu_cond_broadcast(env->halt_cond);
+    if (kvm_enabled())
+        qemu_thread_signal(env->thread, SIGUSR1);
+}
+
+int qemu_cpu_self(void *env)
+{
+    return (cpu_single_env != NULL);
+}
+
+static void cpu_signal(int sig)
+{
+    if (cpu_single_env)
+        cpu_exit(cpu_single_env);
+}
+
+static void block_io_signals(void)
+{
+    sigset_t set;
+    struct sigaction sigact;
+
+    sigemptyset(&set);
+    sigaddset(&set, SIGUSR2);
+    sigaddset(&set, SIGIO);
+    sigaddset(&set, SIGALRM);
+    pthread_sigmask(SIG_BLOCK, &set, NULL);
+
+    sigemptyset(&set);
+    sigaddset(&set, SIGUSR1);
+    pthread_sigmask(SIG_UNBLOCK, &set, NULL);
+
+    memset(&sigact, 0, sizeof(sigact));
+    sigact.sa_handler = cpu_signal;
+    sigaction(SIGUSR1, &sigact, NULL);
+}
+
+static void unblock_io_signals(void)
+{
+    sigset_t set;
+
+    sigemptyset(&set);
+    sigaddset(&set, SIGUSR2);
+    sigaddset(&set, SIGIO);
+    sigaddset(&set, SIGALRM);
+    pthread_sigmask(SIG_UNBLOCK, &set, NULL);
+
+    sigemptyset(&set);
+    sigaddset(&set, SIGUSR1);
+    pthread_sigmask(SIG_BLOCK, &set, NULL);
+}
+
+static void qemu_signal_lock(unsigned int msecs)
+{
+    qemu_mutex_lock(&qemu_fair_mutex);
+
+    while (qemu_mutex_trylock(&qemu_global_mutex)) {
+        qemu_thread_signal(tcg_cpu_thread, SIGUSR1);
+        if (!qemu_mutex_timedlock(&qemu_global_mutex, msecs))
+            break;
+    }
+    qemu_mutex_unlock(&qemu_fair_mutex);
+}
+
+static void qemu_mutex_lock_iothread(void)
+{
+    if (kvm_enabled()) {
+        qemu_mutex_lock(&qemu_fair_mutex);
+        qemu_mutex_lock(&qemu_global_mutex);
+        qemu_mutex_unlock(&qemu_fair_mutex);
+    } else
+        qemu_signal_lock(100);
+}
+
+static void qemu_mutex_unlock_iothread(void)
+{
+    qemu_mutex_unlock(&qemu_global_mutex);
+}
+
+static int all_vcpus_paused(void)
+{
+    CPUState *penv = first_cpu;
+
+    while (penv) {
+        if (!penv->stopped)
+            return 0;
+        penv = (CPUState *)penv->next_cpu;
+    }
+
+    return 1;
+}
+
+static void pause_all_vcpus(void)
+{
+    CPUState *penv = first_cpu;
+
+    while (penv) {
+        penv->stop = 1;
+        qemu_thread_signal(penv->thread, SIGUSR1);
+        qemu_cpu_kick(penv);
+        penv = (CPUState *)penv->next_cpu;
+    }
+
+    while (!all_vcpus_paused()) {
+        qemu_cond_timedwait(&qemu_pause_cond, &qemu_global_mutex, 100);
+        penv = first_cpu;
+        while (penv) {
+            qemu_thread_signal(penv->thread, SIGUSR1);
+            penv = (CPUState *)penv->next_cpu;
+        }
+    }
+}
+
+static void resume_all_vcpus(void)
+{
+    CPUState *penv = first_cpu;
+
+    while (penv) {
+        penv->stop = 0;
+        penv->stopped = 0;
+        qemu_thread_signal(penv->thread, SIGUSR1);
+        qemu_cpu_kick(penv);
+        penv = (CPUState *)penv->next_cpu;
+    }
+}
+
+static void tcg_init_vcpu(void *_env)
+{
+    CPUState *env = _env;
+    /* share a single thread for all cpus with TCG */
+    if (!tcg_cpu_thread) {
+        env->thread = qemu_mallocz(sizeof(QemuThread));
+        env->halt_cond = qemu_mallocz(sizeof(QemuCond));
+        qemu_cond_init(env->halt_cond);
+        qemu_thread_create(env->thread, tcg_cpu_thread_fn, env);
+        while (env->created == 0)
+            qemu_cond_timedwait(&qemu_cpu_cond, &qemu_global_mutex, 100);
+        tcg_cpu_thread = env->thread;
+        tcg_halt_cond = env->halt_cond;
+    } else {
+        env->thread = tcg_cpu_thread;
+        env->halt_cond = tcg_halt_cond;
+    }
+}
+
+static void kvm_start_vcpu(CPUState *env)
+{
+    kvm_init_vcpu(env);
+    env->thread = qemu_mallocz(sizeof(QemuThread));
+    env->halt_cond = qemu_mallocz(sizeof(QemuCond));
+    qemu_cond_init(env->halt_cond);
+    qemu_thread_create(env->thread, kvm_cpu_thread_fn, env);
+    while (env->created == 0)
+        qemu_cond_timedwait(&qemu_cpu_cond, &qemu_global_mutex, 100);
+}
+
+void qemu_init_vcpu(void *_env)
+{
+    CPUState *env = _env;
+
+    if (kvm_enabled())
+        kvm_start_vcpu(env);
+    else
+        tcg_init_vcpu(env);
+}
+
+void qemu_notify_event(void)
+{
+    qemu_event_increment();
+}
+
+void vm_stop(int reason)
+{
+    QemuThread me;
+    qemu_thread_self(&me);
+
+    if (!qemu_thread_equal(&me, &io_thread)) {
+        qemu_system_vmstop_request(reason);
+        /*
+         * FIXME: should not return to device code in case
+         * vm_stop() has been requested.
+         */
+        if (cpu_single_env) {
+            cpu_exit(cpu_single_env);
+            cpu_single_env->stop = 1;
+        }
+        return;
+    }
+    do_vm_stop(reason);
+}
+
+#endif
+
+
 #ifdef _WIN32
 static void host_main_loop_wait(int *timeout)
 {
@@ -3846,9 +4187,11 @@ void main_loop_wait(int timeout)
     }
 
     /* vm time timers */
-    if (vm_running && likely(!(cur_cpu->singlestep_enabled & SSTEP_NOTIMER)))
-        qemu_run_timers(&active_timers[QEMU_TIMER_VIRTUAL],
-                        qemu_get_clock(vm_clock));
+    if (vm_running) {
+        if (!cur_cpu || likely(!(cur_cpu->singlestep_enabled & SSTEP_NOTIMER)))
+            qemu_run_timers(&active_timers[QEMU_TIMER_VIRTUAL],
+                qemu_get_clock(vm_clock));
+    }
 
     /* real time timers */
     qemu_run_timers(&active_timers[QEMU_TIMER_REALTIME],
@@ -3902,7 +4245,7 @@ static int qemu_cpu_exec(CPUState *env)
 
 static void tcg_cpu_exec(void)
 {
-    int ret;
+    int ret = 0;
 
     if (next_cpu == NULL)
         next_cpu = first_cpu;
@@ -3915,7 +4258,8 @@ static void tcg_cpu_exec(void)
             timer_alarm_pending = 0;
             break;
         }
-        ret = qemu_cpu_exec(env);
+        if (cpu_can_run(env))
+            ret = qemu_cpu_exec(env);
         if (ret == EXCP_DEBUG) {
             gdb_set_stop_cpu(env);
             debug_requested = 1;
@@ -3926,6 +4270,10 @@ static void tcg_cpu_exec(void)
 
 static int cpu_has_work(CPUState *env)
 {
+    if (env->stop)
+        return 1;
+    if (env->stopped)
+        return 0;
     if (!env->halted)
         return 1;
     if (qemu_cpu_has_work(env))
@@ -4009,16 +4357,27 @@ static void main_loop(void)
 {
     int r;
 
+#ifdef CONFIG_IOTHREAD
+    qemu_system_ready = 1;
+    qemu_cond_broadcast(&qemu_system_cond);
+#endif
+
     for (;;) {
         do {
 #ifdef CONFIG_PROFILER
             int64_t ti;
 #endif
+#ifndef CONFIG_IOTHREAD
             tcg_cpu_exec();
+#endif
 #ifdef CONFIG_PROFILER
             ti = profile_getclock();
 #endif
+#ifdef CONFIG_IOTHREAD
+            main_loop_wait(1000);
+#else
             main_loop_wait(qemu_calculate_timeout());
+#endif
 #ifdef CONFIG_PROFILER
             dev_time += profile_getclock() - ti;
 #endif
@@ -4033,13 +4392,17 @@ static void main_loop(void)
             } else
                 break;
         }
-        if (qemu_reset_requested())
+        if (qemu_reset_requested()) {
+            pause_all_vcpus();
             qemu_system_reset();
+            resume_all_vcpus();
+        }
         if (qemu_powerdown_requested())
             qemu_system_powerdown();
         if ((r = qemu_vmstop_requested()))
             vm_stop(r);
     }
+    pause_all_vcpus();
 }
 
 static void version(void)
cpu-defs.h +5 -0
Index: qemu-iothread-4/cpu-defs.h
===================================================================
@@ -170,6 +170,8 @@ typedef struct CPUWatchpoint {
     target_ulong mem_io_vaddr; /* target virtual addr at which the      \
                                      memory was accessed */             \
     uint32_t halted; /* Nonzero if the CPU is in suspend state */       \
+    uint32_t stop;   /* Stop request */                                 \
+    uint32_t stopped; /* Artificially stopped */                        \
     uint32_t interrupt_request;                                         \
     volatile sig_atomic_t exit_request;                                 \
     /* The meaning of the MMU modes is defined in the target code. */   \
@@ -209,6 +211,9 @@ typedef struct CPUWatchpoint {
     /* user data */                                                     \
     void *opaque;                                                       \
                                                                         \
+    uint32_t created;                                                   \
+    struct QemuThread *thread;                                          \
+    struct QemuCond *halt_cond;                                         \
     const char *cpu_model_str;                                          \
     struct KVMState *kvm_state;                                         \
     struct kvm_run *kvm_run;                                            \