iothread: create gcontext unconditionally
cover iothread: create gcontext unconditionally
When I first read the iothread code, the gcontext confused me for quite a while. Meanwhile, I've been tackling with some races due to this complexity as well. How much we'll pay for creating the gcontext unconditionally? Do we really need this flexibitily (or is it really a flexibility after all)? I don't see much gain of existing code, but I might be wrong. Anyway, I wrote this patchset to see how the list would think about it.
This series directly originates from previous discussion with Marc-Andre where there's a slightly hacky way to try to acquire the gcontext:
https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05460.html
Now with this series logically above patch is not needed any more. Please read patch 4 for more information.
And if this patchset can survive... how about running gcontext directly in iothread_run()? I believe there could be a bit more things to clean but I'll see.
Make check passes for me.
Comments welcomed. Thanks,
Peter Xu (4):
iothread: replace init_done_cond with a semaphore
iothread: create the gcontext onconditionally
iothread: create main loop unconditionally
iothread: push gcontext earlier in the thread_fn
include/sysemu/iothread.h | 5 +-- iothread.c | 77 +++++++++++++++------------------------ 2 files changed, 32 insertions(+), 50 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/4 iothread: replace init_done_cond with a semaphore
Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore.
Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner.
--- include/sysemu/iothread.h | 3 +-- iothread.c | 17 ++++------------- 2 files changed, 5 insertions(+), 15 deletions(-)
include/sysemu/iothread.h +1 -2
@@ -27,8 +27,7 @@ typedef struct {GMainContext *worker_context;GMainLoop *main_loop;GOnce once; -QemuMutex init_done_lock; -QemuCond init_done_cond;/* is thread initialization done? */ +QemuSemaphore init_done_sem;/* is thread init done? */ bool stopping;/* has iothread_stop() been called? */ bool running;/* should iothread_run() continue? */ int thread_id;
iothread.c +4 -13
@@ -55,10 +55,8 @@ static void *iothread_run(void *opaque)rcu_register_thread (); my_iothread = iothread; -qemu_mutex_lock (&iothread->init_done_lock); iothread->thread_id =qemu_get_thread_id (); -qemu_cond_signal (&iothread->init_done_cond); -qemu_mutex_unlock (&iothread->init_done_lock); +qemu_sem_post (&iothread->init_done_sem);while (iothread->running) {aio_poll (iothread->ctx, true);
@@ -115,6 +113,7 @@ static void iothread_instance_init(Object *obj) iothread->poll_max_ns =IOTHREAD_POLL_MAX_NS_DEFAULT ; iothread->thread_id =-1 ; +qemu_sem_init (&iothread->init_done_sem,0 ); }static void iothread_instance_finalize (Object *obj)
@@ -123,10 +122,6 @@ static void iothread_instance_finalize(Object *obj)iothread_stop (iothread); -if (iothread->thread_id !=-1 ) { -qemu_cond_destroy (&iothread->init_done_cond); -qemu_mutex_destroy (&iothread->init_done_lock); - }/* * Before glib2 2.33.10, there is a glib2 bug that GSource context * pointer may not be cleared even if the context has already been
@@ -145,6 +140,7 @@ static void iothread_instance_finalize(Object *obj)g_main_context_unref (iothread->worker_context); iothread->worker_context =NULL ; } +qemu_sem_destroy (&iothread->init_done_sem); }static void iothread_complete (UserCreatable *obj,Error **errp)
@@ -173,8 +169,6 @@ static void iothread_complete(UserCreatable *obj, Error **errp)return ; } -qemu_mutex_init (&iothread->init_done_lock); -qemu_cond_init (&iothread->init_done_cond); iothread->once = (GOnce )G_ONCE_INIT ;/* This assumes we are called from a thread with useful CPU affinity for us
@@ -188,12 +182,9 @@ static void iothread_complete(UserCreatable *obj, Error **errp)g_free (name);/* Wait for initialization to complete */ -qemu_mutex_lock (&iothread->init_done_lock);while (iothread->thread_id ==-1 ) { -qemu_cond_wait (&iothread->init_done_cond, - &iothread->init_done_lock); +qemu_sem_wait (&iothread->init_done_sem); } -qemu_mutex_unlock (&iothread->init_done_lock); } typedef struct {
Signature · 1 line
2.17.1
Re: [Qemu-devel] [PATCH 1/4] iothread: replace init_done_cond with a semaphore
Hi
Quoted 8 lines · Peter Xu
Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore.
Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner.
Signed-off-by Peter Xu <[email protected]>
The lock is also protecting thread_id.
Quoted 88 lines
--- include/sysemu/iothread.h | 3 +-- iothread.c | 17 ++++------------- 2 files changed, 5 insertions(+), 15 deletions(-)include/sysemu/iothread.h +1
-2@@ -27,8 +27,7 @@ typedef struct {GMainContext *worker_context;GMainLoop *main_loop;GOnce once; -QemuMutex init_done_lock; -QemuCond init_done_cond;/* is thread initialization done? */ +QemuSemaphore init_done_sem;/* is thread init done? */ bool stopping;/* has iothread_stop() been called? */ bool running;/* should iothread_run() continue? */ int thread_id;iothread.c +4
-14@@ -55,10 +55,8 @@ static void *iothread_run(void *opaque)rcu_register_thread (); my_iothread = iothread; -qemu_mutex_lock (&iothread->init_done_lock); iothread->thread_id =qemu_get_thread_id (); -qemu_cond_signal (&iothread->init_done_cond); -qemu_mutex_unlock (&iothread->init_done_lock); +qemu_sem_post (&iothread->init_done_sem);while (iothread->running) {aio_poll (iothread->ctx, true);@@ -115,6 +113,7 @@ static void iothread_instance_init(Object *obj) iothread->poll_max_ns =IOTHREAD_POLL_MAX_NS_DEFAULT ; iothread->thread_id =-1 ; +qemu_sem_init (&iothread->init_done_sem,0 ); }static void iothread_instance_finalize (Object *obj)@@ -123,10 +122,6 @@ static void iothread_instance_finalize(Object *obj)iothread_stop (iothread); -if (iothread->thread_id !=-1 ) { -qemu_cond_destroy (&iothread->init_done_cond); -qemu_mutex_destroy (&iothread->init_done_lock); - }/* * Before glib2 2.33.10, there is a glib2 bug that GSource context * pointer may not be cleared even if the context has already been @@ -145,6 +140,7 @@ static void iothread_instance_finalize(Object *obj)g_main_context_unref (iothread->worker_context); iothread->worker_context =NULL ; } +qemu_sem_destroy (&iothread->init_done_sem); }static void iothread_complete (UserCreatable *obj,Error **errp)@@ -173,8 +169,6 @@ static void iothread_complete(UserCreatable *obj, Error **errp)return ; } -qemu_mutex_init (&iothread->init_done_lock); -qemu_cond_init (&iothread->init_done_cond); iothread->once = (GOnce )G_ONCE_INIT ;/* This assumes we are called from a thread with useful CPU affinity for us @@ -188,12 +182,9 @@ static void iothread_complete(UserCreatable *obj, Error **errp)g_free (name);/* Wait for initialization to complete */ -qemu_mutex_lock (&iothread->init_done_lock);while (iothread->thread_id ==-1 ) { -qemu_cond_wait (&iothread->init_done_cond, - &iothread->init_done_lock); +qemu_sem_wait (&iothread->init_done_sem); } -qemu_mutex_unlock (&iothread->init_done_lock); } typedef struct { --2.17.1
Re: [Qemu-devel] [PATCH 1/4] iothread: replace init_done_cond with a semaphore
Quoted 7 lines · Marc-André Lureau
Hi
Quoted 4 lines · Peter Xu
Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore.
Quoted 6 lines
Quoted 4 lines
Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner.
Signed-off-by Peter Xu <[email protected]>The lock is also protecting
thread_id.
IMHO it's fine because thread_id is only changed at the beginning of iothread_run where the caller will definitely wait for the thread_id to be generated. Here qemu_sem_post() should at least contain one write memory barrier there to make sure the waker will read the correct value after sem_wait() and then later on thread_id is never changed.
Regards,
Signature · 1 line
Peter Xu
Re: [Qemu-devel] [PATCH 1/4] iothread: replace init_done_cond with a semaphore
Quoted 13 lines · Peter Xu
Quoted 12 lines · Marc-André Lureau
Hi
Quoted 8 lines · Peter Xu
Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore.
Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner.
Signed-off-by Peter Xu <[email protected]>
Quoted 8 lines
Quoted 1 line
The lock is also protecting
thread_id.IMHO it's fine because
thread_idis only changed at the beginning ofiothread_runwhere the caller will definitely wait for thethread_idto be generated. Hereqemu_sem_post()should at least contain one write memory barrier there to make sure the waker will read the correct value aftersem_wait()and then later onthread_idis never changed.
Yes, qemu_sem_post() is a "release" operation. Anything that happens before it is visible to the thread that does qemu_sem_wait().
In fact, thread_id is accessed outside the lock in iothread_instance_finalize, so Peter's change is overall an improvement.
Paolo
Re: [Qemu-devel] [PATCH 1/4] iothread: replace init_done_cond with a semaphore
Hi
Quoted 24 lines · Paolo Bonzini
Quoted 13 lines · Peter Xu
Quoted 12 lines · Marc-André Lureau
Hi
Quoted 8 lines · Peter Xu
Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore.
Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner.
Signed-off-by Peter Xu <[email protected]>Quoted 8 lines
Quoted 1 line
The lock is also protecting
thread_id.IMHO it's fine because
thread_idis only changed at the beginning ofiothread_runwhere the caller will definitely wait for thethread_idto be generated. Hereqemu_sem_post()should at least contain one write memory barrier there to make sure the waker will read the correct value aftersem_wait()and then later onthread_idis never changed.
Quoted 2 lines
Yes,
qemu_sem_post()is a "release" operation. Anything that happens before it is visible to the thread that doesqemu_sem_wait().
ok
Quoted 3 lines
In fact,
thread_idis accessed outside the lock iniothread_instance_finalize, so Peter's change is overall an improvement.
finalize() stops the threadm though
Signature · 1 line
Marc-André Lureau
Re: [Qemu-devel] [PATCH 1/4] iothread: replace init_done_cond with a semaphore
In reply to Peter XuQuoted 11 lines · Peter Xu
Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore.
Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner.
Signed-off-by Peter Xu <[email protected]>--- include/sysemu/iothread.h | 3 +-- iothread.c | 17 ++++------------- 2 files changed, 5 insertions(+), 15 deletions(-)
PATCH 2/4 iothread: create the gcontext onconditionally
In reply to Peter XuIn existing code we create the gcontext dynamically at the first access of the gcontext from caller. That can bring some complexity and potential races during using iothread. Since the context itself is not that big a resource, and we won't have millions of iothread, let's simply create the gcontext unconditionally.
This will also be a preparation work further to move the thread context push operation earlier than before (now it's only pushed right before we want to start running the gmainloop).
Removing the g_once since it's not necessary, while introducing a new run_gcontext boolean to show whether we want to run the gcontext.
--- include/sysemu/iothread.h | 2 +- iothread.c | 43 +++++++++++++++++++-------------------- 2 files changed, 22 insertions(+), 23 deletions(-)
include/sysemu/iothread.h +1 -1
@@ -24,9 +24,9 @@ typedef struct {QemuThread thread;AioContext *ctx; +bool run_gcontext;/* whether we should run gcontext */ GMainContext *worker_context;GMainLoop *main_loop; -GOnce once;QemuSemaphore init_done_sem;/* is thread init done? */ bool stopping;/* has iothread_stop() been called? */ bool running;/* should iothread_run() continue? */
iothread.c +21 -22
81 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [Qemu-devel] [PATCH 2/4] iothread: create the gcontext onconditionally
Hi
Quoted 49 lines · Peter Xu
In existing code we create the gcontext dynamically at the first access of the gcontext from caller. That can bring some complexity and potential races during using iothread. Since the context itself is not that big a resource, and we won't have millions of iothread, let's simply create the gcontext unconditionally.
This will also be a preparation work further to move the thread context push operation earlier than before (now it's only pushed right before we want to start running the gmainloop).
Removing the
g_oncesince it's not necessary, while introducing a newrun_gcontextboolean to show whether we want to run the gcontext.Signed-off-by Peter Xu <[email protected]>--- include/sysemu/iothread.h | 2 +- iothread.c | 43 +++++++++++++++++++-------------------- 2 files changed, 22 insertions(+), 23 deletions(-)include/sysemu/iothread.h +1
-1@@ -24,9 +24,9 @@ typedef struct {QemuThread thread;AioContext *ctx; +bool run_gcontext;/* whether we should run gcontext */ GMainContext *worker_context;GMainLoop *main_loop; -GOnce once;QemuSemaphore init_done_sem;/* is thread init done? */ bool stopping;/* has iothread_stop() been called? */ bool running;/* should iothread_run() continue? */ iothread.c +1
-1@@ -65,7 +65,7 @@ static void *iothread_run(void *opaque)* We must check the running state again in case it was * changed in previous aio_poll() */ -if (iothread->running &&atomic_read (&iothread->worker_context)) { +if (iothread->running &&atomic_read (&iothread->run_gcontext)) {GMainLoop *loop;g_main_context_push_thread_default (iothread->worker_context);
iothread.c static void iothread_instance_init(Object *obj) · line 118
Quoted 5 lines
iothread.c +2
-0@@ -114,6 +114,8 @@ static void iothread_instance_init(Object *obj) iothread->poll_max_ns =IOTHREAD_POLL_MAX_NS_DEFAULT ; iothread->thread_id =-1 ;qemu_sem_init (&iothread->init_done_sem,0 ); +/* By default, we don't run gcontext */ +atomic_set (&iothread->run_gcontext,0 );
I think that initialization isn't really necessary, your call.
Quoted 73 lines
iothread.c +18
-22}static void iothread_instance_finalize (Object *obj)@@ -143,6 +145,16 @@ static void iothread_instance_finalize(Object *obj)qemu_sem_destroy (&iothread->init_done_sem); } +static void iothread_init_gcontext (IOThread *iothread) +{ +GSource *source; + + iothread->worker_context =g_main_context_new (); + source =aio_get_g_source (iothread_get_aio_context (iothread)); +g_source_attach (source, iothread->worker_context); +g_source_unref (source); +} +static void iothread_complete (UserCreatable *obj,Error **errp) {Error *local_error =NULL ;@@ -157,6 +169,12 @@ static void iothread_complete(UserCreatable *obj, Error **errp)return ; } +/* +* Init one GMainContext for the iothread unconditionally, even if +* it's not used +*/ +iothread_init_gcontext (iothread); + aio_context_set_poll_params(iothread->ctx, iothread->poll_max_ns, iothread->poll_grow,@@ -169,8 +187,6 @@ static void iothread_complete(UserCreatable *obj, Error **errp)return ; } - iothread->once = (GOnce )G_ONCE_INIT ; -/* This assumes we are called from a thread with useful CPU affinity for us * to inherit. */ @@ -333,27 +349,10 @@ IOThreadInfoList *qmp_query_iothreads(Error **errp)return head; } -static gpointer iothread_g_main_context_init (gpointer opaque) -{ -AioContext *ctx; -IOThread *iothread = opaque; -GSource *source; - - iothread->worker_context =g_main_context_new (); - - ctx =iothread_get_aio_context (iothread); - source =aio_get_g_source (ctx); -g_source_attach (source, iothread->worker_context); -g_source_unref (source); - -aio_notify (iothread->ctx); -return NULL ; -} -GMainContext *iothread_get_g_main_context (IOThread *iothread) { -g_once (&iothread->once, iothread_g_main_context_init, iothread); - +atomic_set (&iothread->run_gcontext,1 ); +aio_notify (iothread->ctx);return iothread->worker_context; } --
Quoted 2 lines
2.17.1
looks good otherwise,
Re: [Qemu-devel] [PATCH 2/4] iothread: create the gcontext onconditionally
On Fri, Feb 22, 2019 at 07:29:09AM +0100, Marc-André Lureau wrote:
[...]
Quoted 12 lines
Quoted 12 lines
iothread.c +1
-1@@ -65,7 +65,7 @@ static void *iothread_run(void *opaque)* We must check the running state again in case it was * changed in previous aio_poll() */ -if (iothread->running &&atomic_read (&iothread->worker_context)) { +if (iothread->running &&atomic_read (&iothread->run_gcontext)) {GMainLoop *loop;
Quoted 9 lines
iothread.c static void iothread_instance_init(Object *obj) · line 118
Quoted 7 lines
iothread.c +2
-0g_main_context_push_thread_default (iothread->worker_context);@@ -114,6 +114,8 @@ static void iothread_instance_init(Object *obj) iothread->poll_max_ns =IOTHREAD_POLL_MAX_NS_DEFAULT ; iothread->thread_id =-1 ;qemu_sem_init (&iothread->init_done_sem,0 ); +/* By default, we don't run gcontext */ +atomic_set (&iothread->run_gcontext,0 );I think that initialization isn't really necessary, your call.
True; it's more a hint for readers who suddenly jumped into iothread_run. Thanks for being kind, since I would prefer to have it for now. ;)
[...]
Quoted 2 lines
looks good otherwise,
Reviewed-by Marc-André Lureau <[email protected]>
Thanks for the quick review,
Signature · 1 line
Peter Xu
PATCH 3/4 iothread: create main loop unconditionally
In reply to Peter XuSince we've have the gcontext always there, create the main loop altogether. The iothread_run() is even cleaner.
--- iothread.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
iothread.c +3 -9
@@ -66,17 +66,8 @@ static void *iothread_run(void *opaque)* changed in previous aio_poll() */ if (iothread->running &&atomic_read (&iothread->run_gcontext)) { -GMainLoop *loop; -g_main_context_push_thread_default (iothread->worker_context); - iothread->main_loop = -g_main_loop_new (iothread->worker_context, TRUE); - loop = iothread->main_loop; -g_main_loop_run (iothread->main_loop); - iothread->main_loop =NULL ; -g_main_loop_unref (loop); -g_main_context_pop_thread_default (iothread->worker_context); } }
@@ -141,6 +132,8 @@ static void iothread_instance_finalize(Object *obj)if (iothread->worker_context) {g_main_context_unref (iothread->worker_context); iothread->worker_context =NULL ; +g_main_loop_unref (iothread->main_loop); + iothread->main_loop =NULL ; }qemu_sem_destroy (&iothread->init_done_sem); }
@@ -153,6 +146,7 @@ static void iothread_init_gcontext(IOThread *iothread) source =aio_get_g_source (iothread_get_aio_context (iothread));g_source_attach (source, iothread->worker_context);g_source_unref (source); + iothread->main_loop =g_main_loop_new (iothread->worker_context, TRUE); }static void iothread_complete (UserCreatable *obj,Error **errp)
Signature · 1 line
2.17.1
Re: [Qemu-devel] [PATCH 3/4] iothread: create main loop unconditionally
Quoted 5 lines · Peter Xu
Since we've have the gcontext always there, create the main loop altogether. The
iothread_run()is even cleaner.Signed-off-by Peter Xu <[email protected]>
Quoted 46 lines
--- iothread.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)iothread.c +3
-10@@ -66,17 +66,8 @@ static void *iothread_run(void *opaque)* changed in previous aio_poll() */ if (iothread->running &&atomic_read (&iothread->run_gcontext)) { -GMainLoop *loop; -g_main_context_push_thread_default (iothread->worker_context); - iothread->main_loop = -g_main_loop_new (iothread->worker_context, TRUE); - loop = iothread->main_loop; -g_main_loop_run (iothread->main_loop); - iothread->main_loop =NULL ; -g_main_loop_unref (loop); -g_main_context_pop_thread_default (iothread->worker_context); } }@@ -141,6 +132,8 @@ static void iothread_instance_finalize(Object *obj)if (iothread->worker_context) {g_main_context_unref (iothread->worker_context); iothread->worker_context =NULL ; +g_main_loop_unref (iothread->main_loop); + iothread->main_loop =NULL ; }qemu_sem_destroy (&iothread->init_done_sem); }@@ -153,6 +146,7 @@ static void iothread_init_gcontext(IOThread *iothread) source =aio_get_g_source (iothread_get_aio_context (iothread));g_source_attach (source, iothread->worker_context);g_source_unref (source); + iothread->main_loop =g_main_loop_new (iothread->worker_context, TRUE); }static void iothread_complete (UserCreatable *obj,Error **errp) --2.17.1
PATCH 4/4 iothread: push gcontext earlier in the thread_fn
In reply to Peter XuWe were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before init_done_sem, so as long as the iothread user calls iothread_create() and completes, we know that the thread stack is ready.
--- iothread.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
iothread.c +2 -3
@@ -53,7 +53,7 @@ static void *iothread_run(void *opaque)IOThread *iothread = opaque;rcu_register_thread (); - +g_main_context_push_thread_default (iothread->worker_context); my_iothread = iothread; iothread->thread_id =qemu_get_thread_id ();qemu_sem_post (&iothread->init_done_sem);
@@ -66,12 +66,11 @@ static void *iothread_run(void *opaque)* changed in previous aio_poll() */ if (iothread->running &&atomic_read (&iothread->run_gcontext)) { -g_main_context_push_thread_default (iothread->worker_context);g_main_loop_run (iothread->main_loop); -g_main_context_pop_thread_default (iothread->worker_context); } } +g_main_context_pop_thread_default (iothread->worker_context);rcu_unregister_thread ();return NULL ; }
Signature · 1 line
2.17.1
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from iothread_get_g_main_context() would have default context associated to it.
I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 35 lines
Signed-off-by Peter Xu <[email protected]>--- iothread.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)iothread.c +2
-4@@ -53,7 +53,7 @@ static void *iothread_run(void *opaque)IOThread *iothread = opaque;rcu_register_thread (); - +g_main_context_push_thread_default (iothread->worker_context); my_iothread = iothread; iothread->thread_id =qemu_get_thread_id ();qemu_sem_post (&iothread->init_done_sem);@@ -66,12 +66,11 @@ static void *iothread_run(void *opaque)* changed in previous aio_poll() */ if (iothread->running &&atomic_read (&iothread->run_gcontext)) { -g_main_context_push_thread_default (iothread->worker_context);g_main_loop_run (iothread->main_loop); -g_main_context_pop_thread_default (iothread->worker_context); } } +g_main_context_pop_thread_default (iothread->worker_context);rcu_unregister_thread ();return NULL ; } --2.17.1
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.
Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
Stefan?
Thanks,
Signature · 1 line
Peter Xu
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Quoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
The block layer does not use GSource at all.
Paolo
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
In reply to Peter XuQuoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
I don't understand why Patch 4 is desirable. The comment about init_done_sem isn't clear to me but I also wondered the same thing as Marc-André.
Can you explain why we should apply this patch?
Stefan
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Quoted 30 lines · Stefan Hajnoczi
Quoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
Quoted 5 lines
I don't understand why Patch 4 is desirable. The comment about
init_done_semisn't clear to me but I also wondered the same thing as Marc-André.Can you explain why we should apply this patch?
Hi, Stefan,
The patch 4 itself does not help much for current QEMU, but it should be required to replace the patch that Marc-Andre proposed below:
https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05460.html [1]
And IMHO patch 4 along with this whole series should be a cleaner approach comparing to the one proposed in [1]. Here if my understanding is correct the problem is that g_main_context_push_thread_default() is really designed to be called at the very beginning of a thread creation but not dynamically called during the execution of a thread (prove is that it even does not have any error to return when failed to acquire the context so the caller will never know if it failed! see [2] below), in that sense this patch 4 can be seen as a tiny cleanup too.
g_main_context_push_thread_default (GMainContext *context) {
GQueue *stack;
gboolean acquired_context;
acquired_context = g_main_context_acquire (context); g_return_if_fail (acquired_context); <------------- [2]
...
}
Regards,
Signature · 1 line
Peter Xu
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Quoted 59 lines · Peter Xu
Quoted 30 lines · Stefan Hajnoczi
Quoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
Quoted 5 lines
I don't understand why Patch 4 is desirable. The comment about
init_done_semisn't clear to me but I also wondered the same thing as Marc-André.Can you explain why we should apply this patch?
Hi, Stefan,
The patch 4 itself does not help much for current QEMU, but it should be required to replace the patch that Marc-Andre proposed below:
https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05460.html [1]
And IMHO patch 4 along with this whole series should be a cleaner approach comparing to the one proposed in [1]. Here if my understanding is correct the problem is that
g_main_context_push_thread_default()is really designed to be called at the very beginning of a thread creation but not dynamically called during the execution of a thread (prove is that it even does not have any error to return when failed to acquire the context so the caller will never know if it failed! see [2] below), in that sense this patch 4 can be seen as a tiny cleanup too.
g_main_context_push_thread_default(GMainContext *context) {
GQueue *stack;
gbooleanacquired_context;
Quoted 5 lines
acquired_context = g_main_context_acquire (context); g_return_if_fail (acquired_context); <------------- [2]...
}
I see. This explains why you want to call it early. If you're worried about that then there should also be a comment warning people that this must happen first before anything implicitly uses the thread's GMainContext.
What about Marc-André's concern about the change in behavior? Now this thread is associated with the GMainContext that isn't processed at in aio_poll(). Previously the default main context would be used.
Stefan
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Quoted 63 lines · Stefan Hajnoczi
Quoted 62 lines · Peter Xu
Quoted 30 lines · Stefan Hajnoczi
Quoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
Quoted 5 lines
I don't understand why Patch 4 is desirable. The comment about
init_done_semisn't clear to me but I also wondered the same thing as Marc-André.Can you explain why we should apply this patch?
Hi, Stefan,
The patch 4 itself does not help much for current QEMU, but it should be required to replace the patch that Marc-Andre proposed below:
https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05460.html [1]
And IMHO patch 4 along with this whole series should be a cleaner approach comparing to the one proposed in [1]. Here if my understanding is correct the problem is that
g_main_context_push_thread_default()is really designed to be called at the very beginning of a thread creation but not dynamically called during the execution of a thread (prove is that it even does not have any error to return when failed to acquire the context so the caller will never know if it failed! see [2] below), in that sense this patch 4 can be seen as a tiny cleanup too.
g_main_context_push_thread_default(GMainContext *context) {
GQueue *stack;
gbooleanacquired_context;acquired_context = g_main_context_acquire (context); g_return_if_fail (acquired_context); <------------- [2]
Quoted 7 lines
Quoted 2 lines
...
}I see. This explains why you want to call it early. If you're worried about that then there should also be a comment warning people that this must happen first before anything implicitly uses the thread's GMainContext.
Sure, I can add a comment above g_main_context_push_thread_default() to emphasize why it's preferred at the entry.
Quoted 4 lines
What about Marc-André's concern about the change in behavior? Now this thread is associated with the GMainContext that isn't processed at in
aio_poll(). Previously the default main context would be used.
IIUC Paolo has answered this question (Message-ID:
<[email protected]>) - if the block layer (or say, the explicit aio_poll in the iothread_run) does not use the GMainContext at all then it should affect nothing, and with that there should have no real functional change.
Regards,
Signature · 1 line
Peter Xu
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Hi
Quoted 81 lines · Peter Xu
Quoted 63 lines · Stefan Hajnoczi
Quoted 62 lines · Peter Xu
Quoted 30 lines · Stefan Hajnoczi
Quoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
Quoted 5 lines
I don't understand why Patch 4 is desirable. The comment about
init_done_semisn't clear to me but I also wondered the same thing as Marc-André.Can you explain why we should apply this patch?
Hi, Stefan,
The patch 4 itself does not help much for current QEMU, but it should be required to replace the patch that Marc-Andre proposed below:
https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05460.html [1]
And IMHO patch 4 along with this whole series should be a cleaner approach comparing to the one proposed in [1]. Here if my understanding is correct the problem is that
g_main_context_push_thread_default()is really designed to be called at the very beginning of a thread creation but not dynamically called during the execution of a thread (prove is that it even does not have any error to return when failed to acquire the context so the caller will never know if it failed! see [2] below), in that sense this patch 4 can be seen as a tiny cleanup too.
g_main_context_push_thread_default(GMainContext *context) {
GQueue *stack;
gbooleanacquired_context;acquired_context = g_main_context_acquire (context); g_return_if_fail (acquired_context); <------------- [2]Quoted 7 lines
Quoted 2 lines
...
}I see. This explains why you want to call it early. If you're worried about that then there should also be a comment warning people that this must happen first before anything implicitly uses the thread's GMainContext.
Sure, I can add a comment above
g_main_context_push_thread_default()to emphasize why it's preferred at the entry.Quoted 4 lines
What about Marc-André's concern about the change in behavior? Now this thread is associated with the GMainContext that isn't processed at in
aio_poll(). Previously the default main context would be used.
Quoted 6 lines
IIUC Paolo has answered this question (Message-ID:
<[email protected]>) - if the block layer (or say, the explicitaio_pollin theiothread_run) does not use the GMainContext at all then it should affect nothing, and with that there should have no real functional change.
It's a bold claim though, iothread isn't used only by the block layer. It's used also by colo in qemu tree, and I used to have a branch for virgl rendering for example. There might be other experimental work or usage I missed. Furthermore, even the block layer, and its various dependencies, is hard to review thoroughly. But it looks like it doesn't rely on glib main context, I agree. But I would get prepared for some weird (difficult to debug) regressions eventually.
Re: [Qemu-devel] [PATCH 4/4] iothread: push gcontext earlier in the thread_fn
Quoted 49 lines · Marc-André Lureau
Hi
Quoted 46 lines · Peter Xu
Quoted 44 lines · Stefan Hajnoczi
Quoted 43 lines · Peter Xu
Quoted 30 lines · Stefan Hajnoczi
Quoted 23 lines · Peter Xu
Quoted 13 lines · Marc-André Lureau
Hi
Quoted 9 lines · Peter Xu
We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier.
One benefit is that now it's done even before
init_done_sem, so as long as the iothread user callsiothread_create()and completes, we know that the thread stack is ready.Quoted 8 lines
This will change the default context in the iothread, for code running there. This may not be a good idea. Until now, only sources dispatched from
iothread_get_g_main_context()would have default context associated to it.I don't know if the current behaviour is intentional, but it has some logic. With this change, you may create hidden races, by changing the default context of sources to the iothread.
Quoted 5 lines
Yes I agree that the behavior will be changed in this patch that even if the iothread user does not use the gcontext they'll also have the context set. I would think it should be ok because IMHO events hooked onto the aio context should not depend on the gcontext, but indeed I'd like to get some confirmation from others, especially the block layer.
Quoted 5 lines
I don't understand why Patch 4 is desirable. The comment about
init_done_semisn't clear to me but I also wondered the same thing as Marc-André.Can you explain why we should apply this patch?
Hi, Stefan,
The patch 4 itself does not help much for current QEMU, but it should be required to replace the patch that Marc-Andre proposed below:
https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05460.html [1]
[1]
Quoted 42 lines
Quoted 35 lines
Quoted 19 lines
Quoted 19 lines
And IMHO patch 4 along with this whole series should be a cleaner approach comparing to the one proposed in [1]. Here if my understanding is correct the problem is that
g_main_context_push_thread_default()is really designed to be called at the very beginning of a thread creation but not dynamically called during the execution of a thread (prove is that it even does not have any error to return when failed to acquire the context so the caller will never know if it failed! see [2] below), in that sense this patch 4 can be seen as a tiny cleanup too.
g_main_context_push_thread_default(GMainContext *context) {
GQueue *stack;
gbooleanacquired_context;acquired_context = g_main_context_acquire (context); g_return_if_fail (acquired_context); <------------- [2]Quoted 7 lines
Quoted 2 lines
...
}I see. This explains why you want to call it early. If you're worried about that then there should also be a comment warning people that this must happen first before anything implicitly uses the thread's GMainContext.
Sure, I can add a comment above
g_main_context_push_thread_default()to emphasize why it's preferred at the entry.Quoted 4 lines
What about Marc-André's concern about the change in behavior? Now this thread is associated with the GMainContext that isn't processed at in
aio_poll(). Previously the default main context would be used.Quoted 6 lines
IIUC Paolo has answered this question (Message-ID:
<[email protected]>) - if the block layer (or say, the explicitaio_pollin theiothread_run) does not use the GMainContext at all then it should affect nothing, and with that there should have no real functional change.
Quoted 7 lines
It's a bold claim though, iothread isn't used only by the block layer. It's used also by colo in qemu tree, and I used to have a branch for virgl rendering for example. There might be other experimental work or usage I missed. Furthermore, even the block layer, and its various dependencies, is hard to review thoroughly. But it looks like it doesn't rely on glib main context, I agree. But I would get prepared for some weird (difficult to debug) regressions eventually.
For COLO: it should be using GMainContext all the time (IIRC it's the one who introduced this "push default gcontext" stuff to iothread world...) so it should be fine too? After all it's the only real user of that so far!
For virgl rendering: I'm not sure whether there's assumption on running them on the main thread and whether there would be race against it if running on the iothread (AFAIU that's what the "push default gcontext will affect"), but I would assume there's no such requirement otherwise IMHO it should not need a dedicated iothread at all if it needs serialization on main thread tasks...
For other "experimental work": I can't tell because I never know them.
I'm fine to drop this patch if any of us worries that this patch will break something and I cannot guarantee anything for sure if we're even considering experimental works... but still if any of us wants to move on with [1] above I would still prefer to consider this patch 4.
Regards,
Signature · 1 line
Peter Xu
Quoted 3 lines · Peter Xu
And if this patchset can survive... how about running gcontext directly in
iothread_run()? I believe there could be a bit more things to clean but I'll see.
Do you mean instead of aio_poll? The problem is that GMainContext is quite a bit slower than aio_poll.
Frediano and I tried to bring some of the optimizations of aio_poll to GMainContext
(https://github.com/GNOME/glib/commit/e91c11841808ccca408da96136f433a82b2e2145),
but they broke webkit. :(
Paolo
Quoted 6 lines · Paolo Bonzini
Quoted 3 lines · Peter Xu
And if this patchset can survive... how about running gcontext directly in
iothread_run()? I believe there could be a bit more things to clean but I'll see.Do you mean instead of
aio_poll?
Yes.
Quoted 2 lines
The problem is that GMainContext is
quite a bit slower thanaio_poll.
That's really what I wanted to know; so it's about performance.
We should mention it somewhere in iothread_run. I can do that after I know how this series will go.
Quoted 5 lines
Frediano and I tried to bring some of the optimizations of
aio_pollto GMainContext
(https://github.com/GNOME/glib/commit/e91c11841808ccca408da96136f433a82b2e2145),
but they broke webkit. :(
What a pity!
Signature · 1 line
Peter Xu
Quoted 7 lines · Peter Xu
Comments welcomed. Thanks,
Peter Xu (4):
iothread: replaceinit_done_condwith a semaphore
iothread: create the gcontext onconditionally
iothread: create main loop unconditionally
iothread: push gcontext earlier in thethread_fn
What is the status of this series? Will you send another revision?
Stefan
Quoted 10 lines · Stefan Hajnoczi
Quoted 7 lines · Peter Xu
Comments welcomed. Thanks,
Peter Xu (4):
iothread: replaceinit_done_condwith a semaphore
iothread: create the gcontext onconditionally
iothread: create main loop unconditionally
iothread: push gcontext earlier in thethread_fnWhat is the status of this series? Will you send another revision?
I'll add some comment to patch 4 and repost, probably with another patch to document why we can't drop aio_poll in iothread_run.
Regards,
Signature · 1 line
Peter Xu