Skip to contentSkip to conversation

dataplane: add query-iothreads QMP command

PATCH Applied Stefan Hajnoczi 4 participants 7 messages Feb 27, 2014 – Mar 11, 2014

lore
Share & tools
lore
#0 lore raw

v3 cover dataplane: add query-iothreads QMP command

v3:
* Loop until predicate is true to handle pthread_cond spurious wakeups [eblake]
* Fix additional instances of "thread_id" [eblake]

v2:
* Use "thread-id" instead of "thread_id" in QAPI [eblake]
* Avoid mutex unlock/destroy race between threads [pbonzini]

This series applies on top of "[PATCH v4 0/6] dataplane: switch to N:M devices-per-thread model".

The new "query-iothreads" command allows QMP clients to fetch the thread IDs for dataplane threads. This will allow libvirt and friends to bind dataplane threads.

The approach is similar to "query-cpus" which is also used for vcpu thread binding (among other things).

Stefan Hajnoczi (2):
iothread: stash thread ID away
qmp: add query-iothreads command

 iothread.c       | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json | 29 ++++++++++++++++++++++++++++
 qmp-commands.hx  | 39 +++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
Signature · 1 line
1.8.5.3
#1 lore raw

PATCH v3 1/2 iothread: stash thread ID away

Keep the thread ID around so we can report it via QMP.

There's only one problem: qemu_get_thread_id() (gettid() wrapper on Linux) must be called from the thread itself. There is no way to get the thread ID outside the thread.

This patch uses a condvar to wait for iothread_run() to populate the thread_id inside the thread.

---
 iothread.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
iothread.c +22 -0
index 033de7f..0a3c048 100644
@@ -25,7 +25,10 @@ struct IOThread {
     Object parent;
     QemuThread thread;
     AioContext *ctx;
+    QemuMutex init_done_lock;
+    QemuCond init_done_cond;    /* is thread initialization done? */
     bool stopping;
+    int thread_id;
 };
 
 #define IOTHREAD_GET_CLASS(obj) \
@@ -37,6 +40,11 @@ static void *iothread_run(void *opaque)
 {
     IOThread *iothread = opaque;
 
+    qemu_mutex_lock(&iothread->init_done_lock);
+    iothread->thread_id = qemu_get_thread_id();
+    qemu_cond_signal(&iothread->init_done_cond);
+    qemu_mutex_unlock(&iothread->init_done_lock);
+
     while (!iothread->stopping) {
         aio_context_acquire(iothread->ctx);
         while (!iothread->stopping && aio_poll(iothread->ctx, true)) {
@@ -54,6 +62,8 @@ static void iothread_instance_finalize(Object *obj)
     iothread->stopping = true;
     aio_notify(iothread->ctx);
     qemu_thread_join(&iothread->thread);
+    qemu_cond_destroy(&iothread->init_done_cond);
+    qemu_mutex_destroy(&iothread->init_done_lock);
     aio_context_unref(iothread->ctx);
 }
 
@@ -63,12 +73,24 @@ static void iothread_complete(UserCreatable *obj, Error **errp)
 
     iothread->stopping = false;
     iothread->ctx = aio_context_new();
+    iothread->thread_id = -1;
+
+    qemu_mutex_init(&iothread->init_done_lock);
+    qemu_cond_init(&iothread->init_done_cond);
 
     /* This assumes we are called from a thread with useful CPU affinity for us
      * to inherit.
      */
     qemu_thread_create(&iothread->thread, iothread_run,
                        iothread, QEMU_THREAD_JOINABLE);
+
+    /* Wait for initialization to complete */
+    qemu_mutex_lock(&iothread->init_done_lock);
+    while (iothread->thread_id == -1) {
+        qemu_cond_wait(&iothread->init_done_cond,
+                       &iothread->init_done_lock);
+    }
+    qemu_mutex_unlock(&iothread->init_done_lock);
 }
 
 static void iothread_class_init(ObjectClass *klass, void *class_data)
Signature · 1 line
1.8.5.3
#2 lore raw

PATCH v3 2/2 qmp: add query-iothreads command

In reply to Stefan Hajnoczi

The "query-iothreads" command returns a list of information about iothreads. See the patch for API documentation.

---
 iothread.c       | 36 ++++++++++++++++++++++++++++++++++++
 qapi-schema.json | 29 +++++++++++++++++++++++++++++
 qmp-commands.hx  | 39 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 104 insertions(+)
iothread.c +36 -0
index 0a3c048..e348706 100644
@@ -17,6 +17,7 @@
 #include "qemu/thread.h"
 #include "block/aio.h"
 #include "sysemu/iothread.h"
+#include "qmp-commands.h"
 
 #define IOTHREADS_PATH "/objects"
 
@@ -143,3 +144,38 @@ AioContext *iothread_get_aio_context(IOThread *iothread)
 {
     return iothread->ctx;
 }
+
+static int query_one_iothread(Object *object, void *opaque)
+{
+    IOThreadInfoList ***prev = opaque;
+    IOThreadInfoList *elem;
+    IOThreadInfo *info;
+    IOThread *iothread;
+
+    iothread = (IOThread *)object_dynamic_cast(object, TYPE_IOTHREAD);
+    if (!iothread) {
+        return 0;
+    }
+
+    info = g_new0(IOThreadInfo, 1);
+    info->id = iothread_get_id(iothread);
+    info->thread_id = iothread->thread_id;
+
+    elem = g_new0(IOThreadInfoList, 1);
+    elem->value = info;
+    elem->next = NULL;
+
+    **prev = elem;
+    *prev = &elem->next;
+    return 0;
+}
+
+IOThreadInfoList *qmp_query_iothreads(Error **errp)
+{
+    IOThreadInfoList *head = NULL;
+    IOThreadInfoList **prev = &head;
+    Object *container = container_get(object_get_root(), IOTHREADS_PATH);
+
+    object_child_foreach(container, query_one_iothread, &prev);
+    return head;
+}
qapi-schema.json +29 -0
index ac8ad24..9888418 100644
@@ -883,6 +883,35 @@
 { 'command': 'query-cpus', 'returns': ['CpuInfo'] }
 
 ##
+# @IOThreadInfo:
+#
+# Information about an iothread
+#
+# @id: the identifier of the iothread
+#
+# @thread-id: ID of the underlying host thread
+#
+# Since: 2.0
+##
+{ 'type': 'IOThreadInfo',
+  'data': {'id': 'str', 'thread-id': 'int'} }
+
+##
+# @query-iothreads:
+#
+# Returns a list of information about each iothread.
+#
+# Note this list excludes the QEMU main loop thread, which is not declared
+# using the -object iothread command-line option.  It is always the main thread
+# of the process.
+#
+# Returns: a list of @IOThreadInfo for each iothread
+#
+# Since: 2.0
+##
+{ 'command': 'query-iothreads', 'returns': ['IOThreadInfo'] }
+
+##
 # @BlockDeviceInfo:
 #
 # Information about the backing device for a block device.
qmp-commands.hx +39 -0
index 8a0e832..f3407da 100644
@@ -2304,6 +2304,45 @@ EQMP
     },
 
 SQMP
+query-iothreads
+---------------
+
+Returns a list of information about each iothread.
+
+Note this list excludes the QEMU main loop thread, which is not declared
+using the -object iothread command-line option.  It is always the main thread
+of the process.
+
+Return a json-array. Each iothread is represented by a json-object, which contains:
+
+- "id": name of iothread (json-str)
+- "thread-id": ID of the underlying host thread (json-int)
+
+Example:
+
+-> { "execute": "query-iothreads" }
+<- {
+      "return":[
+         {
+            "id":"iothread0",
+            "thread-id":3134
+         },
+         {
+            "id":"iothread1",
+            "thread-id":3135
+         }
+      ]
+   }
+
+EQMP
+
+    {
+        .name       = "query-iothreads",
+        .args_type  = "",
+        .mhandler.cmd_new = qmp_marshal_input_query_iothreads,
+    },
+
+SQMP
 query-pci
 ---------
 
Signature · 1 line
1.8.5.3
#3 lore raw
In reply to Stefan Hajnoczi
Quoted 10 lines · Stefan Hajnoczi

v3:
* Loop until predicate is true to handle pthread_cond spurious wakeups [eblake]
* Fix additional instances of "thread_id" [eblake]

v2:
* Use "thread-id" instead of "thread_id" in QAPI [eblake]
* Avoid mutex unlock/destroy race between threads [pbonzini]

This series applies on top of "[PATCH v4 0/6] dataplane: switch to N:M devices-per-thread model".

Series: Reviewed-by: Eric Blake <[email protected]>

Quoted 17 lines

The new "query-iothreads" command allows QMP clients to fetch the thread IDs for dataplane threads. This will allow libvirt and friends to bind dataplane threads.

The approach is similar to "query-cpus" which is also used for vcpu thread binding (among other things).

Stefan Hajnoczi (2):
iothread: stash thread ID away
qmp: add query-iothreads command

 iothread.c       | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json | 29 ++++++++++++++++++++++++++++
 qmp-commands.hx  | 39 +++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#4 lore raw
In reply to Stefan Hajnoczi
Quoted 10 lines · On Thu, 27 Feb 2014 11:48:40 +0100 Stefan Hajnoczi <[email protected]> wrote:

v3:
* Loop until predicate is true to handle pthread_cond spurious wakeups [eblake]
* Fix additional instances of "thread_id" [eblake]

v2:
* Use "thread-id" instead of "thread_id" in QAPI [eblake]
* Avoid mutex unlock/destroy race between threads [pbonzini]

This series applies on top of "[PATCH v4 0/6] dataplane: switch to N:M devices-per-thread model".

What's the merge plan for both series? IMO, they should go in through the block layer tree. As far as the QMP interface is concerned, Eric's review is all you need.

Quoted 17 lines

The new "query-iothreads" command allows QMP clients to fetch the thread IDs for dataplane threads. This will allow libvirt and friends to bind dataplane threads.

The approach is similar to "query-cpus" which is also used for vcpu thread binding (among other things).

Stefan Hajnoczi (2):
iothread: stash thread ID away
qmp: add query-iothreads command

 iothread.c       | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json | 29 ++++++++++++++++++++++++++++
 qmp-commands.hx  | 39 +++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
#5 lore raw
Quoted 11 lines · Luiz Capitulino
Quoted 8 lines · On Thu, 27 Feb 2014 11:48:40 +0100 Stefan Hajnoczi <[email protected]> wrote:

v3:
* Loop until predicate is true to handle pthread_cond spurious wakeups [eblake]
* Fix additional instances of "thread_id" [eblake]

v2:
* Use "thread-id" instead of "thread_id" in QAPI [eblake]
* Avoid mutex unlock/destroy race between threads [pbonzini]

Quoted 6 lines
Quoted 2 lines

This series applies on top of "[PATCH v4 0/6] dataplane: switch to N:M devices-per-thread model".

What's the merge plan for both series? IMO, they should go in through the block layer tree. As far as the QMP interface is concerned, Eric's review is all you need.

Okay, thanks.

They can go through Kevin if he's ready to merge them this week, or me next week otherwise.

Stefan

#6 lore raw
In reply to Stefan Hajnoczi
Quoted 18 lines · Stefan Hajnoczi

v3:
* Loop until predicate is true to handle pthread_cond spurious wakeups [eblake]
* Fix additional instances of "thread_id" [eblake]

v2:
* Use "thread-id" instead of "thread_id" in QAPI [eblake]
* Avoid mutex unlock/destroy race between threads [pbonzini]

This series applies on top of "[PATCH v4 0/6] dataplane: switch to N:M devices-per-thread model".

The new "query-iothreads" command allows QMP clients to fetch the thread IDs for dataplane threads. This will allow libvirt and friends to bind dataplane threads.

The approach is similar to "query-cpus" which is also used for vcpu thread binding (among other things).

Quoted 8 lines

Stefan Hajnoczi (2):
iothread: stash thread ID away
qmp: add query-iothreads command

 iothread.c       | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qapi-schema.json | 29 ++++++++++++++++++++++++++++
 qmp-commands.hx  | 39 +++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)

Applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan