qemu-coroutine: use a ring per thread for the pool

This reverts commit ac2662a913ee5854b1269256adbdc14e57ba480a.

Quoted 1 line

This reverts commit ac2662a913ee5854b1269256adbdc14e57ba480a.

Justification?

Won't we hit the same problem as the global pool: either pool size is too large or too small. If you exceed the pool size performance drops dramatically.

qemu-coroutine.c enum { · line 31

Quoted 6 lines

qemu-coroutine.c +5 -0

+static __thread struct CoRoutinePool {
+    Coroutine *ptrs[POOL_MAX_SIZE];
+    unsigned int size;
+    unsigned int nextfree;
+} CoPool;

The per-thread ring unfortunately didn't work well last time it was tested. Devices that do not use ioeventfd (not just the slow ones, even decently performing ones like ahci, nvme or megasas) will create the coroutine in the VCPU thread, and destroy it in the iothread. The result is that coroutines cannot be reused.

Can you check if this is still the case?

Quoted 18 lines

Am 27.11.2014 um 17:40 schrieb Paolo Bonzini:

Quoted 13 lines · Peter Lieven

qemu-coroutine.c enum { · line 31

Quoted 6 lines

qemu-coroutine.c +5 -0

+static __thread struct CoRoutinePool {
+    Coroutine *ptrs[POOL_MAX_SIZE];
+    unsigned int size;
+    unsigned int nextfree;
+} CoPool;

The per-thread ring unfortunately didn't work well last time it was tested. Devices that do not use ioeventfd (not just the slow ones, even decently performing ones like ahci, nvme or megasas) will create the coroutine in the VCPU thread, and destroy it in the iothread. The result is that coroutines cannot be reused.

Quoted 1 line

Can you check if this is still the case?

I already tested at least for IDE and for ioeventfd=off. The coroutine is created in the vCPU thread and destroyed in the I/O thread.

Quoted 6 lines

I also havea more complicated version which sets per therad coroutine pool only for dataplane. Avoiding the lock for dedicated iothreads.

For those who want to take a look:

https://github.com/plieven/qemu/commit/325bc4ef5c7039337fa785744b145e2bdbb7b62e

Can you test it against the patch I just sent in Kevin's linux-aio coroutine thread?

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

Am 28.11.2014 um 12:14 schrieb Paolo Bonzini:

Quoted 7 lines

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Quoted 3 lines

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

indeed:

Run operation 40000000 iterations 10.138684 s, 3945K operations/s, 253ns per coroutine

Quoted 8 lines

Am 28.11.2014 um 12:14 schrieb Paolo Bonzini:

Quoted 7 lines

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Quoted 6 lines

Quoted 3 lines

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

indeed:

Run operation 40000000 iterations 10.138684 s, 3945K operations/s, 253ns per coroutine

Your perf_master2 uses the ring buffer unconditionally, right? I wonder if we can use a similar algorithm but with arrays instead of lists...

Quoted 8 lines

Am 28.11.2014 um 12:14 schrieb Paolo Bonzini:

Quoted 7 lines

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Quoted 6 lines

Quoted 3 lines

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

indeed:

Run operation 40000000 iterations 10.138684 s, 3945K operations/s, 253ns per coroutine

Your perf_master2 uses the ring buffer unconditionally, right? I wonder if we can use a similar algorithm but with arrays instead of lists...

Am 28.11.2014 um 12:23 schrieb Paolo Bonzini:

Quoted 9 lines · Peter Lieven

Quoted 8 lines

Am 28.11.2014 um 12:14 schrieb Paolo Bonzini:

Quoted 7 lines

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

indeed:

Run operation 40000000 iterations 10.138684 s, 3945K operations/s, 253ns per coroutine

Your perf_master2 uses the ring buffer unconditionally, right? I wonder if we can use a similar algorithm but with arrays instead of lists...

Why do you set pool_size = 0 in the create path?

When I do the following:

qemu-coroutine.c +1 -1

index 6bee354..c79ee78 100644
@@ -44,7 +44,7 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
                  * and the actual size of alloc_pool.  But it is just a heuristic,
                  * it does not need to be perfect.
                  */
-                pool_size = 0;
+                atomic_dec(&pool_size);
                 QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
                 co = QSLIST_FIRST(&alloc_pool);
 

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more than one thread in need of a lot of coroutines?

Quoted 4 lines

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more than one thread in need of a lot of coroutines?

Overall the algorithm is expected to adapt. The N threads contribute to the global release pool, so the pool will fill up N times faster than if you had only one thread. There can be some variance, which is why the maximum size of the pool is twice the threshold (and probably could be tuned better).

Benchmarks are needed on real I/O too, of course, especially with high queue depth.

Am 28.11.2014 um 13:26 schrieb Paolo Bonzini:

Quoted 12 lines · Peter Lieven

Quoted 4 lines

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more than one thread in need of a lot of coroutines?

Overall the algorithm is expected to adapt. The N threads contribute to the global release pool, so the pool will fill up N times faster than if you had only one thread. There can be some variance, which is why the maximum size of the pool is twice the threshold (and probably could be tuned better).

Quoted 2 lines

Benchmarks are needed on real I/O too, of course, especially with high queue depth.

Yes, cool. The atomic operations are a bit tricky at the first glance ;-)

Question:
Why is the pool_size increment atomic and the set to zero not?

Idea:
If the release_pool is full why not put the coroutine in the thread alloc_pool instead of throwing it away? :-)

Run operation 40000000 iterations 9.057805 s, 4416K operations/s, 226ns per coroutine

qemu-coroutine.c +14 -6

index 6bee354..edea162 100644
@@ -25,8 +25,9 @@ enum {
 
 /** Free list to speed up creation */
 static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
-static unsigned int pool_size;
+static unsigned int release_pool_size;
 static __thread QSLIST_HEAD(, Coroutine) alloc_pool = QSLIST_HEAD_INITIALIZER(pool);
+static __thread unsigned int alloc_pool_size;
 
 /* The GPrivate is only used to invoke coroutine_pool_cleanup.  */
 static void coroutine_pool_cleanup(void *value);

@@ -39,12 +40,12 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
     if (CONFIG_COROUTINE_POOL) {
         co = QSLIST_FIRST(&alloc_pool);
         if (!co) {
-            if (pool_size > POOL_BATCH_SIZE) {
-                /* This is not exact; there could be a little skew between pool_size
+            if (release_pool_size > POOL_BATCH_SIZE) {
+                /* This is not exact; there could be a little skew between release_pool_size
                  * and the actual size of alloc_pool.  But it is just a heuristic,
                  * it does not need to be perfect.
                  */
-                pool_size = 0;
+                alloc_pool_size = atomic_fetch_and(&release_pool_size, 0);
                 QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
                 co = QSLIST_FIRST(&alloc_pool);
 

@@ -53,6 +54,8 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
                  */
                 g_private_set(&dummy_key, &dummy_key);
             }
+        } else {
+            alloc_pool_size--;
         }
         if (co) {
             QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);

@@ -71,10 +74,15 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
 static void coroutine_delete(Coroutine *co)
 {
     if (CONFIG_COROUTINE_POOL) {
-        if (pool_size < POOL_BATCH_SIZE * 2) {
+        if (release_pool_size < POOL_BATCH_SIZE * 2) {
             co->caller = NULL;
             QSLIST_INSERT_HEAD_ATOMIC(&release_pool, co, pool_next);
-            atomic_inc(&pool_size);
+            atomic_inc(&release_pool_size);
+            return;
+        } else if (alloc_pool_size < POOL_BATCH_SIZE) {
+            co->caller = NULL;
+            QSLIST_INSERT_HEAD(&alloc_pool, co, pool_next);
+            alloc_pool_size++;
             return;
         }
     }

Bug?:
The release_pool is not cleanup up on termination I think.

Quoted 12 lines

Am 28.11.2014 um 13:26 schrieb Paolo Bonzini:

Quoted 11 lines · Peter Lieven

Quoted 4 lines

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more than one thread in need of a lot of coroutines?

Overall the algorithm is expected to adapt. The N threads contribute to the global release pool, so the pool will fill up N times faster than if you had only one thread. There can be some variance, which is why the maximum size of the pool is twice the threshold (and probably could be tuned better).

Quoted 6 lines

Quoted 2 lines

Benchmarks are needed on real I/O too, of course, especially with high queue depth.

Yes, cool. The atomic operations are a bit tricky at the first glance ;-)

Question:
Why is the pool_size increment atomic and the set to zero not?

Because the set to zero is not a read-modify-write operation, so it is always atomic. It's just not sequentially-consistent (see docs/atomics.txt for some info on what that means).

Quoted 2 lines

Idea:
If the release_pool is full why not put the coroutine in the thread alloc_pool instead of throwing it away? :-)

Because you can only waste 64 coroutines per thread. But numbers cannot be sneezed at, so it's worth doing it as a separate patch.

Quoted 3 lines

Quoted 3 lines

Idea:

Quoted 1 line

Quoted 1 line

If the release_pool is full why not put the coroutine in the thread alloc_pool instead of throwing it away? :-)

Because you can only waste 64 coroutines per thread. But numbers cannot

Quoted 1 line

Quoted 1 line

be sneezed at, so it's worth doing it as a separate patch.

What do you mean by that? If I use dataplane I will fill the global pool and never use it okay, but then I use thread local storage only. So I get the same numbers as in my thread local storage only version.

Maybe it is an idea to tweak the POOL_BATCH_SIZE * 2 according to what is really attached. If we have only dataplane or ioeventfd it can be POOL_BATCH_SIZE * 0 and we even won't waste those coroutines oxidating in the global pool.

Quoted 12 lines

Am 28.11.2014 um 13:26 schrieb Paolo Bonzini:

Quoted 11 lines · Peter Lieven

Quoted 4 lines

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more than one thread in need of a lot of coroutines?

Overall the algorithm is expected to adapt. The N threads contribute to the global release pool, so the pool will fill up N times faster than if you had only one thread. There can be some variance, which is why the maximum size of the pool is twice the threshold (and probably could be tuned better).

Quoted 6 lines

Quoted 2 lines

Benchmarks are needed on real I/O too, of course, especially with high queue depth.

Yes, cool. The atomic operations are a bit tricky at the first glance ;-)

Question:
Why is the pool_size increment atomic and the set to zero not?

Because the set to zero is not a read-modify-write operation, so it is always atomic. It's just not sequentially-consistent (see docs/atomics.txt for some info on what that means).

Quoted 2 lines

Idea:
If the release_pool is full why not put the coroutine in the thread alloc_pool instead of throwing it away? :-)

Because you can only waste 64 coroutines per thread. But numbers cannot be sneezed at, so it's worth doing it as a separate patch.

Quoted 62 lines

Run operation 40000000 iterations 9.057805 s, 4416K operations/s, 226ns per coroutine

qemu-coroutine.c +14 -6

index 6bee354..edea162 100644
@@ -25,8 +25,9 @@ enum {
 
 /** Free list to speed up creation */
 static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
-static unsigned int pool_size;
+static unsigned int release_pool_size;
 static __thread QSLIST_HEAD(, Coroutine) alloc_pool = QSLIST_HEAD_INITIALIZER(pool);
+static __thread unsigned int alloc_pool_size;
 
 /* The GPrivate is only used to invoke coroutine_pool_cleanup.  */
 static void coroutine_pool_cleanup(void *value);

@@ -39,12 +40,12 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
     if (CONFIG_COROUTINE_POOL) {
         co = QSLIST_FIRST(&alloc_pool);
         if (!co) {
-            if (pool_size > POOL_BATCH_SIZE) {
-                /* This is not exact; there could be a little skew between pool_size
+            if (release_pool_size > POOL_BATCH_SIZE) {
+                /* This is not exact; there could be a little skew between release_pool_size
                  * and the actual size of alloc_pool.  But it is just a heuristic,
                  * it does not need to be perfect.
                  */
-                pool_size = 0;
+                alloc_pool_size = atomic_fetch_and(&release_pool_size, 0);
                 QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
                 co = QSLIST_FIRST(&alloc_pool);
 

@@ -53,6 +54,8 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
                  */
                 g_private_set(&dummy_key, &dummy_key);
             }
+        } else {
+            alloc_pool_size--;
         }
         if (co) {
             QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);

@@ -71,10 +74,15 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
 static void coroutine_delete(Coroutine *co)
 {
     if (CONFIG_COROUTINE_POOL) {
-        if (pool_size < POOL_BATCH_SIZE * 2) {
+        if (release_pool_size < POOL_BATCH_SIZE * 2) {
             co->caller = NULL;
             QSLIST_INSERT_HEAD_ATOMIC(&release_pool, co, pool_next);
-            atomic_inc(&pool_size);
+            atomic_inc(&release_pool_size);
+            return;
+        } else if (alloc_pool_size < POOL_BATCH_SIZE) {
+            co->caller = NULL;
+            QSLIST_INSERT_HEAD(&alloc_pool, co, pool_next);
+            alloc_pool_size++;
             return;
         }
     }

Quoted 2 lines

Bug?:
The release_pool is not cleanup up on termination I think.

That's not necessary, it is global.

Quoted 2 lines

Quoted 2 lines

Quoted 1 line

Quoted 1 line

The release_pool is not cleanup up on termination I think.

That's not necessary, it is global.

I don't see where you iterate over release_pool and destroy all coroutines?

Quoted 3 lines

Quoted 2 lines

Quoted 2 lines

Quoted 1 line

Quoted 1 line

The release_pool is not cleanup up on termination I think.

That's not necessary, it is global.

I don't see where you iterate over release_pool and destroy all coroutines?

The OS does that for us when we exit.

Am 28.11.2014 um 13:26 schrieb Paolo Bonzini:

Quoted 5 lines · Peter Lieven

Quoted 4 lines

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Ok, understood, it "steals" the whole pool, right? Isn't that bad if we have more than one thread in need of a lot of coroutines?

Quoted 8 lines

Overall the algorithm is expected to adapt. The N threads contribute to the global release pool, so the pool will fill up N times faster than if you had only one thread. There can be some variance, which is why the maximum size of the pool is twice the threshold (and probably could be tuned better).

Benchmarks are needed on real I/O too, of course, especially with high queue depth.

Yes, cool. The atomic operations are a bit tricky at the first glance ;-)

Question:
Why is the pool_size increment atomic and the set to zero not?

Idea:
If the release_pool is full why not put the coroutine in the thread alloc_pool instead of throwing it away? :-)

Run operation 40000000 iterations 9.057805 s, 4416K operations/s, 226ns per coroutine

qemu-coroutine.c +8 -5

index 6bee354..edea162 100644
@@ -25,8 +25,9 @@ enum {
 
 /** Free list to speed up creation */
 static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
-static unsigned int pool_size;
+static unsigned int release_pool_size;
 static __thread QSLIST_HEAD(, Coroutine) alloc_pool = QSLIST_HEAD_INITIALIZER(pool);
+static __thread unsigned int alloc_pool_size;
 
 /* The GPrivate is only used to invoke coroutine_pool_cleanup.  */
 static void coroutine_pool_cleanup(void *value);

@@ -39,12 +40,12 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
     if (CONFIG_COROUTINE_POOL) {
         co = QSLIST_FIRST(&alloc_pool);
         if (!co) {
-            if (pool_size > POOL_BATCH_SIZE) {
-                /* This is not exact; there could be a little skew between pool_size
+            if (release_pool_size > POOL_BATCH_SIZE) {
+                /* This is not exact; there could be a little skew between release_pool_size
                  * and the actual size of alloc_pool.  But it is just a heuristic,
                  * it does not need to be perfect.
                  */
-                pool_size = 0;
+                alloc_pool_size = atomic_fetch_and(&release_pool_size, 0);
                 QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
                 co = QSLIST_FIRST(&alloc_pool);
 

@@ -53,6 +54,8 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
                  */
                 g_private_set(&dummy_key, &dummy_key);
             }
+        } else {
+            alloc_pool_size--;
         }
         if (co) {
             QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);

@@ -71,10 +74,15 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
 static void coroutine_delete(Coroutine *co)
 {
     if (CONFIG_COROUTINE_POOL) {
-        if (pool_size < POOL_BATCH_SIZE * 2) {
+        if (release_pool_size < POOL_BATCH_SIZE * 2) {
             co->caller = NULL;
             QSLIST_INSERT_HEAD_ATOMIC(&release_pool, co, pool_next);

Am 28.11.2014 um 12:23 schrieb Paolo Bonzini:

Quoted 10 lines · Peter Lieven

Quoted 8 lines

Am 28.11.2014 um 12:14 schrieb Paolo Bonzini:

Quoted 7 lines

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

indeed:

Run operation 40000000 iterations 10.138684 s, 3945K operations/s, 253ns per coroutine

Your perf_master2 uses the ring buffer unconditionally, right? I wonder if we can use a similar algorithm but with arrays instead of lists...

Why do you set pool_size = 0 in the create path?

When I do the following:

qemu-coroutine.c +1 -1

index 6bee354..c79ee78 100644
@@ -44,7 +44,7 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
                  * and the actual size of alloc_pool.  But it is just a heuristic,
                  * it does not need to be perfect.
                  */
-                pool_size = 0;
+                atomic_dec(&pool_size);
                 QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
                 co = QSLIST_FIRST(&alloc_pool);
 

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Quoted 35 lines

Am 28.11.2014 um 12:23 schrieb Paolo Bonzini:

Quoted 9 lines · Peter Lieven

Quoted 8 lines

Am 28.11.2014 um 12:14 schrieb Paolo Bonzini:

Quoted 7 lines

Quoted 5 lines

master:
Run operation 40000000 iterations 12.851414 s, 3112K operations/s, 321ns per coroutine

paolo:
Run operation 40000000 iterations 11.951720 s, 3346K operations/s, 298ns per coroutine

Nice. :)

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

Can you please try "coroutine: Use __thread … " together, too? I still see 11% time spent in pthread_getspecific, and I get ~10% more indeed if I apply it here (my times are 191/160/145).

indeed:

Run operation 40000000 iterations 10.138684 s, 3945K operations/s, 253ns per coroutine

Your perf_master2 uses the ring buffer unconditionally, right? I wonder if we can use a similar algorithm but with arrays instead of lists...

Why do you set pool_size = 0 in the create path?

When I do the following:

qemu-coroutine.c +1 -1

index 6bee354..c79ee78 100644
@@ -44,7 +44,7 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
                  * and the actual size of alloc_pool.  But it is just a heuristic,
                  * it does not need to be perfect.
                  */
-                pool_size = 0;
+                atomic_dec(&pool_size);
                 QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
                 co = QSLIST_FIRST(&alloc_pool);
 

Quoted 2 lines

I get:
Run operation 40000000 iterations 9.883958 s, 4046K operations/s, 247ns per coroutine

Because pool_size is the (approximate) number of coroutines in the pool.
It is zero after QSLIST_MOVE_ATOMIC has NULL-ed out release_pool.slh_first.

iothread.c +1 -0

index 342a23f..b53529b 100644
@@ -15,6 +15,7 @@
 #include "qom/object_interfaces.h"
 #include "qemu/module.h"
 #include "block/aio.h"
+#include "block/coroutine.h"
 #include "sysemu/iothread.h"
 #include "qmp-commands.h"
 #include "qemu/error-report.h"