Skip to contentSkip to conversation

block-copy: make helper APIs thread safe

PATCH Applied Emanuele Giuseppe Esposito 2 participants 12 messages Jun 14, 2021 – Jun 25, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover block-copy: make helper APIs thread safe

This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask.
The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

What's missing for block-copy to be fully thread-safe is fixing the CoSleep API to allow cross-thread sleep and wakeup. Paolo is working on it and will post the patches once his new CoSleep API is accepted.

Patches 1-3 work on ratelimit, 4 covers progressmeter and 5 co-shared-resources.

---
v3:
* Rebase on current master (had conflicts in block-copy), remove based-on in cover letter

Emanuele Giuseppe Esposito (2):
progressmeter: protect with a mutex
co-shared-resource: protect with a mutex

Paolo Bonzini (3):
ratelimit: treat zero speed as unlimited
block-copy: let ratelimit handle a speed of 0
blockjob: let ratelimit handle a speed of 0

 block/block-copy.c                | 28 ++++++--------
 block/meson.build                 |  1 +
 block/progress_meter.c            | 64 +++++++++++++++++++++++++++++++
 blockjob.c                        | 46 +++++++++++++---------
 include/qemu/co-shared-resource.h |  4 +-
 include/qemu/progress_meter.h     | 34 ++++++++--------
 include/qemu/ratelimit.h          | 12 +++++-
 job-qmp.c                         |  8 +++-
 job.c                             |  3 ++
 qemu-img.c                        |  9 +++--
 util/qemu-co-shared-resource.c    | 24 +++++++++---
 11 files changed, 168 insertions(+), 65 deletions(-)
 create mode 100644 block/progress_meter.c
Signature · 1 line
2.31.1
#1 lore raw

PATCH v3 1/5 ratelimit: treat zero speed as unlimited

From: Paolo Bonzini <[email protected]>

Both users of RateLimit, block-copy.c and blockjob.c, treat a speed of zero as unlimited, while RateLimit treats it as "as slow as possible". The latter is nicer from the code point of view but pretty useless, so disable rate limiting if a speed of zero is provided.

---
 include/qemu/ratelimit.h | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
include/qemu/ratelimit.h +10 -2
index 003ea6d5a3..48bf59e857 100644
@@ -43,7 +43,11 @@ static inline int64_t ratelimit_calculate_delay(RateLimit *limit, uint64_t n)
     double delay_slices;
 
     QEMU_LOCK_GUARD(&limit->lock);
-    assert(limit->slice_quota && limit->slice_ns);
+    if (!limit->slice_quota) {
+        /* Throttling disabled.  */
+        return 0;
+    }
+    assert(limit->slice_ns);
 
     if (limit->slice_end_time < now) {
         /* Previous, possibly extended, time slice finished; reset the
@@ -83,7 +87,11 @@ static inline void ratelimit_set_speed(RateLimit *limit, uint64_t speed,
 {
     QEMU_LOCK_GUARD(&limit->lock);
     limit->slice_ns = slice_ns;
-    limit->slice_quota = MAX(((double)speed * slice_ns) / 1000000000ULL, 1);
+    if (speed == 0) {
+        limit->slice_quota = 0;
+    } else {
+        limit->slice_quota = MAX(((double)speed * slice_ns) / 1000000000ULL, 1);
+    }
 }
 
 #endif
Signature · 1 line
2.31.1
#2 lore raw

PATCH v3 2/5 block-copy: let ratelimit handle a speed of 0

In reply to Emanuele Giuseppe Esposito

From: Paolo Bonzini <[email protected]>

---
 block/block-copy.c | 28 +++++++++++-----------------
 1 file changed, 11 insertions(+), 17 deletions(-)
block/block-copy.c +11 -17
index 5808cfe657..943e30b7e6 100644
@@ -114,7 +114,6 @@ typedef struct BlockCopyState {
 
     SharedResource *mem;
 
-    uint64_t speed;
     RateLimit rate_limit;
 } BlockCopyState;
 
@@ -647,21 +646,19 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
             task->copy_range = false;
         }
 
-        if (s->speed) {
-            if (!call_state->ignore_ratelimit) {
-                uint64_t ns = ratelimit_calculate_delay(&s->rate_limit, 0);
-                if (ns > 0) {
-                    block_copy_task_end(task, -EAGAIN);
-                    g_free(task);
-                    qemu_co_sleep_ns_wakeable(&call_state->sleep,
-                                              QEMU_CLOCK_REALTIME, ns);
-                    continue;
-                }
+        if (!call_state->ignore_ratelimit) {
+            uint64_t ns = ratelimit_calculate_delay(&s->rate_limit, 0);
+            if (ns > 0) {
+                block_copy_task_end(task, -EAGAIN);
+                g_free(task);
+                qemu_co_sleep_ns_wakeable(&call_state->sleep,
+                                            QEMU_CLOCK_REALTIME, ns);
+                continue;
             }
-
-            ratelimit_calculate_delay(&s->rate_limit, task->bytes);
         }
 
+        ratelimit_calculate_delay(&s->rate_limit, task->bytes);
+
         trace_block_copy_process(s, task->offset);
 
         co_get_from_shres(s->mem, task->bytes);
@@ -853,10 +850,7 @@ void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
 
 void block_copy_set_speed(BlockCopyState *s, uint64_t speed)
 {
-    s->speed = speed;
-    if (speed > 0) {
-        ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME);
-    }
+    ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME);
 
     /*
      * Note: it's good to kick all call states from here, but it should be done
Signature · 1 line
2.31.1
#3 lore raw

Re: [PATCH v3 2/5] block-copy: let ratelimit handle a speed of 0

Quoted 32 lines · Emanuele Giuseppe Esposito

From: Paolo Bonzini <[email protected]>

---
  block/block-copy.c | 28 +++++++++++-----------------
  1 file changed, 11 insertions(+), 17 deletions(-)
block/block-copy.c +0 -8
index 5808cfe657..943e30b7e6 100644
@@ -114,7 +114,6 @@ typedef struct BlockCopyState {
 
      SharedResource *mem;
 
-    uint64_t speed;
      RateLimit rate_limit;
  } BlockCopyState;
 
@@ -647,21 +646,19 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
              task->copy_range = false;
          }
 
-        if (s->speed) {
-            if (!call_state->ignore_ratelimit) {
-                uint64_t ns = ratelimit_calculate_delay(&s->rate_limit, 0);
-                if (ns > 0) {
-                    block_copy_task_end(task, -EAGAIN);
-                    g_free(task);
-                    qemu_co_sleep_ns_wakeable(&call_state->sleep,

block/block-copy.c block_copy_dirty_clusters(BlockCopyCallState *call_state) · line 655

Quoted 10 lines
block/block-copy.c +7 -3
-                                              QEMU_CLOCK_REALTIME, ns);
-                    continue;
-                }
+        if (!call_state->ignore_ratelimit) {
+            uint64_t ns = ratelimit_calculate_delay(&s->rate_limit, 0);
+            if (ns > 0) {
+                block_copy_task_end(task, -EAGAIN);
+                g_free(task);
+                qemu_co_sleep_ns_wakeable(&call_state->sleep,
+                                            QEMU_CLOCK_REALTIME, ns);

Indentation broken. Can be fixed when applying, not worth resending

Quoted 24 lines
block/block-copy.c +4 -6
+                continue;
              }
-
-            ratelimit_calculate_delay(&s->rate_limit, task->bytes);
          }
 
+        ratelimit_calculate_delay(&s->rate_limit, task->bytes);
+
          trace_block_copy_process(s, task->offset);
 
          co_get_from_shres(s->mem, task->bytes);
@@ -853,10 +850,7 @@ void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
 
  void block_copy_set_speed(BlockCopyState *s, uint64_t speed)
  {
-    s->speed = speed;
-    if (speed > 0) {
-        ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME);
-    }
+    ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME);
 
      /*
       * Note: it's good to kick all call states from here, but it should be done
Signature · 2 lines
Best regards,
Vladimir
#4 lore raw

PATCH v3 3/5 blockjob: let ratelimit handle a speed of 0

In reply to Emanuele Giuseppe Esposito

From: Paolo Bonzini <[email protected]>

---
 blockjob.c | 13 +++----------
 1 file changed, 3 insertions(+), 10 deletions(-)
blockjob.c +3 -10
index dc1d9e0e46..22e5bb9b1f 100644
@@ -300,10 +300,6 @@ bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
 
 int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n)
 {
-    if (!job->speed) {
-        return 0;
-    }
-
     return ratelimit_calculate_delay(&job->limit, n);
 }
 
@@ -472,12 +468,9 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,
     blk_set_disable_request_queuing(blk, true);
     blk_set_allow_aio_context_change(blk, true);
 
-    /* Only set speed when necessary to avoid NotSupported error */
-    if (speed != 0) {
-        if (!block_job_set_speed(job, speed, errp)) {
-            job_early_fail(&job->job);
-            return NULL;
-        }
+    if (!block_job_set_speed(job, speed, errp)) {
+        job_early_fail(&job->job);
+        return NULL;
     }
 
     return job;
Signature · 1 line
2.31.1
#5 lore raw

PATCH v3 4/5 progressmeter: protect with a mutex

In reply to Emanuele Giuseppe Esposito

Progressmeter is protected by the AioContext mutex, which is taken by the block jobs and their caller (like blockdev).

We would like to remove the dependency of block layer code on the AioContext mutex, since most drivers and the core I/O code are already not relying on it.

Create a new C file to implement the ProgressMeter API, but keep the struct as public, to avoid forcing allocation on the heap.

Also add a mutex to be able to provide an accurate snapshot of the progress values to the caller.

---
 block/meson.build             |  1 +
 block/progress_meter.c        | 64 +++++++++++++++++++++++++++++++++++
 blockjob.c                    | 33 +++++++++++++-----
 include/qemu/progress_meter.h | 34 +++++++++++--------
 job-qmp.c                     |  8 +++--
 job.c                         |  3 ++
 qemu-img.c                    |  9 +++--
 7 files changed, 124 insertions(+), 28 deletions(-)
 create mode 100644 block/progress_meter.c
block/meson.build +1 -0
index 01861e1545..ef1ba3d973 100644
@@ -13,6 +13,7 @@ block_ss.add(files(
   'commit.c',
   'copy-on-read.c',
   'preallocate.c',
+  'progress_meter.c',
   'create.c',
   'crypto.c',
   'dirty-bitmap.c',
block/progress_meter.c new file +64 -0

64 lines not on this page · raw message

blockjob.c +25 -8
index 22e5bb9b1f..4bad1408cb 100644
@@ -306,18 +306,23 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n)
 BlockJobInfo *block_job_query(BlockJob *job, Error **errp)
 {
     BlockJobInfo *info;
+    uint64_t progress_current, progress_total;
 
     if (block_job_is_internal(job)) {
         error_setg(errp, "Cannot query QEMU internal jobs");
         return NULL;
     }
+
+    progress_get_snapshot(&job->job.progress, &progress_current,
+                          &progress_total);
+
     info = g_new0(BlockJobInfo, 1);
     info->type      = g_strdup(job_type_str(&job->job));
     info->device    = g_strdup(job->job.id);
     info->busy      = qatomic_read(&job->job.busy);
     info->paused    = job->job.pause_count > 0;
-    info->offset    = job->job.progress.current;
-    info->len       = job->job.progress.total;
+    info->offset    = progress_current;
+    info->len       = progress_total;
     info->speed     = job->speed;
     info->io_status = job->iostatus;
     info->ready     = job_is_ready(&job->job),
@@ -344,15 +349,19 @@ static void block_job_iostatus_set_err(BlockJob *job, int error)
 static void block_job_event_cancelled(Notifier *n, void *opaque)
 {
     BlockJob *job = opaque;
+    uint64_t progress_current, progress_total;
 
     if (block_job_is_internal(job)) {
         return;
     }
 
+    progress_get_snapshot(&job->job.progress, &progress_current,
+                          &progress_total);
+
     qapi_event_send_block_job_cancelled(job_type(&job->job),
                                         job->job.id,
-                                        job->job.progress.total,
-                                        job->job.progress.current,
+                                        progress_total,
+                                        progress_current,
                                         job->speed);
 }
 
@@ -360,6 +369,7 @@ static void block_job_event_completed(Notifier *n, void *opaque)
 {
     BlockJob *job = opaque;
     const char *msg = NULL;
+    uint64_t progress_current, progress_total;
 
     if (block_job_is_internal(job)) {
         return;
@@ -369,10 +379,13 @@ static void block_job_event_completed(Notifier *n, void *opaque)
         msg = error_get_pretty(job->job.err);
     }
 
+    progress_get_snapshot(&job->job.progress, &progress_current,
+                          &progress_total);
+
     qapi_event_send_block_job_completed(job_type(&job->job),
                                         job->job.id,
-                                        job->job.progress.total,
-                                        job->job.progress.current,
+                                        progress_total,
+                                        progress_current,
                                         job->speed,
                                         !!msg,
                                         msg);
@@ -393,15 +406,19 @@ static void block_job_event_pending(Notifier *n, void *opaque)
 static void block_job_event_ready(Notifier *n, void *opaque)
 {
     BlockJob *job = opaque;
+    uint64_t progress_current, progress_total;
 
     if (block_job_is_internal(job)) {
         return;
     }
 
+    progress_get_snapshot(&job->job.progress, &progress_current,
+                          &progress_total);
+
     qapi_event_send_block_job_ready(job_type(&job->job),
                                     job->job.id,
-                                    job->job.progress.total,
-                                    job->job.progress.current,
+                                    progress_total,
+                                    progress_current,
                                     job->speed);
 }
 
include/qemu/progress_meter.h +19 -15
index 9a23ff071c..dadf822bbf 100644
@@ -27,6 +27,8 @@
 #ifndef QEMU_PROGRESS_METER_H
 #define QEMU_PROGRESS_METER_H
 
+#include "qemu/lockable.h"
+
 typedef struct ProgressMeter {
     /**
      * Current progress. The unit is arbitrary as long as the ratio between
@@ -37,22 +39,24 @@ typedef struct ProgressMeter {
 
     /** Estimated current value at the completion of the process */
     uint64_t total;
+
+    QemuMutex lock; /* protects concurrent access to above fields */
 } ProgressMeter;
 
-static inline void progress_work_done(ProgressMeter *pm, uint64_t done)
-{
-    pm->current += done;
-}
-
-static inline void progress_set_remaining(ProgressMeter *pm, uint64_t remaining)
-{
-    pm->total = pm->current + remaining;
-}
-
-static inline void progress_increase_remaining(ProgressMeter *pm,
-                                               uint64_t delta)
-{
-    pm->total += delta;
-}
+void progress_init(ProgressMeter *pm);
+void progress_destroy(ProgressMeter *pm);
+
+/* Get a snapshot of internal current and total values  */
+void progress_get_snapshot(ProgressMeter *pm, uint64_t *current,
+                           uint64_t *total);
+
+/* Increases the amount of work done so far by @done */
+void progress_work_done(ProgressMeter *pm, uint64_t done);
+
+/* Sets how much work has to be done to complete to @remaining */
+void progress_set_remaining(ProgressMeter *pm, uint64_t remaining);
+
+/* Increases the total work to do by @delta */
+void progress_increase_remaining(ProgressMeter *pm, uint64_t delta);
 
 #endif /* QEMU_PROGRESS_METER_H */
job-qmp.c +6 -2
index 34c4da094f..829a28aa70 100644
@@ -144,16 +144,20 @@ void qmp_job_dismiss(const char *id, Error **errp)
 static JobInfo *job_query_single(Job *job, Error **errp)
 {
     JobInfo *info;
+    uint64_t progress_current;
+    uint64_t progress_total;
 
     assert(!job_is_internal(job));
+    progress_get_snapshot(&job->progress, &progress_current,
+                          &progress_total);
 
     info = g_new(JobInfo, 1);
     *info = (JobInfo) {
         .id                 = g_strdup(job->id),
         .type               = job_type(job),
         .status             = job->status,
-        .current_progress   = job->progress.current,
-        .total_progress     = job->progress.total,
+        .current_progress   = progress_current,
+        .total_progress     = progress_total,
         .has_error          = !!job->err,
         .error              = job->err ? \
                               g_strdup(error_get_pretty(job->err)) : NULL,
job.c +3 -0
index 8775c1803b..e7a5d28854 100644
@@ -339,6 +339,8 @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn,
     job->cb            = cb;
     job->opaque        = opaque;
 
+    progress_init(&job->progress);
+
     notifier_list_init(&job->on_finalize_cancelled);
     notifier_list_init(&job->on_finalize_completed);
     notifier_list_init(&job->on_pending);
@@ -382,6 +384,7 @@ void job_unref(Job *job)
 
         QLIST_REMOVE(job, job_list);
 
+        progress_destroy(&job->progress);
         error_free(job->err);
         g_free(job->id);
         g_free(job);
qemu-img.c +6 -3
index a5993682aa..7956a89965 100644
@@ -900,6 +900,7 @@ static void common_block_job_cb(void *opaque, int ret)
 
 static void run_block_job(BlockJob *job, Error **errp)
 {
+    uint64_t progress_current, progress_total;
     AioContext *aio_context = blk_get_aio_context(job->blk);
     int ret = 0;
 
@@ -908,9 +909,11 @@ static void run_block_job(BlockJob *job, Error **errp)
     do {
         float progress = 0.0f;
         aio_poll(aio_context, true);
-        if (job->job.progress.total) {
-            progress = (float)job->job.progress.current /
-                       job->job.progress.total * 100.f;
+
+        progress_get_snapshot(&job->job.progress, &progress_current,
+                              &progress_total);
+        if (progress_total) {
+            progress = (float)progress_current / progress_total * 100.f;
         }
         qemu_progress_print(progress, 0);
     } while (!job_is_ready(&job->job) && !job_is_completed(&job->job));
Signature · 1 line
2.31.1
Reviewed-by on v3 4/5
#6 lore raw

Re: [PATCH v3 4/5] progressmeter: protect with a mutex

Quoted 7 lines · Emanuele Giuseppe Esposito

Progressmeter is protected by the AioContext mutex, which is taken by the block jobs and their caller (like blockdev).

We would like to remove the dependency of block layer code on the AioContext mutex, since most drivers and the core I/O code are already not relying on it.

Quoted 8 lines

Create a new C file to implement the ProgressMeter API, but keep the struct as public, to avoid forcing allocation on the heap.

Also add a mutex to be able to provide an accurate snapshot of the progress values to the caller.

Signature · 2 lines
Best regards,
Vladimir
#7 lore raw

PATCH v3 5/5 co-shared-resource: protect with a mutex

In reply to Emanuele Giuseppe Esposito

co-shared-resource is currently not thread-safe, as also reported in co-shared-resource.h. Add a QemuMutex because co_try_get_from_shres can also be invoked from non-coroutine context.

---
 include/qemu/co-shared-resource.h |  4 +---
 util/qemu-co-shared-resource.c    | 24 +++++++++++++++++++-----
 2 files changed, 20 insertions(+), 8 deletions(-)
include/qemu/co-shared-resource.h +1 -3
index 4e4503004c..78ca5850f8 100644
@@ -26,15 +26,13 @@
 #ifndef QEMU_CO_SHARED_RESOURCE_H
 #define QEMU_CO_SHARED_RESOURCE_H
 
-
+/* Accesses to co-shared-resource API are thread-safe */
 typedef struct SharedResource SharedResource;
 
 /*
  * Create SharedResource structure
  *
  * @total: total amount of some resource to be shared between clients
- *
- * Note: this API is not thread-safe.
  */
 SharedResource *shres_create(uint64_t total);
 
util/qemu-co-shared-resource.c +19 -5
index 1c83cd9d29..a66cc07e75 100644
@@ -28,10 +28,13 @@
 #include "qemu/co-shared-resource.h"
 
 struct SharedResource {
-    uint64_t total;
-    uint64_t available;
+    uint64_t total; /* Set in shres_create() and not changed anymore */
 
+    /* State fields protected by lock */
+    uint64_t available;
     CoQueue queue;
+
+    QemuMutex lock;
 };
 
 SharedResource *shres_create(uint64_t total)
@@ -40,6 +43,7 @@ SharedResource *shres_create(uint64_t total)
 
     s->total = s->available = total;
     qemu_co_queue_init(&s->queue);
+    qemu_mutex_init(&s->lock);
 
     return s;
 }
@@ -47,10 +51,12 @@ SharedResource *shres_create(uint64_t total)
 void shres_destroy(SharedResource *s)
 {
     assert(s->available == s->total);
+    qemu_mutex_destroy(&s->lock);
     g_free(s);
 }
 
-bool co_try_get_from_shres(SharedResource *s, uint64_t n)
+/* Called with lock held. */
+static bool co_try_get_from_shres_locked(SharedResource *s, uint64_t n)
 {
     if (s->available >= n) {
         s->available -= n;
@@ -60,16 +66,24 @@ bool co_try_get_from_shres(SharedResource *s, uint64_t n)
     return false;
 }
 
+bool co_try_get_from_shres(SharedResource *s, uint64_t n)
+{
+    QEMU_LOCK_GUARD(&s->lock);
+    return co_try_get_from_shres_locked(s, n);
+}
+
 void coroutine_fn co_get_from_shres(SharedResource *s, uint64_t n)
 {
     assert(n <= s->total);
-    while (!co_try_get_from_shres(s, n)) {
-        qemu_co_queue_wait(&s->queue, NULL);
+    QEMU_LOCK_GUARD(&s->lock);
+    while (!co_try_get_from_shres_locked(s, n)) {
+        qemu_co_queue_wait(&s->queue, &s->lock);
     }
 }
 
 void coroutine_fn co_put_to_shres(SharedResource *s, uint64_t n)
 {
+    QEMU_LOCK_GUARD(&s->lock);
     assert(s->total - s->available >= n);
     s->available += n;
     qemu_co_queue_restart_all(&s->queue);
Signature · 1 line
2.31.1
#8 lore raw
In reply to Emanuele Giuseppe Esposito
Quoted 12 lines · Emanuele Giuseppe Esposito

This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask.
The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

What's missing for block-copy to be fully thread-safe is fixing the CoSleep API to allow cross-thread sleep and wakeup. Paolo is working on it and will post the patches once his new CoSleep API is accepted.

Quoted 8 lines

Patches 1-3 work on ratelimit, 4 covers progressmeter and 5 co-shared-resources.

---
v3:
* Rebase on current master (had conflicts in block-copy), remove based-on in cover letter

Hi Kevin & Max,

I think this series ha been reviewed and I just rebased it to current master. Can you give it a look and let me know if it can be merged?

Thank you,
Emanuele

Quoted 24 lines

Emanuele Giuseppe Esposito (2):
progressmeter: protect with a mutex
co-shared-resource: protect with a mutex

Paolo Bonzini (3):
ratelimit: treat zero speed as unlimited
block-copy: let ratelimit handle a speed of 0
blockjob: let ratelimit handle a speed of 0

  block/block-copy.c                | 28 ++++++--------
  block/meson.build                 |  1 +
  block/progress_meter.c            | 64 +++++++++++++++++++++++++++++++
  blockjob.c                        | 46 +++++++++++++---------
  include/qemu/co-shared-resource.h |  4 +-
  include/qemu/progress_meter.h     | 34 ++++++++--------
  include/qemu/ratelimit.h          | 12 +++++-
  job-qmp.c                         |  8 +++-
  job.c                             |  3 ++
  qemu-img.c                        |  9 +++--
  util/qemu-co-shared-resource.c    | 24 +++++++++---
  11 files changed, 168 insertions(+), 65 deletions(-)

create mode 100644 block/progress_meter.c

#9 lore raw
Quoted 24 lines · Emanuele Giuseppe Esposito
Quoted 12 lines · Emanuele Giuseppe Esposito

This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask.
The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

What's missing for block-copy to be fully thread-safe is fixing the CoSleep API to allow cross-thread sleep and wakeup. Paolo is working on it and will post the patches once his new CoSleep API is accepted.

Quoted 8 lines

Patches 1-3 work on ratelimit, 4 covers progressmeter and 5 co-shared-resources.

---
v3:
* Rebase on current master (had conflicts in block-copy), remove based-on in cover letter

Quoted 6 lines

Hi Kevin & Max,

I think this series ha been reviewed and I just rebased it to current master. Can you give it a look and let me know if it can be merged?

Thank you,
Emanuele

I think, I can queue it myself as a block-job series. ratelimit and progressmeter are not mentioned in Block Jobs sections of MAINTAINERS, but actually these APIs used only by block-jobs.

I remember, Stefan had a complain against patch 5 and against general design of adding mutex to every structure.. Stefan, what do you think now? Paolo, is this v3 OK for you?

If everybody silent, I don't see a reason to slow down these series anymore and will make a pull request on Tuesday.

Signature · 2 lines
Best regards,
Vladimir
#10 lore raw
Quoted 33 lines · Vladimir Sementsov-Ogievskiy
Quoted 24 lines · Emanuele Giuseppe Esposito
Quoted 12 lines · Emanuele Giuseppe Esposito

This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask.
The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

What's missing for block-copy to be fully thread-safe is fixing the CoSleep API to allow cross-thread sleep and wakeup. Paolo is working on it and will post the patches once his new CoSleep API is accepted.

Quoted 8 lines

Patches 1-3 work on ratelimit, 4 covers progressmeter and 5 co-shared-resources.

---
v3:
* Rebase on current master (had conflicts in block-copy), remove based-on in cover letter

Quoted 6 lines

Hi Kevin & Max,

I think this series ha been reviewed and I just rebased it to current master. Can you give it a look and let me know if it can be merged?

Thank you,
Emanuele

Quoted 6 lines

I think, I can queue it myself as a block-job series. ratelimit and progressmeter are not mentioned in Block Jobs sections of MAINTAINERS, but actually these APIs used only by block-jobs.

I remember, Stefan had a complain against patch 5 and against general design of adding mutex to every structure.. Stefan, what do you think now? Paolo, is this v3 OK for you?

If everybody silent, I don't see a reason to slow down these series anymore and will make a pull request on Tuesday.

Hmm, actually, I'll wait for final version of "[PATCH v4 0/6] block-copy: protect block-copy internal structures" which seems to be close, to pull them together.

Signature · 2 lines
Best regards,
Vladimir
#11 lore raw
In reply to Emanuele Giuseppe Esposito
Quoted 12 lines · Emanuele Giuseppe Esposito

This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask.
The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

What's missing for block-copy to be fully thread-safe is fixing the CoSleep API to allow cross-thread sleep and wakeup. Paolo is working on it and will post the patches once his new CoSleep API is accepted.

Quoted 4 lines

Patches 1-3 work on ratelimit, 4 covers progressmeter and 5 co-shared-resources.

Thanks, applied to the jobs branch

Signature · 2 lines
Best regards,
Vladimir