Generic background jobs
Subsystems (2)
cover Generic background jobs
Before we can make x-blockdev-create a background job, we need to generalise the job infrastructure so that it can be used without any associated block node.
This series extracts a Job object from the block job infrastructure, which should contain everything related to jobs that doesn't require the block layer to be involved.
Job creation with a centralised job-create command (if we even want this) as well as the actual conversion of x-blockdev-create to Job is left for another series. With 42 patches, the series must be perfect and I can't add any more patches.
Kevin Wolf (42):
blockjob: Fix assertion in block_job_finalize()
blockjob: Wrappers for progress counter access
blockjob: Move RateLimit to BlockJob
blockjob: Implement block_job_set_speed() centrally
blockjob: Introduce block_job_ratelimit_get_delay()
blockjob: Add block_job_driver()
blockjob: Remove block_job_pause/resume_all()
job: Create Job, JobDriver and job_create()
job: Rename BlockJobType into JobType
job: Add JobDriver.job_type
job: Add job_delete()
job: Maintain a list of all jobs
job: Move state transitions to Job
job: Add reference counting
job: Move cancelled to Job
job: Add Job.aio_context
job: Move defer_to_main_loop to Job
job: Move coroutine and related code to Job
job: Add job_sleep_ns()
job: Move pause/resume functions to Job
job: Replace BlockJob.completed with job_is_completed()
job: Move BlockJobCreateFlags to Job
blockjob: Split block_job_event_pending()
job: Add job_event_*()
job: Move single job finalisation to Job
job: Convert block_job_cancel_async() to Job
job: Add job_drain()
job: Move .complete callback to Job
job: Move job_finish_sync() to Job
job: Switch transactions to JobTxn
job: Move transactions to Job
job: Move completion and cancellation to Job
job: Add job_yield()
job: Add job_dismiss()
job: Add job_is_ready()
job: Add job_transition_to_ready()
job: Move progress fields to Job
job: Add JOB_STATUS_CHANGE QMP event
job: Add lifecycle QMP commands
job: Add query-jobs QMP command
iotests: Move qmp_to_opts() to VM
qemu-iotests: Test job-* with block jobs
qapi/block-core.json | 81 +-- qapi/job.json | 236 +++++++++ qapi/qapi-schema.json | 1 + include/block/block_int.h | 2 +- include/block/blockjob.h | 315 +----------- include/block/blockjob_int.h | 187 +------ include/qemu/job.h | 561 +++++++++++++++++++++ block.c | 2 +- block/backup.c | 111 ++--- block/commit.c | 75 ++- block/mirror.c | 139 +++--- block/replication.c | 10 +- block/stream.c | 68 ++- blockdev.c | 66 +-- blockjob.c | 1095 ++++++----------------------------------- job-qmp.c | 194 ++++++++ job.c | 994 +++++++++++++++++++++++++++++++++++++ qemu-img.c | 22 +- tests/test-bdrv-drain.c | 63 +-- tests/test-blockjob-txn.c | 74 +-- tests/test-blockjob.c | 141 +++--- MAINTAINERS | 3 + Makefile | 9 + Makefile.objs | 6 +- block/trace-events | 5 - tests/qemu-iotests/030 | 6 +- tests/qemu-iotests/040 | 2 + tests/qemu-iotests/041 | 23 +- tests/qemu-iotests/095 | 2 +- tests/qemu-iotests/095.out | 6 + tests/qemu-iotests/109 | 2 +- tests/qemu-iotests/109.out | 178 ++++++- tests/qemu-iotests/124 | 8 + tests/qemu-iotests/127.out | 7 + tests/qemu-iotests/141 | 10 +- tests/qemu-iotests/141.out | 29 ++ tests/qemu-iotests/144 | 2 +- tests/qemu-iotests/144.out | 7 + tests/qemu-iotests/155 | 2 +- tests/qemu-iotests/156 | 2 +- tests/qemu-iotests/156.out | 7 + tests/qemu-iotests/185 | 12 +- tests/qemu-iotests/185.out | 10 + tests/qemu-iotests/191 | 4 +- tests/qemu-iotests/191.out | 132 +++++ tests/qemu-iotests/219 | 201 ++++++++ tests/qemu-iotests/219.out | 327 ++++++++++++ tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 45 +- trace-events | 14 + 50 files changed, 3568 insertions(+), 1931 deletions(-) create mode 100644 qapi/job.json create mode 100644 include/qemu/job.h create mode 100644 job-qmp.c create mode 100644 job.c create mode 100755 tests/qemu-iotests/219 create mode 100644 tests/qemu-iotests/219.out
Signature · 1 line
2.13.6
PATCH 1/42 blockjob: Fix assertion in block_job_finalize()
Every job gets a non-NULL job->txn on creation, but it doesn't necessarily keep it until it is decommissioned: Finalising a job removes it from its transaction. Therefore, calling 'blockdev-job-finalize' a second time on an already concluded job causes an assertion failure.
Remove job->txn from the assertion in block_job_finalize() to fix this. block_job_do_finalize() still has the same assertion, but if a job is already removed from its transaction, block_job_apply_verb() will already error out before we run into that assertion.
--- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
blockjob.c +1 -1
@@ -702,7 +702,7 @@ void block_job_complete(BlockJob *job, Error **errp)void block_job_finalize (BlockJob *job,Error **errp) { -assert (job && job->id && job->txn); +assert (job && job->id);if (block_job_apply_verb (job,BLOCK_JOB_VERB_FINALIZE , errp)) {return ; }
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 01/42] blockjob: Fix assertion in block_job_finalize()
Quoted 9 lines · Kevin Wolf
Every job gets a non-NULL
job->txnon creation, but it doesn't necessarily keep it until it is decommissioned: Finalising a job removes it from its transaction. Therefore, calling 'blockdev-job-finalize' a second time on an already concluded job causes an assertion failure.Remove
job->txnfrom the assertion inblock_job_finalize()to fix this.block_job_do_finalize()still has the same assertion, but if a job is already removed from its transaction,block_job_apply_verb()will already error out before we run into that assertion.
Quoted 6 lines
Signed-off-by Kevin Wolf <[email protected]>--- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
blockjob.c +1
-1@@ -702,7 +702,7 @@ void block_job_complete(BlockJob *job, Error **errp)void block_job_finalize (BlockJob *job,Error **errp) { -assert (job && job->id && job->txn); +assert (job && job->id);if (block_job_apply_verb (job,BLOCK_JOB_VERB_FINALIZE , errp)) {return ; }
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 01/42] blockjob: Fix assertion in block_job_finalize()
In reply to Kevin WolfQuoted 10 lines · Kevin Wolf
Every job gets a non-NULL
job->txnon creation, but it doesn't necessarily keep it until it is decommissioned: Finalising a job removes it from its transaction. Therefore, calling 'blockdev-job-finalize' a second time on an already concluded job causes an assertion failure.Remove
job->txnfrom the assertion inblock_job_finalize()to fix this.block_job_do_finalize()still has the same assertion, but if a job is already removed from its transaction,block_job_apply_verb()will already error out before we run into that assertion.
Quoted 4 lines
Signed-off-by Kevin Wolf <[email protected]>--- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Re: [Qemu-devel] [PATCH 01/42] blockjob: Fix assertion in block_job_finalize()
In reply to Kevin WolfQuoted 11 lines · Kevin Wolf
Every job gets a non-NULL
job->txnon creation, but it doesn't necessarily keep it until it is decommissioned: Finalising a job removes it from its transaction. Therefore, calling 'blockdev-job-finalize' a second time on an already concluded job causes an assertion failure.Remove
job->txnfrom the assertion inblock_job_finalize()to fix this.block_job_do_finalize()still has the same assertion, but if a job is already removed from its transaction,block_job_apply_verb()will already error out before we run into that assertion.Signed-off-by Kevin Wolf <[email protected]>
Quoted 10 lines
--- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)blockjob.c +0
-0
blockjob.c void block_job_complete(BlockJob *job, Error **errp) · line 708
Quoted 8 lines
blockjob.c +1
-1@@ -702,7 +702,7 @@ void block_job_complete(BlockJob *job, Error **errp)void block_job_finalize (BlockJob *job,Error **errp) { -assert (job && job->id && job->txn); +assert (job && job->id);if (block_job_apply_verb (job,BLOCK_JOB_VERB_FINALIZE , errp)) {return ; }
Oh, ack. Good catch.
PATCH 2/42 blockjob: Wrappers for progress counter access
In reply to Kevin WolfBlock job drivers are not expected to mess with the internals of the BlockJob object, so provide wrapper functions for one of the cases where they still do it: Updating the progress counter.
--- include/block/blockjob.h | 19 +++++++++++++++++++ block/backup.c | 22 +++++++++++++--------- block/commit.c | 16 ++++++++-------- block/mirror.c | 11 +++++------ block/stream.c | 14 ++++++++------ blockjob.c | 10 ++++++++++ 6 files changed, 63 insertions(+), 29 deletions(-)
include/block/blockjob.h +19 -0
@@ -278,6 +278,25 @@ void block_job_finalize(BlockJob *job, Error **errp);void block_job_dismiss (BlockJob **job,Error **errp);/** +* block_job_progress_update: +* @job: The job that has made progress +* @done: How much progress the job made +* +* Updates the progress counter of the job. +*/ +void block_job_progress_update (BlockJob *job,uint64_t done); + +/** +* block_job_progress_set_remaining: +* @job: The job whose expected progress end value is set +* @remaining: Expected end value of the progress counter of the job +* +* Sets the expected end value of the progress counter of a job so that a +* completion percentage can be calculated when the progress is updated. +*/ +void block_job_progress_set_remaining (BlockJob *job,uint64_t remaining); + +/** * block_job_query: * @job: The job to get information about. *
block/backup.c +13 -9
@@ -39,6 +39,7 @@ typedef struct BackupBlockJob {BlockdevOnError on_source_error;BlockdevOnError on_target_error;CoRwlock flush_rwlock; +uint64_t len;uint64_t bytes_read;int64_t cluster_size;bool compress;
@@ -118,7 +119,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,trace_backup_do_cow_process (job, start); - n =MIN (job->cluster_size, job->common.len - start); + n =MIN (job->cluster_size, job->len - start);if (!bounce_buffer) { bounce_buffer =blk_blockalign (blk, job->cluster_size);
@@ -159,7 +160,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,* offset field is an opaque progress value, it is not a disk offset. */ job->bytes_read += n; - job->common.offset += n; +block_job_progress_update (&job->common, n); } out:
@@ -261,7 +262,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)return ; } - len =DIV_ROUND_UP (backup_job->common.len, backup_job->cluster_size); + len =DIV_ROUND_UP (backup_job->len, backup_job->cluster_size);hbitmap_set (backup_job->copy_bitmap,0 , len); }
@@ -420,8 +421,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)bdrv_set_dirty_iter (dbi, next_cluster * job->cluster_size); } - job->common.offset = job->common.len - -hbitmap_count (job->copy_bitmap) * job->cluster_size; +/* TODO block_job_progress_set_remaining() would make more sense */ +block_job_progress_update (&job->common, + job->len -hbitmap_count (job->copy_bitmap) * job->cluster_size);bdrv_dirty_iter_free (dbi); }
@@ -437,7 +439,9 @@ static void coroutine_fn backup_run(void *opaque)QLIST_INIT (&job->inflight_reqs);qemu_co_rwlock_init (&job->flush_rwlock); - nb_clusters =DIV_ROUND_UP (job->common.len, job->cluster_size); + nb_clusters =DIV_ROUND_UP (job->len, job->cluster_size); +block_job_progress_set_remaining (&job->common, job->len); + job->copy_bitmap =hbitmap_alloc (nb_clusters,0 );if (job->sync_mode ==MIRROR_SYNC_MODE_INCREMENTAL ) {backup_incremental_init_copy_bitmap (job);
@@ -461,7 +465,7 @@ static void coroutine_fn backup_run(void *opaque) ret =backup_run_incremental (job); }else {/* Both FULL and TOP SYNC_MODE's require copying.. */ -for (offset =0 ; offset < job->common.len; +for (offset =0 ; offset < job->len; offset += job->cluster_size) {bool error_is_read;int alloced =0 ;
@@ -620,7 +624,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } -/* job->common.len is fixed, so we can't allow resize */ +/* job->len is fixed, so we can't allow resize */ job = block_job_create(job_id, &backup_job_driver, txn, bs, BLK_PERM_CONSISTENT_READ, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE |
@@ -676,7 +680,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,/* Required permissions are already taken with target's blk_new() */ block_job_add_bdrv (&job->common,"target" , target,0 ,BLK_PERM_ALL , &error_abort); - job->common.len = len; + job->len = len;return &job->common;
block/commit.c +8 -8
@@ -146,21 +146,21 @@ static void coroutine_fn commit_run(void *opaque)int64_t n =0 ;/* bytes */ void *buf =NULL ;int bytes_written =0 ; -int64_t base_len; +int64_t len, base_len; - ret = s->common.len =blk_getlength (s->top); - -if (s->common.len <0 ) { + ret = len =blk_getlength (s->top); +if (len <0 ) { goto out; } +block_job_progress_set_remaining (&s->common, len); ret = base_len =blk_getlength (s->base);if (base_len <0 ) { goto out; } -if (base_len < s->common.len) { - ret =blk_truncate (s->base, s->common.len,PREALLOC_MODE_OFF ,NULL ); +if (base_len < len) { + ret =blk_truncate (s->base, len,PREALLOC_MODE_OFF ,NULL );if (ret) { goto out; }
@@ -168,7 +168,7 @@ static void coroutine_fn commit_run(void *opaque) buf =blk_blockalign (s->top,COMMIT_BUFFER_SIZE ); -for (offset =0 ; offset < s->common.len; offset += n) { +for (offset =0 ; offset < len; offset += n) {bool copy;/* Note that even when no rate limit is applied we need to yield
@@ -198,7 +198,7 @@ static void coroutine_fn commit_run(void *opaque) } }/* Publish progress */ - s->common.offset += n; +block_job_progress_update (&s->common, n);if (copy && s->common.speed) { delay_ns =ratelimit_calculate_delay (&s->limit, n);
block/mirror.c +5 -6
@@ -121,7 +121,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret)bitmap_set (s->cow_bitmap, chunk_num, nb_chunks); }if (!s->initial_zeroing_ongoing) { - s->common.offset += op->bytes; +block_job_progress_update (&s->common, op->bytes); } }qemu_iovec_destroy (&op->qiov);
@@ -792,11 +792,10 @@ static void coroutine_fn mirror_run(void *opaque)block_job_pause_point (&s->common); cnt =bdrv_get_dirty_count (s->dirty_bitmap); -/* s->common.offset contains the number of bytes already processed so -* far, cnt is the number of dirty bytes remaining and -* s->bytes_in_flight is the number of bytes currently being -* processed; together those are the current total operation length */ - s->common.len = s->common.offset + s->bytes_in_flight + cnt; +/* cnt is the number of dirty bytes remaining and s->bytes_in_flight is +* the number of bytes currently being processed; together those are +* the current total operation length */ +block_job_progress_set_remaining (&s->common, s->bytes_in_flight + cnt);/* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() returns.
block/stream.c +8 -6
@@ -107,6 +107,7 @@ static void coroutine_fn stream_run(void *opaque)BlockBackend *blk = s->common.blk;BlockDriverState *bs =blk_bs (blk);BlockDriverState *base = s->base; +int64_t len;int64_t offset =0 ;uint64_t delay_ns =0 ;int error =0 ;
@@ -118,11 +119,12 @@ static void coroutine_fn stream_run(void *opaque) goto out; } - s->common.len =bdrv_getlength (bs); -if (s->common.len <0 ) { - ret = s->common.len; + len =bdrv_getlength (bs); +if (len <0 ) { + ret = len; goto out; } +block_job_progress_set_remaining (&s->common, len); buf =qemu_blockalign (bs,STREAM_BUFFER_SIZE );
@@ -135,7 +137,7 @@ static void coroutine_fn stream_run(void *opaque)bdrv_enable_copy_on_read (bs); } -for ( ; offset < s->common.len; offset += n) { +for ( ; offset < len; offset += n) {bool copy;/* Note that even when no rate limit is applied we need to yield
@@ -159,7 +161,7 @@ static void coroutine_fn stream_run(void *opaque)/* Finish early if end of backing file has been reached */ if (ret ==0 && n ==0 ) { - n = s->common.len - offset; + n = len - offset; } copy = (ret ==1 );
@@ -185,7 +187,7 @@ static void coroutine_fn stream_run(void *opaque) ret =0 ;/* Publish progress */ - s->common.offset += n; +block_job_progress_update (&s->common, n);if (copy && s->common.speed) { delay_ns =ratelimit_calculate_delay (&s->limit, n); }else {
blockjob.c +10 -0
@@ -810,6 +810,16 @@ int block_job_complete_sync(BlockJob *job, Error **errp)return block_job_finish_sync (job, &block_job_complete, errp); } +void block_job_progress_update (BlockJob *job,uint64_t done) +{ + job->offset += done; +} + +void block_job_progress_set_remaining (BlockJob *job,uint64_t remaining) +{ + job->len = job->offset + remaining; +} +BlockJobInfo *block_job_query (BlockJob *job,Error **errp) {BlockJobInfo *info;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 02/42] blockjob: Wrappers for progress counter access
Quoted 15 lines · Kevin Wolf
Block job drivers are not expected to mess with the internals of the BlockJob object, so provide wrapper functions for one of the cases where they still do it: Updating the progress counter.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>--- include/block/blockjob.h | 19 +++++++++++++++++++ block/backup.c | 22 +++++++++++++--------- block/commit.c | 16 ++++++++-------- block/mirror.c | 11 +++++------ block/stream.c | 14 ++++++++------ blockjob.c | 10 ++++++++++ 6 files changed, 63 insertions(+), 29 deletions(-)
[...]
Quoted 5 lines
block/backup.c +0
-0
[...]
block/backup.c static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) · line 424
Quoted 7 lines
block/backup.c +1
-2@@ -420,8 +421,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)bdrv_set_dirty_iter (dbi, next_cluster * job->cluster_size); } - job->common.offset = job->common.len - -hbitmap_count (job->copy_bitmap) * job->cluster_size; +/* TODO block_job_progress_set_remaining() would make more sense */
Extremely true, especially considering that at least there was an assignment before.
block/backup.c static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) · line 426
Quoted 2 lines
block/backup.c +2
-0+block_job_progress_update (&job->common, + job->len -hbitmap_count (job->copy_bitmap) * job->cluster_size);
Now, with an incremental update, you have to know that the progress was 0 before this call to make any sense of it.
I could ask: Why don't you just resolve the TODO immediately with
block_job_progress_set_remaining(&job->common,
hbitmap_count(job->copy_bitmap) * job->cluster_size);?
I suppose one possible answer is that this series has 42 patches as it is, but I have to say that it took me more time to figure this hunk out than it would have taken me to acknowledge the above change.
Considering that job->len and job->common.len are now separate after this patch, and that there is only a single other block_job_progress_update() call in this file, I can't see any side effects.
Quoted 3 lines
block/backup.c +0
-0bdrv_dirty_iter_free (dbi); }
[...]
Quoted 4 lines
block/mirror.c +0
-0
[...]
block/mirror.c static void coroutine_fn mirror_run(void *opaque) · line 797
Quoted 12 lines
block/mirror.c +3
-5@@ -792,11 +792,10 @@ static void coroutine_fn mirror_run(void *opaque)block_job_pause_point (&s->common); cnt =bdrv_get_dirty_count (s->dirty_bitmap); -/* s->common.offset contains the number of bytes already processed so -* far, cnt is the number of dirty bytes remaining and -* s->bytes_in_flight is the number of bytes currently being -* processed; together those are the current total operation length */ - s->common.len = s->common.offset + s->bytes_in_flight + cnt; +/* cnt is the number of dirty bytes remaining and s->bytes_in_flight is +* the number of bytes currently being processed; together those are +* the current total operation length */
No, together, those are the current remaining operation length.
With that fixed:
Quoted 4 lines
block/mirror.c +1
-0+block_job_progress_set_remaining (&s->common, s->bytes_in_flight + cnt);/* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() returns.
Re: [Qemu-devel] [PATCH 02/42] blockjob: Wrappers for progress counter access
Am 12.05.2018 um 00:12 hat Max Reitz geschrieben:
Quoted 51 lines · Kevin Wolf
Quoted 15 lines
Block job drivers are not expected to mess with the internals of the BlockJob object, so provide wrapper functions for one of the cases where they still do it: Updating the progress counter.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>--- include/block/blockjob.h | 19 +++++++++++++++++++ block/backup.c | 22 +++++++++++++--------- block/commit.c | 16 ++++++++-------- block/mirror.c | 11 +++++------ block/stream.c | 14 ++++++++------ blockjob.c | 10 ++++++++++ 6 files changed, 63 insertions(+), 29 deletions(-)[...]
Quoted 5 lines
block/backup.c +0
-0[...]
block/backup.c static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) · line 424
Quoted 7 lines
block/backup.c +1
-2@@ -420,8 +421,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)bdrv_set_dirty_iter (dbi, next_cluster * job->cluster_size); } - job->common.offset = job->common.len - -hbitmap_count (job->copy_bitmap) * job->cluster_size; +/* TODO block_job_progress_set_remaining() would make more sense */ Extremely true, especially considering that at least there was an assignment before.
block/backup.c static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) · line 426
Quoted 2 lines
block/backup.c +2
-0+block_job_progress_update (&job->common, + job->len -hbitmap_count (job->copy_bitmap) * job->cluster_size);Now, with an incremental update, you have to know that the progress was 0 before this call to make any sense of it.
I could ask: Why don't you just resolve the TODO immediately with
block_job_progress_set_remaining(&job->common, hbitmap_count(job->copy_bitmap) * job->cluster_size);?
Quoted 7 lines
I suppose one possible answer is that this series has 42 patches as it is, but I have to say that it took me more time to figure this hunk out than it would have taken me to acknowledge the above change.
Considering that
job->lenandjob->common.lenare now separate after this patch, and that there is only a single otherblock_job_progress_update()call in this file, I can't see any side effects.
Basically just because I tried to make the naive change whenever I had to touch something that isn't what the patch changes as its main purpose. The old code changed offset rather than len, so I used the function that does the same.
If I reduced len instead of increasing offset, I suppose that at least a few test cases would have to be updated etc. and who knows what else (QMP clients shouldn't rely on the current way, but do they?).
I'd rather not make such a change as a side effect of a patch that tries to do something quite different.
Quoted 16 lines
Quoted 3 lines
block/backup.c +0
-0bdrv_dirty_iter_free (dbi); }[...]
Quoted 4 lines
block/mirror.c +0
-0[...]
Quoted 3 lines
block/mirror.c +0
-0@@ -792,11 +792,10 @@ static void coroutine_fn mirror_run(void *opaque)block_job_pause_point (&s->common);
Quoted 11 lines
block/mirror.c static void coroutine_fn mirror_run(void *opaque) · line 797
Quoted 9 lines
block/mirror.c +3
-5cnt =bdrv_get_dirty_count (s->dirty_bitmap); -/* s->common.offset contains the number of bytes already processed so -* far, cnt is the number of dirty bytes remaining and -* s->bytes_in_flight is the number of bytes currently being -* processed; together those are the current total operation length */ - s->common.len = s->common.offset + s->bytes_in_flight + cnt; +/* cnt is the number of dirty bytes remaining and s->bytes_in_flight is +* the number of bytes currently being processed; together those are +* the current total operation length */ No, together, those are the current remaining operation length.
Thanks, will fix.
Kevin
Re: [Qemu-devel] [PATCH 02/42] blockjob: Wrappers for progress counter access
In reply to Kevin WolfQuoted 6 lines · Kevin Wolf
Block job drivers are not expected to mess with the internals of the BlockJob object, so provide wrapper functions for one of the cases where they still do it: Updating the progress counter.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
The backup code takes a hot second to read, but it seems correct. Having both a common len and a backup len is a bit awful, but only in this patch diff where you have to deal with both.
PATCH 3/42 blockjob: Move RateLimit to BlockJob
In reply to Kevin WolfEvery block job has a RateLimit, and they all do the exact same thing with it, so it should be common infrastructure. Move the struct field for a start.
--- include/block/blockjob.h | 4 ++++ block/backup.c | 5 ++--- block/commit.c | 5 ++--- block/mirror.c | 6 +++--- block/stream.c | 5 ++--- 5 files changed, 13 insertions(+), 12 deletions(-)
include/block/blockjob.h +4 -0
@@ -27,6 +27,7 @@#define BLOCKJOB_H #include "block/block.h" +#include "qemu/ratelimit.h" typedef struct BlockJobDriver BlockJobDriver ;typedef struct BlockJobTxn BlockJobTxn ;
@@ -118,6 +119,9 @@ typedef struct BlockJob {/** Speed that was set with @block_job_set_speed. */ int64_t speed; +/** Rate limiting data structure for implementing @speed. */ +RateLimit limit; +/** The completion function that will be called when the job completes. */ BlockCompletionFunc *cb;
block/backup.c +2 -3
@@ -35,7 +35,6 @@ typedef struct BackupBlockJob {/* bitmap for sync=incremental */ BdrvDirtyBitmap *sync_bitmap;MirrorSyncMode sync_mode; -RateLimit limit;BlockdevOnError on_source_error;BlockdevOnError on_target_error;CoRwlock flush_rwlock;
@@ -199,7 +198,7 @@ static void backup_set_speed(BlockJob *job, int64_t speed, Error **errp)error_setg (errp,QERR_INVALID_PARAMETER ,"speed" );return ; } -ratelimit_set_speed (&s->limit, speed,SLICE_TIME ); +ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); }static void backup_cleanup_sync_bitmap (BackupBlockJob *job,int ret)
@@ -346,7 +345,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job)* (without, VM does not reboot) */ if (job->common.speed) { -uint64_t delay_ns =ratelimit_calculate_delay (&job->limit, +uint64_t delay_ns =ratelimit_calculate_delay (&job->common.limit, job->bytes_read); job->bytes_read =0 ;block_job_sleep_ns (&job->common, delay_ns);
block/commit.c +2 -3
@@ -35,7 +35,6 @@ enum { typedef struct CommitBlockJob { BlockJob common; - RateLimit limit; BlockDriverState *commit_top_bs; BlockBackend *top; BlockBackend *base;
@@ -201,7 +200,7 @@ static void coroutine_fn commit_run(void *opaque)block_job_progress_update (&s->common, n);if (copy && s->common.speed) { - delay_ns =ratelimit_calculate_delay (&s->limit, n); + delay_ns =ratelimit_calculate_delay (&s->common.limit, n); }else { delay_ns =0 ; }
@@ -225,7 +224,7 @@ static void commit_set_speed(BlockJob *job, int64_t speed, Error **errp)error_setg (errp,QERR_INVALID_PARAMETER ,"speed" );return ; } -ratelimit_set_speed (&s->limit, speed,SLICE_TIME ); +ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); }static const BlockJobDriver commit_job_driver = {
block/mirror.c +3 -3
@@ -36,7 +36,6 @@ typedef struct MirrorBuffer { typedef struct MirrorBlockJob { BlockJob common; - RateLimit limit; BlockBackend *target; BlockDriverState *mirror_top_bs; BlockDriverState *source;
@@ -450,7 +449,8 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) offset += io_bytes; nb_chunks -=DIV_ROUND_UP (io_bytes, s->granularity);if (s->common.speed) { - delay_ns =ratelimit_calculate_delay (&s->limit, io_bytes_acct); + delay_ns =ratelimit_calculate_delay (&s->common.limit, + io_bytes_acct); } }return delay_ns;
@@ -916,7 +916,7 @@ static void mirror_set_speed(BlockJob *job, int64_t speed, Error **errp)error_setg (errp,QERR_INVALID_PARAMETER ,"speed" );return ; } -ratelimit_set_speed (&s->limit, speed,SLICE_TIME ); +ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); }static void mirror_complete (BlockJob *job,Error **errp)
block/stream.c +2 -3
@@ -33,7 +33,6 @@ enum { typedef struct StreamBlockJob { BlockJob common; - RateLimit limit; BlockDriverState *base; BlockdevOnError on_error; char *backing_file_str;
@@ -189,7 +188,7 @@ static void coroutine_fn stream_run(void *opaque)/* Publish progress */ block_job_progress_update (&s->common, n);if (copy && s->common.speed) { - delay_ns =ratelimit_calculate_delay (&s->limit, n); + delay_ns =ratelimit_calculate_delay (&s->common.limit, n); }else { delay_ns =0 ; }
@@ -219,7 +218,7 @@ static void stream_set_speed(BlockJob *job, int64_t speed, Error **errp)error_setg (errp,QERR_INVALID_PARAMETER ,"speed" );return ; } -ratelimit_set_speed (&s->limit, speed,SLICE_TIME ); +ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); }static const BlockJobDriver stream_job_driver = {
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 03/42] blockjob: Move RateLimit to BlockJob
Quoted 6 lines · Kevin Wolf
Every block job has a RateLimit, and they all do the exact same thing with it, so it should be common infrastructure. Move the struct field for a start.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
Quoted 7 lines
--- include/block/blockjob.h | 4 ++++ block/backup.c | 5 ++--- block/commit.c | 5 ++--- block/mirror.c | 6 +++--- block/stream.c | 5 ++--- 5 files changed, 13 insertions(+), 12 deletions(-)
Instead of finally getting rid of block job throttling, you make it central functionality? Bah, I say, bah! ;-)
Re: [Qemu-devel] [PATCH 03/42] blockjob: Move RateLimit to BlockJob
In reply to Kevin WolfQuoted 6 lines · Kevin Wolf
Every block job has a RateLimit, and they all do the exact same thing with it, so it should be common infrastructure. Move the struct field for a start.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
PATCH 4/42 blockjob: Implement block_job_set_speed() centrally
In reply to Kevin WolfAll block job drivers support .set_speed and all of them duplicate the same code to implement it. Move that code to blockjob.c and remove the now useless callback.
--- include/block/blockjob.h | 2 ++ include/block/blockjob_int.h | 3 --- block/backup.c | 13 ------------- block/commit.c | 14 -------------- block/mirror.c | 14 -------------- block/stream.c | 14 -------------- blockjob.c | 12 ++++-------- 7 files changed, 6 insertions(+), 66 deletions(-)
include/block/blockjob.h +2 -0
@@ -29,6 +29,8 @@#include "block/block.h" #include "qemu/ratelimit.h" +#define SLICE_TIME 100000000ULL/* ns */ +typedef struct BlockJobDriver BlockJobDriver ;typedef struct BlockJobTxn BlockJobTxn ;
include/block/blockjob_int.h +0 -3
@@ -41,9 +41,6 @@ struct BlockJobDriver {/** String describing the operation, part of query-block-jobs QMP API */ BlockJobType job_type; -/** Optional callback for job types that support setting a speed limit */ -void (*set_speed)(BlockJob *job,int64_t speed,Error **errp); -/** Mandatory: Entrypoint for the Coroutine. */ CoroutineEntry *start;
block/backup.c +0 -13
@@ -27,7 +27,6 @@#include "qemu/error-report.h" #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) -#define SLICE_TIME 100000000ULL/* ns */ typedef struct BackupBlockJob { BlockJob common;
@@ -190,17 +189,6 @@ static int coroutine_fn backup_before_write_notify(return backup_do_cow (job, req->offset, req->bytes,NULL , true); } -static void backup_set_speed (BlockJob *job,int64_t speed,Error **errp) -{ -BackupBlockJob *s =container_of (job, BackupBlockJob, common); - -if (speed <0 ) { -error_setg (errp,QERR_INVALID_PARAMETER ,"speed" ); -return ; - } -ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); -} -static void backup_cleanup_sync_bitmap (BackupBlockJob *job,int ret) {BdrvDirtyBitmap *bm;
@@ -540,7 +528,6 @@ static const BlockJobDriver backup_job_driver = { .instance_size = sizeof(BackupBlockJob), .job_type = BLOCK_JOB_TYPE_BACKUP, .start = backup_run, - .set_speed = backup_set_speed, .commit = backup_commit, .abort = backup_abort, .clean = backup_clean,
block/commit.c +0 -14
@@ -31,8 +31,6 @@ enum { COMMIT_BUFFER_SIZE = 512 * 1024, /* in bytes */ }; -#define SLICE_TIME 100000000ULL/* ns */ - typedef struct CommitBlockJob { BlockJob common; BlockDriverState *commit_top_bs;
@@ -216,21 +214,9 @@ out:block_job_defer_to_main_loop (&s->common, commit_complete, data); } -static void commit_set_speed (BlockJob *job,int64_t speed,Error **errp) -{ -CommitBlockJob *s =container_of (job, CommitBlockJob, common); - -if (speed <0 ) { -error_setg (errp,QERR_INVALID_PARAMETER ,"speed" ); -return ; - } -ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); -} -static const BlockJobDriver commit_job_driver = { .instance_size =sizeof (CommitBlockJob), .job_type =BLOCK_JOB_TYPE_COMMIT , - .set_speed = commit_set_speed, .start = commit_run, };
block/mirror.c +0 -14
@@ -22,7 +22,6 @@#include "qemu/ratelimit.h" #include "qemu/bitmap.h" -#define SLICE_TIME 100000000ULL/* ns */ #define MAX_IN_FLIGHT 16#define MAX_IO_BYTES (1 << 20)/* 1 Mb */ #define DEFAULT_MIRROR_BUF_SIZE (MAX_IN_FLIGHT * MAX_IO_BYTES)
@@ -908,17 +907,6 @@ immediate_exit:block_job_defer_to_main_loop (&s->common, mirror_exit, data); } -static void mirror_set_speed (BlockJob *job,int64_t speed,Error **errp) -{ -MirrorBlockJob *s =container_of (job, MirrorBlockJob, common); - -if (speed <0 ) { -error_setg (errp,QERR_INVALID_PARAMETER ,"speed" ); -return ; - } -ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); -} -static void mirror_complete (BlockJob *job,Error **errp) {MirrorBlockJob *s =container_of (job, MirrorBlockJob, common);
@@ -1003,7 +991,6 @@ static void mirror_drain(BlockJob *job)static const BlockJobDriver mirror_job_driver = { .instance_size =sizeof (MirrorBlockJob), .job_type =BLOCK_JOB_TYPE_MIRROR , - .set_speed = mirror_set_speed, .start = mirror_run, .complete = mirror_complete, .pause = mirror_pause,
@@ -1014,7 +1001,6 @@ static const BlockJobDriver mirror_job_driver = {static const BlockJobDriver commit_active_job_driver = { .instance_size =sizeof (MirrorBlockJob), .job_type =BLOCK_JOB_TYPE_COMMIT , - .set_speed = mirror_set_speed, .start = mirror_run, .complete = mirror_complete, .pause = mirror_pause,
block/stream.c +0 -14
@@ -29,8 +29,6 @@ enum { STREAM_BUFFER_SIZE = 512 * 1024, /* in bytes */ }; -#define SLICE_TIME 100000000ULL/* ns */ - typedef struct StreamBlockJob { BlockJob common; BlockDriverState *base;
@@ -210,21 +208,9 @@ out:block_job_defer_to_main_loop (&s->common, stream_complete, data); } -static void stream_set_speed (BlockJob *job,int64_t speed,Error **errp) -{ -StreamBlockJob *s =container_of (job, StreamBlockJob, common); - -if (speed <0 ) { -error_setg (errp,QERR_INVALID_PARAMETER ,"speed" ); -return ; - } -ratelimit_set_speed (&s->common.limit, speed,SLICE_TIME ); -} -static const BlockJobDriver stream_job_driver = { .instance_size =sizeof (StreamBlockJob), .job_type =BLOCK_JOB_TYPE_STREAM , - .set_speed = stream_set_speed, .start = stream_run, };
blockjob.c +4 -8
@@ -659,22 +659,18 @@ static bool block_job_timer_pending(BlockJob *job)void block_job_set_speed (BlockJob *job,int64_t speed,Error **errp) { -Error *local_err =NULL ;int64_t old_speed = job->speed; -if (!job->driver->set_speed) { -error_setg (errp,QERR_UNSUPPORTED ); -return ; - }if (block_job_apply_verb (job,BLOCK_JOB_VERB_SET_SPEED , errp)) {return ; } - job->driver->set_speed (job, speed, &local_err); -if (local_err) { -error_propagate (errp, local_err); +if (speed <0 ) { +error_setg (errp,QERR_INVALID_PARAMETER ,"speed" );return ; } +ratelimit_set_speed (&job->limit, speed,SLICE_TIME ); + job->speed = speed;if (speed && speed <= old_speed) {return ;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 04/42] blockjob: Implement block_job_set_speed() centrally
Quoted 21 lines · Kevin Wolf
All block job drivers support .set_speed and all of them duplicate the same code to implement it. Move that code to
blockjob.cand remove the now useless callback.Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>--- include/block/blockjob.h | 2 ++ include/block/blockjob_int.h | 3 --- block/backup.c | 13 ------------- block/commit.c | 14 -------------- block/mirror.c | 14 -------------- block/stream.c | 14 -------------- blockjob.c | 12 ++++-------- 7 files changed, 6 insertions(+), 66 deletions(-)include/block/blockjob.h +0
-0
include/block/blockjob.h line 35
Quoted 7 lines
include/block/blockjob.h +2
-0@@ -29,6 +29,8 @@#include "block/block.h" #include "qemu/ratelimit.h" +#define SLICE_TIME 100000000ULL/* ns */ +typedef struct BlockJobDriver BlockJobDriver ;typedef struct BlockJobTxn BlockJobTxn ;
SLICE_TIME can be anything. I don't like something that can be anything to be in a header file. I can see that you still need it in mirror, so it needs to be in a header; but maybe rename it to... THROTTLE_SLICE_TIME? At least JOB_SLICE_TIME?
Apart from that:
Re: [Qemu-devel] [PATCH 04/42] blockjob: Implement block_job_set_speed() centrally
In reply to Kevin WolfQuoted 3 lines · Kevin Wolf
All block job drivers support .set_speed and all of them duplicate the same code to implement it. Move that code to
blockjob.cand remove the now useless callback.
🎊
Quoted 3 lines
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
PATCH 5/42 blockjob: Introduce block_job_ratelimit_get_delay()
In reply to Kevin WolfThis gets us rid of more direct accesses to BlockJob fields from the job drivers.
--- include/block/blockjob_int.h | 8 ++++++++ block/backup.c | 18 +++++++----------- block/commit.c | 4 ++-- block/mirror.c | 5 +---- block/stream.c | 4 ++-- blockjob.c | 9 +++++++++ 6 files changed, 29 insertions(+), 19 deletions(-)
include/block/blockjob_int.h +8 -0
@@ -166,6 +166,14 @@ void block_job_sleep_ns(BlockJob *job, int64_t ns);void block_job_yield (BlockJob *job);/** +* block_job_ratelimit_get_delay: +* +* Calculate and return delay for the next request in ns. See the documentation +* of ratelimit_calculate_delay() for details. +*/ +int64_t block_job_ratelimit_get_delay (BlockJob *job,uint64_t n); + +/** * block_job_pause_all: * * Asynchronously pause all jobs.
block/backup.c +7 -11
@@ -325,21 +325,17 @@ static void backup_complete(BlockJob *job, void *opaque) static bool coroutine_fn yield_and_check(BackupBlockJob *job) { +uint64_t delay_ns; +if (block_job_is_cancelled (&job->common)) {return true; } -/* we need to yield so that bdrv_drain_all() returns. -* (without, VM does not reboot) -*/ -if (job->common.speed) { -uint64_t delay_ns =ratelimit_calculate_delay (&job->common.limit, - job->bytes_read); - job->bytes_read =0 ; -block_job_sleep_ns (&job->common, delay_ns); - }else { -block_job_sleep_ns (&job->common,0 ); - } +/* We need to yield even for delay_ns = 0 so that bdrv_drain_all() can +* return. Without a yield, the VM would not reboot. */ + delay_ns =block_job_ratelimit_get_delay (&job->common, job->bytes_read); + job->bytes_read =0 ; +block_job_sleep_ns (&job->common, delay_ns);if (block_job_is_cancelled (&job->common)) {return true;
block/commit.c +2 -2
@@ -197,8 +197,8 @@ static void coroutine_fn commit_run(void *opaque)/* Publish progress */ block_job_progress_update (&s->common, n); -if (copy && s->common.speed) { - delay_ns =ratelimit_calculate_delay (&s->common.limit, n); +if (copy) { + delay_ns =block_job_ratelimit_get_delay (&s->common, n); }else { delay_ns =0 ; }
block/mirror.c +1 -4
@@ -447,10 +447,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)assert (io_bytes); offset += io_bytes; nb_chunks -=DIV_ROUND_UP (io_bytes, s->granularity); -if (s->common.speed) { - delay_ns =ratelimit_calculate_delay (&s->common.limit, - io_bytes_acct); - } + delay_ns =block_job_ratelimit_get_delay (&s->common, io_bytes_acct); }return delay_ns; }
block/stream.c +2 -2
@@ -185,8 +185,8 @@ static void coroutine_fn stream_run(void *opaque)/* Publish progress */ block_job_progress_update (&s->common, n); -if (copy && s->common.speed) { - delay_ns =ratelimit_calculate_delay (&s->common.limit, n); +if (copy) { + delay_ns =block_job_ratelimit_get_delay (&s->common, n); }else { delay_ns =0 ; }
blockjob.c +9 -0
@@ -680,6 +680,15 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)block_job_enter_cond (job, block_job_timer_pending); } +int64_t block_job_ratelimit_get_delay (BlockJob *job,uint64_t n) +{ +if (!job->speed) { +return 0 ; + } + +return ratelimit_calculate_delay (&job->limit, n); +} +void block_job_complete (BlockJob *job,Error **errp) {/* Should not be reachable via external interface for internal jobs */
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 05/42] blockjob: Introduce block_job_ratelimit_get_delay()
Quoted 5 lines · Kevin Wolf
This gets us rid of more direct accesses to BlockJob fields from the job drivers.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
Quoted 8 lines
--- include/block/blockjob_int.h | 8 ++++++++ block/backup.c | 18 +++++++----------- block/commit.c | 4 ++-- block/mirror.c | 5 +---- block/stream.c | 4 ++-- blockjob.c | 9 +++++++++ 6 files changed, 29 insertions(+), 19 deletions(-)
Re: [Qemu-devel] [PATCH 05/42] blockjob: Introduce block_job_ratelimit_get_delay()
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
This gets us rid of more direct accesses to BlockJob fields from the job drivers.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
Quoted 9 lines
--- include/block/blockjob_int.h | 8 ++++++++ block/backup.c | 18 +++++++----------- block/commit.c | 4 ++-- block/mirror.c | 5 +---- block/stream.c | 4 ++-- blockjob.c | 9 +++++++++ 6 files changed, 29 insertions(+), 19 deletions(-)
[...]
Quoted 7 lines
block/backup.c +0
-0@@ -325,21 +325,17 @@ static void backup_complete(BlockJob *job, void *opaque) static bool coroutine_fn yield_and_check(BackupBlockJob *job)
block/backup.c static void backup_complete(BlockJob *job, void *opaque) · line 334
Quoted 10 lines
block/backup.c +2
-3{ +uint64_t delay_ns; +if (block_job_is_cancelled (&job->common)) {return true; } -/* we need to yield so that bdrv_drain_all() returns. -* (without, VM does not reboot) -*/
I'll kind of miss this terse comment. "without, VM does not reboot" is very direct and to the point. :)
Quoted 16 lines
block/backup.c +5
-8-if (job->common.speed) { -uint64_t delay_ns =ratelimit_calculate_delay (&job->common.limit, - job->bytes_read); - job->bytes_read =0 ; -block_job_sleep_ns (&job->common, delay_ns); - }else { -block_job_sleep_ns (&job->common,0 ); - } +/* We need to yield even for delay_ns = 0 so that bdrv_drain_all() can +* return. Without a yield, the VM would not reboot. */ + delay_ns =block_job_ratelimit_get_delay (&job->common, job->bytes_read); + job->bytes_read =0 ; +block_job_sleep_ns (&job->common, delay_ns);if (block_job_is_cancelled (&job->common)) {return true;
PATCH 6/42 blockjob: Add block_job_driver()
In reply to Kevin WolfThe backup block job directly accesses the driver field in BlockJob. Add a wrapper for getting it.
--- include/block/blockjob.h | 7 +++++++ block/backup.c | 8 +++++--- blockjob.c | 5 +++++ 3 files changed, 17 insertions(+), 3 deletions(-)
include/block/blockjob.h +7 -0
@@ -452,4 +452,11 @@ void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job);*/ bool block_job_is_internal (BlockJob *job); +/** +* block_job_driver: +* +* Returns the driver associated with a block job. +*/ +const BlockJobDriver *block_job_driver (BlockJob *job); +#endif
block/backup.c +5 -3
@@ -47,6 +47,8 @@ typedef struct BackupBlockJob {HBitmap *copy_bitmap; } BackupBlockJob; +static const BlockJobDriver backup_job_driver; +/* See if in-flight requests overlap and wait for them to complete */ static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, int64_t start,
@@ -241,7 +243,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)BackupBlockJob *backup_job =container_of (job, BackupBlockJob, common);int64_t len; -assert (job->driver->job_type ==BLOCK_JOB_TYPE_BACKUP ); +assert (block_job_driver (job) == &backup_job_driver);if (backup_job->sync_mode !=MIRROR_SYNC_MODE_NONE ) { error_setg(errp, "The backup job only supports block checkpoint in"
@@ -259,7 +261,7 @@ void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset,BackupBlockJob *backup_job =container_of (job, BackupBlockJob, common);int64_t start, end; -assert (job->driver->job_type ==BLOCK_JOB_TYPE_BACKUP ); +assert (block_job_driver (job) == &backup_job_driver); start =QEMU_ALIGN_DOWN (offset, backup_job->cluster_size); end =QEMU_ALIGN_UP (offset + bytes, backup_job->cluster_size);
@@ -272,7 +274,7 @@ void backup_cow_request_begin(CowRequest *req, BlockJob *job,BackupBlockJob *backup_job =container_of (job, BackupBlockJob, common);int64_t start, end; -assert (job->driver->job_type ==BLOCK_JOB_TYPE_BACKUP ); +assert (block_job_driver (job) == &backup_job_driver); start =QEMU_ALIGN_DOWN (offset, backup_job->cluster_size); end =QEMU_ALIGN_UP (offset + bytes, backup_job->cluster_size);
blockjob.c +5 -0
@@ -359,6 +359,11 @@ static bool block_job_started(BlockJob *job)return job->co; } +const BlockJobDriver *block_job_driver (BlockJob *job) +{ +return job->driver; +} +/** * All jobs must allow a pause point before entering their job proper. This * ensures that jobs can be paused prior to being started, then resumed later.
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 06/42] blockjob: Add block_job_driver()
Quoted 10 lines · Kevin Wolf
The backup block job directly accesses the driver field in BlockJob. Add a wrapper for getting it.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>--- include/block/blockjob.h | 7 +++++++ block/backup.c | 8 +++++--- blockjob.c | 5 +++++ 3 files changed, 17 insertions(+), 3 deletions(-)
Re: [Qemu-devel] [PATCH 06/42] blockjob: Add block_job_driver()
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
The backup block job directly accesses the driver field in BlockJob. Add a wrapper for getting it.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
I'm not too far into this series yet but I suppose it turns out that it's more useful to get the driver than to ask the job system what the job type is.
Probably there's some QAPI reasons for why that's true, too...
PATCH 7/42 blockjob: Remove block_job_pause/resume_all()
In reply to Kevin WolfCommit 81193349 removed the only use of block_job_pause/resume_all(), which was in bdrv_drain_all(). The functions are now unused and can be removed.
--- include/block/blockjob_int.h | 14 -------------- blockjob.c | 27 --------------------------- 2 files changed, 41 deletions(-)
include/block/blockjob_int.h +0 -14
@@ -174,20 +174,6 @@ void block_job_yield(BlockJob *job);int64_t block_job_ratelimit_get_delay (BlockJob *job,uint64_t n);/** -* block_job_pause_all: -* -* Asynchronously pause all jobs. -*/ -void block_job_pause_all (void ); - -/** -* block_job_resume_all: -* -* Resume all block jobs. Must be paired with a preceding block_job_pause_all. -*/ -void block_job_resume_all (void ); - -/** * block_job_early_fail: * @bs: The block device. *
blockjob.c +0 -27
@@ -1010,19 +1010,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,return job; } -void block_job_pause_all (void ) -{ -BlockJob *job =NULL ; -while ((job =block_job_next (job))) { -AioContext *aio_context =blk_get_aio_context (job->blk); - -aio_context_acquire (aio_context); -block_job_ref (job); -block_job_pause (job); -aio_context_release (aio_context); - } -} -void block_job_early_fail (BlockJob *job) {assert (job->status ==BLOCK_JOB_STATUS_CREATED );
@@ -1100,20 +1087,6 @@ void coroutine_fn block_job_pause_point(BlockJob *job) } } -void block_job_resume_all (void ) -{ -BlockJob *job, *next; - - QLIST_FOREACH_SAFE(job, &block_jobs, job_list, next) { -AioContext *aio_context =blk_get_aio_context (job->blk); - -aio_context_acquire (aio_context); -block_job_resume (job); -block_job_unref (job); -aio_context_release (aio_context); - } -} -/* * Conditionally enter a block_job pending a call to fn() while * under the block_job_lock critical section.
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 07/42] blockjob: Remove block_job_pause/resume_all()
Quoted 9 lines · Kevin Wolf
Commit 81193349 removed the only use of block_job_pause/resume_all(), which was in
bdrv_drain_all(). The functions are now unused and can be removed.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 14 -------------- blockjob.c | 27 --------------------------- 2 files changed, 41 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 07/42] blockjob: Remove block_job_pause/resume_all()
In reply to Kevin WolfQuoted 3 lines · Kevin Wolf
Commit 81193349 removed the only use of block_job_pause/resume_all(), which was in
bdrv_drain_all(). The functions are now unused and can be removed.
I have a strange liking for all-digit commit hash prefixes.
Quoted 5 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 14 -------------- blockjob.c | 27 --------------------------- 2 files changed, 41 deletions(-)
John Snow
Reviewed-byRe: [Qemu-devel] [PATCH 07/42] blockjob: Remove block_job_pause/resume_all()
Quoted 13 lines · Max Reitz
Quoted 3 lines · Kevin Wolf
Commit 81193349 removed the only use of block_job_pause/resume_all(), which was in
bdrv_drain_all(). The functions are now unused and can be removed.I have a strange liking for all-digit commit hash prefixes.
Quoted 5 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 14 -------------- blockjob.c | 27 --------------------------- 2 files changed, 41 deletions(-)
And all deletion diffstats.
Quoted 2 lines
Reviewed-by Max Reitz <[email protected]>
Re: [Qemu-devel] [PATCH 07/42] blockjob: Remove block_job_pause/resume_all()
In reply to Max ReitzQuoted 6 lines · Max Reitz
Quoted 3 lines · Kevin Wolf
Commit 81193349 removed the only use of block_job_pause/resume_all(), which was in
bdrv_drain_all(). The functions are now unused and can be removed.I have a strange liking for all-digit commit hash prefixes.
Especially when the first digit is 0, but then the rest includes an 8 or 9. (in the past, I've actually had a script transiently break when parsing commit ids, and it took me quite a while to figure out why the problem disappeared after I rebased, until I finally realized that it was because the script was choking on an attempt to parse an invalid octal number, only when I got lucky enough to land on a problematic commit id)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 8/42 job: Create Job, JobDriver and job_create()
In reply to Kevin WolfThis is the first step towards creating an infrastructure for generic background jobs that aren't tied to a block device. For now, Job only stores its ID and JobDriver, the rest stays in BlockJob.
The following patches will move over more parts of BlockJob to Job if they are meaningful outside the context of a block job.
--- include/block/blockjob.h | 9 +++---- include/block/blockjob_int.h | 4 +-- include/qemu/job.h | 60 ++++++++++++++++++++++++++++++++++++++++++++ block/backup.c | 4 ++- block/commit.c | 4 ++- block/mirror.c | 10 +++++--- block/stream.c | 4 ++- blockjob.c | 46 ++++++++++++++++----------------- job.c | 48 +++++++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 4 ++- tests/test-blockjob-txn.c | 4 ++- tests/test-blockjob.c | 12 ++++++--- MAINTAINERS | 2 ++ Makefile.objs | 2 +- 14 files changed, 169 insertions(+), 44 deletions(-) create mode 100644 include/qemu/job.h create mode 100644 job.c
include/block/blockjob.h +4 -5
@@ -26,6 +26,7 @@#ifndef BLOCKJOB_H #define BLOCKJOB_H +#include "qemu/job.h" #include "block/block.h" #include "qemu/ratelimit.h"
@@ -40,6 +41,9 @@ typedef struct BlockJobTxn BlockJobTxn;* Long-running operation on a BlockDriverState. */ typedef struct BlockJob { + /** Data belonging to the generic Job infrastructure */ + Job job; + /** The job type, including the job vtable. */ const BlockJobDriver *driver;
@@ -47,11 +51,6 @@ typedef struct BlockJob {BlockBackend *blk;/** -* The ID of the block job. May be NULL for internal jobs. -*/ -char *id; - -/** * The coroutine that executes the job. If not NULL, it is * reentered when busy is false and the job is cancelled. */
include/block/blockjob_int.h +2 -2
@@ -35,8 +35,8 @@* A class type for block job driver. */ struct BlockJobDriver { -/** Derived BlockJob struct size */ -size_t instance_size; +/** Generic JobDriver callbacks and settings */ +JobDriver job_driver;/** String describing the operation, part of query-block-jobs QMP API */ BlockJobType job_type;
include/qemu/job.h new file +60 -0
60 lines not on this page · raw message
block/backup.c +3 -1
@@ -523,7 +523,9 @@ static void coroutine_fn backup_run(void *opaque) }static const BlockJobDriver backup_job_driver = { - .instance_size =sizeof (BackupBlockJob), + .job_driver = { + .instance_size =sizeof (BackupBlockJob), + }, .job_type =BLOCK_JOB_TYPE_BACKUP , .start = backup_run, .commit = backup_commit,
block/commit.c +3 -1
@@ -215,7 +215,9 @@ out: }static const BlockJobDriver commit_job_driver = { - .instance_size =sizeof (CommitBlockJob), + .job_driver = { + .instance_size =sizeof (CommitBlockJob), + }, .job_type =BLOCK_JOB_TYPE_COMMIT , .start = commit_run, };
block/mirror.c +7 -3
@@ -913,7 +913,7 @@ static void mirror_complete(BlockJob *job, Error **errp)if (!s->synced) {error_setg (errp,"The active block job '%s' cannot be completed" , - job->id); + job->job.id);return ; }
@@ -986,7 +986,9 @@ static void mirror_drain(BlockJob *job) }static const BlockJobDriver mirror_job_driver = { - .instance_size =sizeof (MirrorBlockJob), + .job_driver = { + .instance_size =sizeof (MirrorBlockJob), + }, .job_type =BLOCK_JOB_TYPE_MIRROR , .start = mirror_run, .complete = mirror_complete,
@@ -996,7 +998,9 @@ static const BlockJobDriver mirror_job_driver = { };static const BlockJobDriver commit_active_job_driver = { - .instance_size =sizeof (MirrorBlockJob), + .job_driver = { + .instance_size =sizeof (MirrorBlockJob), + }, .job_type =BLOCK_JOB_TYPE_COMMIT , .start = mirror_run, .complete = mirror_complete,
block/stream.c +3 -1
@@ -209,7 +209,9 @@ out: }static const BlockJobDriver stream_job_driver = { - .instance_size =sizeof (StreamBlockJob), + .job_driver = { + .instance_size =sizeof (StreamBlockJob), + }, .job_type =BLOCK_JOB_TYPE_STREAM , .start = stream_run, };
blockjob.c +22 -24
154 lines not on this page · raw message
job.c new file +48 -0
48 lines not on this page · raw message
tests/test-bdrv-drain.c +3 -1
@@ -520,7 +520,9 @@ static void test_job_complete(BlockJob *job, Error **errp) }BlockJobDriver test_job_driver = { - .instance_size =sizeof (TestBlockJob), + .job_driver = { + .instance_size =sizeof (TestBlockJob), + }, .start = test_job_start, .complete = test_job_complete, };
tests/test-blockjob-txn.c +3 -1
@@ -74,7 +74,9 @@ static void test_block_job_cb(void *opaque, int ret) }static const BlockJobDriver test_block_job_driver = { - .instance_size =sizeof (TestBlockJob), + .job_driver = { + .instance_size =sizeof (TestBlockJob), + }, .start = test_block_job_run, };
tests/test-blockjob.c +8 -4
@@ -17,7 +17,9 @@#include "sysemu/block-backend.h" static const BlockJobDriver test_block_job_driver = { - .instance_size =sizeof (BlockJob), + .job_driver = { + .instance_size =sizeof (BlockJob), + }, };static void block_job_cb (void *opaque,int ret)
@@ -38,9 +40,9 @@ static BlockJob *mk_job(BlockBackend *blk, const char *id,g_assert_null (errp);g_assert_nonnull (job);if (id) { - g_assert_cmpstr(job->id, ==, id); + g_assert_cmpstr(job->job.id, ==, id); }else { - g_assert_cmpstr(job->id, ==, blk_name(blk)); + g_assert_cmpstr(job->job.id, ==, blk_name(blk)); } }else {g_assert_nonnull (errp);
@@ -192,7 +194,9 @@ static void coroutine_fn cancel_job_start(void *opaque) }static const BlockJobDriver test_cancel_driver = { - .instance_size =sizeof (CancelJob), + .job_driver = { + .instance_size =sizeof (CancelJob), + }, .start = cancel_job_start, .complete = cancel_job_complete, };
MAINTAINERS +2 -0
@@ -1369,6 +1369,8 @@ L: [email protected] S: Supported F: blockjob.c F: include/block/blockjob.h +F: job.c +F: include/block/job.h F: block/backup.c F: block/commit.c F: block/stream.c
Makefile.objs +1 -1
@@ -63,7 +63,7 @@ chardev-obj-y = chardev/ # block-obj-y is code used by both qemu system emulation and qemu-img block-obj-y += nbd/ -block-obj-y += block.o blockjob.o +block-obj-y += block.o blockjob.o job.o block-obj-y += block/ scsi/ block-obj-y += qemu-io-cmds.o block-obj-$(CONFIG_REPLICATION) += replication.o
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 08/42] job: Create Job, JobDriver and job_create()
Quoted 31 lines · Kevin Wolf
This is the first step towards creating an infrastructure for generic background jobs that aren't tied to a block device. For now, Job only stores its ID and JobDriver, the rest stays in BlockJob.
The following patches will move over more parts of BlockJob to Job if they are meaningful outside the context of a block job.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 9 +++---- include/block/blockjob_int.h | 4 +-- include/qemu/job.h | 60 ++++++++++++++++++++++++++++++++++++++++++++ block/backup.c | 4 ++- block/commit.c | 4 ++- block/mirror.c | 10 +++++--- block/stream.c | 4 ++- blockjob.c | 46 ++++++++++++++++----------------- job.c | 48 +++++++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 4 ++- tests/test-blockjob-txn.c | 4 ++- tests/test-blockjob.c | 12 ++++++--- MAINTAINERS | 2 ++ Makefile.objs | 2 +- 14 files changed, 169 insertions(+), 44 deletions(-) create mode 100644 include/qemu/job.h create mode 100644 job.cinclude/block/blockjob.h +0
-0
[...]
include/block/blockjob.h typedef struct BlockJobTxn BlockJobTxn; · line 48
Quoted 9 lines
include/block/blockjob.h +3
-0@@ -40,6 +41,9 @@ typedef struct BlockJobTxn BlockJobTxn;* Long-running operation on a BlockDriverState. */ typedef struct BlockJob { + /** Data belonging to the generic Job infrastructure */ + Job job; + /** The job type, including the job vtable. */ const BlockJobDriver *driver;
Any reason why you keep this field around? Shouldn't it be just DO_UPCAST(const BlockJobDriver, job_driver, job.driver)?
Quoted 13 lines
include/block/blockjob.h +0
-5@@ -47,11 +51,6 @@ typedef struct BlockJob {BlockBackend *blk;/** -* The ID of the block job. May be NULL for internal jobs. -*/ -char *id; - -/** * The coroutine that executes the job. If not NULL, it is * reentered when busy is false and the job is cancelled. */
[...]
Quoted 4 lines
blockjob.c +0
-0
[...]
blockjob.c void block_job_unref(BlockJob *job) · line 250
Quoted 6 lines
blockjob.c +1
-1@@ -247,7 +247,7 @@ void block_job_unref(BlockJob *job) block_job_detach_aio_context, job);blk_unref (job->blk);error_free (job->blocker); -g_free (job->id); +g_free (job->job.id);
Err. OK. I put my faith in patch 11.
Although I do wonder about BlockJob.driver. It seems to stay even after this series...
Quoted 3 lines
blockjob.c +0
-0assert (!timer_pending (&job->sleep_timer));g_free (job); }
[...]
MAINTAINERS L: [email protected]
Quoted 5 lines
MAINTAINERS +0
-0@@ -1369,6 +1369,8 @@ L: [email protected]
(Cut off here:
M: Jeff Cody <[email protected]>
Clever!)
Quoted 8 lines
MAINTAINERS +2
-0S: Supported F: blockjob.c F: include/block/blockjob.h +F: job.c +F: include/block/job.h F: block/backup.c F: block/commit.c F: block/stream.c
Re: [Qemu-devel] [PATCH 08/42] job: Create Job, JobDriver and job_create()
Am 12.05.2018 um 00:46 hat Max Reitz geschrieben:
Quoted 35 lines · Kevin Wolf
Quoted 31 lines
This is the first step towards creating an infrastructure for generic background jobs that aren't tied to a block device. For now, Job only stores its ID and JobDriver, the rest stays in BlockJob.
The following patches will move over more parts of BlockJob to Job if they are meaningful outside the context of a block job.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 9 +++---- include/block/blockjob_int.h | 4 +-- include/qemu/job.h | 60 ++++++++++++++++++++++++++++++++++++++++++++ block/backup.c | 4 ++- block/commit.c | 4 ++- block/mirror.c | 10 +++++--- block/stream.c | 4 ++- blockjob.c | 46 ++++++++++++++++----------------- job.c | 48 +++++++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 4 ++- tests/test-blockjob-txn.c | 4 ++- tests/test-blockjob.c | 12 ++++++--- MAINTAINERS | 2 ++ Makefile.objs | 2 +- 14 files changed, 169 insertions(+), 44 deletions(-) create mode 100644 include/qemu/job.h create mode 100644 job.cinclude/block/blockjob.h +0
-0[...]
Quoted 12 lines
include/block/blockjob.h typedef struct BlockJobTxn BlockJobTxn; · line 48
Quoted 9 lines
include/block/blockjob.h +3
-0@@ -40,6 +41,9 @@ typedef struct BlockJobTxn BlockJobTxn;* Long-running operation on a BlockDriverState. */ typedef struct BlockJob { + /** Data belonging to the generic Job infrastructure */ + Job job; + /** The job type, including the job vtable. */ const BlockJobDriver *driver;Any reason why you keep this field around? Shouldn't it be just
DO_UPCAST(const BlockJobDriver, job_driver, job.driver)?
I left it around to avoid unnecessary churn at this point. However, I intended to remove it at the end of the series, which I seem to have forgotten. I'll add a patch to this effect.
Kevin
Re: [Qemu-devel] [PATCH 08/42] job: Create Job, JobDriver and job_create()
In reply to Kevin WolfQuoted 8 lines · Kevin Wolf
This is the first step towards creating an infrastructure for generic background jobs that aren't tied to a block device. For now, Job only stores its ID and JobDriver, the rest stays in BlockJob.
The following patches will move over more parts of BlockJob to Job if they are meaningful outside the context of a block job.
Signed-off-by Kevin Wolf <[email protected]>
I think I'll be reviewing this series in a manner that trusts that all of the obvious cleanups get handled later, and only yelp if something looks distinctly wrong.
Anything that simply doesn't get cleaned up is something we can fix later, so unless it looks like it's painting us into a corner, it probably doesn't need to be addressed right away.
PATCH 9/42 job: Rename BlockJobType into JobType
In reply to Kevin WolfQAPI types aren't externally visible, so we can rename them without causing problems. Before we add a job type to Job, rename the enum so it can be used for more than just block jobs.
--- qapi/block-core.json | 14 +++++++------- include/block/blockjob_int.h | 2 +- block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 6 +++--- 7 files changed, 16 insertions(+), 16 deletions(-)
qapi/block-core.json +7 -7
@@ -1050,9 +1050,9 @@ 'data': ['top', 'full', 'none', 'incremental'] } ## -# @BlockJobType: +# @JobType: # -# Type of a block job. +# Type of a background job. # # @commit: block commit job type, see "block-commit" #
@@ -1064,7 +1064,7 @@ # # Since: 1.7 ## -{ 'enum': 'BlockJobType', +{ 'enum': 'JobType', 'data': ['commit', 'stream', 'mirror', 'backup'] } ##
@@ -4489,7 +4489,7 @@ # ## { 'event': 'BLOCK_JOB_COMPLETED', - 'data': { 'type' : 'BlockJobType', + 'data': { 'type' : 'JobType', 'device': 'str', 'len' : 'int', 'offset': 'int',
@@ -4525,7 +4525,7 @@ # ## { 'event': 'BLOCK_JOB_CANCELLED', - 'data': { 'type' : 'BlockJobType', + 'data': { 'type' : 'JobType', 'device': 'str', 'len' : 'int', 'offset': 'int',
@@ -4590,7 +4590,7 @@ # ## { 'event': 'BLOCK_JOB_READY', - 'data': { 'type' : 'BlockJobType', + 'data': { 'type' : 'JobType', 'device': 'str', 'len' : 'int', 'offset': 'int',
@@ -4617,7 +4617,7 @@ # ## { 'event': 'BLOCK_JOB_PENDING', - 'data': { 'type' : 'BlockJobType', + 'data': { 'type' : 'JobType', 'id' : 'str' } } ##
include/block/blockjob_int.h +1 -1
@@ -39,7 +39,7 @@ struct BlockJobDriver {JobDriver job_driver;/** String describing the operation, part of query-block-jobs QMP API */ -BlockJobType job_type; +JobType job_type;/** Mandatory: Entrypoint for the Coroutine. */ CoroutineEntry *start;
block/backup.c +1 -1
@@ -526,7 +526,7 @@ static const BlockJobDriver backup_job_driver = { .job_driver = { .instance_size = sizeof(BackupBlockJob), }, - .job_type = BLOCK_JOB_TYPE_BACKUP, + .job_type = JOB_TYPE_BACKUP, .start = backup_run, .commit = backup_commit, .abort = backup_abort,
block/commit.c +1 -1
@@ -218,7 +218,7 @@ static const BlockJobDriver commit_job_driver = { .job_driver = { .instance_size = sizeof(CommitBlockJob), }, - .job_type = BLOCK_JOB_TYPE_COMMIT, + .job_type = JOB_TYPE_COMMIT, .start = commit_run, };
block/mirror.c +2 -2
@@ -989,7 +989,7 @@ static const BlockJobDriver mirror_job_driver = { .job_driver = { .instance_size = sizeof(MirrorBlockJob), }, - .job_type = BLOCK_JOB_TYPE_MIRROR, + .job_type = JOB_TYPE_MIRROR, .start = mirror_run, .complete = mirror_complete, .pause = mirror_pause,
@@ -1001,7 +1001,7 @@ static const BlockJobDriver commit_active_job_driver = { .job_driver = { .instance_size = sizeof(MirrorBlockJob), }, - .job_type = BLOCK_JOB_TYPE_COMMIT, + .job_type = JOB_TYPE_COMMIT, .start = mirror_run, .complete = mirror_complete, .pause = mirror_pause,
block/stream.c +1 -1
@@ -212,7 +212,7 @@ static const BlockJobDriver stream_job_driver = { .job_driver = { .instance_size = sizeof(StreamBlockJob), }, - .job_type = BLOCK_JOB_TYPE_STREAM, + .job_type = JOB_TYPE_STREAM, .start = stream_run, };
blockjob.c +3 -3
@@ -296,7 +296,7 @@ static char *child_job_get_parent_desc(BdrvChild *c) {BlockJob *job = c->opaque;return g_strdup_printf ("%s job '%s'" , -BlockJobType_str (job->driver->job_type), +JobType_str (job->driver->job_type), job->job.id); }
@@ -839,7 +839,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp)return NULL ; } info =g_new0 (BlockJobInfo,1 ); - info->type =g_strdup (BlockJobType_str (job->driver->job_type)); + info->type =g_strdup (JobType_str (job->driver->job_type)); info->device =g_strdup (job->job.id); info->len = job->len; info->busy =atomic_read (&job->busy);
@@ -972,7 +972,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, block_job_sleep_timer_cb, job);error_setg (&job->blocker,"block device is in use by block job: %s" , -BlockJobType_str (driver->job_type)); +JobType_str (driver->job_type));block_job_add_bdrv (job,"main node" , bs,0 ,BLK_PERM_ALL , &error_abort); bs->job = job;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 09/42] job: Rename BlockJobType into JobType
Quoted 6 lines · Kevin Wolf
QAPI types aren't externally visible, so we can rename them without causing problems. Before we add a job type to Job, rename the enum so it can be used for more than just block jobs.
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
Quoted 9 lines
--- qapi/block-core.json | 14 +++++++------- include/block/blockjob_int.h | 2 +- block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 6 +++--- 7 files changed, 16 insertions(+), 16 deletions(-)
Re: [Qemu-devel] [PATCH 09/42] job: Rename BlockJobType into JobType
In reply to Kevin WolfQuoted 4 lines · Kevin Wolf
QAPI types aren't externally visible, so we can rename them without causing problems. Before we add a job type to Job, rename the enum so it can be used for more than just block jobs.
So we will be registering all jobs in a central place.
Quoted 2 lines
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>
Seems fine.
PATCH 10/42 job: Add JobDriver.job_type
In reply to Kevin WolfThis moves the job_type field from BlockJobDriver to JobDriver.
--- include/block/blockjob_int.h | 3 --- include/qemu/job.h | 11 +++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 16 +++++++--------- job.c | 10 ++++++++++ 8 files changed, 33 insertions(+), 17 deletions(-)
include/block/blockjob_int.h +0 -3
@@ -38,9 +38,6 @@ struct BlockJobDriver {/** Generic JobDriver callbacks and settings */ JobDriver job_driver; -/** String describing the operation, part of query-block-jobs QMP API */ -JobType job_type; -/** Mandatory: Entrypoint for the Coroutine. */ CoroutineEntry *start;
include/qemu/job.h +11 -0
@@ -26,6 +26,8 @@#ifndef JOB_H #define JOB_H +#include "qapi/qapi-types-block-core.h" +typedef struct JobDriver JobDriver ;/**
@@ -45,6 +47,9 @@ typedef struct Job {struct JobDriver {/** Derived Job struct size */ size_t instance_size; + +/** Enum describing the operation */ +JobType job_type; };
@@ -57,4 +62,10 @@ struct JobDriver {*/ void *job_create (const char *job_id,const JobDriver *driver,Error **errp); +/** Returns the JobType of a given Job. */ +JobType job_type (Job *job); + +/** Returns the enum string for the JobType of a given Job. */ +const char *job_type_str (Job *job); +#endif
block/backup.c +1 -1
@@ -525,8 +525,8 @@ static void coroutine_fn backup_run(void *opaque)static const BlockJobDriver backup_job_driver = { .job_driver = { .instance_size =sizeof (BackupBlockJob), + .job_type =JOB_TYPE_BACKUP , }, - .job_type =JOB_TYPE_BACKUP , .start = backup_run, .commit = backup_commit, .abort = backup_abort,
block/commit.c +1 -1
@@ -217,8 +217,8 @@ out:static const BlockJobDriver commit_job_driver = { .job_driver = { .instance_size =sizeof (CommitBlockJob), + .job_type =JOB_TYPE_COMMIT , }, - .job_type =JOB_TYPE_COMMIT , .start = commit_run, };
block/mirror.c +2 -2
@@ -988,8 +988,8 @@ static void mirror_drain(BlockJob *job)static const BlockJobDriver mirror_job_driver = { .job_driver = { .instance_size =sizeof (MirrorBlockJob), + .job_type =JOB_TYPE_MIRROR , }, - .job_type =JOB_TYPE_MIRROR , .start = mirror_run, .complete = mirror_complete, .pause = mirror_pause,
@@ -1000,8 +1000,8 @@ static const BlockJobDriver mirror_job_driver = {static const BlockJobDriver commit_active_job_driver = { .job_driver = { .instance_size =sizeof (MirrorBlockJob), + .job_type =JOB_TYPE_COMMIT , }, - .job_type =JOB_TYPE_COMMIT , .start = mirror_run, .complete = mirror_complete, .pause = mirror_pause,
block/stream.c +1 -1
@@ -211,8 +211,8 @@ out:static const BlockJobDriver stream_job_driver = { .job_driver = { .instance_size =sizeof (StreamBlockJob), + .job_type =JOB_TYPE_STREAM , }, - .job_type =JOB_TYPE_STREAM , .start = stream_run, };
blockjob.c +7 -9
@@ -295,9 +295,7 @@ static void block_job_detach_aio_context(void *opaque)static char *child_job_get_parent_desc (BdrvChild *c) {BlockJob *job = c->opaque; -return g_strdup_printf ("%s job '%s'" , -JobType_str (job->driver->job_type), - job->job.id); +return g_strdup_printf ("%s job '%s'" ,job_type_str (&job->job), job->job.id); }static void child_job_drained_begin (BdrvChild *c)
@@ -839,7 +837,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp)return NULL ; } info =g_new0 (BlockJobInfo,1 ); - info->type =g_strdup (JobType_str (job->driver->job_type)); + info->type =g_strdup (job_type_str (&job->job)); info->device =g_strdup (job->job.id); info->len = job->len; info->busy =atomic_read (&job->busy);
@@ -870,7 +868,7 @@ static void block_job_event_cancelled(BlockJob *job)return ; } - qapi_event_send_block_job_cancelled(job->driver->job_type, + qapi_event_send_block_job_cancelled(job_type(&job->job), job->job.id, job->len, job->offset,
@@ -884,7 +882,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg)return ; } - qapi_event_send_block_job_completed(job->driver->job_type, + qapi_event_send_block_job_completed(job_type(&job->job), job->job.id, job->len, job->offset,
@@ -898,7 +896,7 @@ static int block_job_event_pending(BlockJob *job) {block_job_state_transition (job,BLOCK_JOB_STATUS_PENDING );if (!job->auto_finalize && !block_job_is_internal (job)) { -qapi_event_send_block_job_pending (job->driver->job_type, +qapi_event_send_block_job_pending (job_type (&job->job), job->job.id, &error_abort); }
@@ -972,7 +970,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, block_job_sleep_timer_cb, job);error_setg (&job->blocker,"block device is in use by block job: %s" , -JobType_str (driver->job_type)); +job_type_str (&job->job));block_job_add_bdrv (job,"main node" , bs,0 ,BLK_PERM_ALL , &error_abort); bs->job = job;
@@ -1176,7 +1174,7 @@ void block_job_event_ready(BlockJob *job)return ; } - qapi_event_send_block_job_ready(job->driver->job_type, + qapi_event_send_block_job_ready(job_type(&job->job), job->job.id, job->len, job->offset,
job.c +10 -0
@@ -29,6 +29,16 @@#include "qemu/job.h" #include "qemu/id.h" +JobType job_type (Job *job) +{ +return job->driver->job_type; +} + +const char *job_type_str (Job *job) +{ +return JobType_str (job_type (job)); +} +void *job_create (const char *job_id,const JobDriver *driver,Error **errp) {Job *job;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 10/42] job: Add JobDriver.job_type
Quoted 14 lines · Kevin Wolf
This moves the
job_typefield from BlockJobDriver to JobDriver.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 3 --- include/qemu/job.h | 11 +++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 16 +++++++--------- job.c | 10 ++++++++++ 8 files changed, 33 insertions(+), 17 deletions(-)
[...]
Quoted 4 lines
include/qemu/job.h +0
-0
[...]
include/qemu/job.h struct JobDriver { · line 70
Quoted 10 lines
include/qemu/job.h +6
-0@@ -57,4 +62,10 @@ struct JobDriver {*/ void *job_create (const char *job_id,const JobDriver *driver,Error **errp); +/** Returns the JobType of a given Job. */ +JobType job_type (Job *job); + +/** Returns the enum string for the JobType of a given Job. */ +const char *job_type_str (Job *job); +
Is there a good reason for these not to take a const Job *?
Depending on the answer:
Quoted 1 line
include/qemu/job.h +0
-0#endif
Re: [Qemu-devel] [PATCH 10/42] job: Add JobDriver.job_type
Am 12.05.2018 um 00:53 hat Max Reitz geschrieben:
Quoted 38 lines · Kevin Wolf
Quoted 14 lines
This moves the
job_typefield from BlockJobDriver to JobDriver.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 3 --- include/qemu/job.h | 11 +++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 16 +++++++--------- job.c | 10 ++++++++++ 8 files changed, 33 insertions(+), 17 deletions(-)[...]
Quoted 4 lines
include/qemu/job.h +0
-0[...]
include/qemu/job.h struct JobDriver { · line 70
Quoted 10 lines
include/qemu/job.h +6
-0@@ -57,4 +62,10 @@ struct JobDriver {*/ void *job_create (const char *job_id,const JobDriver *driver,Error **errp); +/** Returns the JobType of a given Job. */ +JobType job_type (Job *job); + +/** Returns the enum string for the JobType of a given Job. */ +const char *job_type_str (Job *job); +Is there a good reason for these not to take a const Job *?
Quoted 3 lines
Depending on the answer:
Reviewed-by Max Reitz <[email protected]>
Not really. I'll change it and take your R-b.
Kevin
John Snow
Reviewed-byRe: [Qemu-devel] [PATCH 10/42] job: Add JobDriver.job_type
In reply to Max ReitzQuoted 4 lines · Max Reitz
Quoted 3 lines · Kevin Wolf
This moves the
job_typefield from BlockJobDriver to JobDriver.Signed-off-by Kevin Wolf <[email protected]>
Quoted 34 lines
Quoted 11 lines
--- include/block/blockjob_int.h | 3 --- include/qemu/job.h | 11 +++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 16 +++++++--------- job.c | 10 ++++++++++ 8 files changed, 33 insertions(+), 17 deletions(-)[...]
Quoted 4 lines
include/qemu/job.h +0
-0[...]
include/qemu/job.h struct JobDriver { · line 70
Quoted 10 lines
include/qemu/job.h +6
-0@@ -57,4 +62,10 @@ struct JobDriver {*/ void *job_create (const char *job_id,const JobDriver *driver,Error **errp); +/** Returns the JobType of a given Job. */ +JobType job_type (Job *job); + +/** Returns the enum string for the JobType of a given Job. */ +const char *job_type_str (Job *job); +Is there a good reason for these not to take a const Job *?
Quoted 6 lines
Depending on the answer:
Reviewed-by Max Reitz <[email protected]>Quoted 1 line
include/qemu/job.h +0
-0#endif
Max is just very excited to talk about Rust some more, I can tell.
PATCH 11/42 job: Add job_delete()
In reply to Kevin WolfThis moves freeing the Job object and its fields from block_job_unref() to job_delete().
--- include/qemu/job.h | 3 +++ blockjob.c | 3 +-- job.c | 6 ++++++ 3 files changed, 10 insertions(+), 2 deletions(-)
include/qemu/job.h +3 -0
@@ -62,6 +62,9 @@ struct JobDriver {*/ void *job_create (const char *job_id,const JobDriver *driver,Error **errp); +/** Frees the @job object. */ +void job_delete (Job *job); +/** Returns the JobType of a given Job. */ JobType job_type (Job *job);
blockjob.c +1 -2
@@ -247,9 +247,8 @@ void block_job_unref(BlockJob *job) block_job_detach_aio_context, job);blk_unref (job->blk);error_free (job->blocker); -g_free (job->job.id);assert (!timer_pending (&job->sleep_timer)); -g_free (job); +job_delete (&job->job); } }
job.c +6 -0
@@ -56,3 +56,9 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp)return job; } + +void job_delete (Job *job) +{ +g_free (job->id); +g_free (job); +}
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 11/42] job: Add job_delete()
Quoted 9 lines · Kevin Wolf
This moves freeing the Job object and its fields from
block_job_unref()tojob_delete().Signed-off-by Kevin Wolf <[email protected]>--- include/qemu/job.h | 3 +++ blockjob.c | 3 +-- job.c | 6 ++++++ 3 files changed, 10 insertions(+), 2 deletions(-)
Re: [Qemu-devel] [PATCH 11/42] job: Add job_delete()
In reply to Kevin WolfQuoted 4 lines · Kevin Wolf
This moves freeing the Job object and its fields from
block_job_unref()tojob_delete().Signed-off-by Kevin Wolf <[email protected]>
I guess the reference counting comes later.
*peeks*
Oh, soon!
Re: [Qemu-devel] [PATCH 11/42] job: Add job_delete()
In reply to Kevin WolfQuoted 11 lines · Kevin Wolf
This moves freeing the Job object and its fields from
block_job_unref()tojob_delete().Signed-off-by Kevin Wolf <[email protected]>--- include/qemu/job.h | 3 +++ blockjob.c | 3 +-- job.c | 6 ++++++ 3 files changed, 10 insertions(+), 2 deletions(-)
job.c void *job_create(const char *job_id, const JobDriver *driver, Error **errp) · line 64
Quoted 6 lines
job.c +6
-0+ +void job_delete (Job *job) +{ +g_free (job->id); +g_free (job); +}
Should this be free-like, acting as a no-op when job == NULL on input?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 12/42 job: Maintain a list of all jobs
In reply to Kevin WolfThis moves the job list from BlockJob to Job. Now we can check for duplicate IDs in job_create().
--- include/block/blockjob.h | 3 --- include/qemu/job.h | 19 +++++++++++++++++++ blockjob.c | 47 +++++++++++++++++++++++++---------------------- job.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 25 deletions(-)
include/block/blockjob.h +0 -3
@@ -105,9 +105,6 @@ typedef struct BlockJob {*/ bool deferred_to_main_loop; -/** Element of the list of block jobs */ -QLIST_ENTRY (BlockJob) job_list; -/** Status that is published by the query-block-jobs QMP API */ BlockDeviceIoStatus iostatus;
include/qemu/job.h +19 -0
@@ -27,6 +27,7 @@#define JOB_H #include "qapi/qapi-types-block-core.h" +#include "qemu/queue.h" typedef struct JobDriver JobDriver ;
@@ -39,6 +40,9 @@ typedef struct Job {/** The type of this job. */ const JobDriver *driver; + +/** Element of the list of jobs */ +QLIST_ENTRY (Job) job_list; } Job;/**
@@ -71,4 +75,19 @@ JobType job_type(Job *job);/** Returns the enum string for the JobType of a given Job. */ const char *job_type_str (Job *job); +/** +* Get the next element from the list of block jobs after @job, or the +* first one if @job is %NULL. +* +* Returns the requested job, or %NULL if there are no more jobs left. +*/ +Job *job_next (Job *job); + +/** +* Get the job identified by @id (which must not be %NULL). +* +* Returns the requested job, or %NULL if it doesn't exist. +*/ +Job *job_get (const char *id); +#endif
blockjob.c +25 -22
96 lines not on this page · raw message
job.c +31 -0
@@ -29,6 +29,8 @@#include "qemu/job.h" #include "qemu/id.h" +static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); +JobType job_type (Job *job) {return job->driver->job_type;
@@ -39,6 +41,27 @@ const char *job_type_str(Job *job)return JobType_str (job_type (job)); } +Job *job_next (Job *job) +{ +if (!job) { +return QLIST_FIRST (&jobs); + } +return QLIST_NEXT (job, job_list); +} + +Job *job_get (const char *id) +{ +Job *job; + + QLIST_FOREACH(job, &jobs, job_list) { +if (job->id && !strcmp (id, job->id)) { +return job; + } + } + +return NULL ; +} +void *job_create (const char *job_id,const JobDriver *driver,Error **errp) {Job *job;
@@ -48,17 +71,25 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp)error_setg (errp,"Invalid job ID '%s'" , job_id);return NULL ; } +if (job_get (job_id)) { +error_setg (errp,"Job ID '%s' already in use" , job_id); +return NULL ; + } } job =g_malloc0 (driver->instance_size); job->driver = driver; job->id =g_strdup (job_id); +QLIST_INSERT_HEAD (&jobs, job, job_list); +return job; }void job_delete (Job *job) { +QLIST_REMOVE (job, job_list); +g_free (job->id);g_free (job); }
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 12/42] job: Maintain a list of all jobs
Quoted 10 lines · Kevin Wolf
This moves the job list from BlockJob to Job. Now we can check for duplicate IDs in
job_create().Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 3 --- include/qemu/job.h | 19 +++++++++++++++++++ blockjob.c | 47 +++++++++++++++++++++++++---------------------- job.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 25 deletions(-)
Re: [Qemu-devel] [PATCH 12/42] job: Maintain a list of all jobs
In reply to Kevin WolfQuoted 4 lines · Kevin Wolf
This moves the job list from BlockJob to Job. Now we can check for duplicate IDs in
job_create().Signed-off-by Kevin Wolf <[email protected]>
Re: [Qemu-devel] [PATCH 12/42] job: Maintain a list of all jobs
In reply to Kevin WolfQuoted 12 lines · Kevin Wolf
This moves the job list from BlockJob to Job. Now we can check for duplicate IDs in
job_create().Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 3 --- include/qemu/job.h | 19 +++++++++++++++++++ blockjob.c | 47 +++++++++++++++++++++++++---------------------- job.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 25 deletions(-)
include/qemu/job.h JobType job_type(Job *job); · line 84
Quoted 10 lines
include/qemu/job.h +7
-0@@ -71,4 +75,19 @@ JobType job_type(Job *job);/** Returns the enum string for the JobType of a given Job. */ const char *job_type_str (Job *job); +/** +* Get the next element from the list of block jobs after @job, or the +* first one if @job is %NULL. +* +* Returns the requested job, or %NULL if there are no more jobs left. +*/ +Job *job_next (Job *job);
The new code is supposed to allow you to prime the search by passing in NULL. This looks similar to the semantics originally held by block_job_next(), where
blockjob.c static QLIST_HEAD(, BlockJob) block_jobs = QLIST_HEAD_INITIALIZER(block_jobs); · line 153
Quoted 6 lines
blockjob.c +1
-4-BlockJob *block_job_next (BlockJob *job) +static bool is_block_job (Job *job) { -if (!job) { -return QLIST_FIRST (&block_jobs); - }
the old code did so by special-casing an incoming NULL.
blockjob.c static QLIST_HEAD(, BlockJob) block_jobs = QLIST_HEAD_INITIALIZER(block_jobs); · line 157
Quoted 10 lines
blockjob.c +9
-1-return QLIST_NEXT (job, job_list); +return job_type (job) ==JOB_TYPE_BACKUP || +job_type (job) ==JOB_TYPE_COMMIT || +job_type (job) ==JOB_TYPE_MIRROR || +job_type (job) ==JOB_TYPE_STREAM ; +} + +BlockJob *block_job_next (BlockJob *bjob) +{ +Job *job = &bjob->job;
But the new code blindly dereferences what may be a NULL bjob. Taking the address of a member of the NULL pointer may happen to work if that member is at offset 0, but is also likely to trigger compiler or sanitizer warnings; and does not work if the member is not at offset 0.
I think all you have to do is:
Job *job = bjob ? &bjob->job : NULL;
blockjob.c static QLIST_HEAD(, BlockJob) block_jobs = QLIST_HEAD_INITIALIZER(block_jobs); · line 164
Quoted 7 lines
blockjob.c +7
-0+ +do { + job =job_next (job); + }while (job && !is_block_job (job)); + + +return job ?container_of (job, BlockJob, job) :NULL ;
Why two blank lines?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 13/42 job: Move state transitions to Job
In reply to Kevin WolfThis moves BlockJob.status and the closely related functions (block_)job_state_transition() and (block_)job_apply_verb to Job. The two QAPI enums are renamed to JobStatus and JobVerb.
--- qapi/block-core.json | 14 +++---- include/block/blockjob.h | 3 -- include/qemu/job.h | 13 ++++++ blockjob.c | 102 +++++++++++------------------------------------ job.c | 56 ++++++++++++++++++++++++++ tests/test-blockjob.c | 39 +++++++++--------- block/trace-events | 2 - trace-events | 4 ++ 8 files changed, 122 insertions(+), 111 deletions(-)
qapi/block-core.json +7 -7
@@ -1068,9 +1068,9 @@ 'data': ['commit', 'stream', 'mirror', 'backup'] } ## -# @BlockJobVerb: +# @JobVerb: # -# Represents command verbs that can be applied to a blockjob. +# Represents command verbs that can be applied to a job. # # @cancel: see @block-job-cancel #
@@ -1088,14 +1088,14 @@ # # Since: 2.12 ## -{ 'enum': 'BlockJobVerb', +{ 'enum': 'JobVerb', 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', 'finalize' ] } ## -# @BlockJobStatus: +# @JobStatus: # -# Indicates the present state of a given blockjob in its lifetime. +# Indicates the present state of a given job in its lifetime. # # @undefined: Erroneous, default state. Should not ever be visible. #
@@ -1134,7 +1134,7 @@ # # Since: 2.12 ## -{ 'enum': 'BlockJobStatus', +{ 'enum': 'JobStatus', 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', 'waiting', 'pending', 'aborting', 'concluded', 'null' ] }
@@ -1181,7 +1181,7 @@ 'data': {'type': 'str', 'device': 'str', 'len': 'int', 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int', 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', - 'status': 'BlockJobStatus', + 'status': 'JobStatus', 'auto-finalize': 'bool', 'auto-dismiss': 'bool', '*error': 'str' } }
include/block/blockjob.h +0 -3
@@ -147,9 +147,6 @@ typedef struct BlockJob {*/ QEMUTimer sleep_timer; -/** Current state; See @BlockJobStatus for details. */ -BlockJobStatus status; -/** True if this job should automatically finalize itself */ bool auto_finalize;
include/qemu/job.h +13 -0
@@ -41,6 +41,9 @@ typedef struct Job {/** The type of this job. */ const JobDriver *driver; +/** Current state; See @JobStatus for details. */ +JobStatus status; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;
@@ -90,4 +93,14 @@ Job *job_next(Job *job);*/ Job *job_get (const char *id); +/** +* Check whether the verb @bv can be applied to @job in its current state. +* Returns 0 if the verb can be applied; otherwise errp is set and -EPERM +* returned. +*/ +int job_apply_verb (Job *job,JobVerb bv,Error **errp); + +/* TODO To be removed from the public interface */ +void job_state_transition (Job *job,JobStatus s1); +#endif
blockjob.c +23 -79
231 lines not on this page · raw message
job.c +56 -0
71 lines not on this page · raw message
tests/test-blockjob.c +19 -20
@@ -211,7 +211,7 @@ static CancelJob *create_common(BlockJob **pjob) job =mk_job (blk,"Steve" , &test_cancel_driver, true,BLOCK_JOB_MANUAL_FINALIZE |BLOCK_JOB_MANUAL_DISMISS );block_job_ref (job); -assert (job->status ==BLOCK_JOB_STATUS_CREATED ); +assert (job->job.status ==JOB_STATUS_CREATED ); s =container_of (job, CancelJob, common); s->blk = blk;
@@ -223,15 +223,14 @@ static void cancel_common(CancelJob *s) {BlockJob *job = &s->common;BlockBackend *blk = s->blk; -BlockJobStatus sts = job->status; +JobStatus sts = job->job.status;block_job_cancel_sync (job); -if ((sts !=BLOCK_JOB_STATUS_CREATED ) && - (sts !=BLOCK_JOB_STATUS_CONCLUDED )) { +if (sts !=JOB_STATUS_CREATED && sts !=JOB_STATUS_CONCLUDED ) {BlockJob *dummy = job;block_job_dismiss (&dummy, &error_abort); } -assert (job->status ==BLOCK_JOB_STATUS_NULL ); +assert (job->job.status ==JOB_STATUS_NULL );block_job_unref (job);destroy_blk (blk); }
@@ -253,7 +252,7 @@ static void test_cancel_running(void) s =create_common (&job);block_job_start (job); -assert (job->status ==BLOCK_JOB_STATUS_RUNNING ); +assert (job->job.status ==JOB_STATUS_RUNNING );cancel_common (s); }
@@ -266,11 +265,11 @@ static void test_cancel_paused(void) s =create_common (&job);block_job_start (job); -assert (job->status ==BLOCK_JOB_STATUS_RUNNING ); +assert (job->job.status ==JOB_STATUS_RUNNING );block_job_user_pause (job, &error_abort);block_job_enter (job); -assert (job->status ==BLOCK_JOB_STATUS_PAUSED ); +assert (job->job.status ==JOB_STATUS_PAUSED );cancel_common (s); }
@@ -283,11 +282,11 @@ static void test_cancel_ready(void) s =create_common (&job);block_job_start (job); -assert (job->status ==BLOCK_JOB_STATUS_RUNNING ); +assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;block_job_enter (job); -assert (job->status ==BLOCK_JOB_STATUS_READY ); +assert (job->job.status ==JOB_STATUS_READY );cancel_common (s); }
@@ -300,15 +299,15 @@ static void test_cancel_standby(void) s =create_common (&job);block_job_start (job); -assert (job->status ==BLOCK_JOB_STATUS_RUNNING ); +assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;block_job_enter (job); -assert (job->status ==BLOCK_JOB_STATUS_READY ); +assert (job->job.status ==JOB_STATUS_READY );block_job_user_pause (job, &error_abort);block_job_enter (job); -assert (job->status ==BLOCK_JOB_STATUS_STANDBY ); +assert (job->job.status ==JOB_STATUS_STANDBY );cancel_common (s); }
@@ -321,18 +320,18 @@ static void test_cancel_pending(void) s =create_common (&job);block_job_start (job); -assert (job->status ==BLOCK_JOB_STATUS_RUNNING ); +assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;block_job_enter (job); -assert (job->status ==BLOCK_JOB_STATUS_READY ); +assert (job->job.status ==JOB_STATUS_READY );block_job_complete (job, &error_abort);block_job_enter (job);while (!s->completed) {aio_poll (qemu_get_aio_context (), true); } -assert (job->status ==BLOCK_JOB_STATUS_PENDING ); +assert (job->job.status ==JOB_STATUS_PENDING );cancel_common (s); }
@@ -345,21 +344,21 @@ static void test_cancel_concluded(void) s =create_common (&job);block_job_start (job); -assert (job->status ==BLOCK_JOB_STATUS_RUNNING ); +assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;block_job_enter (job); -assert (job->status ==BLOCK_JOB_STATUS_READY ); +assert (job->job.status ==JOB_STATUS_READY );block_job_complete (job, &error_abort);block_job_enter (job);while (!s->completed) {aio_poll (qemu_get_aio_context (), true); } -assert (job->status ==BLOCK_JOB_STATUS_PENDING ); +assert (job->job.status ==JOB_STATUS_PENDING );block_job_finalize (job, &error_abort); -assert (job->status ==BLOCK_JOB_STATUS_CONCLUDED ); +assert (job->job.status ==JOB_STATUS_CONCLUDED );cancel_common (s); }
block/trace-events +0 -2
@@ -6,8 +6,6 @@ bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" # blockjob.c block_job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" -block_job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" -block_job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" # block/block-backend.c blk_co_preadv(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x"
trace-events +4 -0
@@ -104,6 +104,10 @@ gdbstub_err_invalid_rle(void) "got invalid RLE sequence" gdbstub_err_checksum_invalid(uint8_t ch) "got invalid command checksum digit: 0x%02x" gdbstub_err_checksum_incorrect(uint8_t expected, uint8_t got) "got command packet with incorrect checksum, expected=0x%02x, received=0x%02x" +# job.c +job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" +job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" + ### Guest events, keep at bottom
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 13/42] job: Move state transitions to Job
Quoted 5 lines · Kevin Wolf
This moves BlockJob.status and the closely related functions (block_)job_state_transition() and (block_)job_apply_verb to Job. The two QAPI enums are renamed to JobStatus and JobVerb.
Signed-off-by Kevin Wolf <[email protected]>
Quoted 10 lines
--- qapi/block-core.json | 14 +++---- include/block/blockjob.h | 3 -- include/qemu/job.h | 13 ++++++ blockjob.c | 102 +++++++++++------------------------------------ job.c | 56 ++++++++++++++++++++++++++ tests/test-blockjob.c | 39 +++++++++--------- block/trace-events | 2 - trace-events | 4 ++ 8 files changed, 122 insertions(+), 111 deletions(-)
block-job-dismiss's documentation still mentions BLOCK_JOB_STATUS_CONCLUDED, but that's cleaned up in patch 39, so I don't mind giving a:
Re: [Qemu-devel] [PATCH 13/42] job: Move state transitions to Job
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
This moves BlockJob.status and the closely related functions (block_)job_state_transition() and (block_)job_apply_verb to Job. The two QAPI enums are renamed to JobStatus and JobVerb.
Signed-off-by Kevin Wolf <[email protected]>
In good faith that the TODOs dissolve by the end of the series:
(As a bonus, removing 'BLOCK' out of all of those enum names made the table a little thinner column-wise. nice!)
Re: [Qemu-devel] [PATCH 13/42] job: Move state transitions to Job
In reply to Kevin WolfQuoted 18 lines · Kevin Wolf
This moves BlockJob.status and the closely related functions (block_)job_state_transition() and (block_)job_apply_verb to Job. The two QAPI enums are renamed to JobStatus and JobVerb.
Signed-off-by Kevin Wolf <[email protected]>--- qapi/block-core.json | 14 +++---- include/block/blockjob.h | 3 -- include/qemu/job.h | 13 ++++++ blockjob.c | 102 +++++++++++------------------------------------ job.c | 56 ++++++++++++++++++++++++++ tests/test-blockjob.c | 39 +++++++++--------- block/trace-events | 2 - trace-events | 4 ++ 8 files changed, 122 insertions(+), 111 deletions(-)
include/qemu/job.h Job *job_next(Job *job); · line 101
Quoted 9 lines
include/qemu/job.h +6
-0@@ -90,4 +93,14 @@ Job *job_next(Job *job);*/ Job *job_get (const char *id); +/** +* Check whether the verb @bv can be applied to @job in its current state. +* Returns 0 if the verb can be applied; otherwise errp is set and -EPERM +* returned. +*/ +int job_apply_verb (Job *job,JobVerb bv,Error **errp);
Why 'JobVerb bv' rather than 'jv' or 'verb'?
blockjob.c void *block_job_create(const char *job_id, const BlockJobDriver *driver, · line 971
Quoted 5 lines
blockjob.c +0
-1@@ -968,7 +913,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->refcnt =1 ; job->auto_finalize = !(flags &BLOCK_JOB_MANUAL_FINALIZE ); job->auto_dismiss = !(flags &BLOCK_JOB_MANUAL_DISMISS ); -block_job_state_transition (job,BLOCK_JOB_STATUS_CREATED );
No replacement to the counterpart job_state_transition() here because that should be handled in the parent class now, right? [1]
...
Review void *block_job_create(const char *job_id, const BlockJobDriver *driver, · line 917
Quoted 2 lines
(quoted diff) +2
-0+ +int job_apply_verb(Job *job, JobVerb bv, Error **errp)
Again, the name bv made sense for BlockJobVerb, but less so now.
job.c void *job_create(const char *job_id, const JobDriver *driver, Error **errp) · line 139
Quoted 6 lines
job.c +2
-0@@ -81,6 +135,8 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) job->driver = driver; job->id =g_strdup (job_id); +job_state_transition (job,JOB_STATUS_CREATED ); +
[1] Yes, just took me a while to get to it.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 14/42 job: Add reference counting
In reply to Kevin WolfThis moves reference counting from BlockJob to Job.
In order to keep calling the BlockJob cleanup code when the job is deleted via job_unref(), introduce a new JobDriver.free callback. Every block job must use block_job_free() for this callback, this is asserted in block_job_create().
--- include/block/blockjob.h | 21 ------------------- include/block/blockjob_int.h | 7 +++++++ include/qemu/job.h | 19 ++++++++++++++++-- block/backup.c | 1 + block/commit.c | 1 + block/mirror.c | 2 ++ block/stream.c | 1 + blockjob.c | 48 +++++++++++++++++++------------------------- job.c | 22 ++++++++++++++++---- qemu-img.c | 4 ++-- tests/test-bdrv-drain.c | 1 + tests/test-blockjob-txn.c | 1 + tests/test-blockjob.c | 6 ++++-- 13 files changed, 76 insertions(+), 58 deletions(-)
include/block/blockjob.h +0 -21
@@ -132,9 +132,6 @@ typedef struct BlockJob {/** The opaque value that is passed to the completion function. */ void *opaque; -/** Reference count of the block job */ -int refcnt; -/** True when job has reported completion by calling block_job_completed. */ bool completed;
@@ -400,24 +397,6 @@ void block_job_iostatus_reset(BlockJob *job);BlockJobTxn *block_job_txn_new (void );/** -* block_job_ref: -* -* Add a reference to BlockJob refcnt, it will be decreased with -* block_job_unref, and then be freed if it comes to be the last -* reference. -*/ -void block_job_ref (BlockJob *job); - -/** -* block_job_unref: -* -* Release a reference that was previously acquired with block_job_ref -* or block_job_create. If it's the last reference to the object, it will be -* freed. -*/ -void block_job_unref (BlockJob *job); - -/** * block_job_txn_unref: * * Release a reference that was previously acquired with block_job_txn_add_job
include/block/blockjob_int.h +7 -0
@@ -144,6 +144,13 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, BlockCompletionFunc *cb, void *opaque, Error **errp);/** +* block_job_free: +* Callback to be used for JobDriver.free in all block jobs. Frees block job +* specific resources in @job. +*/ +void block_job_free (Job *job); + +/** * block_job_sleep_ns: * @job: The job that calls the function. * @ns: How many nanoseconds to stop for.
include/qemu/job.h +17 -2
@@ -41,6 +41,9 @@ typedef struct Job {/** The type of this job. */ const JobDriver *driver; +/** Reference count of the block job */ +int refcnt; +/** Current state; See @JobStatus for details. */ JobStatus status;
@@ -57,6 +60,9 @@ struct JobDriver {/** Enum describing the operation */ JobType job_type; + +/** Called when the job is freed */ +void (*free)(Job *job); };
@@ -69,8 +75,17 @@ struct JobDriver {*/ void *job_create (const char *job_id,const JobDriver *driver,Error **errp); -/** Frees the @job object. */ -void job_delete (Job *job); +/** +* Add a reference to Job refcnt, it will be decreased with job_unref, and then +* be freed if it comes to be the last reference. +*/ +void job_ref (Job *job); + +/** +* Release a reference that was previously acquired with job_ref() or +* job_create(). If it's the last reference to the object, it will be freed. +*/ +void job_unref (Job *job);/** Returns the JobType of a given Job. */ JobType job_type (Job *job);
block/backup.c +1 -0
@@ -526,6 +526,7 @@ static const BlockJobDriver backup_job_driver = { .job_driver = { .instance_size = sizeof(BackupBlockJob), .job_type = JOB_TYPE_BACKUP, + .free = block_job_free, }, .start = backup_run, .commit = backup_commit,
block/commit.c +1 -0
@@ -218,6 +218,7 @@ static const BlockJobDriver commit_job_driver = { .job_driver = { .instance_size = sizeof(CommitBlockJob), .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, }, .start = commit_run, };
block/mirror.c +2 -0
@@ -989,6 +989,7 @@ static const BlockJobDriver mirror_job_driver = { .job_driver = { .instance_size = sizeof(MirrorBlockJob), .job_type = JOB_TYPE_MIRROR, + .free = block_job_free, }, .start = mirror_run, .complete = mirror_complete,
@@ -1001,6 +1002,7 @@ static const BlockJobDriver commit_active_job_driver = { .job_driver = { .instance_size = sizeof(MirrorBlockJob), .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, }, .start = mirror_run, .complete = mirror_complete,
block/stream.c +1 -0
@@ -212,6 +212,7 @@ static const BlockJobDriver stream_job_driver = { .job_driver = { .instance_size = sizeof(StreamBlockJob), .job_type = JOB_TYPE_STREAM, + .free = block_job_free, }, .start = stream_run, };
blockjob.c +21 -27
107 lines not on this page · raw message
job.c +18 -4
@@ -134,6 +134,7 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) job =g_malloc0 (driver->instance_size); job->driver = driver; job->id =g_strdup (job_id); + job->refcnt =1 ;job_state_transition (job,JOB_STATUS_CREATED );
@@ -142,10 +143,23 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp)return job; } -void job_delete (Job *job) +void job_ref (Job *job) { -QLIST_REMOVE (job, job_list); + ++job->refcnt; +} + +void job_unref (Job *job) +{ +if (--job->refcnt ==0 ) { +assert (job->status ==JOB_STATUS_NULL ); -g_free (job->id); -g_free (job); +if (job->driver->free) { + job->driver->free (job); + } + +QLIST_REMOVE (job, job_list); + +g_free (job->id); +g_free (job); + } }
qemu-img.c +2 -2
@@ -862,7 +862,7 @@ static void run_block_job(BlockJob *job, Error **errp)int ret =0 ;aio_context_acquire (aio_context); -block_job_ref (job); +job_ref (&job->job); do { aio_poll(aio_context, true); qemu_progress_print(job->len ?
@@ -874,7 +874,7 @@ static void run_block_job(BlockJob *job, Error **errp) }else { ret = job->ret; } -block_job_unref (job); +job_unref (&job->job);aio_context_release (aio_context);/* publish completion progress only when success */
tests/test-bdrv-drain.c +1 -0
@@ -522,6 +522,7 @@ static void test_job_complete(BlockJob *job, Error **errp)BlockJobDriver test_job_driver = { .job_driver = { .instance_size =sizeof (TestBlockJob), + .free = block_job_free, }, .start = test_job_start, .complete = test_job_complete,
tests/test-blockjob-txn.c +1 -0
@@ -76,6 +76,7 @@ static void test_block_job_cb(void *opaque, int ret)static const BlockJobDriver test_block_job_driver = { .job_driver = { .instance_size =sizeof (TestBlockJob), + .free = block_job_free, }, .start = test_block_job_run, };
tests/test-blockjob.c +4 -2
@@ -19,6 +19,7 @@static const BlockJobDriver test_block_job_driver = { .job_driver = { .instance_size =sizeof (BlockJob), + .free = block_job_free, }, };
@@ -196,6 +197,7 @@ static void coroutine_fn cancel_job_start(void *opaque)static const BlockJobDriver test_cancel_driver = { .job_driver = { .instance_size =sizeof (CancelJob), + .free = block_job_free, }, .start = cancel_job_start, .complete = cancel_job_complete,
@@ -210,7 +212,7 @@ static CancelJob *create_common(BlockJob **pjob) blk =create_blk (NULL ); job =mk_job (blk,"Steve" , &test_cancel_driver, true,BLOCK_JOB_MANUAL_FINALIZE |BLOCK_JOB_MANUAL_DISMISS ); -block_job_ref (job); +job_ref (&job->job);assert (job->job.status ==JOB_STATUS_CREATED ); s =container_of (job, CancelJob, common); s->blk = blk;
@@ -231,7 +233,7 @@ static void cancel_common(CancelJob *s)block_job_dismiss (&dummy, &error_abort); }assert (job->job.status ==JOB_STATUS_NULL ); -block_job_unref (job); +job_unref (&job->job);destroy_blk (blk); }
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 14/42] job: Add reference counting
Quoted 8 lines · Kevin Wolf
This moves reference counting from BlockJob to Job.
In order to keep calling the BlockJob cleanup code when the job is deleted via
job_unref(), introduce a new JobDriver.free callback. Every block job must useblock_job_free()for this callback, this is asserted inblock_job_create().Signed-off-by Kevin Wolf <[email protected]>
Quoted 15 lines
--- include/block/blockjob.h | 21 ------------------- include/block/blockjob_int.h | 7 +++++++ include/qemu/job.h | 19 ++++++++++++++++-- block/backup.c | 1 + block/commit.c | 1 + block/mirror.c | 2 ++ block/stream.c | 1 + blockjob.c | 48 +++++++++++++++++++------------------------- job.c | 22 ++++++++++++++++---- qemu-img.c | 4 ++-- tests/test-bdrv-drain.c | 1 + tests/test-blockjob-txn.c | 1 + tests/test-blockjob.c | 6 ++++-- 13 files changed, 76 insertions(+), 58 deletions(-)
Re: [Qemu-devel] [PATCH 14/42] job: Add reference counting
In reply to Kevin WolfQuoted 8 lines · Kevin Wolf
This moves reference counting from BlockJob to Job.
In order to keep calling the BlockJob cleanup code when the job is deleted via
job_unref(), introduce a new JobDriver.free callback. Every block job must useblock_job_free()for this callback, this is asserted inblock_job_create().Signed-off-by Kevin Wolf <[email protected]>
So far so good, though it does look a little silly that presently every last job has the exact same free callback.
Also, I forgot to reply to #13 with this, but I suppose that the approach you're taking -- of a fairly straightforward mechanical refactor -- means we don't really have the opportunity to change any of the pretty dumb names or existing peculiarities of design we've evolved.
I had hoped we'd be able to change a few things, but certainly keeping them as-is makes the whole re-factoring process an awful lot simpler...
...Or maybe I'm getting ahead of myself, there's a lot of series left to go.
Ah, anyway:
Re: [Qemu-devel] [PATCH 14/42] job: Add reference counting
Am 14.05.2018 um 23:34 hat John Snow geschrieben:
Quoted 20 lines · Kevin Wolf
Quoted 8 lines
This moves reference counting from BlockJob to Job.
In order to keep calling the BlockJob cleanup code when the job is deleted via
job_unref(), introduce a new JobDriver.free callback. Every block job must useblock_job_free()for this callback, this is asserted inblock_job_create().Signed-off-by Kevin Wolf <[email protected]>So far so good, though it does look a little silly that presently every last job has the exact same free callback.
Also, I forgot to reply to #13 with this, but I suppose that the approach you're taking -- of a fairly straightforward mechanical refactor -- means we don't really have the opportunity to change any of the pretty dumb names or existing peculiarities of design we've evolved.
Quoted 5 lines
I had hoped we'd be able to change a few things, but certainly keeping them as-is makes the whole re-factoring process an awful lot simpler...
...Or maybe I'm getting ahead of myself, there's a lot of series left to go.
No, I think you're right. My approach was that we'd separate the generic and the block-specific parts in this series without changing much in the structure or the names, to keep the conversion as obvious as possible (the series is long enough already).
However, I intentionally kept the QMP interface minimal where I wasn't quite sure that we want to keep the old one. If you see anything in the QMP interface that you'd like to see changed, please do comment. Anything else is an implementation detail and can still be changed on top of this series.
Kevin
Re: [Qemu-devel] [PATCH 14/42] job: Add reference counting
In reply to Kevin WolfQuoted 12 lines · Kevin Wolf
This moves reference counting from BlockJob to Job.
In order to keep calling the BlockJob cleanup code when the job is deleted via
job_unref(), introduce a new JobDriver.free callback. Every block job must useblock_job_free()for this callback, this is asserted inblock_job_create().Signed-off-by Kevin Wolf <[email protected]>---
+++
b/job.c
job.c void *job_create(const char *job_id, const JobDriver *driver, Error **errp) · line 153
Quoted 4 lines
job.c +4
-0+ +void job_unref (Job *job) +{ +if (--job->refcnt ==0 ) {
Should this be free()-like and allow an incoming job == NULL as a no-op?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 14/42] job: Add reference counting
Am 16.05.2018 um 20:17 hat Eric Blake geschrieben:
Quoted 11 lines · Kevin Wolf
Quoted 9 lines
This moves reference counting from BlockJob to Job.
In order to keep calling the BlockJob cleanup code when the job is deleted via
job_unref(), introduce a new JobDriver.free callback. Every block job must useblock_job_free()for this callback, this is asserted inblock_job_create().Signed-off-by Kevin Wolf <[email protected]>---
Quoted 8 lines
job.c void *job_create(const char *job_id, const JobDriver *driver, Error **errp) · line 153
Quoted 6 lines
+++
b/job.cjob.c +4
-0+ +void job_unref (Job *job) +{ +if (--job->refcnt ==0 ) {Should this be free()-like and allow an incoming job == NULL as a no-op?
This behaves like block_job_unref() always behavec, and I don't see a single caller having a NULL check before calling job_unref(), so is it worth it?
Kevin
Re: [Qemu-devel] [PATCH 14/42] job: Add reference counting
Quoted 10 lines · Kevin Wolf
Quoted 6 lines
job.c void *job_create(const char *job_id, const JobDriver *driver, Error **errp) · line 153
Quoted 4 lines
job.c +4
-0+ +void job_unref (Job *job) +{ +if (--job->refcnt ==0 ) {Should this be free()-like and allow an incoming job == NULL as a no-op?
This behaves like
block_job_unref()always behavec, and I don't see a single caller having a NULL check before callingjob_unref(), so is it worth it?
Only if it makes it easier to clean up a partially-constructed object (which is the most likely case of wanting to pass in NULL)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 15/42 job: Move cancelled to Job
In reply to Kevin WolfWe cannot yet move the whole logic around job cancelling to Job because it depends on quite a few other things that are still only in BlockJob, but we can move the cancelled field at least.
--- include/block/blockjob.h | 8 -------- include/block/blockjob_int.h | 8 -------- include/qemu/job.h | 11 +++++++++++ block/backup.c | 6 +++--- block/commit.c | 4 ++-- block/mirror.c | 20 ++++++++++---------- block/stream.c | 4 ++-- blockjob.c | 28 +++++++++++++--------------- job.c | 5 +++++ tests/test-blockjob-txn.c | 6 +++--- tests/test-blockjob.c | 2 +- 11 files changed, 50 insertions(+), 52 deletions(-)
include/block/blockjob.h +0 -8
@@ -57,14 +57,6 @@ typedef struct BlockJob {Coroutine *co;/** -* Set to true if the job should cancel itself. The flag must -* always be tested just before toggling the busy flag from false -* to true. After a job has been cancelled, it should only yield -* if #aio_poll will ("sooner or later") reenter the coroutine. -*/ -bool cancelled; - -/** * Set to true if the job should abort immediately without waiting * for data to be in sync. */
include/block/blockjob_int.h +0 -8
@@ -196,14 +196,6 @@ void block_job_early_fail(BlockJob *job);void block_job_completed (BlockJob *job,int ret);/** -* block_job_is_cancelled: -* @job: The job being queried. -* -* Returns whether the job is scheduled for cancellation. -*/ -bool block_job_is_cancelled (BlockJob *job); - -/** * block_job_pause_point: * @job: The job that is ready to pause. *
include/qemu/job.h +11 -0
@@ -47,6 +47,14 @@ typedef struct Job {/** Current state; See @JobStatus for details. */ JobStatus status; +/** +* Set to true if the job should cancel itself. The flag must +* always be tested just before toggling the busy flag from false +* to true. After a job has been cancelled, it should only yield +* if #aio_poll will ("sooner or later") reenter the coroutine. +*/ +bool cancelled; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;
@@ -93,6 +101,9 @@ JobType job_type(Job *job);/** Returns the enum string for the JobType of a given Job. */ const char *job_type_str (Job *job); +/** Returns whether the job is scheduled for cancellation. */ +bool job_is_cancelled (Job *job); +/** * Get the next element from the list of block jobs after @job, or the * first one if @job is %NULL.
block/backup.c +3 -3
@@ -329,7 +329,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) {uint64_t delay_ns; -if (block_job_is_cancelled (&job->common)) { +if (job_is_cancelled (&job->common.job)) {return true; }
@@ -339,7 +339,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) job->bytes_read =0 ;block_job_sleep_ns (&job->common, delay_ns); -if (block_job_is_cancelled (&job->common)) { +if (job_is_cancelled (&job->common.job)) {return true; }
@@ -441,7 +441,7 @@ static void coroutine_fn backup_run(void *opaque)if (job->sync_mode ==MIRROR_SYNC_MODE_NONE ) {/* All bits are set in copy_bitmap to allow any cluster to be copied. * This does not actually require them to be copied. */ -while (!block_job_is_cancelled (&job->common)) { +while (!job_is_cancelled (&job->common.job)) {/* Yield until the job is cancelled. We just let our before_write * notify callback service CoW requests. */ block_job_yield (&job->common);
block/commit.c +2 -2
@@ -90,7 +90,7 @@ static void commit_complete(BlockJob *job, void *opaque)* the normal backing chain can be restored. */ blk_unref (s->base); -if (!block_job_is_cancelled (&s->common) && ret ==0 ) { +if (!job_is_cancelled (&s->common.job) && ret ==0 ) {/* success */ ret =bdrv_drop_intermediate (s->commit_top_bs, base, s->backing_file_str);
@@ -172,7 +172,7 @@ static void coroutine_fn commit_run(void *opaque)* with no pending I/O here so that bdrv_drain_all() returns. */ block_job_sleep_ns (&s->common, delay_ns); -if (block_job_is_cancelled (&s->common)) { +if (job_is_cancelled (&s->common.job)) {break ; }/* Copy if allocated above the base */
block/mirror.c +10 -10
@@ -622,7 +622,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)mirror_throttle (s); -if (block_job_is_cancelled (&s->common)) { +if (job_is_cancelled (&s->common.job)) { s->initial_zeroing_ongoing = false;return 0 ; }
@@ -650,7 +650,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)mirror_throttle (s); -if (block_job_is_cancelled (&s->common)) { +if (job_is_cancelled (&s->common.job)) {return 0 ; }
@@ -695,7 +695,7 @@ static void coroutine_fn mirror_run(void *opaque)checking for a NULL string */ int ret =0 ; -if (block_job_is_cancelled (&s->common)) { +if (job_is_cancelled (&s->common.job)) { goto immediate_exit; }
@@ -729,10 +729,10 @@ static void coroutine_fn mirror_run(void *opaque)/* Report BLOCK_JOB_READY and wait for complete. */ block_job_event_ready (&s->common); s->synced = true; -while (!block_job_is_cancelled (&s->common) && !s->should_complete) { +while (!job_is_cancelled (&s->common.job) && !s->should_complete) {block_job_yield (&s->common); } - s->common.cancelled = false; + s->common.job.cancelled = false; goto immediate_exit; }
@@ -768,7 +768,7 @@ static void coroutine_fn mirror_run(void *opaque) s->last_pause_ns =qemu_clock_get_ns (QEMU_CLOCK_REALTIME );if (!s->is_none_mode) { ret =mirror_dirty_init (s); -if (ret <0 ||block_job_is_cancelled (&s->common)) { +if (ret <0 ||job_is_cancelled (&s->common.job)) { goto immediate_exit; } }
@@ -829,7 +829,7 @@ static void coroutine_fn mirror_run(void *opaque) } should_complete = s->should_complete || -block_job_is_cancelled (&s->common); +job_is_cancelled (&s->common.job); cnt =bdrv_get_dirty_count (s->dirty_bitmap); }
@@ -857,7 +857,7 @@ static void coroutine_fn mirror_run(void *opaque)* completion. */ assert (QLIST_EMPTY (&bs->tracked_requests)); - s->common.cancelled = false; + s->common.job.cancelled = false; need_drain = false;break ; }
@@ -869,7 +869,7 @@ static void coroutine_fn mirror_run(void *opaque) }trace_mirror_before_sleep (s, cnt, s->synced, delay_ns);block_job_sleep_ns (&s->common, delay_ns); -if (block_job_is_cancelled (&s->common) && +if (job_is_cancelled (&s->common.job) && (!s->synced || s->common.force)) {break ;
@@ -884,7 +884,7 @@ immediate_exit:* the target is a copy of the source. */ assert (ret <0 || ((s->common.force || !s->synced) && -block_job_is_cancelled (&s->common))); +job_is_cancelled (&s->common.job)));assert (need_drain);mirror_wait_for_all_io (s); }
block/stream.c +2 -2
@@ -66,7 +66,7 @@ static void stream_complete(BlockJob *job, void *opaque)BlockDriverState *base = s->base;Error *local_err =NULL ; -if (!block_job_is_cancelled (&s->common) && bs->backing && +if (!job_is_cancelled (&s->common.job) && bs->backing && data->ret ==0 ) {const char *base_id =NULL , *base_fmt =NULL ;if (base) {
@@ -141,7 +141,7 @@ static void coroutine_fn stream_run(void *opaque)* with no pending I/O here so that bdrv_drain_all() returns. */ block_job_sleep_ns (&s->common, delay_ns); -if (block_job_is_cancelled (&s->common)) { +if (job_is_cancelled (&s->common.job)) {break ; }
blockjob.c +13 -15
@@ -366,7 +366,7 @@ static void block_job_conclude(BlockJob *job)static void block_job_update_rc (BlockJob *job) { -if (!job->ret &&block_job_is_cancelled (job)) { +if (!job->ret &&job_is_cancelled (&job->job)) { job->ret = -ECANCELED ; }if (job->ret) {
@@ -425,7 +425,7 @@ static int block_job_finalize_single(BlockJob *job)/* Emit events only if we actually started */ if (block_job_started (job)) { -if (block_job_is_cancelled (job)) { +if (job_is_cancelled (&job->job)) {block_job_event_cancelled (job); }else {const char *msg =NULL ;
@@ -451,7 +451,7 @@ static void block_job_cancel_async(BlockJob *job, bool force) job->user_paused = false; job->pause_count--; } - job->cancelled = true; + job->job.cancelled = true;/* To prevent 'force == false' overriding a previous 'force == true' */ job->force |= force; }
@@ -506,7 +506,8 @@ static int block_job_finish_sync(BlockJob *job,while (!job->completed) {aio_poll (qemu_get_aio_context (), true); } - ret = (job->cancelled && job->ret ==0 ) ? -ECANCELED : job->ret; + ret = (job_is_cancelled (&job->job) && job->ret ==0 ) + ? -ECANCELED : job->ret;job_unref (&job->job);return ret; }
@@ -544,7 +545,7 @@ static void block_job_completed_txn_abort(BlockJob *job) other_job =QLIST_FIRST (&txn->jobs); ctx =blk_get_aio_context (other_job->blk);if (!other_job->completed) { -assert (other_job->cancelled); +assert (job_is_cancelled (&other_job->job));block_job_finish_sync (other_job,NULL ,NULL ); }block_job_finalize_single (other_job);
@@ -644,7 +645,9 @@ void block_job_complete(BlockJob *job, Error **errp)if (job_apply_verb (&job->job,JOB_VERB_COMPLETE , errp)) {return ; } -if (job->pause_count || job->cancelled || !job->driver->complete) { +if (job->pause_count ||job_is_cancelled (&job->job) || + !job->driver->complete) + {error_setg (errp,"The active block job '%s' cannot be completed" , job->job.id);return ;
@@ -999,7 +1002,7 @@ void coroutine_fn block_job_pause_point(BlockJob *job)if (!block_job_should_pause (job)) {return ; } -if (block_job_is_cancelled (job)) { +if (job_is_cancelled (&job->job)) {return ; }
@@ -1007,7 +1010,7 @@ void coroutine_fn block_job_pause_point(BlockJob *job) job->driver->pause (job); } -if (block_job_should_pause (job) && !block_job_is_cancelled (job)) { +if (block_job_should_pause (job) && !job_is_cancelled (&job->job)) {JobStatus status = job->job.status; job_state_transition(&job->job, status == JOB_STATUS_READY ? JOB_STATUS_STANDBY
@@ -1059,17 +1062,12 @@ void block_job_enter(BlockJob *job)block_job_enter_cond (job,NULL ); } -bool block_job_is_cancelled (BlockJob *job) -{ -return job->cancelled; -} -void block_job_sleep_ns (BlockJob *job,int64_t ns) {assert (job->busy);/* Check cancellation *before* setting busy = false, too! */ -if (block_job_is_cancelled (job)) { +if (job_is_cancelled (&job->job)) {return ; }
@@ -1085,7 +1083,7 @@ void block_job_yield(BlockJob *job)assert (job->busy);/* Check cancellation *before* setting busy = false, too! */ -if (block_job_is_cancelled (job)) { +if (job_is_cancelled (&job->job)) {return ; }
job.c +5 -0
@@ -95,6 +95,11 @@ const char *job_type_str(Job *job)return JobType_str (job_type (job)); } +bool job_is_cancelled (Job *job) +{ +return job->cancelled; +} +Job *job_next (Job *job) {if (!job) {
tests/test-blockjob-txn.c +3 -3
@@ -29,7 +29,7 @@ static void test_block_job_complete(BlockJob *job, void *opaque)BlockDriverState *bs =blk_bs (job->blk);int rc = (intptr_t )opaque; -if (block_job_is_cancelled (job)) { +if (job_is_cancelled (&job->job)) { rc = -ECANCELED ; }
@@ -49,7 +49,7 @@ static void coroutine_fn test_block_job_run(void *opaque)block_job_yield (job); } -if (block_job_is_cancelled (job)) { +if (job_is_cancelled (&job->job)) {break ; } }
@@ -66,7 +66,7 @@ typedef struct {static void test_block_job_cb (void *opaque,int ret) {TestBlockJobCBData *data = opaque; -if (!ret &&block_job_is_cancelled (&data->job->common)) { +if (!ret &&job_is_cancelled (&data->job->common.job)) { ret = -ECANCELED ; } *data->result = ret;
tests/test-blockjob.c +1 -1
@@ -179,7 +179,7 @@ static void coroutine_fn cancel_job_start(void *opaque)CancelJob *s = opaque;while (!s->should_complete) { -if (block_job_is_cancelled (&s->common)) { +if (job_is_cancelled (&s->common.job)) { goto defer; }
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 15/42] job: Move cancelled to Job
Quoted 5 lines · Kevin Wolf
We cannot yet move the whole logic around job cancelling to Job because it depends on quite a few other things that are still only in BlockJob, but we can move the cancelled field at least.
Signed-off-by Kevin Wolf <[email protected]>
Quoted 13 lines
--- include/block/blockjob.h | 8 -------- include/block/blockjob_int.h | 8 -------- include/qemu/job.h | 11 +++++++++++ block/backup.c | 6 +++--- block/commit.c | 4 ++-- block/mirror.c | 20 ++++++++++---------- block/stream.c | 4 ++-- blockjob.c | 28 +++++++++++++--------------- job.c | 5 +++++ tests/test-blockjob-txn.c | 6 +++--- tests/test-blockjob.c | 2 +- 11 files changed, 50 insertions(+), 52 deletions(-)
Re: [Qemu-devel] [PATCH 15/42] job: Move cancelled to Job
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
We cannot yet move the whole logic around job cancelling to Job because it depends on quite a few other things that are still only in BlockJob, but we can move the cancelled field at least.
Signed-off-by Kevin Wolf <[email protected]>
PATCH 16/42 job: Add Job.aio_context
In reply to Kevin WolfWhen block jobs need an AioContext, they just take it from their main block node. Generic jobs don't have a main block node, so we need to assign them an AioContext explicitly.
--- include/qemu/job.h | 7 ++++++- blockjob.c | 5 ++++- job.c | 4 +++- 3 files changed, 13 insertions(+), 3 deletions(-)
include/qemu/job.h +6 -1
@@ -47,6 +47,9 @@ typedef struct Job {/** Current state; See @JobStatus for details. */ JobStatus status; +/** AioContext to run the job coroutine in */ +AioContext *aio_context; +/** * Set to true if the job should cancel itself. The flag must * always be tested just before toggling the busy flag from false
@@ -79,9 +82,11 @@ struct JobDriver {* * @job_id: The id of the newly-created job, or %NULL for internal jobs * @driver: The class object for the newly-created job. +* @ctx: The AioContext to run the job coroutine in. * @errp: Error object. */ -void *job_create (const char *job_id,const JobDriver *driver,Error **errp); +void *job_create (const char *job_id,const JobDriver *driver,AioContext *ctx, +Error **errp);/** * Add a reference to Job refcnt, it will be decreased with job_unref, and then
blockjob.c +4 -1
@@ -203,6 +203,7 @@ static void block_job_attached_aio_context(AioContext *new_context, {BlockJob *job = opaque; + job->job.aio_context = new_context;if (job->driver->attached_aio_context) { job->driver->attached_aio_context (job, new_context); }
@@ -234,6 +235,7 @@ static void block_job_detach_aio_context(void *opaque)block_job_drain (job); } + job->job.aio_context =NULL ;job_unref (&job->job); }
@@ -892,7 +894,8 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,return NULL ; } - job =job_create (job_id, &driver->job_driver, errp); + job =job_create (job_id, &driver->job_driver,blk_get_aio_context (blk), + errp);if (job ==NULL ) {blk_unref (blk);return NULL ;
job.c +3 -1
@@ -121,7 +121,8 @@ Job *job_get(const char *id)return NULL ; } -void *job_create (const char *job_id,const JobDriver *driver,Error **errp) +void *job_create (const char *job_id,const JobDriver *driver,AioContext *ctx, +Error **errp) {Job *job;
@@ -140,6 +141,7 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) job->driver = driver; job->id =g_strdup (job_id); job->refcnt =1 ; + job->aio_context = ctx;job_state_transition (job,JOB_STATUS_CREATED );
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 16/42] job: Add Job.aio_context
Quoted 10 lines · Kevin Wolf
When block jobs need an AioContext, they just take it from their main block node. Generic jobs don't have a main block node, so we need to assign them an AioContext explicitly.
Signed-off-by Kevin Wolf <[email protected]>--- include/qemu/job.h | 7 ++++++- blockjob.c | 5 ++++- job.c | 4 +++- 3 files changed, 13 insertions(+), 3 deletions(-)
Re: [Qemu-devel] [PATCH 16/42] job: Add Job.aio_context
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
When block jobs need an AioContext, they just take it from their main block node. Generic jobs don't have a main block node, so we need to assign them an AioContext explicitly.
Signed-off-by Kevin Wolf <[email protected]>
Nothing uses the field yet, but so far so good.
PATCH 17/42 job: Move defer_to_main_loop to Job
In reply to Kevin Wolf--- include/block/blockjob.h | 5 ---- include/block/blockjob_int.h | 19 --------------- include/qemu/job.h | 20 ++++++++++++++++ block/backup.c | 7 +++--- block/commit.c | 11 +++++---- block/mirror.c | 15 ++++++------ block/stream.c | 14 +++++------ blockjob.c | 57 ++++---------------------------------------- job.c | 33 +++++++++++++++++++++++++ tests/test-bdrv-drain.c | 7 +++--- tests/test-blockjob-txn.c | 13 +++++----- tests/test-blockjob.c | 7 +++--- 12 files changed, 98 insertions(+), 110 deletions(-)
include/block/blockjob.h +0 -5
@@ -92,11 +92,6 @@ typedef struct BlockJob {*/ bool ready; -/** -* Set to true when the job has deferred work to the main loop. -*/ -bool deferred_to_main_loop; -/** Status that is published by the query-block-jobs QMP API */ BlockDeviceIoStatus iostatus;
include/block/blockjob_int.h +0 -19
@@ -233,23 +233,4 @@ void block_job_event_ready(BlockJob *job);BlockErrorAction block_job_error_action (BlockJob *job,BlockdevOnError on_err,int is_read,int error); -typedef void BlockJobDeferToMainLoopFn (BlockJob *job,void *opaque); - -/** -* block_job_defer_to_main_loop: -* @job: The job -* @fn: The function to run in the main loop -* @opaque: The opaque value that is passed to @fn -* -* This function must be called by the main job coroutine just before it -* returns. @fn is executed in the main loop with the BlockDriverState -* AioContext acquired. Block jobs must call bdrv_unref(), bdrv_close(), and -* anything that uses bdrv_drain_all() in the main loop. -* -* The @job AioContext is held while @fn executes. -*/ -void block_job_defer_to_main_loop (BlockJob *job, -BlockJobDeferToMainLoopFn *fn, -void *opaque); -#endif
include/qemu/job.h +20 -0
@@ -58,6 +58,9 @@ typedef struct Job {*/ bool cancelled; +/** Set to true when the job has deferred work to the main loop. */ +bool deferred_to_main_loop; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;
@@ -131,6 +134,23 @@ Job *job_get(const char *id);*/ int job_apply_verb (Job *job,JobVerb bv,Error **errp); +typedef void JobDeferToMainLoopFn (Job *job,void *opaque); + +/** +* @job: The job +* @fn: The function to run in the main loop +* @opaque: The opaque value that is passed to @fn +* +* This function must be called by the main job coroutine just before it +* returns. @fn is executed in the main loop with the job AioContext acquired. +* +* Block jobs must call bdrv_unref(), bdrv_close(), and anything that uses +* bdrv_drain_all() in the main loop. +* +* The @job AioContext is held while @fn executes. +*/ +void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque); +/* TODO To be removed from the public interface */ void job_state_transition (Job *job,JobStatus s1);
block/backup.c +4 -3
@@ -317,11 +317,12 @@ typedef struct {int ret; } BackupCompleteData; -static void backup_complete (BlockJob *job,void *opaque) +static void backup_complete (Job *job,void *opaque) { +BlockJob *bjob =container_of (job, BlockJob, job);BackupCompleteData *data = opaque; -block_job_completed (job, data->ret); +block_job_completed (bjob, data->ret);g_free (data); }
@@ -519,7 +520,7 @@ static void coroutine_fn backup_run(void *opaque) data =g_malloc (sizeof (*data)); data->ret = ret; -block_job_defer_to_main_loop (&job->common, backup_complete, data); +job_defer_to_main_loop (&job->common.job, backup_complete, data); }static const BlockJobDriver backup_job_driver = {
block/commit.c +6 -5
@@ -72,9 +72,10 @@ typedef struct {int ret; } CommitCompleteData; -static void commit_complete (BlockJob *job,void *opaque) +static void commit_complete (Job *job,void *opaque) { -CommitBlockJob *s =container_of (job, CommitBlockJob, common); +CommitBlockJob *s =container_of (job, CommitBlockJob, common.job); +BlockJob *bjob = &s->common;CommitCompleteData *data = opaque;BlockDriverState *top =blk_bs (s->top);BlockDriverState *base =blk_bs (s->base);
@@ -90,7 +91,7 @@ static void commit_complete(BlockJob *job, void *opaque)* the normal backing chain can be restored. */ blk_unref (s->base); -if (!job_is_cancelled (&s->common.job) && ret ==0 ) { +if (!job_is_cancelled (job) && ret ==0 ) {/* success */ ret =bdrv_drop_intermediate (s->commit_top_bs, base, s->backing_file_str);
@@ -114,7 +115,7 @@ static void commit_complete(BlockJob *job, void *opaque)* block_job_finish_sync()), block_job_completed() won't free it and * therefore the blockers on the intermediate nodes remain. This would * cause bdrv_set_backing_hd() to fail. */ -block_job_remove_all_bdrv (job); +block_job_remove_all_bdrv (bjob);block_job_completed (&s->common, ret);g_free (data);
@@ -211,7 +212,7 @@ out: data =g_malloc (sizeof (*data)); data->ret = ret; -block_job_defer_to_main_loop (&s->common, commit_complete, data); +job_defer_to_main_loop (&s->common.job, commit_complete, data); }static const BlockJobDriver commit_job_driver = {
block/mirror.c +8 -7
@@ -484,9 +484,10 @@ typedef struct {int ret; } MirrorExitData; -static void mirror_exit (BlockJob *job,void *opaque) +static void mirror_exit (Job *job,void *opaque) { -MirrorBlockJob *s =container_of (job, MirrorBlockJob, common); +MirrorBlockJob *s =container_of (job, MirrorBlockJob, common.job); +BlockJob *bjob = &s->common;MirrorExitData *data = opaque;AioContext *replace_aio_context =NULL ;BlockDriverState *src = s->source;
@@ -568,7 +569,7 @@ static void mirror_exit(BlockJob *job, void *opaque)* the blockers on the intermediate nodes so that the resulting state is * valid. Also give up permissions on mirror_top_bs->backing, which might * block the removal. */ -block_job_remove_all_bdrv (job); +block_job_remove_all_bdrv (bjob);bdrv_child_try_set_perm (mirror_top_bs->backing,0 ,BLK_PERM_ALL , &error_abort);bdrv_replace_node (mirror_top_bs,backing_bs (mirror_top_bs), &error_abort);
@@ -576,9 +577,9 @@ static void mirror_exit(BlockJob *job, void *opaque)/* We just changed the BDS the job BB refers to (with either or both of the * bdrv_replace_node() calls), so switch the BB back so the cleanup does * the right thing. We don't need any permissions any more now. */ -blk_remove_bs (job->blk); -blk_set_perm (job->blk,0 ,BLK_PERM_ALL , &error_abort); -blk_insert_bs (job->blk, mirror_top_bs, &error_abort); +blk_remove_bs (bjob->blk); +blk_set_perm (bjob->blk,0 ,BLK_PERM_ALL , &error_abort); +blk_insert_bs (bjob->blk, mirror_top_bs, &error_abort);block_job_completed (&s->common, data->ret);
@@ -901,7 +902,7 @@ immediate_exit:if (need_drain) {bdrv_drained_begin (bs); } -block_job_defer_to_main_loop (&s->common, mirror_exit, data); +job_defer_to_main_loop (&s->common.job, mirror_exit, data); }static void mirror_complete (BlockJob *job,Error **errp)
block/stream.c +7 -7
@@ -58,16 +58,16 @@ typedef struct {int ret; } StreamCompleteData; -static void stream_complete (BlockJob *job,void *opaque) +static void stream_complete (Job *job,void *opaque) { -StreamBlockJob *s =container_of (job, StreamBlockJob, common); +StreamBlockJob *s =container_of (job, StreamBlockJob, common.job); +BlockJob *bjob = &s->common;StreamCompleteData *data = opaque; -BlockDriverState *bs =blk_bs (job->blk); +BlockDriverState *bs =blk_bs (bjob->blk);BlockDriverState *base = s->base;Error *local_err =NULL ; -if (!job_is_cancelled (&s->common.job) && bs->backing && - data->ret ==0 ) { +if (!job_is_cancelled (job) && bs->backing && data->ret ==0 ) {const char *base_id =NULL , *base_fmt =NULL ;if (base) { base_id = s->backing_file_str;
@@ -88,7 +88,7 @@ out:/* Reopen the image back in read-only mode if necessary */ if (s->bs_flags !=bdrv_get_flags (bs)) {/* Give up write permissions before making it read-only */ -blk_set_perm (job->blk,0 ,BLK_PERM_ALL , &error_abort); +blk_set_perm (bjob->blk,0 ,BLK_PERM_ALL , &error_abort);bdrv_reopen (bs, s->bs_flags,NULL ); }
@@ -205,7 +205,7 @@ out:/* Modify backing chain and close BDSes in main loop */ data =g_malloc (sizeof (*data)); data->ret = ret; -block_job_defer_to_main_loop (&s->common, stream_complete, data); +job_defer_to_main_loop (&s->common.job, stream_complete, data); }static const BlockJobDriver stream_job_driver = {
blockjob.c +5 -52
90 lines not on this page · raw message
job.c +33 -0
@@ -28,6 +28,7 @@#include "qapi/error.h" #include "qemu/job.h" #include "qemu/id.h" +#include "qemu/main-loop.h" #include "trace-root.h" static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs);
@@ -170,3 +171,35 @@ void job_unref(Job *job)g_free (job); } } + +typedef struct { +Job *job; +JobDeferToMainLoopFn *fn; +void *opaque; +}JobDeferToMainLoopData ; + +static void job_defer_to_main_loop_bh (void *opaque) +{ +JobDeferToMainLoopData *data = opaque; +Job *job = data->job; +AioContext *aio_context = job->aio_context; + +/* Prevent race with job_defer_to_main_loop() */ +aio_context_acquire (aio_context); + data->fn (data->job, data->opaque); +aio_context_release (aio_context); + +g_free (data); +} + +void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque) +{ +JobDeferToMainLoopData *data =g_malloc (sizeof (*data)); + data->job = job; + data->fn = fn; + data->opaque = opaque; + job->deferred_to_main_loop = true; + +aio_bh_schedule_oneshot (qemu_get_aio_context (), + job_defer_to_main_loop_bh, data); +}
tests/test-bdrv-drain.c +4 -3
@@ -496,9 +496,10 @@ typedef struct TestBlockJob {bool should_complete; } TestBlockJob; -static void test_job_completed (BlockJob *job,void *opaque) +static void test_job_completed (Job *job,void *opaque) { -block_job_completed (job,0 ); +BlockJob *bjob =container_of (job, BlockJob, job); +block_job_completed (bjob,0 ); } static void coroutine_fn test_job_start(void *opaque)
@@ -510,7 +511,7 @@ static void coroutine_fn test_job_start(void *opaque)block_job_sleep_ns (&s->common,100000 ); } -block_job_defer_to_main_loop (&s->common, test_job_completed,NULL ); +job_defer_to_main_loop (&s->common.job, test_job_completed,NULL ); }static void test_job_complete (BlockJob *job,Error **errp)
tests/test-blockjob-txn.c +7 -6
@@ -24,16 +24,17 @@ typedef struct {int *result; } TestBlockJob; -static void test_block_job_complete (BlockJob *job,void *opaque) +static void test_block_job_complete (Job *job,void *opaque) { -BlockDriverState *bs =blk_bs (job->blk); +BlockJob *bjob =container_of (job, BlockJob, job); +BlockDriverState *bs =blk_bs (bjob->blk);int rc = (intptr_t )opaque; -if (job_is_cancelled (&job->job)) { +if (job_is_cancelled (job)) { rc = -ECANCELED ; } -block_job_completed (job, rc); +block_job_completed (bjob, rc);bdrv_unref (bs); }
@@ -54,8 +55,8 @@ static void coroutine_fn test_block_job_run(void *opaque) } } -block_job_defer_to_main_loop (job, test_block_job_complete, - (void *)(intptr_t )s->rc); +job_defer_to_main_loop (&job->job, test_block_job_complete, + (void *)(intptr_t )s->rc); } typedef struct {
tests/test-blockjob.c +4 -3
@@ -161,11 +161,12 @@ typedef struct CancelJob {bool completed; } CancelJob; -static void cancel_job_completed (BlockJob *job,void *opaque) +static void cancel_job_completed (Job *job,void *opaque) { +BlockJob *bjob =container_of (job, BlockJob, job);CancelJob *s = opaque; s->completed = true; -block_job_completed (job,0 ); +block_job_completed (bjob,0 ); }static void cancel_job_complete (BlockJob *job,Error **errp)
@@ -191,7 +192,7 @@ static void coroutine_fn cancel_job_start(void *opaque) } defer: -block_job_defer_to_main_loop (&s->common, cancel_job_completed, s); +job_defer_to_main_loop (&s->common.job, cancel_job_completed, s); }static const BlockJobDriver test_cancel_driver = {
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
Quoted 15 lines · Kevin Wolf
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 5 ---- include/block/blockjob_int.h | 19 --------------- include/qemu/job.h | 20 ++++++++++++++++ block/backup.c | 7 +++--- block/commit.c | 11 +++++---- block/mirror.c | 15 ++++++------ block/stream.c | 14 +++++------ blockjob.c | 57 ++++---------------------------------------- job.c | 33 +++++++++++++++++++++++++ tests/test-bdrv-drain.c | 7 +++--- tests/test-blockjob-txn.c | 13 +++++----- tests/test-blockjob.c | 7 +++--- 12 files changed, 98 insertions(+), 110 deletions(-)
[...]
block/commit.c typedef struct { · line 77
Quoted 13 lines
block/commit.c +2
-2@@ -72,9 +72,10 @@ typedef struct {int ret; } CommitCompleteData; -static void commit_complete (BlockJob *job,void *opaque) +static void commit_complete (Job *job,void *opaque) { -CommitBlockJob *s =container_of (job, CommitBlockJob, common); +CommitBlockJob *s =container_of (job, CommitBlockJob, common.job);
Now this is just abuse.
...but it's not the first time someone packs two container_of() into one, it appears. So, whatever, I guess.
Quoted 4 lines
block/commit.c +1
-0+BlockJob *bjob = &s->common;CommitCompleteData *data = opaque;BlockDriverState *top =blk_bs (s->top);BlockDriverState *base =blk_bs (s->base);
[...]
Quoted 4 lines
blockjob.c +0
-0
[...]
Quoted 72 lines
blockjob.c +0
-47@@ -1159,50 +1159,3 @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, }return action; } - -typedef struct { -BlockJob *job; -AioContext *aio_context; -BlockJobDeferToMainLoopFn *fn; -void *opaque; -}BlockJobDeferToMainLoopData ; - -static void block_job_defer_to_main_loop_bh (void *opaque) -{ -BlockJobDeferToMainLoopData *data = opaque; -AioContext *aio_context; - -/* Prevent race with block_job_defer_to_main_loop() */ -aio_context_acquire (data->aio_context); - -/* Fetch BDS AioContext again, in case it has changed */ - aio_context =blk_get_aio_context (data->job->blk); -if (aio_context != data->aio_context) { -aio_context_acquire (aio_context); - } - - data->fn (data->job, data->opaque); - -if (aio_context != data->aio_context) { -aio_context_release (aio_context); - } - -aio_context_release (data->aio_context); - -g_free (data); -} - -void block_job_defer_to_main_loop (BlockJob *job, -BlockJobDeferToMainLoopFn *fn, -void *opaque) -{ -BlockJobDeferToMainLoopData *data =g_malloc (sizeof (*data)); - data->job = job; - data->aio_context =blk_get_aio_context (job->blk); - data->fn = fn; - data->opaque = opaque; - job->deferred_to_main_loop = true; - -aio_bh_schedule_oneshot (qemu_get_aio_context (), - block_job_defer_to_main_loop_bh, data); -}job.c +6
-0@@ -28,6 +28,7 @@#include "qapi/error.h" #include "qemu/job.h" #include "qemu/id.h" +#include "qemu/main-loop.h" #include "trace-root.h" static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs);@@ -170,3 +171,35 @@ void job_unref(Job *job)g_free (job); } } + +typedef struct { + Job *job; + JobDeferToMainLoopFn *fn; + void *opaque;
job.c void job_unref(Job *job) · line 188
Quoted 10 lines
job.c +10
-0+} JobDeferToMainLoopData; + +static void job_defer_to_main_loop_bh (void *opaque) +{ +JobDeferToMainLoopData *data = opaque; +Job *job = data->job; +AioContext *aio_context = job->aio_context; + +/* Prevent race with job_defer_to_main_loop() */ +aio_context_acquire (aio_context);
I don't have a good feeling about this. The original code had this comment above an aio_context_acquire() for a context that might decidedly not have anything to do with the BB's context; block_job_defer_to_main_loop()'s description was that it would acquire the latter, so why did it acquire the former at all?
We wouldn't need this comment here at all, because acquiring this AioContext is part of the interface. That's why I don't have a good feeling.
The best explanation I can come up with is that the original code acquired the AioContext both of the block device at the time of the BH (because that needs to be done), and at the time of block_job_defer_to_main_loop() -- because the latter is probably the context the block_job_defer_to_main_loop() call came from, so it should be (b)locked.
But if that's the case, then the same should be done here. The context of the job may change between scheduling the BH and the BH being executed, so we might lock a different context here than the one job_defer_to_main_loop() ran in (i.e., job->aio_context at the time of job_defer_to_main_loop() running). And maybe we should lock that old context, too -- just like block_job_defer_to_main_loop_bh() did.
Max
Quoted 17 lines
job.c +17
-0+ data->fn (data->job, data->opaque); +aio_context_release (aio_context); + +g_free (data); +} + +void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque) +{ +JobDeferToMainLoopData *data =g_malloc (sizeof (*data)); + data->job = job; + data->fn = fn; + data->opaque = opaque; + job->deferred_to_main_loop = true; + +aio_bh_schedule_oneshot (qemu_get_aio_context (), + job_defer_to_main_loop_bh, data); +}
Re: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
Am 14.05.2018 um 17:52 hat Max Reitz geschrieben:
Quoted 35 lines · Kevin Wolf
Quoted 15 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 5 ---- include/block/blockjob_int.h | 19 --------------- include/qemu/job.h | 20 ++++++++++++++++ block/backup.c | 7 +++--- block/commit.c | 11 +++++---- block/mirror.c | 15 ++++++------ block/stream.c | 14 +++++------ blockjob.c | 57 ++++---------------------------------------- job.c | 33 +++++++++++++++++++++++++ tests/test-bdrv-drain.c | 7 +++--- tests/test-blockjob-txn.c | 13 +++++----- tests/test-blockjob.c | 7 +++--- 12 files changed, 98 insertions(+), 110 deletions(-)[...]
block/commit.c typedef struct { · line 77
Quoted 13 lines
block/commit.c +2
-2@@ -72,9 +72,10 @@ typedef struct {int ret; } CommitCompleteData; -static void commit_complete (BlockJob *job,void *opaque) +static void commit_complete (Job *job,void *opaque) { -CommitBlockJob *s =container_of (job, CommitBlockJob, common); +CommitBlockJob *s =container_of (job, CommitBlockJob, common.job);Now this is just abuse.
Quoted 2 lines
...but it's not the first time someone packs two
container_of()into one, it appears. So, whatever, I guess.
I don't think it's abuse. Why wouldn't I directly cast to the type that I really want instead of casting to each of the uninteresting parent classes, too?
Quoted 26 lines
Quoted 9 lines
block/commit.c +5
-0@@ -170,3 +171,35 @@ void job_unref(Job *job)g_free (job); } } + +typedef struct { + Job *job; + JobDeferToMainLoopFn *fn; + void *opaque;block/commit.c void job_unref(Job *job) · line 188
Quoted 10 lines
block/commit.c +10
-0+} JobDeferToMainLoopData; + +static void job_defer_to_main_loop_bh (void *opaque) +{ +JobDeferToMainLoopData *data = opaque; +Job *job = data->job; +AioContext *aio_context = job->aio_context; + +/* Prevent race with job_defer_to_main_loop() */ +aio_context_acquire (aio_context);I don't have a good feeling about this. The original code had this comment above an
aio_context_acquire()for a context that might decidedly not have anything to do with the BB's context; block_job_defer_to_main_loop()'s description was that it would acquire the latter, so why did it acquire the former at all?
Quoted 3 lines
We wouldn't need this comment here at all, because acquiring this AioContext is part of the interface. That's why I don't have a good feeling.
To be honest, I don't fully understand what the old code was trying to do or what race it was talking about, because I don't see any potential race with job_defer_to_main_loop() (neither in the old nor in the new code).
My suspicion is that Stefan solved the race that you reported [1] (and which was not with job_defer_to_main_loop(), but with random code that runs between that and the BH) just by adding some more code that made the scenario safe, but missed that this also made the existing code redundant. In other words, I think taking data->aio_context didn't serve a purpose even in the old code.
The only thing it could possibly protect is the access of data->job->bs, but that's not something that changes between job_defer_to_main_loop() and the execution of the BH.
[1] https://lists.gnu.org/archive/html/qemu-devel/2014-10/msg00260.html
Quoted 7 lines
The best explanation I can come up with is that the original code acquired the AioContext both of the block device at the time of the BH (because that needs to be done), and at the time of
block_job_defer_to_main_loop()-- because the latter is probably the context theblock_job_defer_to_main_loop()call came from, so it should be (b)locked.
Quoted 6 lines
But if that's the case, then the same should be done here. The context of the job may change between scheduling the BH and the BH being executed, so we might lock a different context here than the one
job_defer_to_main_loop()ran in (i.e.,job->aio_contextat the time ofjob_defer_to_main_loop()running). And maybe we should lock that old context, too -- just likeblock_job_defer_to_main_loop_bh()did.
Why should we lock the old context? We don't access anything protected by it. Even the data->job->bs access has gone away because we now have job->aio_context.
Kevin
Max Reitz
Reviewed-byRe: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
Quoted 36 lines · Kevin Wolf
Am 14.05.2018 um 17:52 hat Max Reitz geschrieben:
Quoted 35 lines · Kevin Wolf
Quoted 15 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 5 ---- include/block/blockjob_int.h | 19 --------------- include/qemu/job.h | 20 ++++++++++++++++ block/backup.c | 7 +++--- block/commit.c | 11 +++++---- block/mirror.c | 15 ++++++------ block/stream.c | 14 +++++------ blockjob.c | 57 ++++---------------------------------------- job.c | 33 +++++++++++++++++++++++++ tests/test-bdrv-drain.c | 7 +++--- tests/test-blockjob-txn.c | 13 +++++----- tests/test-blockjob.c | 7 +++--- 12 files changed, 98 insertions(+), 110 deletions(-)[...]
block/commit.c typedef struct { · line 77
Quoted 13 lines
block/commit.c +2
-2@@ -72,9 +72,10 @@ typedef struct {int ret; } CommitCompleteData; -static void commit_complete (BlockJob *job,void *opaque) +static void commit_complete (Job *job,void *opaque) { -CommitBlockJob *s =container_of (job, CommitBlockJob, common); +CommitBlockJob *s =container_of (job, CommitBlockJob, common.job);Now this is just abuse.
Quoted 6 lines
Quoted 2 lines
...but it's not the first time someone packs two
container_of()into one, it appears. So, whatever, I guess.I don't think it's abuse. Why wouldn't I directly cast to the type that I really want instead of casting to each of the uninteresting parent classes, too?
Because the final parameter is called "member" and not "path". :-)
Quoted 35 lines
Quoted 26 lines
Quoted 9 lines
block/commit.c +5
-0@@ -170,3 +171,35 @@ void job_unref(Job *job)g_free (job); } } + +typedef struct { + Job *job; + JobDeferToMainLoopFn *fn; + void *opaque;block/commit.c void job_unref(Job *job) · line 188
Quoted 10 lines
block/commit.c +10
-0+} JobDeferToMainLoopData; + +static void job_defer_to_main_loop_bh (void *opaque) +{ +JobDeferToMainLoopData *data = opaque; +Job *job = data->job; +AioContext *aio_context = job->aio_context; + +/* Prevent race with job_defer_to_main_loop() */ +aio_context_acquire (aio_context);I don't have a good feeling about this. The original code had this comment above an
aio_context_acquire()for a context that might decidedly not have anything to do with the BB's context; block_job_defer_to_main_loop()'s description was that it would acquire the latter, so why did it acquire the former at all?Quoted 3 lines
We wouldn't need this comment here at all, because acquiring this AioContext is part of the interface. That's why I don't have a good feeling.
To be honest, I don't fully understand what the old code was trying to do or what race it was talking about, because I don't see any potential race with
job_defer_to_main_loop()(neither in the old nor in the new code).
Quoted 6 lines
My suspicion is that Stefan solved the race that you reported [1] (and which was not with
job_defer_to_main_loop(), but with random code that runs between that and the BH) just by adding some more code that made the scenario safe, but missed that this also made the existing code redundant. In other words, I think takingdata->aio_contextdidn't serve a purpose even in the old code.
Possible, yes.
Also seems more likely than any of the explanations I tried to come up with.
Quoted 20 lines
The only thing it could possibly protect is the access of
data->job->bs, but that's not something that changes betweenjob_defer_to_main_loop()and the execution of the BH.[1] https://lists.gnu.org/archive/html/qemu-devel/2014-10/msg00260.html
Quoted 7 lines
The best explanation I can come up with is that the original code acquired the AioContext both of the block device at the time of the BH (because that needs to be done), and at the time of
block_job_defer_to_main_loop()-- because the latter is probably the context theblock_job_defer_to_main_loop()call came from, so it should be (b)locked.Quoted 6 lines
But if that's the case, then the same should be done here. The context of the job may change between scheduling the BH and the BH being executed, so we might lock a different context here than the one
job_defer_to_main_loop()ran in (i.e.,job->aio_contextat the time ofjob_defer_to_main_loop()running). And maybe we should lock that old context, too -- just likeblock_job_defer_to_main_loop_bh()did.
Quoted 3 lines
Why should we lock the old context? We don't access anything protected by it. Even the
data->job->bsaccess has gone away because we now havejob->aio_context.
Because the old code did so and I don't know why. O:-)
Your explanation does make sense to me, though, so:
Re: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
Quoted 8 lines · Max Reitz
Quoted 8 lines
Quoted 8 lines
block/commit.c typedef struct { · line 77
Quoted 5 lines
block/commit.c +2
-2-static void commit_complete (BlockJob *job,void *opaque) +static void commit_complete (Job *job,void *opaque) { -CommitBlockJob *s =container_of (job, CommitBlockJob, common); +CommitBlockJob *s =container_of (job, CommitBlockJob, common.job);Now this is just abuse.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
...but it's not the first time someone packs two
container_of()into one, it appears. So, whatever, I guess.I don't think it's abuse. Why wouldn't I directly cast to the type that I really want instead of casting to each of the uninteresting parent classes, too?
Because the final parameter is called "member" and not "path". :-)
container_of() is using offsetof(); and in C99 7.17, the parameter is named "member-designator" which can indeed jump through multiple layers (any valid address constant, as defined in 6.6P9). I don't see this as abuse of the interface.
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
The best explanation I can come up with is that the original code acquired the AioContext both of the block device at the time of the BH (because that needs to be done), and at the time of
block_job_defer_to_main_loop()-- because the latter is probably the context theblock_job_defer_to_main_loop()call came from, so it should be (b)locked.But if that's the case, then the same should be done here. The context of the job may change between scheduling the BH and the BH being executed, so we might lock a different context here than the one
job_defer_to_main_loop()ran in (i.e.,job->aio_contextat the time ofjob_defer_to_main_loop()running). And maybe we should lock that old context, too -- just likeblock_job_defer_to_main_loop_bh()did.
Quoted 7 lines
Quoted 3 lines
Why should we lock the old context? We don't access anything protected by it. Even the
data->job->bsaccess has gone away because we now havejob->aio_context.Because the old code did so and I don't know why. O:-)
Your explanation does make sense to me, though, so:
Then it's best to include that explanation in the commit message itself, to save the future reader the hassle of digging up this thread.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
In reply to Kevin WolfQuoted 1 line · Kevin Wolf
Signed-off-by Kevin Wolf <[email protected]>
Hmm, this one is a bit more than just code motion due to the way the aio_context acquisition has changed. I think at a minimum a good commit message is warranted.
Quoted 388 lines
--- include/block/blockjob.h | 5 ---- include/block/blockjob_int.h | 19 --------------- include/qemu/job.h | 20 ++++++++++++++++ block/backup.c | 7 +++--- block/commit.c | 11 +++++---- block/mirror.c | 15 ++++++------ block/stream.c | 14 +++++------ blockjob.c | 57 ++++---------------------------------------- job.c | 33 +++++++++++++++++++++++++ tests/test-bdrv-drain.c | 7 +++--- tests/test-blockjob-txn.c | 13 +++++----- tests/test-blockjob.c | 7 +++--- 12 files changed, 98 insertions(+), 110 deletions(-)include/block/blockjob.h +0
-5@@ -92,11 +92,6 @@ typedef struct BlockJob {*/ bool ready; -/** -* Set to true when the job has deferred work to the main loop. -*/ -bool deferred_to_main_loop; -/** Status that is published by the query-block-jobs QMP API */ BlockDeviceIoStatus iostatus;include/block/blockjob_int.h +0
-19@@ -233,23 +233,4 @@ void block_job_event_ready(BlockJob *job);BlockErrorAction block_job_error_action (BlockJob *job,BlockdevOnError on_err,int is_read,int error); -typedef void BlockJobDeferToMainLoopFn (BlockJob *job,void *opaque); - -/** -* block_job_defer_to_main_loop: -* @job: The job -* @fn: The function to run in the main loop -* @opaque: The opaque value that is passed to @fn -* -* This function must be called by the main job coroutine just before it -* returns. @fn is executed in the main loop with the BlockDriverState -* AioContext acquired. Block jobs must call bdrv_unref(), bdrv_close(), and -* anything that uses bdrv_drain_all() in the main loop. -* -* The @job AioContext is held while @fn executes. -*/ -void block_job_defer_to_main_loop (BlockJob *job, -BlockJobDeferToMainLoopFn *fn, -void *opaque); -#endif include/qemu/job.h +20
-0@@ -58,6 +58,9 @@ typedef struct Job {*/ bool cancelled; +/** Set to true when the job has deferred work to the main loop. */ +bool deferred_to_main_loop; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;@@ -131,6 +134,23 @@ Job *job_get(const char *id);*/ int job_apply_verb (Job *job,JobVerb bv,Error **errp); +typedef void JobDeferToMainLoopFn (Job *job,void *opaque); + +/** +* @job: The job +* @fn: The function to run in the main loop +* @opaque: The opaque value that is passed to @fn +* +* This function must be called by the main job coroutine just before it +* returns. @fn is executed in the main loop with the job AioContext acquired. +* +* Block jobs must call bdrv_unref(), bdrv_close(), and anything that uses +* bdrv_drain_all() in the main loop. +* +* The @job AioContext is held while @fn executes. +*/ +void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque); +/* TODO To be removed from the public interface */ void job_state_transition (Job *job,JobStatus s1);block/backup.c +4
-3@@ -317,11 +317,12 @@ typedef struct {int ret; } BackupCompleteData; -static void backup_complete (BlockJob *job,void *opaque) +static void backup_complete (Job *job,void *opaque) { +BlockJob *bjob =container_of (job, BlockJob, job);BackupCompleteData *data = opaque; -block_job_completed (job, data->ret); +block_job_completed (bjob, data->ret);g_free (data); }@@ -519,7 +520,7 @@ static void coroutine_fn backup_run(void *opaque) data =g_malloc (sizeof (*data)); data->ret = ret; -block_job_defer_to_main_loop (&job->common, backup_complete, data); +job_defer_to_main_loop (&job->common.job, backup_complete, data); }static const BlockJobDriver backup_job_driver = {block/commit.c +6
-5@@ -72,9 +72,10 @@ typedef struct {int ret; } CommitCompleteData; -static void commit_complete (BlockJob *job,void *opaque) +static void commit_complete (Job *job,void *opaque) { -CommitBlockJob *s =container_of (job, CommitBlockJob, common); +CommitBlockJob *s =container_of (job, CommitBlockJob, common.job); +BlockJob *bjob = &s->common;CommitCompleteData *data = opaque;BlockDriverState *top =blk_bs (s->top);BlockDriverState *base =blk_bs (s->base);@@ -90,7 +91,7 @@ static void commit_complete(BlockJob *job, void *opaque)* the normal backing chain can be restored. */ blk_unref (s->base); -if (!job_is_cancelled (&s->common.job) && ret ==0 ) { +if (!job_is_cancelled (job) && ret ==0 ) {/* success */ ret =bdrv_drop_intermediate (s->commit_top_bs, base, s->backing_file_str);@@ -114,7 +115,7 @@ static void commit_complete(BlockJob *job, void *opaque)* block_job_finish_sync()), block_job_completed() won't free it and * therefore the blockers on the intermediate nodes remain. This would * cause bdrv_set_backing_hd() to fail. */ -block_job_remove_all_bdrv (job); +block_job_remove_all_bdrv (bjob);block_job_completed (&s->common, ret);g_free (data);@@ -211,7 +212,7 @@ out: data =g_malloc (sizeof (*data)); data->ret = ret; -block_job_defer_to_main_loop (&s->common, commit_complete, data); +job_defer_to_main_loop (&s->common.job, commit_complete, data); }static const BlockJobDriver commit_job_driver = {block/mirror.c +8
-7@@ -484,9 +484,10 @@ typedef struct {int ret; } MirrorExitData; -static void mirror_exit (BlockJob *job,void *opaque) +static void mirror_exit (Job *job,void *opaque) { -MirrorBlockJob *s =container_of (job, MirrorBlockJob, common); +MirrorBlockJob *s =container_of (job, MirrorBlockJob, common.job); +BlockJob *bjob = &s->common;MirrorExitData *data = opaque;AioContext *replace_aio_context =NULL ;BlockDriverState *src = s->source;@@ -568,7 +569,7 @@ static void mirror_exit(BlockJob *job, void *opaque)* the blockers on the intermediate nodes so that the resulting state is * valid. Also give up permissions on mirror_top_bs->backing, which might * block the removal. */ -block_job_remove_all_bdrv (job); +block_job_remove_all_bdrv (bjob);bdrv_child_try_set_perm (mirror_top_bs->backing,0 ,BLK_PERM_ALL , &error_abort);bdrv_replace_node (mirror_top_bs,backing_bs (mirror_top_bs), &error_abort);@@ -576,9 +577,9 @@ static void mirror_exit(BlockJob *job, void *opaque)/* We just changed the BDS the job BB refers to (with either or both of the * bdrv_replace_node() calls), so switch the BB back so the cleanup does * the right thing. We don't need any permissions any more now. */ -blk_remove_bs (job->blk); -blk_set_perm (job->blk,0 ,BLK_PERM_ALL , &error_abort); -blk_insert_bs (job->blk, mirror_top_bs, &error_abort); +blk_remove_bs (bjob->blk); +blk_set_perm (bjob->blk,0 ,BLK_PERM_ALL , &error_abort); +blk_insert_bs (bjob->blk, mirror_top_bs, &error_abort);block_job_completed (&s->common, data->ret);@@ -901,7 +902,7 @@ immediate_exit:if (need_drain) {bdrv_drained_begin (bs); } -block_job_defer_to_main_loop (&s->common, mirror_exit, data); +job_defer_to_main_loop (&s->common.job, mirror_exit, data); }static void mirror_complete (BlockJob *job,Error **errp)block/stream.c +7
-7@@ -58,16 +58,16 @@ typedef struct {int ret; } StreamCompleteData; -static void stream_complete (BlockJob *job,void *opaque) +static void stream_complete (Job *job,void *opaque) { -StreamBlockJob *s =container_of (job, StreamBlockJob, common); +StreamBlockJob *s =container_of (job, StreamBlockJob, common.job); +BlockJob *bjob = &s->common;StreamCompleteData *data = opaque; -BlockDriverState *bs =blk_bs (job->blk); +BlockDriverState *bs =blk_bs (bjob->blk);BlockDriverState *base = s->base;Error *local_err =NULL ; -if (!job_is_cancelled (&s->common.job) && bs->backing && - data->ret ==0 ) { +if (!job_is_cancelled (job) && bs->backing && data->ret ==0 ) {const char *base_id =NULL , *base_fmt =NULL ;if (base) { base_id = s->backing_file_str;@@ -88,7 +88,7 @@ out:/* Reopen the image back in read-only mode if necessary */ if (s->bs_flags !=bdrv_get_flags (bs)) {/* Give up write permissions before making it read-only */ -blk_set_perm (job->blk,0 ,BLK_PERM_ALL , &error_abort); +blk_set_perm (bjob->blk,0 ,BLK_PERM_ALL , &error_abort);bdrv_reopen (bs, s->bs_flags,NULL ); }@@ -205,7 +205,7 @@ out:/* Modify backing chain and close BDSes in main loop */ data =g_malloc (sizeof (*data)); data->ret = ret; -block_job_defer_to_main_loop (&s->common, stream_complete, data); +job_defer_to_main_loop (&s->common.job, stream_complete, data); }static const BlockJobDriver stream_job_driver = {blockjob.c +5
-52@@ -347,7 +347,7 @@ static void block_job_decommission(BlockJob *job) job->completed = true; job->busy = false; job->paused = false; - job->deferred_to_main_loop = true; + job->job.deferred_to_main_loop = true;block_job_txn_del_job (job);job_state_transition (&job->job,JOB_STATUS_NULL );job_unref (&job->job);@@ -502,7 +502,7 @@ static int block_job_finish_sync(BlockJob *job,/* block_job_drain calls block_job_enter, and it should be enough to * induce progress until the job completes or moves to the main thread. */ -while (!job->deferred_to_main_loop && !job->completed) { +while (!job->job.deferred_to_main_loop && !job->completed) {block_job_drain (job); }while (!job->completed) {@@ -722,7 +722,7 @@ void block_job_cancel(BlockJob *job, bool force)block_job_cancel_async (job, force);if (!block_job_started (job)) {block_job_completed (job, -ECANCELED ); - }else if (job->deferred_to_main_loop) { + }else if (job->job.deferred_to_main_loop) {block_job_completed_txn_abort (job); }else {block_job_enter (job);@@ -1038,7 +1038,7 @@ static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job))if (!block_job_started (job)) {return ; } -if (job->deferred_to_main_loop) { +if (job->job.deferred_to_main_loop) {return ; }@@ -1053,7 +1053,7 @@ static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job))return ; } -assert (!job->deferred_to_main_loop); +assert (!job->job.deferred_to_main_loop);timer_del (&job->sleep_timer); job->busy = true;block_job_unlock ();@@ -1159,50 +1159,3 @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, }return action; } - -typedef struct { -BlockJob *job; -AioContext *aio_context; -BlockJobDeferToMainLoopFn *fn; -void *opaque; -}BlockJobDeferToMainLoopData ; - -static void block_job_defer_to_main_loop_bh (void *opaque) -{ -BlockJobDeferToMainLoopData *data = opaque; -AioContext *aio_context; - -/* Prevent race with block_job_defer_to_main_loop() */ -aio_context_acquire (data->aio_context); - -/* Fetch BDS AioContext again, in case it has changed */ - aio_context =blk_get_aio_context (data->job->blk); -if (aio_context != data->aio_context) { -aio_context_acquire (aio_context); - } - - data->fn (data->job, data->opaque); - -if (aio_context != data->aio_context) { -aio_context_release (aio_context); - } - -aio_context_release (data->aio_context); - -g_free (data); -} - -void block_job_defer_to_main_loop (BlockJob *job, -BlockJobDeferToMainLoopFn *fn, -void *opaque) -{ -BlockJobDeferToMainLoopData *data =g_malloc (sizeof (*data)); - data->job = job; - data->aio_context =blk_get_aio_context (job->blk); - data->fn = fn; - data->opaque = opaque; - job->deferred_to_main_loop = true; - -aio_bh_schedule_oneshot (qemu_get_aio_context (), - block_job_defer_to_main_loop_bh, data); -}job.c +12
-0@@ -28,6 +28,7 @@#include "qapi/error.h" #include "qemu/job.h" #include "qemu/id.h" +#include "qemu/main-loop.h" #include "trace-root.h" static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs);@@ -170,3 +171,35 @@ void job_unref(Job *job)g_free (job); } } + +typedef struct { +Job *job; +JobDeferToMainLoopFn *fn; +void *opaque; +}JobDeferToMainLoopData ; + +static void job_defer_to_main_loop_bh (void *opaque) +{ +JobDeferToMainLoopData *data = opaque; +Job *job = data->job;
job.c void job_unref(Job *job) · line 194
Quoted 10 lines
job.c +10
-0+AioContext *aio_context = job->aio_context; + +/* Prevent race with job_defer_to_main_loop() */ +aio_context_acquire (aio_context); + data->fn (data->job, data->opaque); +aio_context_release (aio_context); + +g_free (data); +} +
This function showed up in '14, with dec7d42 from Stefan. His first draft looked like Kevin's, until:
https://lists.gnu.org/archive/html/qemu-devel/2014-10/msg00111.html
Max, from 2014:
"I'm not so sure whether it'd be possible to change the BDS's AIO context (in another thread) after the call to bdrv_get_aio_context() in block_job_defer_to_main_loop() and before block_job_defer_to_main_loop_bh() is run. If so, this might create race conditions."
Err, I dunno either.
Quoted 103 lines
job.c +11
-0+void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque) +{ +JobDeferToMainLoopData *data =g_malloc (sizeof (*data)); + data->job = job; + data->fn = fn; + data->opaque = opaque; + job->deferred_to_main_loop = true; + +aio_bh_schedule_oneshot (qemu_get_aio_context (), + job_defer_to_main_loop_bh, data); +}tests/test-bdrv-drain.c +4
-3@@ -496,9 +496,10 @@ typedef struct TestBlockJob {bool should_complete; } TestBlockJob; -static void test_job_completed (BlockJob *job,void *opaque) +static void test_job_completed (Job *job,void *opaque) { -block_job_completed (job,0 ); +BlockJob *bjob =container_of (job, BlockJob, job); +block_job_completed (bjob,0 ); } static void coroutine_fn test_job_start(void *opaque)@@ -510,7 +511,7 @@ static void coroutine_fn test_job_start(void *opaque)block_job_sleep_ns (&s->common,100000 ); } -block_job_defer_to_main_loop (&s->common, test_job_completed,NULL ); +job_defer_to_main_loop (&s->common.job, test_job_completed,NULL ); }static void test_job_complete (BlockJob *job,Error **errp)tests/test-blockjob-txn.c +7
-6@@ -24,16 +24,17 @@ typedef struct {int *result; } TestBlockJob; -static void test_block_job_complete (BlockJob *job,void *opaque) +static void test_block_job_complete (Job *job,void *opaque) { -BlockDriverState *bs =blk_bs (job->blk); +BlockJob *bjob =container_of (job, BlockJob, job); +BlockDriverState *bs =blk_bs (bjob->blk);int rc = (intptr_t )opaque; -if (job_is_cancelled (&job->job)) { +if (job_is_cancelled (job)) { rc = -ECANCELED ; } -block_job_completed (job, rc); +block_job_completed (bjob, rc);bdrv_unref (bs); }@@ -54,8 +55,8 @@ static void coroutine_fn test_block_job_run(void *opaque) } } -block_job_defer_to_main_loop (job, test_block_job_complete, - (void *)(intptr_t )s->rc); +job_defer_to_main_loop (&job->job, test_block_job_complete, + (void *)(intptr_t )s->rc); } typedef struct {tests/test-blockjob.c +4
-3@@ -161,11 +161,12 @@ typedef struct CancelJob {bool completed; } CancelJob; -static void cancel_job_completed (BlockJob *job,void *opaque) +static void cancel_job_completed (Job *job,void *opaque) { +BlockJob *bjob =container_of (job, BlockJob, job);CancelJob *s = opaque; s->completed = true; -block_job_completed (job,0 ); +block_job_completed (bjob,0 ); }static void cancel_job_complete (BlockJob *job,Error **errp)@@ -191,7 +192,7 @@ static void coroutine_fn cancel_job_start(void *opaque) } defer: -block_job_defer_to_main_loop (&s->common, cancel_job_completed, s); +job_defer_to_main_loop (&s->common.job, cancel_job_completed, s); }static const BlockJobDriver test_cancel_driver = {
Re: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
Am 15.05.2018 um 00:33 hat John Snow geschrieben:
Quoted 8 lines · Kevin Wolf
Quoted 1 line
Signed-off-by Kevin Wolf <[email protected]>Hmm, this one is a bit more than just code motion due to the way the
aio_contextacquisition has changed. I think at a minimum a good commit message is warranted.
I know it took a while to convince myself that it's right. I wonder where the commit message has gone...
Quoted 33 lines
Quoted 15 lines
job.c +11
-0@@ -170,3 +171,35 @@ void job_unref(Job *job)g_free (job); } } + +typedef struct { +Job *job; +JobDeferToMainLoopFn *fn; +void *opaque; +}JobDeferToMainLoopData ; + +static void job_defer_to_main_loop_bh (void *opaque) +{ +JobDeferToMainLoopData *data = opaque; +Job *job = data->job;job.c void job_unref(Job *job) · line 194
Quoted 10 lines
job.c +10
-0+AioContext *aio_context = job->aio_context; + +/* Prevent race with job_defer_to_main_loop() */ +aio_context_acquire (aio_context); + data->fn (data->job, data->opaque); +aio_context_release (aio_context); + +g_free (data); +} +This function showed up in '14, with dec7d42 from Stefan. His first draft looked like Kevin's, until:
https://lists.gnu.org/archive/html/qemu-devel/2014-10/msg00111.html
Max, from 2014:
Quoted 7 lines
"I'm not so sure whether it'd be possible to change the BDS's AIO context (in another thread) after the call to
bdrv_get_aio_context()inblock_job_defer_to_main_loop()and beforeblock_job_defer_to_main_loop_bh()is run. If so, this might create race conditions."Err, I dunno either.
This one at least is not a problem in the new code any more because we have job->aio_context now, which is updated when the BDS's context changes. We read job->aio_context only in the BH, so we always get the current one.
In contrast, the old code put it into BlockJobDeferToMainLoopData at the time when the BH was scheduled, so it could be outdated when the BH actually ran.
Kevin
Re: [Qemu-devel] [PATCH 17/42] job: Move defer_to_main_loop to Job
In reply to Kevin WolfQuoted 2 lines · Kevin Wolf
Signed-off-by Kevin Wolf <[email protected]>---
Rather sparse on the commit message, given the other comments in this thread.
Quoted 17 lines
+++
b/include/qemu/job.hinclude/qemu/job.h +5
-0@@ -58,6 +58,9 @@ typedef struct Job {*/ bool cancelled; +/** Set to true when the job has deferred work to the main loop. */ +bool deferred_to_main_loop; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;@@ -131,6 +134,23 @@ Job *job_get(const char *id);*/ int job_apply_verb (Job *job,JobVerb bv,Error **errp); +typedef void JobDeferToMainLoopFn (Job *job,void *opaque); +
include/qemu/job.h Job *job_get(const char *id); · line 148
Quoted 10 lines
include/qemu/job.h +10
-0+/** +* @job: The job +* @fn: The function to run in the main loop +* @opaque: The opaque value that is passed to @fn +* +* This function must be called by the main job coroutine just before it +* returns. @fn is executed in the main loop with the job AioContext acquired. +* +* Block jobs must call bdrv_unref(), bdrv_close(), and anything that uses +* bdrv_drain_all() in the main loop.
Do we still want this block-job-specific comment in the main job.h header?
Quoted 5 lines
include/qemu/job.h +5
-0+* +* The @job AioContext is held while @fn executes. +*/ +void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque); +
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 18/42 job: Move coroutine and related code to Job
In reply to Kevin WolfThis commit moves some core functions for dealing with the job coroutine from BlockJob to Job. This includes primarily entering the coroutine (both for the first and reentering) and yielding explicitly and at pause points.
--- include/block/blockjob.h | 40 --------- include/block/blockjob_int.h | 26 ------ include/qemu/job.h | 76 ++++++++++++++++ block/backup.c | 2 +- block/commit.c | 4 +- block/mirror.c | 22 ++--- block/replication.c | 2 +- block/stream.c | 4 +- blockdev.c | 8 +- blockjob.c | 201 +++++++------------------------------------ job.c | 137 +++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 38 ++++---- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 14 +-- 14 files changed, 296 insertions(+), 290 deletions(-)
include/block/blockjob.h +0 -40
@@ -51,43 +51,18 @@ typedef struct BlockJob {BlockBackend *blk;/** -* The coroutine that executes the job. If not NULL, it is -* reentered when busy is false and the job is cancelled. -*/ -Coroutine *co; - -/** * Set to true if the job should abort immediately without waiting * for data to be in sync. */ bool force;/** -* Counter for pause request. If non-zero, the block job is either paused, -* or if busy == true will pause itself as soon as possible. -*/ -int pause_count; - -/** * Set to true if the job is paused by user. Can be unpaused with the * block-job-resume QMP command. */ bool user_paused;/** -* Set to false by the job while the coroutine has yielded and may be -* re-entered by block_job_enter(). There may still be I/O or event loop -* activity pending. Accessed under block_job_mutex (in blockjob.c). -*/ -bool busy; - -/** -* Set to true by the job while it is in a quiescent state, where -* no I/O or event loop activity is pending. -*/ -bool paused; - -/** * Set to true when the job is ready to be completed. */ bool ready;
@@ -125,12 +100,6 @@ typedef struct BlockJob {/** ret code passed to block_job_completed. */ int ret; -/** -* Timer that is used by @block_job_sleep_ns. Accessed under -* block_job_mutex (in blockjob.c). -*/ -QEMUTimer sleep_timer; -/** True if this job should automatically finalize itself */ bool auto_finalize;
@@ -208,15 +177,6 @@ void block_job_remove_all_bdrv(BlockJob *job);void block_job_set_speed (BlockJob *job,int64_t speed,Error **errp);/** -* block_job_start: -* @job: A job that has not yet been started. -* -* Begins execution of a block job. -* Takes ownership of one reference to the job object. -*/ -void block_job_start (BlockJob *job); - -/** * block_job_cancel: * @job: The job to be canceled. * @force: Quit a job without waiting for data to be in sync.
include/block/blockjob_int.h +0 -26
@@ -38,9 +38,6 @@ struct BlockJobDriver {/** Generic JobDriver callbacks and settings */ JobDriver job_driver; -/** Mandatory: Entrypoint for the Coroutine. */ -CoroutineEntry *start; -/** * Optional callback for job types whose completion must be triggered * manually.
@@ -85,20 +82,6 @@ struct BlockJobDriver {*/ void (*clean)(BlockJob *job); -/** -* If the callback is not NULL, it will be invoked when the job transitions -* into the paused state. Paused jobs must not perform any asynchronous -* I/O or event loop activity. This callback is used to quiesce jobs. -*/ - void coroutine_fn (*pause)(BlockJob *job); - -/** -* If the callback is not NULL, it will be invoked when the job transitions -* out of the paused state. Any asynchronous I/O or event loop activity -* should be restarted from this callback. -*/ - void coroutine_fn (*resume)(BlockJob *job); -/* * If the callback is not NULL, it will be invoked before the job is * resumed in a new AioContext. This is the place to move any resources
@@ -196,15 +179,6 @@ void block_job_early_fail(BlockJob *job);void block_job_completed (BlockJob *job,int ret);/** -* block_job_pause_point: -* @job: The job that is ready to pause. -* -* Pause now if block_job_pause() has been called. Block jobs that perform -* lots of I/O must call this between requests so that the job can be paused. -*/ -void coroutine_fn block_job_pause_point(BlockJob *job); - -/** * block_job_enter: * @job: The job to enter. *
include/qemu/job.h +76 -0
105 lines not on this page · raw message
block/backup.c +1 -1
@@ -528,8 +528,8 @@ static const BlockJobDriver backup_job_driver = { .instance_size = sizeof(BackupBlockJob), .job_type = JOB_TYPE_BACKUP, .free = block_job_free, + .start = backup_run, }, - .start = backup_run, .commit = backup_commit, .abort = backup_abort, .clean = backup_clean,
block/commit.c +2 -2
@@ -220,8 +220,8 @@ static const BlockJobDriver commit_job_driver = { .instance_size = sizeof(CommitBlockJob), .job_type = JOB_TYPE_COMMIT, .free = block_job_free, + .start = commit_run, }, - .start = commit_run, }; static int coroutine_fn bdrv_commit_top_preadv(BlockDriverState *bs,
@@ -371,7 +371,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, s->on_error = on_error;trace_commit_start (bs, base, top, s); -block_job_start (&s->common); +job_start (&s->common.job);return ; fail:
block/mirror.c +11 -11
@@ -126,7 +126,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret)g_free (op);if (s->waiting_for_io) { -qemu_coroutine_enter (s->common.co); +qemu_coroutine_enter (s->common.job.co); } }
@@ -345,7 +345,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)mirror_wait_for_io (s); } -block_job_pause_point (&s->common); +job_pause_point (&s->common.job);/* Find the number of consective dirty chunks following the first dirty * one, and wait for in flight requests in them. */
@@ -597,7 +597,7 @@ static void mirror_throttle(MirrorBlockJob *s) s->last_pause_ns = now;block_job_sleep_ns (&s->common,0 ); }else { -block_job_pause_point (&s->common); +job_pause_point (&s->common.job); } }
@@ -786,7 +786,7 @@ static void coroutine_fn mirror_run(void *opaque) goto immediate_exit; } -block_job_pause_point (&s->common); +job_pause_point (&s->common.job); cnt =bdrv_get_dirty_count (s->dirty_bitmap);/* cnt is the number of dirty bytes remaining and s->bytes_in_flight is
@@ -957,9 +957,9 @@ static void mirror_complete(BlockJob *job, Error **errp)block_job_enter (&s->common); } -static void mirror_pause (BlockJob *job) +static void mirror_pause (Job *job) { -MirrorBlockJob *s =container_of (job, MirrorBlockJob, common); +MirrorBlockJob *s =container_of (job, MirrorBlockJob, common.job);mirror_wait_for_all_io (s); }
@@ -991,10 +991,10 @@ static const BlockJobDriver mirror_job_driver = { .instance_size = sizeof(MirrorBlockJob), .job_type = JOB_TYPE_MIRROR, .free = block_job_free, + .start = mirror_run, + .pause = mirror_pause, }, - .start = mirror_run, .complete = mirror_complete, - .pause = mirror_pause, .attached_aio_context = mirror_attached_aio_context, .drain = mirror_drain, };
@@ -1004,10 +1004,10 @@ static const BlockJobDriver commit_active_job_driver = { .instance_size = sizeof(MirrorBlockJob), .job_type = JOB_TYPE_COMMIT, .free = block_job_free, + .start = mirror_run, + .pause = mirror_pause, }, - .start = mirror_run, .complete = mirror_complete, - .pause = mirror_pause, .attached_aio_context = mirror_attached_aio_context, .drain = mirror_drain, };
@@ -1242,7 +1242,7 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs, }trace_mirror_start (bs, s, opaque); -block_job_start (&s->common); +job_start (&s->common.job);return ; fail:
block/replication.c +1 -1
@@ -576,7 +576,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,aio_context_release (aio_context);return ; } -block_job_start (job); +job_start (&job->job);break ;default :aio_context_release (aio_context);
block/stream.c +2 -2
@@ -213,8 +213,8 @@ static const BlockJobDriver stream_job_driver = { .instance_size = sizeof(StreamBlockJob), .job_type = JOB_TYPE_STREAM, .free = block_job_free, + .start = stream_run, }, - .start = stream_run, }; void stream_start(const char *job_id, BlockDriverState *bs,
@@ -262,7 +262,7 @@ void stream_start(const char *job_id, BlockDriverState *bs, s->on_error = on_error;trace_stream_start (bs, base, s); -block_job_start (&s->common); +job_start (&s->common.job);return ; fail:
blockdev.c +4 -4
@@ -1910,7 +1910,7 @@ static void drive_backup_commit(BlkActionState *common)aio_context_acquire (aio_context);assert (state->job); -block_job_start (state->job); +job_start (&state->job->job);aio_context_release (aio_context); }
@@ -2008,7 +2008,7 @@ static void blockdev_backup_commit(BlkActionState *common)aio_context_acquire (aio_context);assert (state->job); -block_job_start (state->job); +job_start (&state->job->job);aio_context_release (aio_context); }
@@ -3425,7 +3425,7 @@ void qmp_drive_backup(DriveBackup *arg, Error **errp)BlockJob *job; job =do_drive_backup (arg,NULL , errp);if (job) { -block_job_start (job); +job_start (&job->job); } }
@@ -3513,7 +3513,7 @@ void qmp_blockdev_backup(BlockdevBackup *arg, Error **errp)BlockJob *job; job =do_blockdev_backup (arg,NULL , errp);if (job) { -block_job_start (job); +job_start (&job->job); } }
blockjob.c +30 -171
359 lines not on this page · raw message
job.c +137 -0
161 lines not on this page · raw message
tests/test-bdrv-drain.c +19 -19
@@ -524,8 +524,8 @@ BlockJobDriver test_job_driver = { .job_driver = { .instance_size = sizeof(TestBlockJob), .free = block_job_free, + .start = test_job_start, }, - .start = test_job_start, .complete = test_job_complete, };
@@ -549,47 +549,47 @@ static void test_blockjob_common(enum drain_type drain_type) job =block_job_create ("job0" , &test_job_driver,NULL , src,0 ,BLK_PERM_ALL ,0 ,0 ,NULL ,NULL , &error_abort);block_job_add_bdrv (job,"target" , target,0 ,BLK_PERM_ALL , &error_abort); -block_job_start (job); +job_start (&job->job); - g_assert_cmpint(job->pause_count, ==, 0); -g_assert_false (job->paused); -g_assert_false (job->busy);/* We're in block_job_sleep_ns() */ + g_assert_cmpint(job->job.pause_count, ==, 0); +g_assert_false (job->job.paused); +g_assert_false (job->job.busy);/* We're in block_job_sleep_ns() */ do_drain_begin (drain_type, src);if (drain_type ==BDRV_DRAIN_ALL ) {/* bdrv_drain_all() drains both src and target */ - g_assert_cmpint(job->pause_count, ==, 2); + g_assert_cmpint(job->job.pause_count, ==, 2); }else { - g_assert_cmpint(job->pause_count, ==, 1); + g_assert_cmpint(job->job.pause_count, ==, 1); }/* XXX We don't wait until the job is actually paused. Is this okay? */ -/* g_assert_true(job->paused); */ -g_assert_false (job->busy);/* The job is paused */ +/* g_assert_true(job->job.paused); */ +g_assert_false (job->job.busy);/* The job is paused */ do_drain_end (drain_type, src); - g_assert_cmpint(job->pause_count, ==, 0); -g_assert_false (job->paused); -g_assert_false (job->busy);/* We're in block_job_sleep_ns() */ + g_assert_cmpint(job->job.pause_count, ==, 0); +g_assert_false (job->job.paused); +g_assert_false (job->job.busy);/* We're in block_job_sleep_ns() */ do_drain_begin (drain_type, target);if (drain_type ==BDRV_DRAIN_ALL ) {/* bdrv_drain_all() drains both src and target */ - g_assert_cmpint(job->pause_count, ==, 2); + g_assert_cmpint(job->job.pause_count, ==, 2); }else { - g_assert_cmpint(job->pause_count, ==, 1); + g_assert_cmpint(job->job.pause_count, ==, 1); }/* XXX We don't wait until the job is actually paused. Is this okay? */ -/* g_assert_true(job->paused); */ -g_assert_false (job->busy);/* The job is paused */ +/* g_assert_true(job->job.paused); */ +g_assert_false (job->job.busy);/* The job is paused */ do_drain_end (drain_type, target); - g_assert_cmpint(job->pause_count, ==, 0); -g_assert_false (job->paused); -g_assert_false (job->busy);/* We're in block_job_sleep_ns() */ + g_assert_cmpint(job->job.pause_count, ==, 0); +g_assert_false (job->job.paused); +g_assert_false (job->job.busy);/* We're in block_job_sleep_ns() */ ret =block_job_complete_sync (job, &error_abort); g_assert_cmpint(ret, ==, 0);
tests/test-blockjob-txn.c +6 -6
@@ -78,8 +78,8 @@ static const BlockJobDriver test_block_job_driver = { .job_driver = { .instance_size = sizeof(TestBlockJob), .free = block_job_free, + .start = test_block_job_run, }, - .start = test_block_job_run, };/* Create a block job that completes with a given return code after a given
@@ -125,7 +125,7 @@ static void test_single_job(int expected) txn =block_job_txn_new (); job =test_block_job_start (1 , true, expected, &result, txn); -block_job_start (job); +job_start (&job->job);if (expected == -ECANCELED ) {block_job_cancel (job, false);
@@ -165,8 +165,8 @@ static void test_pair_jobs(int expected1, int expected2) txn =block_job_txn_new (); job1 =test_block_job_start (1 , true, expected1, &result1, txn); job2 =test_block_job_start (2 , true, expected2, &result2, txn); -block_job_start (job1); -block_job_start (job2); +job_start (&job1->job); +job_start (&job2->job);/* Release our reference now to trigger as many nice * use-after-free bugs as possible.
@@ -227,8 +227,8 @@ static void test_pair_jobs_fail_cancel_race(void) txn =block_job_txn_new (); job1 =test_block_job_start (1 , true, -ECANCELED , &result1, txn); job2 =test_block_job_start (2 , false,0 , &result2, txn); -block_job_start (job1); -block_job_start (job2); +job_start (&job1->job); +job_start (&job2->job);block_job_cancel (job1, false);
tests/test-blockjob.c +7 -7
@@ -199,8 +199,8 @@ static const BlockJobDriver test_cancel_driver = { .job_driver = { .instance_size = sizeof(CancelJob), .free = block_job_free, + .start = cancel_job_start, }, - .start = cancel_job_start, .complete = cancel_job_complete, };
@@ -254,7 +254,7 @@ static void test_cancel_running(void) s =create_common (&job); -block_job_start (job); +job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING );cancel_common (s);
@@ -267,7 +267,7 @@ static void test_cancel_paused(void) s =create_common (&job); -block_job_start (job); +job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING );block_job_user_pause (job, &error_abort);
@@ -284,7 +284,7 @@ static void test_cancel_ready(void) s =create_common (&job); -block_job_start (job); +job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;
@@ -301,7 +301,7 @@ static void test_cancel_standby(void) s =create_common (&job); -block_job_start (job); +job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;
@@ -322,7 +322,7 @@ static void test_cancel_pending(void) s =create_common (&job); -block_job_start (job); +job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;
@@ -346,7 +346,7 @@ static void test_cancel_concluded(void) s =create_common (&job); -block_job_start (job); +job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 18/42] job: Move coroutine and related code to Job
Quoted 22 lines · Kevin Wolf
This commit moves some core functions for dealing with the job coroutine from BlockJob to Job. This includes primarily entering the coroutine (both for the first and reentering) and yielding explicitly and at pause points.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 40 --------- include/block/blockjob_int.h | 26 ------ include/qemu/job.h | 76 ++++++++++++++++ block/backup.c | 2 +- block/commit.c | 4 +- block/mirror.c | 22 ++--- block/replication.c | 2 +- block/stream.c | 4 +- blockdev.c | 8 +- blockjob.c | 201 +++++++------------------------------------ job.c | 137 +++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 38 ++++---- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 14 +-- 14 files changed, 296 insertions(+), 290 deletions(-)
[...]
Quoted 4 lines
job.c +0
-0
[...]
Quoted 23 lines
job.c +19
-0@@ -172,6 +205,110 @@ void job_unref(Job *job) } } +void job_enter_cond (Job *job,bool (*fn)(Job *job)) +{ +if (!job_started (job)) { +return ; + } +if (job->deferred_to_main_loop) { +return ; + } + +job_lock (); +if (job->busy) { +job_unlock (); +return ; + } + +if (fn && !fn (job)) { +job_unlock (); +return ; + }
job.c void job_unref(Job *job) · line 236
Quoted 10 lines
job.c +10
-0+ +assert (!job->deferred_to_main_loop); +timer_del (&job->sleep_timer); + job->busy = true; +job_unlock (); +aio_co_wake (job->co); +} + +/* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. +* Reentering the job coroutine with block_job_enter() before the timer has
There is no job_enter() yet, but you do reference it below, so this should probably be adjusted here...
job.c void job_unref(Job *job) · line 239
Quoted 3 lines
job.c +3
-0+* expired is allowed and cancels the timer. +* +* If @ns is (uint64_t) -1, no timer is scheduled and block_job_enter() must be
...and here. Or maybe you want to change the job_enter below to a block_job_enter until there is a job_enter().
With that done (or not, because I don't really care about mid-series comments);
Quoted 14 lines
job.c +14
-0+* called explicitly. */ +void coroutine_fn job_do_yield(Job *job, uint64_t ns) +{ +job_lock (); +if (ns !=-1 ) { +timer_mod (&job->sleep_timer, ns); + } + job->busy = false; +job_unlock (); +qemu_coroutine_yield (); + +/* Set by job_enter before re-entering the coroutine. */ +assert (job->busy); +}
Re: [Qemu-devel] [PATCH 18/42] job: Move coroutine and related code to Job
In reply to Kevin WolfQuoted 23 lines · Kevin Wolf
This commit moves some core functions for dealing with the job coroutine from BlockJob to Job. This includes primarily entering the coroutine (both for the first and reentering) and yielding explicitly and at pause points.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 40 --------- include/block/blockjob_int.h | 26 ------ include/qemu/job.h | 76 ++++++++++++++++ block/backup.c | 2 +- block/commit.c | 4 +- block/mirror.c | 22 ++--- block/replication.c | 2 +- block/stream.c | 4 +- blockdev.c | 8 +- blockjob.c | 201 +++++++------------------------------------ job.c | 137 +++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 38 ++++---- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 14 +-- 14 files changed, 296 insertions(+), 290 deletions(-)
[...]
blockjob.c static void block_job_completed_txn_success(BlockJob *job) · line 555
Quoted 4 lines
/* Assumes the
block_job_mutexis held */blockjob.c +1
-1-static bool block_job_timer_pending (BlockJob *job) +static bool job_timer_pending (Job *job)
Is this intentionally left behind in blockjob.c, even once you've changed the name (and moved the state to job.c?)
[...]
job.c void job_unref(Job *job) · line 309
Quoted 10 lines
job.c +10
-0+void job_start (Job *job) +{ +assert (job && !job_started (job) && job->paused && + job->driver && job->driver->start); + job->co =qemu_coroutine_create (job_co_entry, job); + job->pause_count--; + job->busy = true; + job->paused = false; +job_state_transition (job,JOB_STATUS_RUNNING ); +aio_co_enter (job->aio_context, job->co);
I suppose patch 16 was the time to ask this, but are there any detriments to setting the AIO Context at creation time and then using that consistently instead of just fetching the AIO context of the BDS whenever we need it?
I guess if the context changes then we've got it covered in block_job_attached_aio_context.
Re: [Qemu-devel] [PATCH 18/42] job: Move coroutine and related code to Job
Am 15.05.2018 um 01:02 hat John Snow geschrieben:
Quoted 30 lines · Kevin Wolf
Quoted 23 lines
This commit moves some core functions for dealing with the job coroutine from BlockJob to Job. This includes primarily entering the coroutine (both for the first and reentering) and yielding explicitly and at pause points.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 40 --------- include/block/blockjob_int.h | 26 ------ include/qemu/job.h | 76 ++++++++++++++++ block/backup.c | 2 +- block/commit.c | 4 +- block/mirror.c | 22 ++--- block/replication.c | 2 +- block/stream.c | 4 +- blockdev.c | 8 +- blockjob.c | 201 +++++++------------------------------------ job.c | 137 +++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 38 ++++---- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 14 +-- 14 files changed, 296 insertions(+), 290 deletions(-)[...]
Quoted 1 line
Quoted 6 lines
blockjob.c static void block_job_completed_txn_success(BlockJob *job) · line 555
Quoted 3 lines
/* Assumes the
block_job_mutexis held */blockjob.c +1
-1-static bool block_job_timer_pending (BlockJob *job) +static bool job_timer_pending (Job *job)Is this intentionally left behind in
blockjob.c, even once you've changed the name (and moved the state tojob.c?)
Yes, it's just a small callback that is structually part of block_job_set_speed(). Maybe I shouldn't rename it, but it does get a Job rather than a BlockJob now, so I'm not sure.
Kevin
Re: [Qemu-devel] [PATCH 18/42] job: Move coroutine and related code to Job
Quoted 38 lines · Kevin Wolf
Am 15.05.2018 um 01:02 hat John Snow geschrieben:
Quoted 30 lines · Kevin Wolf
Quoted 23 lines
This commit moves some core functions for dealing with the job coroutine from BlockJob to Job. This includes primarily entering the coroutine (both for the first and reentering) and yielding explicitly and at pause points.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 40 --------- include/block/blockjob_int.h | 26 ------ include/qemu/job.h | 76 ++++++++++++++++ block/backup.c | 2 +- block/commit.c | 4 +- block/mirror.c | 22 ++--- block/replication.c | 2 +- block/stream.c | 4 +- blockdev.c | 8 +- blockjob.c | 201 +++++++------------------------------------ job.c | 137 +++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 38 ++++---- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 14 +-- 14 files changed, 296 insertions(+), 290 deletions(-)[...]
Quoted 1 line
Quoted 6 lines
blockjob.c static void block_job_completed_txn_success(BlockJob *job) · line 555
Quoted 3 lines
/* Assumes the
block_job_mutexis held */blockjob.c +1
-1-static bool block_job_timer_pending (BlockJob *job) +static bool job_timer_pending (Job *job)Is this intentionally left behind in
blockjob.c, even once you've changed the name (and moved the state tojob.c?)
Quoted 6 lines
Yes, it's just a small callback that is structually part of
block_job_set_speed(). Maybe I shouldn't rename it, but it does get a Job rather than a BlockJob now, so I'm not sure.Kevin
Your choice; it's not clear to me yet which things truly belong in which file because I haven't had a chance to look at the finished result. I'm sure if it looks like it needs to move later that someone will eventually move it.
--js
PATCH 19/42 job: Add job_sleep_ns()
In reply to Kevin WolfThere is nothing block layer specific about block_job_sleep_ns(), so move the function to Job.
--- include/block/blockjob_int.h | 11 ----------- include/qemu/job.h | 17 +++++++++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 27 --------------------------- job.c | 32 ++++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 8 ++++---- tests/test-blockjob-txn.c | 2 +- tests/test-blockjob.c | 2 +- 11 files changed, 60 insertions(+), 49 deletions(-)
include/block/blockjob_int.h +0 -11
@@ -134,17 +134,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,void block_job_free (Job *job);/** -* block_job_sleep_ns: -* @job: The job that calls the function. -* @ns: How many nanoseconds to stop for. -* -* Put the job to sleep (assuming that it wasn't canceled) for @ns -* %QEMU_CLOCK_REALTIME nanoseconds. Canceling the job will immediately -* interrupt the wait. -*/ -void block_job_sleep_ns (BlockJob *job,int64_t ns); - -/** * block_job_yield: * @job: The job that calls the function. *
include/qemu/job.h +17 -0
@@ -168,6 +168,13 @@ void job_enter_cond(Job *job, bool(*fn)(Job *job));void job_start (Job *job);/** +* @job: The job to enter. +* +* Continue the specified job by entering the coroutine. +*/ +void job_enter (Job *job); + +/** * @job: The job that is ready to pause. * * Pause now if job_pause() has been called. Jobs that perform lots of I/O
@@ -175,6 +182,16 @@ void job_start(Job *job);*/ void coroutine_fn job_pause_point(Job *job); +/** +* @job: The job that calls the function. +* @ns: How many nanoseconds to stop for. +* +* Put the job to sleep (assuming that it wasn't canceled) for @ns +* %QEMU_CLOCK_REALTIME nanoseconds. Canceling the job will immediately +* interrupt the wait. +*/ +void coroutine_fn job_sleep_ns(Job *job, int64_t ns); +/** Returns the JobType of a given Job. */ JobType job_type (Job *job);
block/backup.c +1 -1
@@ -338,7 +338,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job)* return. Without a yield, the VM would not reboot. */ delay_ns =block_job_ratelimit_get_delay (&job->common, job->bytes_read); job->bytes_read =0 ; -block_job_sleep_ns (&job->common, delay_ns); +job_sleep_ns (&job->common.job, delay_ns);if (job_is_cancelled (&job->common.job)) {return true;
block/commit.c +1 -1
@@ -172,7 +172,7 @@ static void coroutine_fn commit_run(void *opaque)/* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. */ -block_job_sleep_ns (&s->common, delay_ns); +job_sleep_ns (&s->common.job, delay_ns);if (job_is_cancelled (&s->common.job)) {break ; }
block/mirror.c +2 -2
@@ -595,7 +595,7 @@ static void mirror_throttle(MirrorBlockJob *s)if (now - s->last_pause_ns >SLICE_TIME ) { s->last_pause_ns = now; -block_job_sleep_ns (&s->common,0 ); +job_sleep_ns (&s->common.job,0 ); }else {job_pause_point (&s->common.job); }
@@ -869,7 +869,7 @@ static void coroutine_fn mirror_run(void *opaque) delay_ns = (s->in_flight ==0 && cnt ==0 ?SLICE_TIME :0 ); }trace_mirror_before_sleep (s, cnt, s->synced, delay_ns); -block_job_sleep_ns (&s->common, delay_ns); +job_sleep_ns (&s->common.job, delay_ns);if (job_is_cancelled (&s->common.job) && (!s->synced || s->common.force)) {
block/stream.c +1 -1
@@ -140,7 +140,7 @@ static void coroutine_fn stream_run(void *opaque)/* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. */ -block_job_sleep_ns (&s->common, delay_ns); +job_sleep_ns (&s->common.job, delay_ns);if (job_is_cancelled (&s->common.job)) {break ; }
blockjob.c +0 -27
@@ -174,7 +174,6 @@ void block_job_free(Job *job) block_job_detach_aio_context, bjob);blk_unref (bjob->blk);error_free (bjob->blocker); -assert (!timer_pending (&bjob->job.sleep_timer)); } static void block_job_attached_aio_context(AioContext *new_context,
@@ -283,13 +282,6 @@ const BlockJobDriver *block_job_driver(BlockJob *job)return job->driver; } -static void block_job_sleep_timer_cb (void *opaque) -{ -BlockJob *job = opaque; - -block_job_enter (job); -} -static void block_job_decommission (BlockJob *job) {assert (job);
@@ -859,9 +851,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->opaque = opaque; job->auto_finalize = !(flags &BLOCK_JOB_MANUAL_FINALIZE ); job->auto_dismiss = !(flags &BLOCK_JOB_MANUAL_DISMISS ); -aio_timer_init (qemu_get_aio_context (), &job->job.sleep_timer, -QEMU_CLOCK_REALTIME ,SCALE_NS , - block_job_sleep_timer_cb, job);error_setg (&job->blocker,"block device is in use by block job: %s" ,job_type_str (&job->job));
@@ -924,22 +913,6 @@ void block_job_enter(BlockJob *job)job_enter_cond (&job->job,NULL ); } -void block_job_sleep_ns (BlockJob *job,int64_t ns) -{ -assert (job->job.busy); - -/* Check cancellation *before* setting busy = false, too! */ -if (job_is_cancelled (&job->job)) { -return ; - } - -if (!job_should_pause (&job->job)) { -job_do_yield (&job->job,qemu_clock_get_ns (QEMU_CLOCK_REALTIME ) + ns); - } - -job_pause_point (&job->job); -} -void block_job_yield (BlockJob *job) {assert (job->job.busy);
job.c +32 -0
@@ -152,6 +152,13 @@ Job *job_get(const char *id)return NULL ; } +static void job_sleep_timer_cb (void *opaque) +{ +Job *job = opaque; + +job_enter (job); +} +void *job_create (const char *job_id,const JobDriver *driver,AioContext *ctx,Error **errp) {
@@ -178,6 +185,9 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, job->pause_count =1 ;job_state_transition (job,JOB_STATUS_CREATED ); +aio_timer_init (qemu_get_aio_context (), &job->sleep_timer, +QEMU_CLOCK_REALTIME ,SCALE_NS , + job_sleep_timer_cb, job);QLIST_INSERT_HEAD (&jobs, job, job_list);
@@ -193,6 +203,7 @@ void job_unref(Job *job) {if (--job->refcnt ==0 ) {assert (job->status ==JOB_STATUS_NULL ); +assert (!timer_pending (&job->sleep_timer));if (job->driver->free) { job->driver->free (job);
@@ -232,6 +243,11 @@ void job_enter_cond(Job *job, bool(*fn)(Job *job))aio_co_wake (job->co); } +void job_enter (Job *job) +{ +job_enter_cond (job,NULL ); +} +/* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. * Reentering the job coroutine with block_job_enter() before the timer has * expired is allowed and cancels the timer.
@@ -283,6 +299,22 @@ void coroutine_fn job_pause_point(Job *job) } } +void coroutine_fn job_sleep_ns(Job *job, int64_t ns) +{ +assert (job->busy); + +/* Check cancellation *before* setting busy = false, too! */ +if (job_is_cancelled (job)) { +return ; + } + +if (!job_should_pause (job)) { +job_do_yield (job,qemu_clock_get_ns (QEMU_CLOCK_REALTIME ) + ns); + } + +job_pause_point (job); +} +/** * All jobs must allow a pause point before entering their job proper. This * ensures that jobs can be paused prior to being started, then resumed later.
tests/test-bdrv-drain.c +4 -4
@@ -508,7 +508,7 @@ static void coroutine_fn test_job_start(void *opaque)block_job_event_ready (&s->common);while (!s->should_complete) { -block_job_sleep_ns (&s->common,100000 ); +job_sleep_ns (&s->common.job,100000 ); }job_defer_to_main_loop (&s->common.job, test_job_completed,NULL );
@@ -553,7 +553,7 @@ static void test_blockjob_common(enum drain_type drain_type) g_assert_cmpint(job->job.pause_count, ==, 0);g_assert_false (job->job.paused); -g_assert_false (job->job.busy);/* We're in block_job_sleep_ns() */ +g_assert_false (job->job.busy);/* We're in job_sleep_ns() */ do_drain_begin (drain_type, src);
@@ -571,7 +571,7 @@ static void test_blockjob_common(enum drain_type drain_type) g_assert_cmpint(job->job.pause_count, ==, 0);g_assert_false (job->job.paused); -g_assert_false (job->job.busy);/* We're in block_job_sleep_ns() */ +g_assert_false (job->job.busy);/* We're in job_sleep_ns() */ do_drain_begin (drain_type, target);
@@ -589,7 +589,7 @@ static void test_blockjob_common(enum drain_type drain_type) g_assert_cmpint(job->job.pause_count, ==, 0);g_assert_false (job->job.paused); -g_assert_false (job->job.busy);/* We're in block_job_sleep_ns() */ +g_assert_false (job->job.busy);/* We're in job_sleep_ns() */ ret =block_job_complete_sync (job, &error_abort); g_assert_cmpint(ret, ==, 0);
tests/test-blockjob-txn.c +1 -1
@@ -45,7 +45,7 @@ static void coroutine_fn test_block_job_run(void *opaque)while (s->iterations--) {if (s->use_timer) { -block_job_sleep_ns (job,0 ); +job_sleep_ns (&job->job,0 ); }else {block_job_yield (job); }
tests/test-blockjob.c +1 -1
@@ -188,7 +188,7 @@ static void coroutine_fn cancel_job_start(void *opaque)block_job_event_ready (&s->common); } -block_job_sleep_ns (&s->common,100000 ); +job_sleep_ns (&s->common.job,100000 ); } defer:
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 19/42] job: Add job_sleep_ns()
Quoted 4 lines · Kevin Wolf
There is nothing block layer specific about
block_job_sleep_ns(), so move the function to Job.Signed-off-by Kevin Wolf <[email protected]>
Quoted 13 lines
--- include/block/blockjob_int.h | 11 ----------- include/qemu/job.h | 17 +++++++++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 27 --------------------------- job.c | 32 ++++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 8 ++++---- tests/test-blockjob-txn.c | 2 +- tests/test-blockjob.c | 2 +- 11 files changed, 60 insertions(+), 49 deletions(-)
The Job.sleep_timer documentation should be adjusted. With that done:
Re: [Qemu-devel] [PATCH 19/42] job: Add job_sleep_ns()
In reply to Kevin WolfQuoted 45 lines · Kevin Wolf
There is nothing block layer specific about
block_job_sleep_ns(), so move the function to Job.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 11 ----------- include/qemu/job.h | 17 +++++++++++++++++ block/backup.c | 2 +- block/commit.c | 2 +- block/mirror.c | 4 ++-- block/stream.c | 2 +- blockjob.c | 27 --------------------------- job.c | 32 ++++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 8 ++++---- tests/test-blockjob-txn.c | 2 +- tests/test-blockjob.c | 2 +- 11 files changed, 60 insertions(+), 49 deletions(-)include/block/blockjob_int.h +0
-11@@ -134,17 +134,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,void block_job_free (Job *job);/** -* block_job_sleep_ns: -* @job: The job that calls the function. -* @ns: How many nanoseconds to stop for. -* -* Put the job to sleep (assuming that it wasn't canceled) for @ns -* %QEMU_CLOCK_REALTIME nanoseconds. Canceling the job will immediately -* interrupt the wait. -*/ -void block_job_sleep_ns (BlockJob *job,int64_t ns); - -/** * block_job_yield: * @job: The job that calls the function. * include/qemu/job.h +0
-0
include/qemu/job.h void job_enter_cond(Job *job, bool(*fn)(Job *job)); · line 176
Quoted 9 lines
include/qemu/job.h +6
-0@@ -168,6 +168,13 @@ void job_enter_cond(Job *job, bool(*fn)(Job *job));void job_start (Job *job);/** +* @job: The job to enter. +* +* Continue the specified job by entering the coroutine. +*/ +void job_enter (Job *job); +
Is this a holdout from #18?
Eh, either way.
PATCH 20/42 job: Move pause/resume functions to Job
In reply to Kevin WolfWhile we already moved the state related to job pausing to Job, the functions to do were still BlockJob only. This commit moves them over to Job.
--- include/block/blockjob.h | 32 ------------------- include/block/blockjob_int.h | 7 +++++ include/qemu/job.h | 37 ++++++++++++++++++++++ block/backup.c | 1 + block/commit.c | 1 + block/mirror.c | 2 ++ block/stream.c | 1 + blockdev.c | 6 ++-- blockjob.c | 73 ++++++++++---------------------------------- job.c | 51 +++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 1 + tests/test-blockjob-txn.c | 1 + tests/test-blockjob.c | 6 ++-- 13 files changed, 125 insertions(+), 94 deletions(-)
include/block/blockjob.h +0 -32
@@ -57,12 +57,6 @@ typedef struct BlockJob {bool force;/** -* Set to true if the job is paused by user. Can be unpaused with the -* block-job-resume QMP command. -*/ -bool user_paused; - -/** * Set to true when the job is ready to be completed. */ bool ready;
@@ -248,32 +242,6 @@ void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining);BlockJobInfo *block_job_query (BlockJob *job,Error **errp);/** -* block_job_user_pause: -* @job: The job to be paused. -* -* Asynchronously pause the specified job. -* Do not allow a resume until a matching call to block_job_user_resume. -*/ -void block_job_user_pause (BlockJob *job,Error **errp); - -/** -* block_job_paused: -* @job: The job to query. -* -* Returns true if the job is user-paused. -*/ -bool block_job_user_paused (BlockJob *job); - -/** -* block_job_user_resume: -* @job: The job to be resumed. -* -* Resume the specified job. -* Must be paired with a preceding block_job_user_pause. -*/ -void block_job_user_resume (BlockJob *job,Error **errp); - -/** * block_job_user_cancel: * @job: The job to be cancelled. * @force: Quit a job without waiting for data to be in sync.
include/block/blockjob_int.h +7 -0
@@ -134,6 +134,13 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,void block_job_free (Job *job);/** +* block_job_user_resume: +* Callback to be used for JobDriver.user_resume in all block jobs. Resets the +* iostatus when the user resumes @job. +*/ +void block_job_user_resume (Job *job); + +/** * block_job_yield: * @job: The job that calls the function. *
include/qemu/job.h +37 -0
@@ -83,6 +83,12 @@ typedef struct Job {bool paused;/** +* Set to true if the job is paused by user. Can be unpaused with the +* block-job-resume QMP command. +*/ +bool user_paused; + +/** * Set to true if the job should cancel itself. The flag must * always be tested just before toggling the busy flag from false * to true. After a job has been cancelled, it should only yield
@@ -124,6 +130,12 @@ struct JobDriver {*/ void coroutine_fn (*resume)(Job *job); +/** +* Called when the job is resumed by the user (i.e. user_paused becomes +* false). .user_resume is called before .resume. +*/ +void (*user_resume)(Job *job); +/** Called when the job is freed */ void (*free)(Job *job); };
@@ -203,6 +215,31 @@ const char *job_type_str(Job *job);bool job_is_cancelled (Job *job);/** +* Request @job to pause at the next pause point. Must be paired with +* job_resume(). If the job is supposed to be resumed by user action, call +* job_user_pause() instead. +*/ +void job_pause (Job *job); + +/** Resumes a @job paused with job_pause. */ +void job_resume (Job *job); + +/** +* Asynchronously pause the specified @job. +* Do not allow a resume until a matching call to job_user_resume. +*/ +void job_user_pause (Job *job,Error **errp); + +/** Returns true if the job is user-paused. */ +bool job_user_paused (Job *job); + +/** +* Resume the specified @job. +* Must be paired with a preceding job_user_pause. +*/ +void job_user_resume (Job *job,Error **errp); + +/** * Get the next element from the list of block jobs after @job, or the * first one if @job is %NULL. *
block/backup.c +1 -0
@@ -528,6 +528,7 @@ static const BlockJobDriver backup_job_driver = { .instance_size = sizeof(BackupBlockJob), .job_type = JOB_TYPE_BACKUP, .free = block_job_free, + .user_resume = block_job_user_resume, .start = backup_run, }, .commit = backup_commit,
block/commit.c +1 -0
@@ -220,6 +220,7 @@ static const BlockJobDriver commit_job_driver = { .instance_size = sizeof(CommitBlockJob), .job_type = JOB_TYPE_COMMIT, .free = block_job_free, + .user_resume = block_job_user_resume, .start = commit_run, }, };
block/mirror.c +2 -0
@@ -991,6 +991,7 @@ static const BlockJobDriver mirror_job_driver = { .instance_size = sizeof(MirrorBlockJob), .job_type = JOB_TYPE_MIRROR, .free = block_job_free, + .user_resume = block_job_user_resume, .start = mirror_run, .pause = mirror_pause, },
@@ -1004,6 +1005,7 @@ static const BlockJobDriver commit_active_job_driver = { .instance_size = sizeof(MirrorBlockJob), .job_type = JOB_TYPE_COMMIT, .free = block_job_free, + .user_resume = block_job_user_resume, .start = mirror_run, .pause = mirror_pause, },
block/stream.c +1 -0
@@ -214,6 +214,7 @@ static const BlockJobDriver stream_job_driver = { .job_type = JOB_TYPE_STREAM, .free = block_job_free, .start = stream_run, + .user_resume = block_job_user_resume, }, };
blockdev.c +3 -3
@@ -3844,7 +3844,7 @@ void qmp_block_job_cancel(const char *device, force = false; } -if (block_job_user_paused (job) && !force) { +if (job_user_paused (&job->job) && !force) {error_setg (errp,"The block job for device '%s' is currently paused" , device); goto out;
@@ -3866,7 +3866,7 @@ void qmp_block_job_pause(const char *device, Error **errp) }trace_qmp_block_job_pause (job); -block_job_user_pause (job, errp); +job_user_pause (&job->job, errp);aio_context_release (aio_context); }
@@ -3880,7 +3880,7 @@ void qmp_block_job_resume(const char *device, Error **errp) }trace_qmp_block_job_resume (job); -block_job_user_resume (job, errp); +job_user_resume (&job->job, errp);aio_context_release (aio_context); }
blockjob.c +16 -57
137 lines not on this page · raw message
job.c +51 -0
57 lines not on this page · raw message
tests/test-bdrv-drain.c +1 -0
@@ -524,6 +524,7 @@ BlockJobDriver test_job_driver = { .job_driver = { .instance_size = sizeof(TestBlockJob), .free = block_job_free, + .user_resume = block_job_user_resume, .start = test_job_start, }, .complete = test_job_complete,
tests/test-blockjob-txn.c +1 -0
@@ -78,6 +78,7 @@ static const BlockJobDriver test_block_job_driver = { .job_driver = { .instance_size = sizeof(TestBlockJob), .free = block_job_free, + .user_resume = block_job_user_resume, .start = test_block_job_run, }, };
tests/test-blockjob.c +4 -2
@@ -20,6 +20,7 @@ static const BlockJobDriver test_block_job_driver = { .job_driver = { .instance_size = sizeof(BlockJob), .free = block_job_free, + .user_resume = block_job_user_resume, }, };
@@ -199,6 +200,7 @@ static const BlockJobDriver test_cancel_driver = { .job_driver = { .instance_size = sizeof(CancelJob), .free = block_job_free, + .user_resume = block_job_user_resume, .start = cancel_job_start, }, .complete = cancel_job_complete,
@@ -270,7 +272,7 @@ static void test_cancel_paused(void)job_start (&job->job);assert (job->job.status ==JOB_STATUS_RUNNING ); -block_job_user_pause (job, &error_abort); +job_user_pause (&job->job, &error_abort);block_job_enter (job);assert (job->job.status ==JOB_STATUS_PAUSED );
@@ -308,7 +310,7 @@ static void test_cancel_standby(void)block_job_enter (job);assert (job->job.status ==JOB_STATUS_READY ); -block_job_user_pause (job, &error_abort); +job_user_pause (&job->job, &error_abort);block_job_enter (job);assert (job->job.status ==JOB_STATUS_STANDBY );
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 20/42] job: Move pause/resume functions to Job
Quoted 5 lines · Kevin Wolf
While we already moved the state related to job pausing to Job, the functions to do were still BlockJob only. This commit moves them over to Job.
Signed-off-by Kevin Wolf <[email protected]>
Quoted 15 lines
--- include/block/blockjob.h | 32 ------------------- include/block/blockjob_int.h | 7 +++++ include/qemu/job.h | 37 ++++++++++++++++++++++ block/backup.c | 1 + block/commit.c | 1 + block/mirror.c | 2 ++ block/stream.c | 1 + blockdev.c | 6 ++-- blockjob.c | 73 ++++++++++---------------------------------- job.c | 51 +++++++++++++++++++++++++++++++ tests/test-bdrv-drain.c | 1 + tests/test-blockjob-txn.c | 1 + tests/test-blockjob.c | 6 ++-- 13 files changed, 125 insertions(+), 94 deletions(-)
Re: [Qemu-devel] [PATCH 20/42] job: Move pause/resume functions to Job
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
While we already moved the state related to job pausing to Job, the functions to do were still BlockJob only. This commit moves them over to Job.
Signed-off-by Kevin Wolf <[email protected]>
I'm slightly sad about the new callback, because it still seems silly to have to specify something that's enforced with an assertion to be ubiquitously true, but I think any other way of fixing that is probably uglier.
So OK.
PATCH 21/42 job: Replace BlockJob.completed with job_is_completed()
In reply to Kevin WolfSince we introduced an explicit status to block job, BlockJob.completed is redundant because it can be derived from the status. Remove the field from BlockJob and add a function to derive it from the status at the Job level.
--- include/block/blockjob.h | 3 --- include/qemu/job.h | 3 +++ blockjob.c | 16 +++++++--------- job.c | 22 ++++++++++++++++++++++ qemu-img.c | 4 ++-- 5 files changed, 34 insertions(+), 14 deletions(-)
include/block/blockjob.h +0 -3
@@ -88,9 +88,6 @@ typedef struct BlockJob {/** The opaque value that is passed to the completion function. */ void *opaque; -/** True when job has reported completion by calling block_job_completed. */ -bool completed; -/** ret code passed to block_job_completed. */ int ret;
include/qemu/job.h +3 -0
@@ -214,6 +214,9 @@ const char *job_type_str(Job *job);/** Returns whether the job is scheduled for cancellation. */ bool job_is_cancelled (Job *job); +/** Returns whether the job is in a completed state. */ +bool job_is_completed (Job *job); +/** * Request @job to pause at the next pause point. Must be paired with * job_resume(). If the job is supposed to be resumed by user action, call
blockjob.c +7 -9
@@ -194,7 +194,7 @@ static void block_job_detach_aio_context(void *opaque)job_pause (&job->job); -while (!job->job.paused && !job->completed) { +while (!job->job.paused && !job_is_completed (&job->job)) {block_job_drain (job); }
@@ -270,7 +270,6 @@ const BlockJobDriver *block_job_driver(BlockJob *job)static void block_job_decommission (BlockJob *job) {assert (job); - job->completed = true; job->job.busy = false; job->job.paused = false; job->job.deferred_to_main_loop = true;
@@ -335,7 +334,7 @@ static void block_job_clean(BlockJob *job)static int block_job_finalize_single (BlockJob *job) { -assert (job->completed); +assert (job_is_completed (&job->job));/* Ensure abort is called for late-transactional failures */ block_job_update_rc (job);
@@ -428,10 +427,10 @@ static int block_job_finish_sync(BlockJob *job,/* block_job_drain calls block_job_enter, and it should be enough to * induce progress until the job completes or moves to the main thread. */ -while (!job->job.deferred_to_main_loop && !job->completed) { +while (!job->job.deferred_to_main_loop && !job_is_completed (&job->job)) {block_job_drain (job); } -while (!job->completed) { +while (!job_is_completed (&job->job)) {aio_poll (qemu_get_aio_context (), true); } ret = (job_is_cancelled (&job->job) && job->ret ==0 )
@@ -472,7 +471,7 @@ static void block_job_completed_txn_abort(BlockJob *job)while (!QLIST_EMPTY (&txn->jobs)) { other_job =QLIST_FIRST (&txn->jobs); ctx =blk_get_aio_context (other_job->blk); -if (!other_job->completed) { +if (!job_is_completed (&other_job->job)) {assert (job_is_cancelled (&other_job->job));block_job_finish_sync (other_job,NULL ,NULL ); }
@@ -514,7 +513,7 @@ static void block_job_completed_txn_success(BlockJob *job)* txn. */ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { -if (!other_job->completed) { +if (!job_is_completed (&other_job->job)) {return ; }assert (other_job->ret ==0 );
@@ -848,9 +847,8 @@ void block_job_early_fail(BlockJob *job)void block_job_completed (BlockJob *job,int ret) { -assert (job && job->txn && !job->completed); +assert (job && job->txn && !job_is_completed (&job->job));assert (blk_bs (job->blk)->job == job); - job->completed = true; job->ret = ret;block_job_update_rc (job);trace_block_job_completed (job, ret, job->ret);
job.c +22 -0
@@ -121,6 +121,28 @@ bool job_is_cancelled(Job *job)return job->cancelled; } +bool job_is_completed (Job *job) +{ +switch (job->status) { +case JOB_STATUS_UNDEFINED : +case JOB_STATUS_CREATED : +case JOB_STATUS_RUNNING : +case JOB_STATUS_PAUSED : +case JOB_STATUS_READY : +case JOB_STATUS_STANDBY : +return false; +case JOB_STATUS_WAITING : +case JOB_STATUS_PENDING : +case JOB_STATUS_ABORTING : +case JOB_STATUS_CONCLUDED : +case JOB_STATUS_NULL : +return true; +default : +g_assert_not_reached (); + } +return false; +} +bool job_started (Job *job) {return job->co;
qemu-img.c +2 -2
@@ -867,9 +867,9 @@ static void run_block_job(BlockJob *job, Error **errp)aio_poll (aio_context, true);qemu_progress_print (job->len ? ((float )job->offset / job->len *100.f ) :0.0f ,0 ); - }while (!job->ready && !job->completed); + }while (!job->ready && !job_is_completed (&job->job)); -if (!job->completed) { +if (!job_is_completed (&job->job)) { ret =block_job_complete_sync (job, errp); }else { ret = job->ret;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 21/42] job: Replace BlockJob.completed with job_is_completed()
Quoted 5 lines · Kevin Wolf
Since we introduced an explicit status to block job, BlockJob.completed is redundant because it can be derived from the status. Remove the field from BlockJob and add a function to derive it from the status at the Job level.
Quoted 8 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 3 --- include/qemu/job.h | 3 +++ blockjob.c | 16 +++++++--------- job.c | 22 ++++++++++++++++++++++ qemu-img.c | 4 ++-- 5 files changed, 34 insertions(+), 14 deletions(-)
Re: [Qemu-devel] [PATCH 21/42] job: Replace BlockJob.completed with job_is_completed()
In reply to Kevin WolfQuoted 5 lines · Kevin Wolf
Since we introduced an explicit status to block job, BlockJob.completed is redundant because it can be derived from the status. Remove the field from BlockJob and add a function to derive it from the status at the Job level.
You're braver than I am.
Quoted 51 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 3 --- include/qemu/job.h | 3 +++ blockjob.c | 16 +++++++--------- job.c | 22 ++++++++++++++++++++++ qemu-img.c | 4 ++-- 5 files changed, 34 insertions(+), 14 deletions(-)include/block/blockjob.h +0
-3@@ -88,9 +88,6 @@ typedef struct BlockJob {/** The opaque value that is passed to the completion function. */ void *opaque; -/** True when job has reported completion by calling block_job_completed. */ -bool completed; -/** ret code passed to block_job_completed. */ int ret;include/qemu/job.h +3
-0@@ -214,6 +214,9 @@ const char *job_type_str(Job *job);/** Returns whether the job is scheduled for cancellation. */ bool job_is_cancelled (Job *job); +/** Returns whether the job is in a completed state. */ +bool job_is_completed (Job *job); +/** * Request @job to pause at the next pause point. Must be paired with * job_resume(). If the job is supposed to be resumed by user action, call blockjob.c +1
-1@@ -194,7 +194,7 @@ static void block_job_detach_aio_context(void *opaque)job_pause (&job->job); -while (!job->job.paused && !job->completed) { +while (!job->job.paused && !job_is_completed (&job->job)) {block_job_drain (job); }
blockjob.c const BlockJobDriver *block_job_driver(BlockJob *job) · line 273
Quoted 4 lines
blockjob.c +0
-1@@ -270,7 +270,6 @@ const BlockJobDriver *block_job_driver(BlockJob *job)static void block_job_decommission (BlockJob *job) {assert (job); - job->completed = true;
Oops. I forgot I left this stuff in here. It's definitely safe to remove, as you know :)
Quoted 67 lines
blockjob.c +6
-7job->job.busy = false; job->job.paused = false; job->job.deferred_to_main_loop = true;@@ -335,7 +334,7 @@ static void block_job_clean(BlockJob *job)static int block_job_finalize_single (BlockJob *job) { -assert (job->completed); +assert (job_is_completed (&job->job));/* Ensure abort is called for late-transactional failures */ block_job_update_rc (job);@@ -428,10 +427,10 @@ static int block_job_finish_sync(BlockJob *job,/* block_job_drain calls block_job_enter, and it should be enough to * induce progress until the job completes or moves to the main thread. */ -while (!job->job.deferred_to_main_loop && !job->completed) { +while (!job->job.deferred_to_main_loop && !job_is_completed (&job->job)) {block_job_drain (job); } -while (!job->completed) { +while (!job_is_completed (&job->job)) {aio_poll (qemu_get_aio_context (), true); } ret = (job_is_cancelled (&job->job) && job->ret ==0 )@@ -472,7 +471,7 @@ static void block_job_completed_txn_abort(BlockJob *job)while (!QLIST_EMPTY (&txn->jobs)) { other_job =QLIST_FIRST (&txn->jobs); ctx =blk_get_aio_context (other_job->blk); -if (!other_job->completed) { +if (!job_is_completed (&other_job->job)) {assert (job_is_cancelled (&other_job->job));block_job_finish_sync (other_job,NULL ,NULL ); }@@ -514,7 +513,7 @@ static void block_job_completed_txn_success(BlockJob *job)* txn. */ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { -if (!other_job->completed) { +if (!job_is_completed (&other_job->job)) {return ; }assert (other_job->ret ==0 );@@ -848,9 +847,8 @@ void block_job_early_fail(BlockJob *job)void block_job_completed (BlockJob *job,int ret) { -assert (job && job->txn && !job->completed); +assert (job && job->txn && !job_is_completed (&job->job));assert (blk_bs (job->blk)->job == job); - job->completed = true; job->ret = ret;block_job_update_rc (job);trace_block_job_completed (job, ret, job->ret);job.c +5
-0@@ -121,6 +121,28 @@ bool job_is_cancelled(Job *job)return job->cancelled; } +bool job_is_completed (Job *job) +{ +switch (job->status) { +case JOB_STATUS_UNDEFINED : +case JOB_STATUS_CREATED :
job.c bool job_is_cancelled(Job *job) · line 138
Quoted 10 lines
job.c +10
-0+case JOB_STATUS_RUNNING : +case JOB_STATUS_PAUSED : +case JOB_STATUS_READY : +case JOB_STATUS_STANDBY : +return false; +case JOB_STATUS_WAITING : +case JOB_STATUS_PENDING : +case JOB_STATUS_ABORTING : +case JOB_STATUS_CONCLUDED : +case JOB_STATUS_NULL :
Well, I'd argue that NULL shouldn't have state that it can answer with one way or the other, but sure. In practice it ought not matter.
Quoted 27 lines
job.c +7
-0+return true; +default : +g_assert_not_reached (); + } +return false; +} +bool job_started (Job *job) {return job->co;qemu-img.c +2
-2@@ -867,9 +867,9 @@ static void run_block_job(BlockJob *job, Error **errp)aio_poll (aio_context, true);qemu_progress_print (job->len ? ((float )job->offset / job->len *100.f ) :0.0f ,0 ); - }while (!job->ready && !job->completed); + }while (!job->ready && !job_is_completed (&job->job)); -if (!job->completed) { +if (!job_is_completed (&job->job)) { ret =block_job_complete_sync (job, errp); }else { ret = job->ret;
PATCH 22/42 job: Move BlockJobCreateFlags to Job
In reply to Kevin WolfThis renames the BlockJobCreateFlags constants, moves a few JOB_INTERNAL checks to job_create() and the auto_{finalize,dismiss} fields from BlockJob to Job.
--- include/block/blockjob.h | 17 ----------------- include/block/blockjob_int.h | 3 +-- include/qemu/job.h | 20 +++++++++++++++++++- block/commit.c | 2 +- block/mirror.c | 2 +- block/replication.c | 4 ++-- block/stream.c | 2 +- blockdev.c | 14 +++++++------- blockjob.c | 27 +++++++-------------------- job.c | 11 ++++++++++- qemu-img.c | 2 +- tests/test-blockjob-txn.c | 2 +- tests/test-blockjob.c | 4 ++-- 13 files changed, 53 insertions(+), 57 deletions(-)
include/block/blockjob.h +0 -17
@@ -91,27 +91,10 @@ typedef struct BlockJob {/** ret code passed to block_job_completed. */ int ret; -/** True if this job should automatically finalize itself */ -bool auto_finalize; - -/** True if this job should automatically dismiss itself */ -bool auto_dismiss; -BlockJobTxn *txn;QLIST_ENTRY (BlockJob) txn_list; } BlockJob; -typedef enum BlockJobCreateFlags { -/* Default behavior */ -BLOCK_JOB_DEFAULT =0x00 , -/* BlockJob is not QMP-created and should not send QMP events */ -BLOCK_JOB_INTERNAL =0x01 , -/* BlockJob requires manual finalize step */ -BLOCK_JOB_MANUAL_FINALIZE =0x02 , -/* BlockJob requires manual dismiss step */ -BLOCK_JOB_MANUAL_DISMISS =0x04 , -}BlockJobCreateFlags ; -/** * block_job_next: * @job: A block job, or %NULL.
include/block/blockjob_int.h +1 -2
@@ -106,8 +106,7 @@ struct BlockJobDriver {* @bs: The block * @perm, @shared_perm: Permissions to request for @bs * @speed: The maximum speed, in bytes per second, or 0 for unlimited. -* @flags: Creation flags for the Block Job. -* See @BlockJobCreateFlags +* @flags: Creation flags for the Block Job. See @JobCreateFlags. * @cb: Completion function for the job. * @opaque: Opaque pointer value passed to @cb. * @errp: Error object.
include/qemu/job.h +19 -1
@@ -99,6 +99,12 @@ typedef struct Job {/** Set to true when the job has deferred work to the main loop. */ bool deferred_to_main_loop; +/** True if this job should automatically finalize itself */ +bool auto_finalize; + +/** True if this job should automatically dismiss itself */ +bool auto_dismiss; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;
@@ -140,6 +146,17 @@ struct JobDriver {void (*free)(Job *job); }; +typedef enum JobCreateFlags { +/* Default behavior */ +JOB_DEFAULT =0x00 , +/* Job is not QMP-created and should not send QMP events */ +JOB_INTERNAL =0x01 , +/* Job requires manual finalize step */ +JOB_MANUAL_FINALIZE =0x02 , +/* Job requires manual dismiss step */ +JOB_MANUAL_DISMISS =0x04 , +}JobCreateFlags ; +/** * Create a new long-running job and return it.
@@ -147,10 +164,11 @@ struct JobDriver {* @job_id: The id of the newly-created job, or %NULL for internal jobs * @driver: The class object for the newly-created job. * @ctx: The AioContext to run the job coroutine in. +* @flags: Creation flags for the job. See @JobCreateFlags. * @errp: Error object. */ void *job_create (const char *job_id,const JobDriver *driver,AioContext *ctx, -Error **errp); +int flags,Error **errp);/** * Add a reference to Job refcnt, it will be decreased with job_unref, and then
block/commit.c +1 -1
@@ -282,7 +282,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, } s =block_job_create (job_id, &commit_job_driver,NULL , bs,0 ,BLK_PERM_ALL , - speed,BLOCK_JOB_DEFAULT ,NULL ,NULL , errp); + speed,JOB_DEFAULT ,NULL ,NULL , errp);if (!s) {return ; }
block/mirror.c +1 -1
@@ -1282,7 +1282,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs, } is_none_mode = mode ==MIRROR_SYNC_MODE_NONE ; base = mode ==MIRROR_SYNC_MODE_TOP ?backing_bs (bs) :NULL ; - mirror_start_job(job_id, bs, BLOCK_JOB_DEFAULT, target, replaces, + mirror_start_job(job_id, bs, JOB_DEFAULT, target, replaces, speed, granularity, buf_size, backing_mode, on_source_error, on_target_error, unmap, NULL, NULL, &mirror_job_driver, is_none_mode, base, false,
block/replication.c +2 -2
@@ -568,7 +568,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, job =backup_job_create (NULL , s->secondary_disk->bs, s->hidden_disk->bs,0 ,MIRROR_SYNC_MODE_NONE ,NULL , false,BLOCKDEV_ON_ERROR_REPORT , -BLOCKDEV_ON_ERROR_REPORT ,BLOCK_JOB_INTERNAL , +BLOCKDEV_ON_ERROR_REPORT ,JOB_INTERNAL , backup_job_completed, bs,NULL , &local_err);if (local_err) {error_propagate (errp, local_err);
@@ -693,7 +693,7 @@ static void replication_stop(ReplicationState *rs, bool failover, Error **errp) s->stage =BLOCK_REPLICATION_FAILOVER ;commit_active_start (NULL , s->active_disk->bs, s->secondary_disk->bs, -BLOCK_JOB_INTERNAL ,0 ,BLOCKDEV_ON_ERROR_REPORT , +JOB_INTERNAL ,0 ,BLOCKDEV_ON_ERROR_REPORT ,NULL , replication_done, bs, true, errp);break ;default :
block/stream.c +1 -1
@@ -242,7 +242,7 @@ void stream_start(const char *job_id, BlockDriverState *bs,BLK_PERM_GRAPH_MOD ,BLK_PERM_CONSISTENT_READ |BLK_PERM_WRITE_UNCHANGED |BLK_PERM_WRITE , - speed, BLOCK_JOB_DEFAULT, NULL, NULL, errp); + speed, JOB_DEFAULT, NULL, NULL, errp);if (!s) { goto fail; }
blockdev.c +7 -7
@@ -3244,7 +3244,7 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, goto out; }commit_active_start (has_job_id ? job_id :NULL , bs, base_bs, -BLOCK_JOB_DEFAULT , speed, on_error, +JOB_DEFAULT , speed, on_error, filter_node_name,NULL ,NULL , false, &local_err); }else {BlockDriverState *overlay_bs =bdrv_find_overlay (bs, top_bs);
@@ -3275,7 +3275,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn,AioContext *aio_context;QDict *options =NULL ;Error *local_err =NULL ; -int flags, job_flags =BLOCK_JOB_DEFAULT ; +int flags, job_flags =JOB_DEFAULT ;int64_t size;bool set_backing_hd = false;
@@ -3398,10 +3398,10 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, } }if (!backup->auto_finalize) { - job_flags |=BLOCK_JOB_MANUAL_FINALIZE ; + job_flags |=JOB_MANUAL_FINALIZE ; }if (!backup->auto_dismiss) { - job_flags |=BLOCK_JOB_MANUAL_DISMISS ; + job_flags |=JOB_MANUAL_DISMISS ; } job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
@@ -3442,7 +3442,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn,Error *local_err =NULL ;AioContext *aio_context;BlockJob *job =NULL ; -int job_flags =BLOCK_JOB_DEFAULT ; +int job_flags =JOB_DEFAULT ;if (!backup->has_speed) { backup->speed =0 ;
@@ -3491,10 +3491,10 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, } }if (!backup->auto_finalize) { - job_flags |=BLOCK_JOB_MANUAL_FINALIZE ; + job_flags |=JOB_MANUAL_FINALIZE ; }if (!backup->auto_dismiss) { - job_flags |=BLOCK_JOB_MANUAL_DISMISS ; + job_flags |=JOB_MANUAL_DISMISS ; } job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, NULL, backup->compress,
blockjob.c +7 -20
@@ -286,7 +286,7 @@ static void block_job_do_dismiss(BlockJob *job)static void block_job_conclude (BlockJob *job) {job_state_transition (&job->job,JOB_STATUS_CONCLUDED ); -if (job->auto_dismiss || !job_started (&job->job)) { +if (job->job.auto_dismiss || !job_started (&job->job)) {block_job_do_dismiss (job); } }
@@ -484,7 +484,7 @@ static void block_job_completed_txn_abort(BlockJob *job)static int block_job_needs_finalize (BlockJob *job) { -return !job->auto_finalize; +return !job->job.auto_finalize; }static void block_job_do_finalize (BlockJob *job)
@@ -689,8 +689,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) info->io_status = job->iostatus; info->ready = job->ready; info->status = job->job.status; - info->auto_finalize = job->auto_finalize; - info->auto_dismiss = job->auto_dismiss; + info->auto_finalize = job->job.auto_finalize; + info->auto_dismiss = job->job.auto_dismiss; info->has_error = job->ret !=0 ; info->error = job->ret ?g_strdup (strerror (-job->ret)) :NULL ;return info;
@@ -737,7 +737,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg)static int block_job_event_pending (BlockJob *job) {job_state_transition (&job->job,JOB_STATUS_PENDING ); -if (!job->auto_finalize && !block_job_is_internal (job)) { +if (!job->job.auto_finalize && !block_job_is_internal (job)) {qapi_event_send_block_job_pending (job_type (&job->job), job->job.id, &error_abort);
@@ -764,19 +764,8 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,return NULL ; } -if (job_id ==NULL && !(flags &BLOCK_JOB_INTERNAL )) { +if (job_id ==NULL && !(flags &JOB_INTERNAL )) { job_id =bdrv_get_device_name (bs); -if (!*job_id) { -error_setg (errp,"An explicit job ID is required for this node" ); -return NULL ; - } - } - -if (job_id) { -if (flags &BLOCK_JOB_INTERNAL ) { -error_setg (errp,"Cannot specify job ID for internal block job" ); -return NULL ; - } } blk =blk_new (perm, shared_perm);
@@ -787,7 +776,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, } job =job_create (job_id, &driver->job_driver,blk_get_aio_context (blk), - errp); + flags, errp);if (job ==NULL ) {blk_unref (blk);return NULL ;
@@ -801,8 +790,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->blk = blk; job->cb = cb; job->opaque = opaque; - job->auto_finalize = !(flags &BLOCK_JOB_MANUAL_FINALIZE ); - job->auto_dismiss = !(flags &BLOCK_JOB_MANUAL_DISMISS );error_setg (&job->blocker,"block device is in use by block job: %s" ,job_type_str (&job->job));
job.c +10 -1
@@ -182,11 +182,15 @@ static void job_sleep_timer_cb(void *opaque) }void *job_create (const char *job_id,const JobDriver *driver,AioContext *ctx, -Error **errp) +int flags,Error **errp) {Job *job;if (job_id) { +if (flags &JOB_INTERNAL ) { +error_setg (errp,"Cannot specify job ID for internal job" ); +return NULL ; + }if (!id_wellformed (job_id)) {error_setg (errp,"Invalid job ID '%s'" , job_id);return NULL ;
@@ -195,6 +199,9 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx,error_setg (errp,"Job ID '%s' already in use" , job_id);return NULL ; } + }else if (!(flags &JOB_INTERNAL )) { +error_setg (errp,"An explicit job ID is required" ); +return NULL ; } job =g_malloc0 (driver->instance_size);
@@ -205,6 +212,8 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, job->busy = false; job->paused = true; job->pause_count =1 ; + job->auto_finalize = !(flags &JOB_MANUAL_FINALIZE ); + job->auto_dismiss = !(flags &JOB_MANUAL_DISMISS );job_state_transition (job,JOB_STATUS_CREATED ); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer,
qemu-img.c +1 -1
@@ -1015,7 +1015,7 @@ static int img_commit(int argc, char **argv) aio_context =bdrv_get_aio_context (bs);aio_context_acquire (aio_context); -commit_active_start ("commit" , bs, base_bs,BLOCK_JOB_DEFAULT ,0 , +commit_active_start ("commit" , bs, base_bs,JOB_DEFAULT ,0 ,BLOCKDEV_ON_ERROR_REPORT ,NULL , common_block_job_cb, &cbi, false, &local_err);aio_context_release (aio_context);
tests/test-blockjob-txn.c +1 -1
@@ -107,7 +107,7 @@ static BlockJob *test_block_job_start(unsigned int iterations,snprintf (job_id,sizeof (job_id),"job%u" , counter++); s =block_job_create (job_id, &test_block_job_driver, txn, bs, -0 ,BLK_PERM_ALL ,0 ,BLOCK_JOB_DEFAULT , +0 ,BLK_PERM_ALL ,0 ,JOB_DEFAULT , test_block_job_cb, data, &error_abort); s->iterations = iterations; s->use_timer = use_timer;
tests/test-blockjob.c +2 -2
@@ -59,7 +59,7 @@ static BlockJob *do_test_id(BlockBackend *blk, const char *id, bool should_succeed) {return mk_job (blk, id, &test_block_job_driver, - should_succeed,BLOCK_JOB_DEFAULT ); + should_succeed,JOB_DEFAULT ); }/* This creates a BlockBackend (optionally with a name) with a
@@ -214,7 +214,7 @@ static CancelJob *create_common(BlockJob **pjob) blk =create_blk (NULL ); job =mk_job (blk,"Steve" , &test_cancel_driver, true, -BLOCK_JOB_MANUAL_FINALIZE |BLOCK_JOB_MANUAL_DISMISS ); +JOB_MANUAL_FINALIZE |JOB_MANUAL_DISMISS );job_ref (&job->job);assert (job->job.status ==JOB_STATUS_CREATED ); s =container_of (job, CancelJob, common);
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 22/42] job: Move BlockJobCreateFlags to Job
Quoted 5 lines · Kevin Wolf
This renames the BlockJobCreateFlags constants, moves a few
JOB_INTERNALchecks tojob_create()and the auto_{finalize,dismiss} fields from BlockJob to Job.Signed-off-by Kevin Wolf <[email protected]>
Quoted 15 lines
--- include/block/blockjob.h | 17 ----------------- include/block/blockjob_int.h | 3 +-- include/qemu/job.h | 20 +++++++++++++++++++- block/commit.c | 2 +- block/mirror.c | 2 +- block/replication.c | 4 ++-- block/stream.c | 2 +- blockdev.c | 14 +++++++------- blockjob.c | 27 +++++++-------------------- job.c | 11 ++++++++++- qemu-img.c | 2 +- tests/test-blockjob-txn.c | 2 +- tests/test-blockjob.c | 4 ++-- 13 files changed, 53 insertions(+), 57 deletions(-)
Re: [Qemu-devel] [PATCH 22/42] job: Move BlockJobCreateFlags to Job
In reply to Kevin WolfQuoted 3 lines · Kevin Wolf
This renames the BlockJobCreateFlags constants, moves a few
JOB_INTERNALchecks tojob_create()and the auto_{finalize,dismiss} fields from BlockJob to Job.
Do we still want to allow auto-finalize on all jobs, or should we keep it just for legacy support on block jobs? Even more so for auto-dismiss (if you use the legacy interface, that's what you expect to happen; but other than for legacy block jobs, any sane management app is going to request auto-dismiss be false, so why expose it to generic jobs?)
Of course, it may still be easiest to plumb up auto- actions in the internal code (where it has to work to keep legacy block jobs from breaking, but no new callers have to enable it), so my argument may not apply until you actually expose a QMP interface for generic jobs.
block/mirror.c void mirror_start(const char *job_id, BlockDriverState *bs, · line 1288
Quoted 10 lines
+++
b/block/mirror.cblock/mirror.c +1
-1@@ -1282,7 +1282,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs, } is_none_mode = mode ==MIRROR_SYNC_MODE_NONE ; base = mode ==MIRROR_SYNC_MODE_TOP ?backing_bs (bs) :NULL ; - mirror_start_job(job_id, bs, BLOCK_JOB_DEFAULT, target, replaces, + mirror_start_job(job_id, bs, JOB_DEFAULT, target, replaces, speed, granularity, buf_size, backing_mode, on_source_error, on_target_error, unmap, NULL, NULL, &mirror_job_driver, is_none_mode, base, false,
Just an observation that this is a lot of parameters; would using boxed QAPI types make these calls any more obvious? But that's a separate cleanup.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 22/42] job: Move BlockJobCreateFlags to Job
Am 16.05.2018 um 21:13 hat Eric Blake geschrieben:
Quoted 11 lines · Kevin Wolf
Quoted 3 lines
This renames the BlockJobCreateFlags constants, moves a few
JOB_INTERNALchecks tojob_create()and the auto_{finalize,dismiss} fields from BlockJob to Job.Do we still want to allow auto-finalize on all jobs, or should we keep it just for legacy support on block jobs? Even more so for auto-dismiss (if you use the legacy interface, that's what you expect to happen; but other than for legacy block jobs, any sane management app is going to request auto-dismiss be false, so why expose it to generic jobs?)
Quoted 4 lines
Of course, it may still be easiest to plumb up auto- actions in the internal code (where it has to work to keep legacy block jobs from breaking, but no new callers have to enable it), so my argument may not apply until you actually expose a QMP interface for generic jobs.
This series doesn't expose it anyway. We can later decide whether we want to add it or not. A sophisticated management tool like libvirt that needs to manage individual nodes and cope with daemon restarts will never make use of them, but they might still be useful in hand-crafted scripts for defined special cases.
Quoted 5 lines
Quoted 5 lines
+++
b/block/mirror.cblock/mirror.c +0
-0@@ -1282,7 +1282,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs, } is_none_mode = mode ==MIRROR_SYNC_MODE_NONE ; base = mode ==MIRROR_SYNC_MODE_TOP ?backing_bs (bs) :NULL ;
Quoted 8 lines
block/mirror.c void mirror_start(const char *job_id, BlockDriverState *bs, · line 1288
Quoted 5 lines
block/mirror.c +1
-1- mirror_start_job(job_id, bs, BLOCK_JOB_DEFAULT, target, replaces, + mirror_start_job(job_id, bs, JOB_DEFAULT, target, replaces, speed, granularity, buf_size, backing_mode, on_source_error, on_target_error, unmap, NULL, NULL, &mirror_job_driver, is_none_mode, base, false,Just an observation that this is a lot of parameters; would using boxed QAPI types make these calls any more obvious? But that's a separate cleanup.
I'm not sure if we have a QAPI type that matches this. But maybe it could become a QAPI struct and very few extra parameters.
Kevin
PATCH 23/42 blockjob: Split block_job_event_pending()
In reply to Kevin Wolfblock_job_event_pending() doesn't only send a QMP event, but it also transitions to the PENDING state. Split the function so that we get one part only sending the event (like other block_job_event_* functions) and another part than does the state transition.
--- blockjob.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-)
blockjob.c +18 -9
@@ -38,7 +38,7 @@static void block_job_event_cancelled (BlockJob *job);static void block_job_event_completed (BlockJob *job,const char *msg); -static int block_job_event_pending (BlockJob *job); +static void block_job_event_pending (BlockJob *job);/* Transactional group of block jobs */ struct BlockJobTxn {
@@ -501,6 +501,15 @@ static void block_job_do_finalize(BlockJob *job) } } +static int block_job_transition_to_pending (BlockJob *job) +{ +job_state_transition (&job->job,JOB_STATUS_PENDING ); +if (!job->job.auto_finalize) { +block_job_event_pending (job); + } +return 0 ; +} +static void block_job_completed_txn_success (BlockJob *job) {BlockJobTxn *txn = job->txn;
@@ -519,7 +528,7 @@ static void block_job_completed_txn_success(BlockJob *job)assert (other_job->ret ==0 ); } -block_job_txn_apply (txn, block_job_event_pending, false); +block_job_txn_apply (txn, block_job_transition_to_pending, false);/* If no jobs need manual finalization, automatically do so */ if (block_job_txn_apply (txn, block_job_needs_finalize, false) ==0 ) {
@@ -734,15 +743,15 @@ static void block_job_event_completed(BlockJob *job, const char *msg) &error_abort); } -static int block_job_event_pending (BlockJob *job) +static void block_job_event_pending (BlockJob *job) { -job_state_transition (&job->job,JOB_STATUS_PENDING ); -if (!job->job.auto_finalize && !block_job_is_internal (job)) { -qapi_event_send_block_job_pending (job_type (&job->job), - job->job.id, - &error_abort); +if (block_job_is_internal (job)) { +return ; } -return 0 ; + +qapi_event_send_block_job_pending (job_type (&job->job), + job->job.id, + &error_abort); }/*
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 23/42] blockjob: Split block_job_event_pending()
Quoted 9 lines · Kevin Wolf
block_job_event_pending()doesn't only send a QMP event, but it also transitions to the PENDING state. Split the function so that we get one part only sending the event (like other block_job_event_* functions) and another part than does the state transition.Signed-off-by Kevin Wolf <[email protected]>--- blockjob.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-)
Re: [Qemu-devel] [PATCH 23/42] blockjob: Split block_job_event_pending()
In reply to Kevin WolfQuoted 4 lines · Kevin Wolf
block_job_event_pending()doesn't only send a QMP event, but it also transitions to the PENDING state. Split the function so that we get one part only sending the event (like other block_job_event_* functions) and another part than does the state transition.
s/than/that/
Quoted 6 lines
Signed-off-by Kevin Wolf <[email protected]>--- blockjob.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 24/42 job: Add job_event_*()
In reply to Kevin WolfGo through the Job layer in order to send QMP events. For the moment, these functions only call a notifier in the BlockJob layer that sends the existing commands.
This uses notifiers rather than JobDriver callbacks because internal users of jobs won't receive QMP events, but might still be interested in getting notified for the events.
--- include/block/blockjob.h | 9 +++++++++ include/qemu/job.h | 18 ++++++++++++++++++ blockjob.c | 42 ++++++++++++++++++++++++++++-------------- job.c | 19 +++++++++++++++++++ 4 files changed, 74 insertions(+), 14 deletions(-)
include/block/blockjob.h +9 -0
@@ -82,6 +82,15 @@ typedef struct BlockJob {/** Block other operations when block job is running */ Error *blocker; +/** Called when a cancelled job is finalised. */ +Notifier finalize_cancelled_notifier; + +/** Called when a successfully completed job is finalised. */ +Notifier finalize_completed_notifier; + +/** Called when the job transitions to PENDING */ +Notifier pending_notifier; +/** BlockDriverStates that are involved in this block job */ GSList *nodes;
include/qemu/job.h +18 -0
@@ -105,6 +105,15 @@ typedef struct Job {/** True if this job should automatically dismiss itself */ bool auto_dismiss; +/** Notifiers called when a cancelled job is finalised */ +NotifierList on_finalize_cancelled; + +/** Notifiers called when a successfully completed job is finalised */ +NotifierList on_finalize_completed; + +/** Notifiers called when the job transitions to PENDING */ +NotifierList on_pending; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; } Job;
@@ -182,6 +191,15 @@ void job_ref(Job *job);*/ void job_unref (Job *job); +/** To be called when a cancelled job is finalised. */ +void job_event_cancelled (Job *job); + +/** To be called when a successfully completed job is finalised. */ +void job_event_completed (Job *job); + +/** To be called when the job transitions to PENDING */ +void job_event_pending (Job *job); +/** * Conditionally enter the job coroutine if the job is ready to run, not * already busy and fn() returns true. fn() is called while under the job_lock
blockjob.c +28 -14
92 lines not on this page · raw message
job.c +19 -0
@@ -215,6 +215,10 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, job->auto_finalize = !(flags &JOB_MANUAL_FINALIZE ); job->auto_dismiss = !(flags &JOB_MANUAL_DISMISS ); +notifier_list_init (&job->on_finalize_cancelled); +notifier_list_init (&job->on_finalize_completed); +notifier_list_init (&job->on_pending); +job_state_transition (job,JOB_STATUS_CREATED ); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, QEMU_CLOCK_REALTIME, SCALE_NS,
@@ -247,6 +251,21 @@ void job_unref(Job *job) } } +void job_event_cancelled (Job *job) +{ +notifier_list_notify (&job->on_finalize_cancelled, job); +} + +void job_event_completed (Job *job) +{ +notifier_list_notify (&job->on_finalize_completed, job); +} + +void job_event_pending (Job *job) +{ +notifier_list_notify (&job->on_pending, job); +} +void job_enter_cond (Job *job,bool (*fn)(Job *job)) {if (!job_started (job)) {
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 24/42] job: Add job_event_*()
Quoted 8 lines · Kevin Wolf
Go through the Job layer in order to send QMP events. For the moment, these functions only call a notifier in the BlockJob layer that sends the existing commands.
This uses notifiers rather than JobDriver callbacks because internal users of jobs won't receive QMP events, but might still be interested in getting notified for the events.
Quoted 7 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 9 +++++++++ include/qemu/job.h | 18 ++++++++++++++++++ blockjob.c | 42 ++++++++++++++++++++++++++++-------------- job.c | 19 +++++++++++++++++++ 4 files changed, 74 insertions(+), 14 deletions(-)
PATCH 25/42 job: Move single job finalisation to Job
In reply to Kevin WolfThis moves the finalisation of a single job from BlockJob to Job.
Some part of this code depends on job transactions, and job transactions call this code, we introduce some temporary calls from Job functions to BlockJob ones. This will be fixed once transactions move to Job, too.
--- include/block/blockjob.h | 9 --- include/block/blockjob_int.h | 36 ----------- include/qemu/job.h | 53 +++++++++++++++- block/backup.c | 22 +++---- block/commit.c | 2 +- block/mirror.c | 2 +- blockjob.c | 142 ++++++++----------------------------------- job.c | 100 +++++++++++++++++++++++++++++- qemu-img.c | 2 +- tests/test-blockjob.c | 10 +-- 10 files changed, 194 insertions(+), 184 deletions(-)
include/block/blockjob.h +0 -9
@@ -76,9 +76,6 @@ typedef struct BlockJob {/** Rate limiting data structure for implementing @speed. */ RateLimit limit; -/** The completion function that will be called when the job completes. */ -BlockCompletionFunc *cb; -/** Block other operations when block job is running */ Error *blocker;
@@ -94,12 +91,6 @@ typedef struct BlockJob {/** BlockDriverStates that are involved in this block job */ GSList *nodes; -/** The opaque value that is passed to the completion function. */ -void *opaque; - -/** ret code passed to block_job_completed. */ -int ret; -BlockJobTxn *txn;QLIST_ENTRY (BlockJob) txn_list; } BlockJob;
include/block/blockjob_int.h +0 -36
@@ -54,34 +54,6 @@ struct BlockJobDriver {*/ int (*prepare)(BlockJob *job); -/** -* If the callback is not NULL, it will be invoked when all the jobs -* belonging to the same transaction complete; or upon this job's -* completion if it is not in a transaction. Skipped if NULL. -* -* All jobs will complete with a call to either .commit() or .abort() but -* never both. -*/ -void (*commit)(BlockJob *job); - -/** -* If the callback is not NULL, it will be invoked when any job in the -* same transaction fails; or upon this job's failure (due to error or -* cancellation) if it is not in a transaction. Skipped if NULL. -* -* All jobs will complete with a call to either .commit() or .abort() but -* never both. -*/ -void (*abort)(BlockJob *job); - -/** -* If the callback is not NULL, it will be invoked after a call to either -* .commit() or .abort(). Regardless of which callback is invoked after -* completion, .clean() will always be called, even if the job does not -* belong to a transaction group. -*/ -void (*clean)(BlockJob *job); -/* * If the callback is not NULL, it will be invoked before the job is * resumed in a new AioContext. This is the place to move any resources
@@ -156,14 +128,6 @@ void block_job_yield(BlockJob *job);int64_t block_job_ratelimit_get_delay (BlockJob *job,uint64_t n);/** -* block_job_early_fail: -* @bs: The block device. -* -* The block job could not be started, free it. -*/ -void block_job_early_fail (BlockJob *job); - -/** * block_job_completed: * @job: The job being completed. * @ret: The status code.
include/qemu/job.h +52 -1
91 lines not on this page · raw message
block/backup.c +11 -11
@@ -207,25 +207,25 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) } } -static void backup_commit (BlockJob *job) +static void backup_commit (Job *job) { -BackupBlockJob *s =container_of (job, BackupBlockJob, common); +BackupBlockJob *s =container_of (job, BackupBlockJob, common.job);if (s->sync_bitmap) {backup_cleanup_sync_bitmap (s,0 ); } } -static void backup_abort (BlockJob *job) +static void backup_abort (Job *job) { -BackupBlockJob *s =container_of (job, BackupBlockJob, common); +BackupBlockJob *s =container_of (job, BackupBlockJob, common.job);if (s->sync_bitmap) {backup_cleanup_sync_bitmap (s,-1 ); } } -static void backup_clean (BlockJob *job) +static void backup_clean (Job *job) { -BackupBlockJob *s =container_of (job, BackupBlockJob, common); +BackupBlockJob *s =container_of (job, BackupBlockJob, common.job);assert (s->target);blk_unref (s->target); s->target =NULL ;
@@ -530,10 +530,10 @@ static const BlockJobDriver backup_job_driver = { .free = block_job_free, .user_resume = block_job_user_resume, .start = backup_run, + .commit = backup_commit, + .abort = backup_abort, + .clean = backup_clean, }, - .commit = backup_commit, - .abort = backup_abort, - .clean = backup_clean, .attached_aio_context = backup_attached_aio_context, .drain = backup_drain, };
@@ -678,8 +678,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,bdrv_reclaim_dirty_bitmap (bs, sync_bitmap,NULL ); }if (job) { -backup_clean (&job->common); -block_job_early_fail (&job->common); +backup_clean (&job->common.job); +job_early_fail (&job->common.job); }return NULL ;
block/commit.c +1 -1
@@ -385,7 +385,7 @@ fail:if (commit_top_bs) {bdrv_replace_node (commit_top_bs, top, &error_abort); } -block_job_early_fail (&s->common); +job_early_fail (&s->common.job); }
block/mirror.c +1 -1
@@ -1255,7 +1255,7 @@ fail:g_free (s->replaces);blk_unref (s->target); -block_job_early_fail (&s->common); +job_early_fail (&s->common.job); } bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL,
blockjob.c +25 -117
233 lines not on this page · raw message
job.c +98 -2
124 lines not on this page · raw message
qemu-img.c +1 -1
@@ -872,7 +872,7 @@ static void run_block_job(BlockJob *job, Error **errp)if (!job_is_completed (&job->job)) { ret =block_job_complete_sync (job, errp); }else { - ret = job->ret; + ret = job->job.ret; }job_unref (&job->job);aio_context_release (aio_context);
tests/test-blockjob.c +5 -5
@@ -128,11 +128,11 @@ static void test_job_ids(void) job[1 ] =do_test_id (blk[1 ],"id0" , false);/* But once job[0] finishes we can reuse its ID */ -block_job_early_fail (job[0 ]); +job_early_fail (&job[0 ]->job); job[1 ] =do_test_id (blk[1 ],"id0" , true);/* No job ID specified, defaults to the backend name ('drive1') */ -block_job_early_fail (job[1 ]); +job_early_fail (&job[1 ]->job); job[1 ] =do_test_id (blk[1 ],NULL , true);/* Duplicate job ID */
@@ -145,9 +145,9 @@ static void test_job_ids(void)/* This one is valid */ job[2 ] =do_test_id (blk[2 ],"id_2" , true); -block_job_early_fail (job[0 ]); -block_job_early_fail (job[1 ]); -block_job_early_fail (job[2 ]); +job_early_fail (&job[0 ]->job); +job_early_fail (&job[1 ]->job); +job_early_fail (&job[2 ]->job);destroy_blk (blk[0 ]);destroy_blk (blk[1 ]);
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 25/42] job: Move single job finalisation to Job
Quoted 19 lines · Kevin Wolf
This moves the finalisation of a single job from BlockJob to Job.
Some part of this code depends on job transactions, and job transactions call this code, we introduce some temporary calls from Job functions to BlockJob ones. This will be fixed once transactions move to Job, too.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 9 --- include/block/blockjob_int.h | 36 ----------- include/qemu/job.h | 53 +++++++++++++++- block/backup.c | 22 +++---- block/commit.c | 2 +- block/mirror.c | 2 +- blockjob.c | 142 ++++++++----------------------------------- job.c | 100 +++++++++++++++++++++++++++++- qemu-img.c | 2 +- tests/test-blockjob.c | 10 +-- 10 files changed, 194 insertions(+), 184 deletions(-)
[...]
Quoted 13 lines
include/qemu/job.h +1
-0@@ -29,6 +29,7 @@#include "qapi/qapi-types-block-core.h" #include "qemu/queue.h" #include "qemu/coroutine.h" +#include "block/aio.h" typedef struct JobDriver JobDriver ;
include/qemu/job.h typedef struct Job { · line 110
Quoted 5 lines
include/qemu/job.h +2
-0@@ -105,6 +106,15 @@ typedef struct Job {/** True if this job should automatically dismiss itself */ bool auto_dismiss; +/** ret code passed to block_job_completed. */ +int ret;
While there is no such function yet, you might not want to mention a block_job_* function here anyway.
Apart from that, as far as I can see, this comment remains unchanged even later in this series when block_job_completed() is removed.
So something needs to be changed somewhere, so here is an anticipating
PATCH 26/42 job: Convert block_job_cancel_async() to Job
In reply to Kevin Wolfblock_job_cancel_async() did two things that were still block job specific:
* Setting job->force. This field makes sense on the Job level, so we can just move it. While at it, rename it to job->force_cancel to make its purpose more obvious.
* Resetting the I/O status. This can't be moved because generic Jobs don't have an I/O status. What the function really implements is a user resume, except without entering the coroutine. Consequently, it makes sense to call the .user_resume driver callback here which already resets the I/O status.
The old block_job_cancel_async() has two separate if statements that check job->iostatus != BLOCK_DEVICE_IO_STATUS_OK and job->user_paused. However, the former condition always implies the latter (as is asserted in block_job_iostatus_reset()), so changing the explicit call of block_job_iostatus_reset() on the former condition with the .user_resume callback on the latter condition is equivalent and doesn't need to access any BlockJob specific state.
--- include/block/blockjob.h | 6 ------ include/qemu/job.h | 6 ++++++ block/mirror.c | 4 ++-- blockjob.c | 25 +++++++++++++------------ 4 files changed, 21 insertions(+), 20 deletions(-)
include/block/blockjob.h +0 -6
@@ -51,12 +51,6 @@ typedef struct BlockJob {BlockBackend *blk;/** -* Set to true if the job should abort immediately without waiting -* for data to be in sync. -*/ -bool force; - -/** * Set to true when the job is ready to be completed. */ bool ready;
include/qemu/job.h +6 -0
@@ -97,6 +97,12 @@ typedef struct Job {*/ bool cancelled; +/** +* Set to true if the job should abort immediately without waiting +* for data to be in sync. +*/ +bool force_cancel; +/** Set to true when the job has deferred work to the main loop. */ bool deferred_to_main_loop;
block/mirror.c +2 -2
@@ -871,7 +871,7 @@ static void coroutine_fn mirror_run(void *opaque)trace_mirror_before_sleep (s, cnt, s->synced, delay_ns);job_sleep_ns (&s->common.job, delay_ns);if (job_is_cancelled (&s->common.job) && - (!s->synced || s->common.force)) + (!s->synced || s->common.job.force_cancel)) {break ; }
@@ -884,7 +884,7 @@ immediate_exit:* or it was cancelled prematurely so that we do not guarantee that * the target is a copy of the source. */ -assert (ret <0 || ((s->common.force || !s->synced) && +assert (ret <0 || ((s->common.job.force_cancel || !s->synced) &&job_is_cancelled (&s->common.job)));assert (need_drain);mirror_wait_for_all_io (s);
blockjob.c +13 -12
@@ -271,19 +271,20 @@ static int block_job_prepare(BlockJob *job)return job->job.ret; } -static void block_job_cancel_async (BlockJob *job,bool force) +static void job_cancel_async (Job *job,bool force) { -if (job->iostatus !=BLOCK_DEVICE_IO_STATUS_OK ) { -block_job_iostatus_reset (job); - } -if (job->job.user_paused) { -/* Do not call block_job_enter here, the caller will handle it. */ - job->job.user_paused = false; - job->job.pause_count--; +if (job->user_paused) { +/* Do not call job_enter here, the caller will handle it. */ + job->user_paused = false; +if (job->driver->user_resume) { + job->driver->user_resume (job); + } +assert (job->pause_count >0 ); + job->pause_count--; } - job->job.cancelled = true; + job->cancelled = true;/* To prevent 'force == false' overriding a previous 'force == true' */ - job->force |= force; + job->force_cancel |= force; }static int block_job_txn_apply (BlockJobTxn *txn,int fn (BlockJob *),bool lock)
@@ -368,7 +369,7 @@ static void block_job_completed_txn_abort(BlockJob *job)* on the caller, so leave it. */ QLIST_FOREACH(other_job, &txn->jobs, txn_list) {if (other_job != job) { -block_job_cancel_async (other_job, false); +job_cancel_async (&other_job->job, false); } }while (!QLIST_EMPTY (&txn->jobs)) {
@@ -528,7 +529,7 @@ void block_job_cancel(BlockJob *job, bool force)job_do_dismiss (&job->job);return ; } -block_job_cancel_async (job, force); +job_cancel_async (&job->job, force);if (!job_started (&job->job)) {block_job_completed (job, -ECANCELED ); }else if (job->job.deferred_to_main_loop) {
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 26/42] job: Convert block_job_cancel_async() to Job
Quoted 21 lines · Kevin Wolf
block_job_cancel_async()did two things that were still block job specific:* Setting
job->force. This field makes sense on the Job level, so we can just move it. While at it, rename it tojob->force_cancelto make its purpose more obvious.* Resetting the I/O status. This can't be moved because generic Jobs don't have an I/O status. What the function really implements is a user resume, except without entering the coroutine. Consequently, it makes sense to call the .user_resume driver callback here which already resets the I/O status.
The old
block_job_cancel_async()has two separate if statements that checkjob->iostatus!=BLOCK_DEVICE_IO_STATUS_OKandjob->user_paused. However, the former condition always implies the latter (as is asserted inblock_job_iostatus_reset()), so changing the explicit call ofblock_job_iostatus_reset()on the former condition with the .user_resume callback on the latter condition is equivalent and doesn't need to access any BlockJob specific state.
Quoted 7 lines
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 6 ------ include/qemu/job.h | 6 ++++++ block/mirror.c | 4 ++-- blockjob.c | 25 +++++++++++++------------ 4 files changed, 21 insertions(+), 20 deletions(-)
I'm not quite sure why you keep this function in blockjob.c, when you've previously moved such static functions over to job.c and made them temporarily public (e.g. job_state_transition()).
But I don't really care either way, in fact keeping the function in the same file makes reviewing easier for me, so:
PATCH 27/42 job: Add job_drain()
In reply to Kevin Wolfblock_job_drain() contains a blk_drain() call which cannot be moved to Job, so add a new JobDriver callback JobDriver.drain which has a common implementation for all BlockJobs. In addition to this we keep the existing BlockJobDriver.drain callback that is called by the common drain implementation for all block jobs.
--- include/block/blockjob_int.h | 12 ++++++++++++ include/qemu/job.h | 13 +++++++++++++ block/backup.c | 1 + block/commit.c | 1 + block/mirror.c | 2 ++ block/stream.c | 1 + blockjob.c | 20 ++++++++++---------- job.c | 11 +++++++++++ tests/test-bdrv-drain.c | 1 + tests/test-blockjob-txn.c | 1 + tests/test-blockjob.c | 2 ++ 11 files changed, 55 insertions(+), 10 deletions(-)
include/block/blockjob_int.h +12 -0
@@ -65,6 +65,10 @@ struct BlockJobDriver {* If the callback is not NULL, it will be invoked when the job has to be * synchronously cancelled or completed; it should drain BlockDriverStates * as required to ensure progress. +* +* Block jobs must use the default implementation for job_driver.drain, +* which will in turn call this callback after doing generic block job +* stuff. */ void (*drain)(BlockJob *job); };
@@ -112,6 +116,14 @@ void block_job_free(Job *job);void block_job_user_resume (Job *job);/** +* block_job_drain: +* Callback to be used for JobDriver.drain in all block jobs. Drains the main +* block node associated with the block jobs and calls BlockJobDriver.drain for +* job-specific actions. +*/ +void block_job_drain (Job *job); + +/** * block_job_yield: * @job: The job that calls the function. *
include/qemu/job.h +13 -0
@@ -167,6 +167,13 @@ struct JobDriver {*/ void (*user_resume)(Job *job); +/* +* If the callback is not NULL, it will be invoked when the job has to be +* synchronously cancelled or completed; it should drain any activities +* as required to ensure progress. +*/ +void (*drain)(Job *job); +/** * If the callback is not NULL, it will be invoked when all the jobs * belonging to the same transaction complete; or upon this job's
@@ -325,6 +332,12 @@ bool job_user_paused(Job *job);*/ void job_user_resume (Job *job,Error **errp); +/* +* Drain any activities as required to ensure progress. This can be called in a +* loop to synchronously complete a job. +*/ +void job_drain (Job *job); +/** * Get the next element from the list of block jobs after @job, or the * first one if @job is %NULL.
block/backup.c +1 -0
@@ -529,6 +529,7 @@ static const BlockJobDriver backup_job_driver = { .job_type = JOB_TYPE_BACKUP, .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = backup_run, .commit = backup_commit, .abort = backup_abort,
block/commit.c +1 -0
@@ -221,6 +221,7 @@ static const BlockJobDriver commit_job_driver = { .job_type = JOB_TYPE_COMMIT, .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = commit_run, }, };
block/mirror.c +2 -0
@@ -992,6 +992,7 @@ static const BlockJobDriver mirror_job_driver = { .job_type = JOB_TYPE_MIRROR, .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = mirror_run, .pause = mirror_pause, },
@@ -1006,6 +1007,7 @@ static const BlockJobDriver commit_active_job_driver = { .job_type = JOB_TYPE_COMMIT, .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = mirror_run, .pause = mirror_pause, },
block/stream.c +1 -0
@@ -215,6 +215,7 @@ static const BlockJobDriver stream_job_driver = { .free = block_job_free, .start = stream_run, .user_resume = block_job_user_resume, + .drain = block_job_drain, }, };
blockjob.c +10 -10
@@ -170,14 +170,13 @@ static void block_job_attached_aio_context(AioContext *new_context,job_resume (&job->job); } -static void block_job_drain (BlockJob *job) +void block_job_drain (Job *job) { -/* If job is !job->job.busy this kicks it into the next pause point. */ -block_job_enter (job); +BlockJob *bjob =container_of (job, BlockJob, job); -blk_drain (job->blk); -if (job->driver->drain) { - job->driver->drain (job); +blk_drain (bjob->blk); +if (bjob->driver->drain) { + bjob->driver->drain (bjob); } }
@@ -191,7 +190,7 @@ static void block_job_detach_aio_context(void *opaque)job_pause (&job->job);while (!job->job.paused && !job_is_completed (&job->job)) { -block_job_drain (job); +job_drain (&job->job); } job->job.aio_context =NULL ;
@@ -328,11 +327,11 @@ static int block_job_finish_sync(BlockJob *job,job_unref (&job->job);return -EBUSY ; } -/* block_job_drain calls block_job_enter, and it should be enough to -* induce progress until the job completes or moves to the main thread. +/* job_drain calls job_enter, and it should be enough to induce progress +* until the job completes or moves to the main thread. */ while (!job->job.deferred_to_main_loop && !job_is_completed (&job->job)) { -block_job_drain (job); +job_drain (&job->job); }while (!job_is_completed (&job->job)) {aio_poll (qemu_get_aio_context (), true);
@@ -714,6 +713,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,assert (is_block_job (&job->job));assert (job->job.driver->free == &block_job_free);assert (job->job.driver->user_resume == &block_job_user_resume); +assert (job->job.driver->drain == &block_job_drain); job->driver = driver; job->blk = blk;
job.c +11 -0
@@ -367,6 +367,17 @@ void coroutine_fn job_sleep_ns(Job *job, int64_t ns)job_pause_point (job); } +void job_drain (Job *job) +{ +/* If job is !busy this kicks it into the next pause point. */ +job_enter (job); + +if (job->driver->drain) { + job->driver->drain (job); + } +} + +/** * All jobs must allow a pause point before entering their job proper. This * ensures that jobs can be paused prior to being started, then resumed later.
tests/test-bdrv-drain.c +1 -0
@@ -525,6 +525,7 @@ BlockJobDriver test_job_driver = { .instance_size = sizeof(TestBlockJob), .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = test_job_start, }, .complete = test_job_complete,
tests/test-blockjob-txn.c +1 -0
@@ -79,6 +79,7 @@ static const BlockJobDriver test_block_job_driver = { .instance_size = sizeof(TestBlockJob), .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = test_block_job_run, }, };
tests/test-blockjob.c +2 -0
@@ -21,6 +21,7 @@ static const BlockJobDriver test_block_job_driver = { .instance_size = sizeof(BlockJob), .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, }, };
@@ -201,6 +202,7 @@ static const BlockJobDriver test_cancel_driver = { .instance_size = sizeof(CancelJob), .free = block_job_free, .user_resume = block_job_user_resume, + .drain = block_job_drain, .start = cancel_job_start, }, .complete = cancel_job_complete,
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 27/42] job: Add job_drain()
Quoted 26 lines · Kevin Wolf
block_job_drain()contains ablk_drain()call which cannot be moved to Job, so add a new JobDriver callback JobDriver.drain which has a common implementation for all BlockJobs. In addition to this we keep the existing BlockJobDriver.drain callback that is called by the common drain implementation for all block jobs.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 12 ++++++++++++ include/qemu/job.h | 13 +++++++++++++ block/backup.c | 1 + block/commit.c | 1 + block/mirror.c | 2 ++ block/stream.c | 1 + blockjob.c | 20 ++++++++++---------- job.c | 11 +++++++++++ tests/test-bdrv-drain.c | 1 + tests/test-blockjob-txn.c | 1 + tests/test-blockjob.c | 2 ++ 11 files changed, 55 insertions(+), 10 deletions(-)include/block/blockjob_int.h +0
-0
include/block/blockjob_int.h struct BlockJobDriver { · line 73
Quoted 9 lines
include/block/blockjob_int.h +4
-0@@ -65,6 +65,10 @@ struct BlockJobDriver {* If the callback is not NULL, it will be invoked when the job has to be * synchronously cancelled or completed; it should drain BlockDriverStates * as required to ensure progress. +* +* Block jobs must use the default implementation for job_driver.drain, +* which will in turn call this callback after doing generic block job +* stuff. */ void (*drain)(BlockJob *job);
I don't really see the point of having two drain callbacks for block jobs. Well, it allows an assert() that block_job_drain() is called at some point, but still. I'd like block jobs to be not very special, but this makes them a bit more special than they need to be.
Maybe I'd like it a bit more if there was a macro to automatically set these mandatory values for block jobs...
But mostly a question of style, so I'll grudgingly give a:
Quoted 1 line
include/block/blockjob_int.h +0
-0};
PATCH 28/42 job: Move .complete callback to Job
In reply to Kevin WolfThis moves the .complete callback that tells a READY job to complete from BlockJobDriver to JobDriver. The wrapper function job_complete() doesn't require anything block job specific any more and can be moved to Job.
--- include/block/blockjob.h | 10 ---------- include/block/blockjob_int.h | 6 ------ include/qemu/job.h | 8 ++++++++ block/mirror.c | 10 +++++----- blockdev.c | 2 +- blockjob.c | 23 +++++------------------ job.c | 16 ++++++++++++++++ tests/test-bdrv-drain.c | 6 +++--- tests/test-blockjob.c | 10 +++++----- 9 files changed, 43 insertions(+), 48 deletions(-)
include/block/blockjob.h +0 -10
@@ -154,16 +154,6 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp);void block_job_cancel (BlockJob *job,bool force);/** -* block_job_complete: -* @job: The job to be completed. -* @errp: Error object. -* -* Asynchronously complete the specified job. -*/ -void block_job_complete (BlockJob *job,Error **errp); - - -/** * block_job_finalize: * @job: The job to fully commit and finish. * @errp: Error object.
include/block/blockjob_int.h +0 -6
@@ -39,12 +39,6 @@ struct BlockJobDriver {JobDriver job_driver;/** -* Optional callback for job types whose completion must be triggered -* manually. -*/ -void (*complete)(BlockJob *job,Error **errp); - -/** * If the callback is not NULL, prepare will be invoked when all the jobs * belonging to the same transaction complete; or upon this job's completion * if it is not in a transaction.
include/qemu/job.h +8 -0
@@ -167,6 +167,12 @@ struct JobDriver {*/ void (*user_resume)(Job *job); +/** +* Optional callback for job types whose completion must be triggered +* manually. +*/ +void (*complete)(Job *job,Error **errp); +/* * If the callback is not NULL, it will be invoked when the job has to be * synchronously cancelled or completed; it should drain any activities
@@ -363,6 +369,8 @@ int job_apply_verb(Job *job, JobVerb bv, Error **errp);/** The @job could not be started, free it. */ void job_early_fail (Job *job); +/** Asynchronously complete the specified @job. */ +void job_complete (Job *job,Error **errp);;typedef void JobDeferToMainLoopFn (Job *job,void *opaque);
block/mirror.c +5 -5
@@ -905,16 +905,16 @@ immediate_exit:job_defer_to_main_loop (&s->common.job, mirror_exit, data); } -static void mirror_complete (BlockJob *job,Error **errp) +static void mirror_complete (Job *job,Error **errp) { -MirrorBlockJob *s =container_of (job, MirrorBlockJob, common); +MirrorBlockJob *s =container_of (job, MirrorBlockJob, common.job);BlockDriverState *target; target =blk_bs (s->target);if (!s->synced) {error_setg (errp,"The active block job '%s' cannot be completed" , - job->job.id); + job->id);return ; }
@@ -995,8 +995,8 @@ static const BlockJobDriver mirror_job_driver = { .drain = block_job_drain, .start = mirror_run, .pause = mirror_pause, + .complete = mirror_complete, }, - .complete = mirror_complete, .attached_aio_context = mirror_attached_aio_context, .drain = mirror_drain, };
@@ -1010,8 +1010,8 @@ static const BlockJobDriver commit_active_job_driver = { .drain = block_job_drain, .start = mirror_run, .pause = mirror_pause, + .complete = mirror_complete, }, - .complete = mirror_complete, .attached_aio_context = mirror_attached_aio_context, .drain = mirror_drain, };
blockdev.c +1 -1
@@ -3894,7 +3894,7 @@ void qmp_block_job_complete(const char *device, Error **errp) }trace_qmp_block_job_complete (job); -block_job_complete (job, errp); +job_complete (&job->job, errp);aio_context_release (aio_context); }
blockjob.c +5 -18
@@ -482,24 +482,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n)return ratelimit_calculate_delay (&job->limit, n); } -void block_job_complete (BlockJob *job,Error **errp) -{ -/* Should not be reachable via external interface for internal jobs */ -assert (job->job.id); -if (job_apply_verb (&job->job,JOB_VERB_COMPLETE , errp)) { -return ; - } -if (job->job.pause_count ||job_is_cancelled (&job->job) || - !job->driver->complete) - { -error_setg (errp,"The active block job '%s' cannot be completed" , - job->job.id); -return ; - } - - job->driver->complete (job, errp); -} -void block_job_finalize (BlockJob *job,Error **errp) {assert (job && job->job.id);
@@ -572,6 +554,11 @@ void block_job_cancel_sync_all(void) } } +static void block_job_complete (BlockJob *job,Error **errp) +{ +job_complete (&job->job, errp); +} +int block_job_complete_sync (BlockJob *job,Error **errp) {return block_job_finish_sync (job, &block_job_complete, errp);
job.c +16 -0
@@ -548,6 +548,22 @@ int job_finalize_single(Job *job)return 0 ; } +void job_complete (Job *job,Error **errp) +{ +/* Should not be reachable via external interface for internal jobs */ +assert (job->id); +if (job_apply_verb (job,JOB_VERB_COMPLETE , errp)) { +return ; + } +if (job->pause_count ||job_is_cancelled (job) || !job->driver->complete) { +error_setg (errp,"The active block job '%s' cannot be completed" , + job->id); +return ; + } + + job->driver->complete (job, errp); +} + typedef struct { Job *job;
tests/test-bdrv-drain.c +3 -3
@@ -514,9 +514,9 @@ static void coroutine_fn test_job_start(void *opaque)job_defer_to_main_loop (&s->common.job, test_job_completed,NULL ); } -static void test_job_complete (BlockJob *job,Error **errp) +static void test_job_complete (Job *job,Error **errp) { -TestBlockJob *s =container_of (job, TestBlockJob, common); +TestBlockJob *s =container_of (job, TestBlockJob, common.job); s->should_complete = true; }
@@ -527,8 +527,8 @@ BlockJobDriver test_job_driver = { .user_resume = block_job_user_resume, .drain = block_job_drain, .start = test_job_start, + .complete = test_job_complete, }, - .complete = test_job_complete, };static void test_blockjob_common (enum drain_type drain_type)
tests/test-blockjob.c +5 -5
@@ -171,9 +171,9 @@ static void cancel_job_completed(Job *job, void *opaque)block_job_completed (bjob,0 ); } -static void cancel_job_complete (BlockJob *job,Error **errp) +static void cancel_job_complete (Job *job,Error **errp) { -CancelJob *s =container_of (job, CancelJob, common); +CancelJob *s =container_of (job, CancelJob, common.job); s->should_complete = true; }
@@ -204,8 +204,8 @@ static const BlockJobDriver test_cancel_driver = { .user_resume = block_job_user_resume, .drain = block_job_drain, .start = cancel_job_start, + .complete = cancel_job_complete, }, - .complete = cancel_job_complete, };static CancelJob *create_common (BlockJob **pjob)
@@ -333,7 +333,7 @@ static void test_cancel_pending(void)block_job_enter (job);assert (job->job.status ==JOB_STATUS_READY ); -block_job_complete (job, &error_abort); +job_complete (&job->job, &error_abort);block_job_enter (job);while (!s->completed) {aio_poll (qemu_get_aio_context (), true);
@@ -357,7 +357,7 @@ static void test_cancel_concluded(void)block_job_enter (job);assert (job->job.status ==JOB_STATUS_READY ); -block_job_complete (job, &error_abort); +job_complete (&job->job, &error_abort);block_job_enter (job);while (!s->completed) {aio_poll (qemu_get_aio_context (), true);
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 28/42] job: Move .complete callback to Job
Quoted 6 lines · Kevin Wolf
This moves the .complete callback that tells a READY job to complete from BlockJobDriver to JobDriver. The wrapper function
job_complete()doesn't require anything block job specific any more and can be moved to Job.Signed-off-by Kevin Wolf <[email protected]>
Quoted 11 lines
--- include/block/blockjob.h | 10 ---------- include/block/blockjob_int.h | 6 ------ include/qemu/job.h | 8 ++++++++ block/mirror.c | 10 +++++----- blockdev.c | 2 +- blockjob.c | 23 +++++------------------ job.c | 16 ++++++++++++++++ tests/test-bdrv-drain.c | 6 +++--- tests/test-blockjob.c | 10 +++++----- 9 files changed, 43 insertions(+), 48 deletions(-)
PATCH 29/42 job: Move job_finish_sync() to Job
In reply to Kevin Wolfblock_job_finish_sync() doesn't contain anything block job specific any more, so it can be moved to Job.
--- include/qemu/job.h | 9 +++++++++ blockjob.c | 55 +++++++++--------------------------------------------- job.c | 28 +++++++++++++++++++++++++++ 3 files changed, 46 insertions(+), 46 deletions(-)
include/qemu/job.h +9 -0
@@ -389,6 +389,15 @@ typedef void JobDeferToMainLoopFn(Job *job, void *opaque);*/ void job_defer_to_main_loop (Job *job,JobDeferToMainLoopFn *fn,void *opaque); +/** +* Synchronously finishes the given @job. If @finish is given, it is called to +* trigger completion or cancellation of the job. +* +* Returns 0 if the job is successfully completed, -ECANCELED if the job was +* cancelled before completing, and -errno in other error cases. +*/ +int job_finish_sync (Job *job,void (*finish)(Job *,Error **errp),Error **errp); +/* TODO To be removed from the public interface */ void job_state_transition (Job *job,JobStatus s1); void coroutine_fn job_do_yield(Job *job, uint64_t ns);
blockjob.c +9 -46
86 lines not on this page · raw message
job.c +28 -0
@@ -596,3 +596,31 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque)aio_bh_schedule_oneshot (qemu_get_aio_context (), job_defer_to_main_loop_bh, data); } + +int job_finish_sync (Job *job,void (*finish)(Job *,Error **errp),Error **errp) +{ +Error *local_err =NULL ; +int ret; + +job_ref (job); + +if (finish) { +finish (job, &local_err); + } +if (local_err) { +error_propagate (errp, local_err); +job_unref (job); +return -EBUSY ; + } +/* job_drain calls job_enter, and it should be enough to induce progress +* until the job completes or moves to the main thread. */ +while (!job->deferred_to_main_loop && !job_is_completed (job)) { +job_drain (job); + } +while (!job_is_completed (job)) { +aio_poll (qemu_get_aio_context (), true); + } + ret = (job_is_cancelled (job) && job->ret ==0 ) ? -ECANCELED : job->ret; +job_unref (job); +return ret; +}
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 29/42] job: Move job_finish_sync() to Job
Quoted 9 lines · Kevin Wolf
block_job_finish_sync()doesn't contain anything block job specific any more, so it can be moved to Job.Signed-off-by Kevin Wolf <[email protected]>--- include/qemu/job.h | 9 +++++++++ blockjob.c | 55 +++++++++--------------------------------------------- job.c | 28 +++++++++++++++++++++++++++ 3 files changed, 46 insertions(+), 46 deletions(-)
PATCH 30/42 job: Switch transactions to JobTxn
In reply to Kevin WolfThis doesn't actually move any transaction code to Job yet, but it renames the type for transactions from BlockJobTxn to JobTxn and makes them contain Jobs rather than BlockJobs
--- include/block/block_int.h | 2 +- include/block/blockjob.h | 11 ++++---- include/block/blockjob_int.h | 2 +- include/qemu/job.h | 3 +++ block/backup.c | 2 +- blockdev.c | 14 +++++------ blockjob.c | 60 +++++++++++++++++++++++--------------------- tests/test-blockjob-txn.c | 8 +++--- 8 files changed, 54 insertions(+), 48 deletions(-)
include/block/block_int.h +1 -1
@@ -1015,7 +1015,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,BlockdevOnError on_target_error, int creation_flags, BlockCompletionFunc *cb, void *opaque, - BlockJobTxn *txn, Error **errp); + JobTxn *txn, Error **errp);void hmp_drive_add_node (Monitor *mon,const char *optstr);
include/block/blockjob.h +5 -6
@@ -33,7 +33,7 @@#define SLICE_TIME 100000000ULL/* ns */ typedef struct BlockJobDriver BlockJobDriver ; -typedef struct BlockJobTxn BlockJobTxn ; +typedef struct JobTxn JobTxn ;/** * BlockJob:
@@ -85,8 +85,7 @@ typedef struct BlockJob {/** BlockDriverStates that are involved in this block job */ GSList *nodes; -BlockJobTxn *txn; -QLIST_ENTRY (BlockJob) txn_list; +JobTxn *txn; } BlockJob;/**
@@ -273,7 +272,7 @@ void block_job_iostatus_reset(BlockJob *job);* group. Jobs wait for each other before completing. Cancelling one job * cancels all jobs in the transaction. */ -BlockJobTxn *block_job_txn_new (void ); +JobTxn *block_job_txn_new (void );/** * block_job_txn_unref:
@@ -282,7 +281,7 @@ BlockJobTxn *block_job_txn_new(void);* or block_job_txn_new. If it's the last reference to the object, it will be * freed. */ -void block_job_txn_unref (BlockJobTxn *txn); +void block_job_txn_unref (JobTxn *txn);/** * block_job_txn_add_job:
@@ -293,7 +292,7 @@ void block_job_txn_unref(BlockJobTxn *txn);* The caller must call either block_job_txn_unref() or block_job_completed() * to release the reference that is automatically grabbed here. */ -void block_job_txn_add_job (BlockJobTxn *txn,BlockJob *job); +void block_job_txn_add_job (JobTxn *txn,BlockJob *job);/** * block_job_is_internal:
include/block/blockjob_int.h +1 -1
@@ -91,7 +91,7 @@ struct BlockJobDriver {* called from a wrapper that is specific to the job type. */ void *block_job_create (const char *job_id,const BlockJobDriver *driver, -BlockJobTxn *txn,BlockDriverState *bs,uint64_t perm, +JobTxn *txn,BlockDriverState *bs,uint64_t perm,uint64_t shared_perm,int64_t speed,int flags,BlockCompletionFunc *cb,void *opaque,Error **errp);
include/qemu/job.h +3 -0
@@ -132,6 +132,9 @@ typedef struct Job {/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list; + +/** Element of the list of jobs in a job transaction */ +QLIST_ENTRY (Job) txn_list; } Job;/**
block/backup.c +1 -1
@@ -547,7 +547,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,BlockdevOnError on_target_error, int creation_flags, BlockCompletionFunc *cb, void *opaque, - BlockJobTxn *txn, Error **errp) + JobTxn *txn, Error **errp) { int64_t len;BlockDriverInfo bdi;
blockdev.c +7 -7
@@ -1446,7 +1446,7 @@ typedef struct BlkActionOps {struct BlkActionState {TransactionAction *action;const BlkActionOps *ops; -BlockJobTxn *block_job_txn; +JobTxn *block_job_txn;TransactionProperties *txn_props;QSIMPLEQ_ENTRY (BlkActionState) entry; };
@@ -1864,7 +1864,7 @@ typedef struct DriveBackupState {BlockJob *job; } DriveBackupState; -static BlockJob *do_drive_backup (DriveBackup *backup,BlockJobTxn *txn, +static BlockJob *do_drive_backup (DriveBackup *backup,JobTxn *txn,Error **errp);static void drive_backup_prepare (BlkActionState *common,Error **errp)
@@ -1954,7 +1954,7 @@ typedef struct BlockdevBackupState {BlockJob *job; } BlockdevBackupState; -static BlockJob *do_blockdev_backup (BlockdevBackup *backup,BlockJobTxn *txn, +static BlockJob *do_blockdev_backup (BlockdevBackup *backup,JobTxn *txn,Error **errp);static void blockdev_backup_prepare (BlkActionState *common,Error **errp)
@@ -2243,7 +2243,7 @@ void qmp_transaction(TransactionActionList *dev_list, Error **errp) {TransactionActionList *dev_entry = dev_list; -BlockJobTxn *block_job_txn =NULL ; +JobTxn *block_job_txn =NULL ;BlkActionState *state, *next;Error *local_err =NULL ;
@@ -2251,7 +2251,7 @@ void qmp_transaction(TransactionActionList *dev_list,QSIMPLEQ_INIT (&snap_bdrv_states);/* Does this transaction get canceled as a group on failure? -* If not, we don't really need to make a BlockJobTxn. +* If not, we don't really need to make a JobTxn. */ props =get_transaction_properties (props);if (props->completion_mode !=ACTION_COMPLETION_MODE_INDIVIDUAL ) {
@@ -3264,7 +3264,7 @@ out:aio_context_release (aio_context); } -static BlockJob *do_drive_backup (DriveBackup *backup,BlockJobTxn *txn, +static BlockJob *do_drive_backup (DriveBackup *backup,JobTxn *txn,Error **errp) {BlockDriverState *bs;
@@ -3434,7 +3434,7 @@ BlockDeviceInfoList *qmp_query_named_block_nodes(Error **errp)return bdrv_named_nodes_list (errp); } -BlockJob *do_blockdev_backup (BlockdevBackup *backup,BlockJobTxn *txn, +BlockJob *do_blockdev_backup (BlockdevBackup *backup,JobTxn *txn,Error **errp) {BlockDriverState *bs;
blockjob.c +32 -28
162 lines not on this page · raw message
tests/test-blockjob-txn.c +4 -4
@@ -93,7 +93,7 @@ static const BlockJobDriver test_block_job_driver = {*/ static BlockJob *test_block_job_start (unsigned int iterations,bool use_timer, -int rc,int *result,BlockJobTxn *txn) +int rc,int *result,JobTxn *txn) {BlockDriverState *bs;TestBlockJob *s;
@@ -122,7 +122,7 @@ static BlockJob *test_block_job_start(unsigned int iterations,static void test_single_job (int expected) {BlockJob *job; -BlockJobTxn *txn; +JobTxn *txn;int result = -EINPROGRESS ; txn =block_job_txn_new ();
@@ -160,7 +160,7 @@ static void test_pair_jobs(int expected1, int expected2) {BlockJob *job1;BlockJob *job2; -BlockJobTxn *txn; +JobTxn *txn;int result1 = -EINPROGRESS ;int result2 = -EINPROGRESS ;
@@ -222,7 +222,7 @@ static void test_pair_jobs_fail_cancel_race(void) {BlockJob *job1;BlockJob *job2; -BlockJobTxn *txn; +JobTxn *txn;int result1 = -EINPROGRESS ;int result2 = -EINPROGRESS ;
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 30/42] job: Switch transactions to JobTxn
Quoted 5 lines · Kevin Wolf
This doesn't actually move any transaction code to Job yet, but it renames the type for transactions from BlockJobTxn to JobTxn and makes them contain Jobs rather than BlockJobs
Signed-off-by Kevin Wolf <[email protected]>
Quoted 10 lines
--- include/block/block_int.h | 2 +- include/block/blockjob.h | 11 ++++---- include/block/blockjob_int.h | 2 +- include/qemu/job.h | 3 +++ block/backup.c | 2 +- blockdev.c | 14 +++++------ blockjob.c | 60 +++++++++++++++++++++++--------------------- tests/test-blockjob-txn.c | 8 +++--- 8 files changed, 54 insertions(+), 48 deletions(-)
PATCH 31/42 job: Move transactions to Job
In reply to Kevin WolfThis moves the logic that implements job transactions from BlockJob to Job.
--- include/block/blockjob.h | 54 ---------- include/block/blockjob_int.h | 10 -- include/qemu/job.h | 71 +++++++++++-- blockdev.c | 6 +- blockjob.c | 238 +------------------------------------------ job.c | 235 ++++++++++++++++++++++++++++++++++++++++-- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 2 +- 8 files changed, 304 insertions(+), 324 deletions(-)
include/block/blockjob.h +0 -54
@@ -33,7 +33,6 @@#define SLICE_TIME 100000000ULL/* ns */ typedef struct BlockJobDriver BlockJobDriver ; -typedef struct JobTxn JobTxn ;/** * BlockJob:
@@ -84,8 +83,6 @@ typedef struct BlockJob {/** BlockDriverStates that are involved in this block job */ GSList *nodes; - -JobTxn *txn; } BlockJob;/**
@@ -153,22 +150,6 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp);void block_job_cancel (BlockJob *job,bool force);/** -* block_job_finalize: -* @job: The job to fully commit and finish. -* @errp: Error object. -* -* For jobs that have finished their work and are pending -* awaiting explicit acknowledgement to commit their work, -* This will commit that work. -* -* FIXME: Make the below statement universally true: -* For jobs that support the manual workflow mode, all graph -* changes that occur as a result will occur after this command -* and before a successful reply. -*/ -void block_job_finalize (BlockJob *job,Error **errp); - -/** * block_job_dismiss: * @job: The job to be dismissed. * @errp: Error object.
@@ -260,41 +241,6 @@ int block_job_complete_sync(BlockJob *job, Error **errp);void block_job_iostatus_reset (BlockJob *job);/** -* block_job_txn_new: -* -* Allocate and return a new block job transaction. Jobs can be added to the -* transaction using block_job_txn_add_job(). -* -* The transaction is automatically freed when the last job completes or is -* cancelled. -* -* All jobs in the transaction either complete successfully or fail/cancel as a -* group. Jobs wait for each other before completing. Cancelling one job -* cancels all jobs in the transaction. -*/ -JobTxn *block_job_txn_new (void ); - -/** -* block_job_txn_unref: -* -* Release a reference that was previously acquired with block_job_txn_add_job -* or block_job_txn_new. If it's the last reference to the object, it will be -* freed. -*/ -void block_job_txn_unref (JobTxn *txn); - -/** -* block_job_txn_add_job: -* @txn: The transaction (may be NULL) -* @job: Job to add to the transaction -* -* Add @job to the transaction. The @job must not already be in a transaction. -* The caller must call either block_job_txn_unref() or block_job_completed() -* to release the reference that is automatically grabbed here. -*/ -void block_job_txn_add_job (JobTxn *txn,BlockJob *job); - -/** * block_job_is_internal: * @job: The job to determine if it is user-visible or not. *
include/block/blockjob_int.h +0 -10
@@ -38,16 +38,6 @@ struct BlockJobDriver {/** Generic JobDriver callbacks and settings */ JobDriver job_driver; -/** -* If the callback is not NULL, prepare will be invoked when all the jobs -* belonging to the same transaction complete; or upon this job's completion -* if it is not in a transaction. -* -* This callback will not be invoked if the job has already failed. -* If it fails, abort and then clean will be called. -*/ -int (*prepare)(BlockJob *job); -/* * If the callback is not NULL, it will be invoked before the job is * resumed in a new AioContext. This is the place to move any resources
include/qemu/job.h +62 -9
125 lines not on this page · raw message
blockdev.c +3 -3
@@ -2255,7 +2255,7 @@ void qmp_transaction(TransactionActionList *dev_list,*/ props =get_transaction_properties (props);if (props->completion_mode !=ACTION_COMPLETION_MODE_INDIVIDUAL ) { - block_job_txn =block_job_txn_new (); + block_job_txn =job_txn_new (); }/* drain all i/o before any operations */
@@ -2314,7 +2314,7 @@ exit:if (!has_props) {qapi_free_TransactionProperties (props); } -block_job_txn_unref (block_job_txn); +job_txn_unref (block_job_txn); } void qmp_eject(bool has_device, const char *device,
@@ -3908,7 +3908,7 @@ void qmp_block_job_finalize(const char *id, Error **errp) }trace_qmp_block_job_finalize (job); -block_job_finalize (job, errp); +job_finalize (&job->job, errp);aio_context_release (aio_context); }
blockjob.c +5 -233
297 lines not on this page · raw message
job.c +227 -8
295 lines not on this page · raw message
tests/test-blockjob-txn.c +6 -6
@@ -125,7 +125,7 @@ static void test_single_job(int expected)JobTxn *txn;int result = -EINPROGRESS ; - txn =block_job_txn_new (); + txn =job_txn_new (); job =test_block_job_start (1 , true, expected, &result, txn);job_start (&job->job);
@@ -138,7 +138,7 @@ static void test_single_job(int expected) } g_assert_cmpint(result, ==, expected); -block_job_txn_unref (txn); +job_txn_unref (txn); }static void test_single_job_success (void )
@@ -164,7 +164,7 @@ static void test_pair_jobs(int expected1, int expected2)int result1 = -EINPROGRESS ;int result2 = -EINPROGRESS ; - txn =block_job_txn_new (); + txn =job_txn_new (); job1 =test_block_job_start (1 , true, expected1, &result1, txn); job2 =test_block_job_start (2 , true, expected2, &result2, txn);job_start (&job1->job);
@@ -173,7 +173,7 @@ static void test_pair_jobs(int expected1, int expected2)/* Release our reference now to trigger as many nice * use-after-free bugs as possible. */ -block_job_txn_unref (txn); +job_txn_unref (txn);if (expected1 == -ECANCELED ) {block_job_cancel (job1, false);
@@ -226,7 +226,7 @@ static void test_pair_jobs_fail_cancel_race(void)int result1 = -EINPROGRESS ;int result2 = -EINPROGRESS ; - txn =block_job_txn_new (); + txn =job_txn_new (); job1 =test_block_job_start (1 , true, -ECANCELED , &result1, txn); job2 =test_block_job_start (2 , false,0 , &result2, txn);job_start (&job1->job);
@@ -247,7 +247,7 @@ static void test_pair_jobs_fail_cancel_race(void) g_assert_cmpint(result1, ==, -ECANCELED); g_assert_cmpint(result2, ==, -ECANCELED); -block_job_txn_unref (txn); +job_txn_unref (txn); }int main (int argc,char **argv)
tests/test-blockjob.c +1 -1
@@ -364,7 +364,7 @@ static void test_cancel_concluded(void) }assert (job->job.status ==JOB_STATUS_PENDING ); -block_job_finalize (job, &error_abort); +job_finalize (&job->job, &error_abort);assert (job->job.status ==JOB_STATUS_CONCLUDED );cancel_common (s);
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 31/42] job: Move transactions to Job
Quoted 14 lines · Kevin Wolf
This moves the logic that implements job transactions from BlockJob to Job.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 54 ---------- include/block/blockjob_int.h | 10 -- include/qemu/job.h | 71 +++++++++++-- blockdev.c | 6 +- blockjob.c | 238 +------------------------------------------ job.c | 235 ++++++++++++++++++++++++++++++++++++++++-- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 2 +- 8 files changed, 304 insertions(+), 324 deletions(-)
[...]
Quoted 4 lines
include/qemu/job.h +0
-0
[...]
Quoted 1 line
include/qemu/job.h +0
-0@@ -227,20 +242,52 @@ typedef enum JobCreateFlags {
[...]
Quoted 6 lines
+/**
+ * @txn: The transaction (may be NULL)
+ * @job: Job to add to the transaction
+ *
+ * Add @job to the transaction. The @job must not already be in a transaction.
+ * The caller must call eitherblock_job_txn_unref()orblock_job_completed()
*job_txn_unref()
(and maybe even job_completed() in preparation for the next patches)
Quoted 5 lines
+ * to release the reference that is automatically grabbed here.
+ *
+ * If @txn is NULL, the function does nothing.
+ */
+voidjob_txn_add_job(JobTxn *txn, Job *job);
[...]
Quoted 4 lines
job.c +0
-0
[...]
Quoted 1 line
job.c +0
-0@@ -80,6 +93,71 @@ static void __attribute__((__constructor__)) job_init(void)
[...]
Quoted 1 line
+static int
job_txn_apply(JobTxn *txn, int fn(Job *), bool lock)
(“6.7.6.3. (8) A declaration of a parameter as ‘function returning type’ shall be adjusted to ‘pointer to function returning type’, as in 6.3.2.1.”
Interesting. Didn't know that worked.)
[...]
Quoted 1 line
job.c +0
-0@@ -542,12 +632,141 @@ int job_finalize_single(Job *job)
[...]
job.c int job_finalize_single(Job *job) · line 705
Quoted 7 lines
job.c +7
-0+static int job_prepare (Job *job) +{ +if (job->ret ==0 && job->driver->prepare) { + job->ret = job->driver->prepare (job); + } +return job->ret; +}
I'd have put this above job_commit() and the like, but it's not like it matters functionally...
Well, you know me. With the comment fixed:
Re: [Qemu-devel] [PATCH 31/42] job: Move transactions to Job
In reply to Kevin WolfQuoted 14 lines · Kevin Wolf
This moves the logic that implements job transactions from BlockJob to Job.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 54 ---------- include/block/blockjob_int.h | 10 -- include/qemu/job.h | 71 +++++++++++-- blockdev.c | 6 +- blockjob.c | 238 +------------------------------------------ job.c | 235 ++++++++++++++++++++++++++++++++++++++++-- tests/test-blockjob-txn.c | 12 +-- tests/test-blockjob.c | 2 +- 8 files changed, 304 insertions(+), 324 deletions(-)
[...]
Quoted 4 lines
job.c +0
-0
[...]
job.c void job_event_completed(Job *job) · line 362
Quoted 12 lines
job.c +2
-1@@ -263,9 +353,10 @@ void job_event_completed(Job *job)notifier_list_notify (&job->on_finalize_completed, job); } -void job_event_pending (Job *job) +static int job_event_pending (Job *job) {notifier_list_notify (&job->on_pending, job); +return 0 ; }void job_enter_cond (Job *job,bool (*fn)(Job *job))
On second thought, I don't know why this hunk is necessary.
Max
PATCH 32/42 job: Move completion and cancellation to Job
In reply to Kevin WolfThis moves the top-level job completion and cancellation functions from BlockJob to Job.
--- include/block/blockjob.h | 55 ------------------------------- include/block/blockjob_int.h | 18 ---------- include/qemu/job.h | 58 ++++++++++++++++++++++++++++---- block.c | 2 +- block/backup.c | 3 +- block/commit.c | 8 ++--- block/mirror.c | 6 ++-- block/replication.c | 4 +-- block/stream.c | 2 +- blockdev.c | 8 ++--- blockjob.c | 76 ------------------------------------------ job.c | 78 +++++++++++++++++++++++++++++++++++++++++--- qemu-img.c | 2 +- tests/test-bdrv-drain.c | 5 ++- tests/test-blockjob-txn.c | 14 ++++---- tests/test-blockjob.c | 21 ++++++------ block/trace-events | 3 -- trace-events | 1 + 18 files changed, 162 insertions(+), 202 deletions(-)
include/block/blockjob.h +0 -55
@@ -141,15 +141,6 @@ void block_job_remove_all_bdrv(BlockJob *job);void block_job_set_speed (BlockJob *job,int64_t speed,Error **errp);/** -* block_job_cancel: -* @job: The job to be canceled. -* @force: Quit a job without waiting for data to be in sync. -* -* Asynchronously cancel the specified job. -*/ -void block_job_cancel (BlockJob *job,bool force); - -/** * block_job_dismiss: * @job: The job to be dismissed. * @errp: Error object.
@@ -186,52 +177,6 @@ void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining);BlockJobInfo *block_job_query (BlockJob *job,Error **errp);/** -* block_job_user_cancel: -* @job: The job to be cancelled. -* @force: Quit a job without waiting for data to be in sync. -* -* Cancels the specified job, but may refuse to do so if the -* operation isn't currently meaningful. -*/ -void block_job_user_cancel (BlockJob *job,bool force,Error **errp); - -/** -* block_job_cancel_sync: -* @job: The job to be canceled. -* -* Synchronously cancel the job. The completion callback is called -* before the function returns. The job may actually complete -* instead of canceling itself; the circumstances under which this -* happens depend on the kind of job that is active. -* -* Returns the return value from the job if the job actually completed -* during the call, or -ECANCELED if it was canceled. -*/ -int block_job_cancel_sync (BlockJob *job); - -/** -* block_job_cancel_sync_all: -* -* Synchronously cancels all jobs using block_job_cancel_sync(). -*/ -void block_job_cancel_sync_all (void ); - -/** -* block_job_complete_sync: -* @job: The job to be completed. -* @errp: Error object which may be set by block_job_complete(); this is not -* necessarily set on every error, the job return value has to be -* checked as well. -* -* Synchronously complete the job. The completion callback is called before the -* function returns, unless it is NULL (which is permissible when using this -* function). -* -* Returns the return value from the job. -*/ -int block_job_complete_sync (BlockJob *job,Error **errp); - -/** * block_job_iostatus_reset: * @job: The job whose I/O status should be reset. *
include/block/blockjob_int.h +0 -18
@@ -124,24 +124,6 @@ void block_job_yield(BlockJob *job);int64_t block_job_ratelimit_get_delay (BlockJob *job,uint64_t n);/** -* block_job_completed: -* @job: The job being completed. -* @ret: The status code. -* -* Call the completion function that was registered at creation time, and -* free @job. -*/ -void block_job_completed (BlockJob *job,int ret); - -/** -* block_job_enter: -* @job: The job to enter. -* -* Continue the specified job by entering the coroutine. -*/ -void block_job_enter (BlockJob *job); - -/** * block_job_event_ready: * @job: The job which is now ready to be completed. *
include/qemu/job.h +52 -6
71 lines not on this page · raw message
block.c +1 -1
@@ -3362,7 +3362,7 @@ static void bdrv_close(BlockDriverState *bs)void bdrv_close_all (void ) { -block_job_cancel_sync_all (); +job_cancel_sync_all ();nbd_export_close_all ();/* Drop references from requests still in flight, such as canceled block
block/backup.c +1 -2
@@ -319,10 +319,9 @@ typedef struct {static void backup_complete (Job *job,void *opaque) { -BlockJob *bjob =container_of (job, BlockJob, job);BackupCompleteData *data = opaque; -block_job_completed (bjob, data->ret); +job_completed (job, data->ret);g_free (data); }
block/commit.c +4 -4
@@ -112,12 +112,12 @@ static void commit_complete(Job *job, void *opaque)blk_unref (s->top);/* If there is more than one reference to the job (e.g. if called from -* block_job_finish_sync()), block_job_completed() won't free it and -* therefore the blockers on the intermediate nodes remain. This would -* cause bdrv_set_backing_hd() to fail. */ +* block_job_finish_sync()), job_completed() won't free it and therefore +* the blockers on the intermediate nodes remain. This would cause +* bdrv_set_backing_hd() to fail. */ block_job_remove_all_bdrv (bjob); -block_job_completed (&s->common, ret); +job_completed (job, ret);g_free (data);/* If bdrv_drop_intermediate() didn't already do that, remove the commit
block/mirror.c +3 -3
@@ -498,7 +498,7 @@ static void mirror_exit(Job *job, void *opaque)bdrv_release_dirty_bitmap (src, s->dirty_bitmap);/* Make sure that the source BDS doesn't go away before we called -* block_job_completed(). */ +* job_completed(). */ bdrv_ref (src);bdrv_ref (mirror_top_bs);bdrv_ref (target_bs);
@@ -581,7 +581,7 @@ static void mirror_exit(Job *job, void *opaque)blk_set_perm (bjob->blk,0 ,BLK_PERM_ALL , &error_abort);blk_insert_bs (bjob->blk, mirror_top_bs, &error_abort); -block_job_completed (&s->common, data->ret); +job_completed (job, data->ret);g_free (data);bdrv_drained_end (src);
@@ -954,7 +954,7 @@ static void mirror_complete(Job *job, Error **errp) } s->should_complete = true; -block_job_enter (&s->common); +job_enter (job); }static void mirror_pause (Job *job)
block/replication.c +2 -2
@@ -145,7 +145,7 @@ static void replication_close(BlockDriverState *bs)replication_stop (s->rs, false,NULL ); }if (s->stage ==BLOCK_REPLICATION_FAILOVER ) { -block_job_cancel_sync (s->active_disk->bs->job); +job_cancel_sync (&s->active_disk->bs->job->job); }if (s->mode ==REPLICATION_MODE_SECONDARY ) {
@@ -681,7 +681,7 @@ static void replication_stop(ReplicationState *rs, bool failover, Error **errp)* disk, secondary disk in backup_job_completed(). */ if (s->secondary_disk->bs->job) { -block_job_cancel_sync (s->secondary_disk->bs->job); +job_cancel_sync (&s->secondary_disk->bs->job->job); }if (!failover) {
block/stream.c +1 -1
@@ -93,7 +93,7 @@ out: }g_free (s->backing_file_str); -block_job_completed (&s->common, data->ret); +job_completed (job, data->ret);g_free (data); }
blockdev.c +4 -4
@@ -150,7 +150,7 @@ void blockdev_mark_auto_del(BlockBackend *blk)aio_context_acquire (aio_context);if (bs->job) { -block_job_cancel (bs->job, false); +job_cancel (&bs->job->job, false); }aio_context_release (aio_context);
@@ -1925,7 +1925,7 @@ static void drive_backup_abort(BlkActionState *common) aio_context =bdrv_get_aio_context (state->bs);aio_context_acquire (aio_context); -block_job_cancel_sync (state->job); +job_cancel_sync (&state->job->job);aio_context_release (aio_context); }
@@ -2023,7 +2023,7 @@ static void blockdev_backup_abort(BlkActionState *common) aio_context =bdrv_get_aio_context (state->bs);aio_context_acquire (aio_context); -block_job_cancel_sync (state->job); +job_cancel_sync (&state->job->job);aio_context_release (aio_context); }
@@ -3851,7 +3851,7 @@ void qmp_block_job_cancel(const char *device, }trace_qmp_block_job_cancel (job); -block_job_user_cancel (job, force, errp); +job_user_cancel (&job->job, force, errp); out:aio_context_release (aio_context); }
blockjob.c +0 -76
88 lines not on this page · raw message
job.c +73 -5
114 lines not on this page · raw message
qemu-img.c +1 -1
@@ -870,7 +870,7 @@ static void run_block_job(BlockJob *job, Error **errp) }while (!job->ready && !job_is_completed (&job->job));if (!job_is_completed (&job->job)) { - ret =block_job_complete_sync (job, errp); + ret =job_complete_sync (&job->job, errp); }else { ret = job->job.ret; }
tests/test-bdrv-drain.c +2 -3
@@ -498,8 +498,7 @@ typedef struct TestBlockJob {static void test_job_completed (Job *job,void *opaque) { -BlockJob *bjob =container_of (job, BlockJob, job); -block_job_completed (bjob,0 ); +job_completed (job,0 ); } static void coroutine_fn test_job_start(void *opaque)
@@ -593,7 +592,7 @@ static void test_blockjob_common(enum drain_type drain_type)g_assert_false (job->job.paused);g_assert_false (job->job.busy);/* We're in job_sleep_ns() */ - ret =block_job_complete_sync (job, &error_abort); + ret =job_complete_sync (&job->job, &error_abort); g_assert_cmpint(ret, ==, 0);blk_unref (blk_src);
tests/test-blockjob-txn.c +7 -7
@@ -34,7 +34,7 @@ static void test_block_job_complete(Job *job, void *opaque) rc = -ECANCELED ; } -block_job_completed (bjob, rc); +job_completed (job, rc);bdrv_unref (bs); }
@@ -130,7 +130,7 @@ static void test_single_job(int expected)job_start (&job->job);if (expected == -ECANCELED ) { -block_job_cancel (job, false); +job_cancel (&job->job, false); }while (result == -EINPROGRESS ) {
@@ -176,10 +176,10 @@ static void test_pair_jobs(int expected1, int expected2)job_txn_unref (txn);if (expected1 == -ECANCELED ) { -block_job_cancel (job1, false); +job_cancel (&job1->job, false); }if (expected2 == -ECANCELED ) { -block_job_cancel (job2, false); +job_cancel (&job2->job, false); }while (result1 == -EINPROGRESS || result2 == -EINPROGRESS ) {
@@ -232,13 +232,13 @@ static void test_pair_jobs_fail_cancel_race(void)job_start (&job1->job);job_start (&job2->job); -block_job_cancel (job1, false); +job_cancel (&job1->job, false);/* Now make job2 finish before the main loop kicks jobs. This simulates * the race between a pending kick and another job completing. */ -block_job_enter (job2); -block_job_enter (job2); +job_enter (&job2->job); +job_enter (&job2->job);while (result1 == -EINPROGRESS || result2 == -EINPROGRESS ) {aio_poll (qemu_get_aio_context (), true);
tests/test-blockjob.c +10 -11
@@ -165,10 +165,9 @@ typedef struct CancelJob {static void cancel_job_completed (Job *job,void *opaque) { -BlockJob *bjob =container_of (job, BlockJob, job);CancelJob *s = opaque; s->completed = true; -block_job_completed (bjob,0 ); +job_completed (job,0 ); }static void cancel_job_complete (Job *job,Error **errp)
@@ -232,7 +231,7 @@ static void cancel_common(CancelJob *s)BlockBackend *blk = s->blk;JobStatus sts = job->job.status; -block_job_cancel_sync (job); +job_cancel_sync (&job->job);if (sts !=JOB_STATUS_CREATED && sts !=JOB_STATUS_CONCLUDED ) {BlockJob *dummy = job;block_job_dismiss (&dummy, &error_abort);
@@ -275,7 +274,7 @@ static void test_cancel_paused(void)assert (job->job.status ==JOB_STATUS_RUNNING );job_user_pause (&job->job, &error_abort); -block_job_enter (job); +job_enter (&job->job);assert (job->job.status ==JOB_STATUS_PAUSED );cancel_common (s);
@@ -292,7 +291,7 @@ static void test_cancel_ready(void)assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true; -block_job_enter (job); +job_enter (&job->job);assert (job->job.status ==JOB_STATUS_READY );cancel_common (s);
@@ -309,11 +308,11 @@ static void test_cancel_standby(void)assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true; -block_job_enter (job); +job_enter (&job->job);assert (job->job.status ==JOB_STATUS_READY );job_user_pause (&job->job, &error_abort); -block_job_enter (job); +job_enter (&job->job);assert (job->job.status ==JOB_STATUS_STANDBY );cancel_common (s);
@@ -330,11 +329,11 @@ static void test_cancel_pending(void)assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true; -block_job_enter (job); +job_enter (&job->job);assert (job->job.status ==JOB_STATUS_READY );job_complete (&job->job, &error_abort); -block_job_enter (job); +job_enter (&job->job);while (!s->completed) {aio_poll (qemu_get_aio_context (), true); }
@@ -354,11 +353,11 @@ static void test_cancel_concluded(void)assert (job->job.status ==JOB_STATUS_RUNNING ); s->should_converge = true; -block_job_enter (job); +job_enter (&job->job);assert (job->job.status ==JOB_STATUS_READY );job_complete (&job->job, &error_abort); -block_job_enter (job); +job_enter (&job->job);while (!s->completed) {aio_poll (qemu_get_aio_context (), true); }
block/trace-events +0 -3
@@ -4,9 +4,6 @@ bdrv_open_common(void *bs, const char *filename, int flags, const char *format_name) "bs %p filename \"%s\" flags 0x%x format_name \"%s\"" bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" -# blockjob.c -block_job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" - # block/block-backend.c blk_co_preadv(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" blk_co_pwritev(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x"
trace-events +1 -0
@@ -107,6 +107,7 @@ gdbstub_err_checksum_incorrect(uint8_t expected, uint8_t got) "got command packe # job.c job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" +job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" ### Guest events, keep at bottom
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 32/42] job: Move completion and cancellation to Job
Quoted 16 lines · Kevin Wolf
This moves the top-level job completion and cancellation functions from BlockJob to Job.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 55 ------------------------------- include/block/blockjob_int.h | 18 ---------- include/qemu/job.h | 58 ++++++++++++++++++++++++++++---- block.c | 2 +- block/backup.c | 3 +- block/commit.c | 8 ++--- block/mirror.c | 6 ++-- block/replication.c | 4 +-- block/stream.c | 2 +- blockdev.c | 8 ++--- blockjob.c | 76 ------------------------------------------
Quoted 8 lines
job.c | 78 +++++++++++++++++++++++++++++++++++++++++--- qemu-img.c | 2 +- tests/test-bdrv-drain.c | 5 ++- tests/test-blockjob-txn.c | 14 ++++---- tests/test-blockjob.c | 21 ++++++------ block/trace-events | 3 -- trace-events | 1 + 18 files changed, 162 insertions(+), 202 deletions(-)
After this patch, there are a couple of places left that mention block_job_enter(), those should be fixed. Well, and those two block_job_completed() mentions.
[...]
block.c static void bdrv_close(BlockDriverState *bs) · line 3365
Quoted 10 lines
block.c +1
-1@@ -3362,7 +3362,7 @@ static void bdrv_close(BlockDriverState *bs)void bdrv_close_all (void ) { -block_job_cancel_sync_all (); +job_cancel_sync_all ();
Do we really want to cancel jobs that might have nothing to do with the block layer?
Quoted 3 lines
block.c +0
-0nbd_export_close_all ();/* Drop references from requests still in flight, such as canceled block
[...]
block/commit.c static void commit_complete(Job *job, void *opaque) · line 115
Quoted 12 lines
block/commit.c +1
-3@@ -112,12 +112,12 @@ static void commit_complete(Job *job, void *opaque)blk_unref (s->top);/* If there is more than one reference to the job (e.g. if called from -* block_job_finish_sync()), block_job_completed() won't free it and -* therefore the blockers on the intermediate nodes remain. This would -* cause bdrv_set_backing_hd() to fail. */ +* block_job_finish_sync()), job_completed() won't free it and therefore
Nice start, but there is more to do in this line. :-)
(Though probably in some other patch.)
Max
Quoted 9 lines
block/commit.c +3
-1+* the blockers on the intermediate nodes remain. This would cause +* bdrv_set_backing_hd() to fail. */ block_job_remove_all_bdrv (bjob); -block_job_completed (&s->common, ret); +job_completed (job, ret);g_free (data);/* If bdrv_drop_intermediate() didn't already do that, remove the commit
Re: [Qemu-devel] [PATCH 32/42] job: Move completion and cancellation to Job
Am 14.05.2018 um 22:53 hat Max Reitz geschrieben:
Quoted 8 lines · Kevin Wolf
Quoted 7 lines
This moves the top-level job completion and cancellation functions from BlockJob to Job.
Signed-off-by Kevin Wolf <[email protected]>(quoted diff) +0
-0@@ -3362,7 +3362,7 @@ static void bdrv_close(BlockDriverState *bs)
Quoted 7 lines
block.c static void bdrv_close(BlockDriverState *bs) · line 3365
Quoted 4 lines
block.c +1
-1void bdrv_close_all (void ) { -block_job_cancel_sync_all (); +job_cancel_sync_all ();Do we really want to cancel jobs that might have nothing to do with the block layer?
Yes, though I admit it's a bit ugly to do it here. Alternatively, I could assert here that no jobs are running and pull the job_cancel_sync_all() into the callers. For vl.c, this is trivial. qemu-nbd.c uses 'atexit(bdrv_close_all);', so I'd have to introduce a wrapper function that calls both job_cancel_sync_all() and bdrv_close_all().
Would you prefer that?
Kevin
Re: [Qemu-devel] [PATCH 32/42] job: Move completion and cancellation to Job
Quoted 17 lines · Kevin Wolf
Am 14.05.2018 um 22:53 hat Max Reitz geschrieben:
Quoted 8 lines · Kevin Wolf
Quoted 7 lines
This moves the top-level job completion and cancellation functions from BlockJob to Job.
Signed-off-by Kevin Wolf <[email protected]>(quoted diff) +0
-0@@ -3362,7 +3362,7 @@ static void bdrv_close(BlockDriverState *bs)Quoted 7 lines
block.c static void bdrv_close(BlockDriverState *bs) · line 3365
Quoted 4 lines
block.c +1
-1void bdrv_close_all (void ) { -block_job_cancel_sync_all (); +job_cancel_sync_all ();Do we really want to cancel jobs that might have nothing to do with the block layer?
Quoted 8 lines
Yes, though I admit it's a bit ugly to do it here. Alternatively, I could assert here that no jobs are running and pull the
job_cancel_sync_all()into the callers. Forvl.c, this is trivial.qemu-nbd.cuses 'atexit(bdrv_close_all);', so I'd have to introduce a wrapper function that calls bothjob_cancel_sync_all()andbdrv_close_all().Would you prefer that?
Yes, I'd like that.
Thanks!
Max
PATCH 33/42 job: Add job_yield()
In reply to Kevin WolfThis moves block_job_yield() to the Job layer.
--- include/block/blockjob_int.h | 8 -------- include/qemu/job.h | 9 +++++++-- block/backup.c | 2 +- block/mirror.c | 2 +- blockjob.c | 16 ---------------- job.c | 20 ++++++++++++++++++-- tests/test-blockjob-txn.c | 2 +- 7 files changed, 28 insertions(+), 31 deletions(-)
include/block/blockjob_int.h +0 -8
@@ -108,14 +108,6 @@ void block_job_user_resume(Job *job);void block_job_drain (Job *job);/** -* block_job_yield: -* @job: The job that calls the function. -* -* Yield the block job coroutine. -*/ -void block_job_yield (BlockJob *job); - -/** * block_job_ratelimit_get_delay: * * Calculate and return delay for the next request in ns. See the documentation
include/qemu/job.h +7 -2
@@ -339,6 +339,13 @@ void coroutine_fn job_pause_point(Job *job);/** * @job: The job that calls the function. +* +* Yield the job coroutine. +*/ +void job_yield (Job *job); + +/** +* @job: The job that calls the function. * @ns: How many nanoseconds to stop for. * * Put the job to sleep (assuming that it wasn't canceled) for @ns
@@ -508,8 +515,6 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp)/* TODO To be removed from the public interface */ void job_state_transition (Job *job,JobStatus s1); -void coroutine_fn job_do_yield(Job *job, uint64_t ns); -bool job_should_pause (Job *job);void job_do_dismiss (Job *job);#endif
block/backup.c +1 -1
@@ -444,7 +444,7 @@ static void coroutine_fn backup_run(void *opaque)while (!job_is_cancelled (&job->common.job)) {/* Yield until the job is cancelled. We just let our before_write * notify callback service CoW requests. */ -block_job_yield (&job->common); +job_yield (&job->common.job); } }else if (job->sync_mode ==MIRROR_SYNC_MODE_INCREMENTAL ) { ret =backup_run_incremental (job);
block/mirror.c +1 -1
@@ -731,7 +731,7 @@ static void coroutine_fn mirror_run(void *opaque)block_job_event_ready (&s->common); s->synced = true;while (!job_is_cancelled (&s->common.job) && !s->should_complete) { -block_job_yield (&s->common); +job_yield (&s->common.job); } s->common.job.cancelled = false; goto immediate_exit;
blockjob.c +0 -16
@@ -433,22 +433,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,return job; } -void block_job_yield (BlockJob *job) -{ -assert (job->job.busy); - -/* Check cancellation *before* setting busy = false, too! */ -if (job_is_cancelled (&job->job)) { -return ; - } - -if (!job_should_pause (&job->job)) { -job_do_yield (&job->job,-1 ); - } - -job_pause_point (&job->job); -} -void block_job_iostatus_reset (BlockJob *job) {if (job->iostatus ==BLOCK_DEVICE_IO_STATUS_OK ) {
job.c +18 -2
@@ -226,7 +226,7 @@ static bool job_started(Job *job)return job->co; } -bool job_should_pause (Job *job) +static bool job_should_pause (Job *job) {return job->pause_count >0 ; }
@@ -397,7 +397,7 @@ void job_enter(Job *job)* * If @ns is (uint64_t) -1, no timer is scheduled and block_job_enter() must be * called explicitly. */ -void coroutine_fn job_do_yield(Job *job, uint64_t ns) +static void coroutine_fn job_do_yield(Job *job, uint64_t ns) {job_lock ();if (ns !=-1 ) {
@@ -442,6 +442,22 @@ void coroutine_fn job_pause_point(Job *job) } } +void job_yield (Job *job) +{ +assert (job->busy); + +/* Check cancellation *before* setting busy = false, too! */ +if (job_is_cancelled (job)) { +return ; + } + +if (!job_should_pause (job)) { +job_do_yield (job,-1 ); + } + +job_pause_point (job); +} + void coroutine_fn job_sleep_ns(Job *job, int64_t ns) {assert (job->busy);
tests/test-blockjob-txn.c +1 -1
@@ -47,7 +47,7 @@ static void coroutine_fn test_block_job_run(void *opaque)if (s->use_timer) {job_sleep_ns (&job->job,0 ); }else { -block_job_yield (job); +job_yield (&job->job); }if (job_is_cancelled (&job->job)) {
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 33/42] job: Add job_yield()
Quoted 12 lines · Kevin Wolf
This moves
block_job_yield()to the Job layer.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob_int.h | 8 -------- include/qemu/job.h | 9 +++++++-- block/backup.c | 2 +- block/mirror.c | 2 +- blockjob.c | 16 ---------------- job.c | 20 ++++++++++++++++++-- tests/test-blockjob-txn.c | 2 +- 7 files changed, 28 insertions(+), 31 deletions(-)
PATCH 34/42 job: Add job_dismiss()
In reply to Kevin WolfThis moves block_job_dismiss() to the Job layer.
--- include/block/blockjob.h | 9 --------- include/qemu/job.h | 7 ++++++- blockdev.c | 8 +++++--- blockjob.c | 13 ------------- job.c | 15 ++++++++++++++- tests/test-blockjob.c | 4 ++-- 6 files changed, 27 insertions(+), 29 deletions(-)
include/block/blockjob.h +0 -9
@@ -141,15 +141,6 @@ void block_job_remove_all_bdrv(BlockJob *job);void block_job_set_speed (BlockJob *job,int64_t speed,Error **errp);/** -* block_job_dismiss: -* @job: The job to be dismissed. -* @errp: Error object. -* -* Remove a concluded job from the query list. -*/ -void block_job_dismiss (BlockJob **job,Error **errp); - -/** * block_job_progress_update: * @job: The job that has made progress * @done: How much progress the job made
include/qemu/job.h +6 -1
@@ -487,6 +487,12 @@ int job_complete_sync(Job *job, Error **errp);*/ void job_finalize (Job *job,Error **errp); +/** +* Remove the concluded @job from the query list and resets the passed pointer +* to %NULL. Returns an error if the job is not actually concluded. +*/ +void job_dismiss (Job **job,Error **errp); +typedef void JobDeferToMainLoopFn (Job *job,void *opaque);/**
@@ -515,6 +521,5 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp)/* TODO To be removed from the public interface */ void job_state_transition (Job *job,JobStatus s1); -void job_do_dismiss (Job *job);#endif
blockdev.c +5 -3
@@ -3915,14 +3915,16 @@ void qmp_block_job_finalize(const char *id, Error **errp)void qmp_block_job_dismiss (const char *id,Error **errp) {AioContext *aio_context; -BlockJob *job =find_block_job (id, &aio_context, errp); +BlockJob *bjob =find_block_job (id, &aio_context, errp); +Job *job; -if (!job) { +if (!bjob) {return ; }trace_qmp_block_job_dismiss (job); -block_job_dismiss (&job, errp); + job = &bjob->job; +job_dismiss (&job, errp);aio_context_release (aio_context); }
blockjob.c +0 -13
@@ -243,19 +243,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n)return ratelimit_calculate_delay (&job->limit, n); } -void block_job_dismiss (BlockJob **jobptr,Error **errp) -{ -BlockJob *job = *jobptr; -/* similarly to _complete, this is QMP-interface only. */ -assert (job->job.id); -if (job_apply_verb (&job->job,JOB_VERB_DISMISS , errp)) { -return ; - } - -job_do_dismiss (&job->job); - *jobptr =NULL ; -} -void block_job_progress_update (BlockJob *job,uint64_t done) { job->offset += done;
job.c +14 -1
@@ -561,7 +561,7 @@ void job_user_resume(Job *job, Error **errp)job_resume (job); } -void job_do_dismiss (Job *job) +static void job_do_dismiss (Job *job) {assert (job); job->busy = false;
@@ -574,6 +574,19 @@ void job_do_dismiss(Job *job)job_unref (job); } +void job_dismiss (Job **jobptr,Error **errp) +{ +Job *job = *jobptr; +/* similarly to _complete, this is QMP-interface only. */ +assert (job->id); +if (job_apply_verb (job,JOB_VERB_DISMISS , errp)) { +return ; + } + +job_do_dismiss (job); + *jobptr =NULL ; +} +void job_early_fail (Job *job) {assert (job->status ==JOB_STATUS_CREATED );
tests/test-blockjob.c +2 -2
@@ -233,8 +233,8 @@ static void cancel_common(CancelJob *s)job_cancel_sync (&job->job);if (sts !=JOB_STATUS_CREATED && sts !=JOB_STATUS_CONCLUDED ) { -BlockJob *dummy = job; -block_job_dismiss (&dummy, &error_abort); +Job *dummy = &job->job; +job_dismiss (&dummy, &error_abort); }assert (job->job.status ==JOB_STATUS_NULL );job_unref (&job->job);
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 34/42] job: Add job_dismiss()
Quoted 11 lines · Kevin Wolf
This moves
block_job_dismiss()to the Job layer.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 9 --------- include/qemu/job.h | 7 ++++++- blockdev.c | 8 +++++--- blockjob.c | 13 ------------- job.c | 15 ++++++++++++++- tests/test-blockjob.c | 4 ++-- 6 files changed, 27 insertions(+), 29 deletions(-)
Re: [Qemu-devel] [PATCH 34/42] job: Add job_dismiss()
In reply to Kevin WolfQuoted 11 lines · Kevin Wolf
This moves
block_job_dismiss()to the Job layer.Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 9 --------- include/qemu/job.h | 7 ++++++- blockdev.c | 8 +++++--- blockjob.c | 13 ------------- job.c | 15 ++++++++++++++- tests/test-blockjob.c | 4 ++-- 6 files changed, 27 insertions(+), 29 deletions(-)
[...]
blockdev.c void qmp_block_job_finalize(const char *id, Error **errp) · line 3919
Quoted 11 lines
blockdev.c +2
-1@@ -3915,14 +3915,16 @@ void qmp_block_job_finalize(const char *id, Error **errp)void qmp_block_job_dismiss (const char *id,Error **errp) {AioContext *aio_context; -BlockJob *job =find_block_job (id, &aio_context, errp); +BlockJob *bjob =find_block_job (id, &aio_context, errp); +Job *job;
clang (probably gcc as well if I had enabled tracing in that build...) complains that this should be initialized to NULL because of...
blockdev.c void qmp_block_job_finalize(const char *id, Error **errp) · line 3925
Quoted 7 lines
blockdev.c +1
-1-if (!job) { +if (!bjob) {return ; }trace_qmp_block_job_dismiss (job);
...this.
Max
Quoted 6 lines
blockdev.c +2
-1-block_job_dismiss (&job, errp); + job = &bjob->job; +job_dismiss (&job, errp);aio_context_release (aio_context); }
PATCH 35/42 job: Add job_is_ready()
In reply to Kevin WolfInstead of having a 'bool ready' in BlockJob, add a function that derives its value from the job status.
At the same time, this fixes the behaviour to match what the QAPI documentation promises for query-block-job: 'true if the job may be completed'. When the ready flag was introduced in commit ef6dbf1e46e, the flag never had to be reset to match the description because after being ready, the jobs would immediately complete and disappear.
Job transactions and manual job finalisation were introduced only later. With these changes, jobs may stay around even after having completed (and they are not ready to be completed a second time), however their patches forgot to reset the ready flag.
--- include/block/blockjob.h | 5 ----- include/qemu/job.h | 3 +++ blockjob.c | 3 +-- job.c | 22 ++++++++++++++++++++++ qemu-img.c | 2 +- tests/test-blockjob.c | 2 +- 6 files changed, 28 insertions(+), 9 deletions(-)
include/block/blockjob.h +0 -5
@@ -49,11 +49,6 @@ typedef struct BlockJob {/** The block device on which the job is operating. */ BlockBackend *blk; -/** -* Set to true when the job is ready to be completed. -*/ -bool ready; -/** Status that is published by the query-block-jobs QMP API */ BlockDeviceIoStatus iostatus;
include/qemu/job.h +3 -0
@@ -367,6 +367,9 @@ bool job_is_cancelled(Job *job);/** Returns whether the job is in a completed state. */ bool job_is_completed (Job *job); +/** Returns whether the job is ready to be completed. */ +bool job_is_ready (Job *job); +/** * Request @job to pause at the next pause point. Must be paired with * job_resume(). If the job is supposed to be resumed by user action, call
blockjob.c +1 -2
@@ -270,7 +270,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) info->offset = job->offset; info->speed = job->speed; info->io_status = job->iostatus; - info->ready = job->ready; + info->ready =job_is_ready (&job->job), info->status = job->job.status; info->auto_finalize = job->job.auto_finalize; info->auto_dismiss = job->job.auto_dismiss;
@@ -438,7 +438,6 @@ void block_job_user_resume(Job *job)void block_job_event_ready (BlockJob *job) {job_state_transition (&job->job,JOB_STATUS_READY ); - job->ready = true;if (block_job_is_internal (job)) {return ;
job.c +22 -0
@@ -199,6 +199,28 @@ bool job_is_cancelled(Job *job)return job->cancelled; } +bool job_is_ready (Job *job) +{ +switch (job->status) { +case JOB_STATUS_UNDEFINED : +case JOB_STATUS_CREATED : +case JOB_STATUS_RUNNING : +case JOB_STATUS_PAUSED : +case JOB_STATUS_WAITING : +case JOB_STATUS_PENDING : +case JOB_STATUS_ABORTING : +case JOB_STATUS_CONCLUDED : +case JOB_STATUS_NULL : +return false; +case JOB_STATUS_READY : +case JOB_STATUS_STANDBY : +return true; +default : +g_assert_not_reached (); + } +return false; +} +bool job_is_completed (Job *job) {switch (job->status) {
qemu-img.c +1 -1
@@ -867,7 +867,7 @@ static void run_block_job(BlockJob *job, Error **errp)aio_poll (aio_context, true);qemu_progress_print (job->len ? ((float )job->offset / job->len *100.f ) :0.0f ,0 ); - }while (!job->ready && !job_is_completed (&job->job)); + }while (!job_is_ready (&job->job) && !job_is_completed (&job->job));if (!job_is_completed (&job->job)) { ret =job_complete_sync (&job->job, errp);
tests/test-blockjob.c +1 -1
@@ -185,7 +185,7 @@ static void coroutine_fn cancel_job_start(void *opaque) goto defer; } -if (!s->common.ready && s->should_converge) { +if (!job_is_ready (&s->common.job) && s->should_converge) {block_job_event_ready (&s->common); }
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 35/42] job: Add job_is_ready()
Quoted 15 lines · Kevin Wolf
Instead of having a 'bool ready' in BlockJob, add a function that derives its value from the job status.
At the same time, this fixes the behaviour to match what the QAPI documentation promises for query-block-job: 'true if the job may be completed'. When the ready flag was introduced in commit
ef6dbf1e46e, the flag never had to be reset to match the description because after being ready, the jobs would immediately complete and disappear.Job transactions and manual job finalisation were introduced only later. With these changes, jobs may stay around even after having completed (and they are not ready to be completed a second time), however their patches forgot to reset the ready flag.
Signed-off-by Kevin Wolf <[email protected]>
Quoted 8 lines
--- include/block/blockjob.h | 5 ----- include/qemu/job.h | 3 +++ blockjob.c | 3 +-- job.c | 22 ++++++++++++++++++++++ qemu-img.c | 2 +- tests/test-blockjob.c | 2 +- 6 files changed, 28 insertions(+), 9 deletions(-)
PATCH 36/42 job: Add job_transition_to_ready()
In reply to Kevin WolfThe transition to the READY state was still performed in the BlockJob layer, in the same function that sent the BLOCK_JOB_READY QMP event.
This patch brings the state transition to the Job layer and implements the QMP event using a notifier called from the Job layer, like we already do for other events related to state transitions.
--- include/block/blockjob.h | 3 +++ include/block/blockjob_int.h | 8 -------- include/qemu/job.h | 9 ++++++--- block/mirror.c | 6 +++--- blockjob.c | 36 +++++++++++++++++++----------------- job.c | 16 +++++++++++++--- tests/test-bdrv-drain.c | 2 +- tests/test-blockjob.c | 2 +- 8 files changed, 46 insertions(+), 36 deletions(-)
include/block/blockjob.h +3 -0
@@ -76,6 +76,9 @@ typedef struct BlockJob {/** Called when the job transitions to PENDING */ Notifier pending_notifier; +/** Called when the job transitions to READY */ +Notifier ready_notifier; +/** BlockDriverStates that are involved in this block job */ GSList *nodes; } BlockJob;
include/block/blockjob_int.h +0 -8
@@ -116,14 +116,6 @@ void block_job_drain(Job *job);int64_t block_job_ratelimit_get_delay (BlockJob *job,uint64_t n);/** -* block_job_event_ready: -* @job: The job which is now ready to be completed. -* -* Send a BLOCK_JOB_READY event for the specified job. -*/ -void block_job_event_ready (BlockJob *job); - -/** * block_job_error_action: * @job: The job to signal an error for. * @on_err: The error action setting.
include/qemu/job.h +6 -3
@@ -132,6 +132,9 @@ typedef struct Job {/** Notifiers called when the job transitions to PENDING */ NotifierList on_pending; +/** Notifiers called when the job transitions to READY */ +NotifierList on_ready; +/** Element of the list of jobs */ QLIST_ENTRY (Job) job_list;
@@ -426,6 +429,9 @@ int job_apply_verb(Job *job, JobVerb bv, Error **errp);/** The @job could not be started, free it. */ void job_early_fail (Job *job); +/** Moves the @job from RUNNING to READY */ +void job_transition_to_ready (Job *job); +/** * @job: The job being completed. * @ret: The status code.
@@ -522,7 +528,4 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque);*/ int job_finish_sync (Job *job,void (*finish)(Job *,Error **errp),Error **errp); -/* TODO To be removed from the public interface */ -void job_state_transition (Job *job,JobStatus s1); -#endif
block/mirror.c +3 -3
@@ -727,8 +727,8 @@ static void coroutine_fn mirror_run(void *opaque) }if (s->bdev_length ==0 ) { -/* Report BLOCK_JOB_READY and wait for complete. */ -block_job_event_ready (&s->common); +/* Transition to the READY state and wait for complete. */ +job_transition_to_ready (&s->common.job); s->synced = true;while (!job_is_cancelled (&s->common.job) && !s->should_complete) {job_yield (&s->common.job);
@@ -825,7 +825,7 @@ static void coroutine_fn mirror_run(void *opaque)* report completion. This way, block-job-cancel will leave * the target in a consistent state. */ -block_job_event_ready (&s->common); +job_transition_to_ready (&s->common.job); s->synced = true; }
blockjob.c +19 -17
@@ -339,6 +339,22 @@ static void block_job_event_pending(Notifier *n, void *opaque) &error_abort); } +static void block_job_event_ready (Notifier *n,void *opaque) +{ +BlockJob *job = opaque; + +if (block_job_is_internal (job)) { +return ; + } + +qapi_event_send_block_job_ready (job_type (&job->job), + job->job.id, + job->len, + job->offset, + job->speed, &error_abort); +} + +/* * API for block job drivers and the block layer. These functions are * declared in blockjob_int.h.
@@ -387,13 +403,14 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->finalize_cancelled_notifier.notify = block_job_event_cancelled; job->finalize_completed_notifier.notify = block_job_event_completed; job->pending_notifier.notify = block_job_event_pending; + job->ready_notifier.notify = block_job_event_ready;notifier_list_add (&job->job.on_finalize_cancelled, &job->finalize_cancelled_notifier);notifier_list_add (&job->job.on_finalize_completed, &job->finalize_completed_notifier); -notifier_list_add (&job->job.on_pending, - &job->pending_notifier); +notifier_list_add (&job->job.on_pending, &job->pending_notifier); +notifier_list_add (&job->job.on_ready, &job->ready_notifier);error_setg (&job->blocker,"block device is in use by block job: %s" ,job_type_str (&job->job));
@@ -435,21 +452,6 @@ void block_job_user_resume(Job *job)block_job_iostatus_reset (bjob); } -void block_job_event_ready (BlockJob *job) -{ -job_state_transition (&job->job,JOB_STATUS_READY ); - -if (block_job_is_internal (job)) { -return ; - } - -qapi_event_send_block_job_ready (job_type (&job->job), - job->job.id, - job->len, - job->offset, - job->speed, &error_abort); -} -BlockErrorAction block_job_error_action (BlockJob *job,BlockdevOnError on_err,int is_read,int error) {
job.c +13 -3
@@ -157,9 +157,7 @@ static int job_txn_apply(JobTxn *txn, int fn(Job *), bool lock)return rc; } - -/* TODO Make static once the whole state machine is in job.c */ -void job_state_transition (Job *job,JobStatus s1) +static void job_state_transition (Job *job,JobStatus s1) {JobStatus s0 = job->status;assert (s1 >=0 && s1 <=JOB_STATUS__MAX );
@@ -321,6 +319,7 @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn,notifier_list_init (&job->on_finalize_cancelled);notifier_list_init (&job->on_finalize_completed);notifier_list_init (&job->on_pending); +notifier_list_init (&job->on_ready);job_state_transition (job,JOB_STATUS_CREATED ); aio_timer_init(qemu_get_aio_context(), &job->sleep_timer,
@@ -381,6 +380,11 @@ static int job_event_pending(Job *job)return 0 ; } +static void job_event_ready (Job *job) +{ +notifier_list_notify (&job->on_ready, job); +} +void job_enter_cond (Job *job,bool (*fn)(Job *job)) {if (!job_started (job)) {
@@ -792,6 +796,12 @@ static int job_transition_to_pending(Job *job)return 0 ; } +void job_transition_to_ready (Job *job) +{ +job_state_transition (job,JOB_STATUS_READY ); +job_event_ready (job); +} +static void job_completed_txn_success (Job *job) {JobTxn *txn = job->txn;
tests/test-bdrv-drain.c +1 -1
@@ -505,7 +505,7 @@ static void coroutine_fn test_job_start(void *opaque) {TestBlockJob *s = opaque; -block_job_event_ready (&s->common); +job_transition_to_ready (&s->common.job);while (!s->should_complete) {job_sleep_ns (&s->common.job,100000 ); }
tests/test-blockjob.c +1 -1
@@ -186,7 +186,7 @@ static void coroutine_fn cancel_job_start(void *opaque) }if (!job_is_ready (&s->common.job) && s->should_converge) { -block_job_event_ready (&s->common); +job_transition_to_ready (&s->common.job); }job_sleep_ns (&s->common.job,100000 );
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 36/42] job: Add job_transition_to_ready()
Quoted 18 lines · Kevin Wolf
The transition to the READY state was still performed in the BlockJob layer, in the same function that sent the
BLOCK_JOB_READYQMP event.This patch brings the state transition to the Job layer and implements the QMP event using a notifier called from the Job layer, like we already do for other events related to state transitions.
Signed-off-by Kevin Wolf <[email protected]>--- include/block/blockjob.h | 3 +++ include/block/blockjob_int.h | 8 -------- include/qemu/job.h | 9 ++++++--- block/mirror.c | 6 +++--- blockjob.c | 36 +++++++++++++++++++----------------- job.c | 16 +++++++++++++--- tests/test-bdrv-drain.c | 2 +- tests/test-blockjob.c | 2 +- 8 files changed, 46 insertions(+), 36 deletions(-)
[...]
Quoted 4 lines
include/qemu/job.h +0
-0
[...]
include/qemu/job.h void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque); · line 527
Quoted 7 lines
include/qemu/job.h +0
-3@@ -522,7 +528,4 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque);*/ int job_finish_sync (Job *job,void (*finish)(Job *,Error **errp),Error **errp); -/* TODO To be removed from the public interface */ -void job_state_transition (Job *job,JobStatus s1); -
\o/
Quoted 1 line
include/qemu/job.h +0
-0#endif
[...]
Quoted 4 lines
blockjob.c +0
-0
[...]
blockjob.c void *block_job_create(const char *job_id, const BlockJobDriver *driver, · line 412
Quoted 13 lines
blockjob.c +2
-2@@ -387,13 +403,14 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->finalize_cancelled_notifier.notify = block_job_event_cancelled; job->finalize_completed_notifier.notify = block_job_event_completed; job->pending_notifier.notify = block_job_event_pending; + job->ready_notifier.notify = block_job_event_ready;notifier_list_add (&job->job.on_finalize_cancelled, &job->finalize_cancelled_notifier);notifier_list_add (&job->job.on_finalize_completed, &job->finalize_completed_notifier); -notifier_list_add (&job->job.on_pending, - &job->pending_notifier); +notifier_list_add (&job->job.on_pending, &job->pending_notifier);
Do you want to move this to the patch that introduced this line?
If you do:
Quoted 4 lines
blockjob.c +1
-0+notifier_list_add (&job->job.on_ready, &job->ready_notifier);error_setg (&job->blocker,"block device is in use by block job: %s" ,job_type_str (&job->job));
PATCH 37/42 job: Move progress fields to Job
In reply to Kevin WolfBlockJob has fields .offset and .len, which are actually misnomers today because they are no longer tied to block device sizes, but just progress counters. As such they make a lot of sense in generic Jobs.
This patch moves the fields to Job and renames them to .progress_current and .progress_total to describe their function better.
--- include/block/blockjob.h | 25 ------------------------- include/qemu/job.h | 27 +++++++++++++++++++++++++++ block/backup.c | 8 ++++---- block/commit.c | 4 ++-- block/mirror.c | 4 ++-- block/stream.c | 4 ++-- blockjob.c | 26 ++++++++------------------ job.c | 10 ++++++++++ qemu-img.c | 8 ++++++-- 9 files changed, 61 insertions(+), 55 deletions(-)
include/block/blockjob.h +0 -25
@@ -52,12 +52,6 @@ typedef struct BlockJob {/** Status that is published by the query-block-jobs QMP API */ BlockDeviceIoStatus iostatus; -/** Offset that is published by the query-block-jobs QMP API */ -int64_t offset; - -/** Length that is published by the query-block-jobs QMP API */ -int64_t len; -/** Speed that was set with @block_job_set_speed. */ int64_t speed;
@@ -139,25 +133,6 @@ void block_job_remove_all_bdrv(BlockJob *job);void block_job_set_speed (BlockJob *job,int64_t speed,Error **errp);/** -* block_job_progress_update: -* @job: The job that has made progress -* @done: How much progress the job made -* -* Updates the progress counter of the job. -*/ -void block_job_progress_update (BlockJob *job,uint64_t done); - -/** -* block_job_progress_set_remaining: -* @job: The job whose expected progress end value is set -* @remaining: Expected end value of the progress counter of the job -* -* Sets the expected end value of the progress counter of a job so that a -* completion percentage can be calculated when the progress is updated. -*/ -void block_job_progress_set_remaining (BlockJob *job,uint64_t remaining); - -/** * block_job_query: * @job: The job to get information about. *
include/qemu/job.h +27 -0
@@ -114,6 +114,16 @@ typedef struct Job {/** True if this job should automatically dismiss itself */ bool auto_dismiss; +/** +* Current progress. The unit is arbitrary as long as the ratio between +* progress_current and progress_total represents the estimated percentage +* of work already done. +*/ +int64_t progress_current; + +/** Estimated progress_current value at the completion of the job */ +int64_t progress_total; +/** ret code passed to block_job_completed. */ int ret;
@@ -304,6 +314,23 @@ void job_ref(Job *job);*/ void job_unref (Job *job); +/** +* @job: The job that has made progress +* @done: How much progress the job made +* +* Updates the progress counter of the job. +*/ +void job_progress_update (Job *job,uint64_t done); + +/** +* @job: The job whose expected progress end value is set +* @remaining: Expected end value of the progress counter of the job +* +* Sets the expected end value of the progress counter of a job so that a +* completion percentage can be calculated when the progress is updated. +*/ +void job_progress_set_remaining (Job *job,uint64_t remaining); +/** To be called when a cancelled job is finalised. */ void job_event_cancelled (Job *job);
block/backup.c +4 -4
@@ -160,7 +160,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,* offset field is an opaque progress value, it is not a disk offset. */ job->bytes_read += n; -block_job_progress_update (&job->common, n); +job_progress_update (&job->common.job, n); } out:
@@ -406,8 +406,8 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)bdrv_set_dirty_iter (dbi, next_cluster * job->cluster_size); } -/* TODO block_job_progress_set_remaining() would make more sense */ -block_job_progress_update (&job->common, +/* TODO job_progress_set_remaining() would make more sense */ +job_progress_update (&job->common.job, job->len -hbitmap_count (job->copy_bitmap) * job->cluster_size);bdrv_dirty_iter_free (dbi);
@@ -425,7 +425,7 @@ static void coroutine_fn backup_run(void *opaque)qemu_co_rwlock_init (&job->flush_rwlock); nb_clusters =DIV_ROUND_UP (job->len, job->cluster_size); -block_job_progress_set_remaining (&job->common, job->len); +job_progress_set_remaining (&job->common.job, job->len); job->copy_bitmap =hbitmap_alloc (nb_clusters,0 );if (job->sync_mode ==MIRROR_SYNC_MODE_INCREMENTAL ) {
block/commit.c +2 -2
@@ -150,7 +150,7 @@ static void coroutine_fn commit_run(void *opaque)if (len <0 ) { goto out; } -block_job_progress_set_remaining (&s->common, len); +job_progress_set_remaining (&s->common.job, len); ret = base_len =blk_getlength (s->base);if (base_len <0 ) {
@@ -196,7 +196,7 @@ static void coroutine_fn commit_run(void *opaque) } }/* Publish progress */ -block_job_progress_update (&s->common, n); +job_progress_update (&s->common.job, n);if (copy) { delay_ns =block_job_ratelimit_get_delay (&s->common, n);
block/mirror.c +2 -2
@@ -119,7 +119,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret)bitmap_set (s->cow_bitmap, chunk_num, nb_chunks); }if (!s->initial_zeroing_ongoing) { -block_job_progress_update (&s->common, op->bytes); +job_progress_update (&s->common.job, op->bytes); } }qemu_iovec_destroy (&op->qiov);
@@ -792,7 +792,7 @@ static void coroutine_fn mirror_run(void *opaque)/* cnt is the number of dirty bytes remaining and s->bytes_in_flight is * the number of bytes currently being processed; together those are * the current total operation length */ -block_job_progress_set_remaining (&s->common, s->bytes_in_flight + cnt); +job_progress_set_remaining (&s->common.job, s->bytes_in_flight + cnt);/* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() returns.
block/stream.c +2 -2
@@ -121,7 +121,7 @@ static void coroutine_fn stream_run(void *opaque) ret = len; goto out; } -block_job_progress_set_remaining (&s->common, len); +job_progress_set_remaining (&s->common.job, len); buf =qemu_blockalign (bs,STREAM_BUFFER_SIZE );
@@ -184,7 +184,7 @@ static void coroutine_fn stream_run(void *opaque) ret =0 ;/* Publish progress */ -block_job_progress_update (&s->common, n); +job_progress_update (&s->common.job, n);if (copy) { delay_ns =block_job_ratelimit_get_delay (&s->common, n); }else {
blockjob.c +8 -18
@@ -243,16 +243,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n)return ratelimit_calculate_delay (&job->limit, n); } -void block_job_progress_update (BlockJob *job,uint64_t done) -{ - job->offset += done; -} - -void block_job_progress_set_remaining (BlockJob *job,uint64_t remaining) -{ - job->len = job->offset + remaining; -} -BlockJobInfo *block_job_query (BlockJob *job,Error **errp) {BlockJobInfo *info;
@@ -264,10 +254,10 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) info =g_new0 (BlockJobInfo,1 ); info->type =g_strdup (job_type_str (&job->job)); info->device =g_strdup (job->job.id); - info->len = job->len; info->busy =atomic_read (&job->job.busy); info->paused = job->job.pause_count >0 ; - info->offset = job->offset; + info->offset = job->job.progress_current; + info->len = job->job.progress_total; info->speed = job->speed; info->io_status = job->iostatus; info->ready = job_is_ready(&job->job),
@@ -297,8 +287,8 @@ static void block_job_event_cancelled(Notifier *n, void *opaque)qapi_event_send_block_job_cancelled (job_type (&job->job), job->job.id, - job->len, - job->offset, + job->job.progress_total, + job->job.progress_current, job->speed, &error_abort); }
@@ -318,8 +308,8 @@ static void block_job_event_completed(Notifier *n, void *opaque) qapi_event_send_block_job_completed(job_type(&job->job), job->job.id, - job->len, - job->offset, + job->job.progress_total, + job->job.progress_current, job->speed, !!msg, msg,
@@ -349,8 +339,8 @@ static void block_job_event_ready(Notifier *n, void *opaque)qapi_event_send_block_job_ready (job_type (&job->job), job->job.id, - job->len, - job->offset, + job->job.progress_total, + job->job.progress_current, job->speed, &error_abort); }
job.c +10 -0
@@ -364,6 +364,16 @@ void job_unref(Job *job) } } +void job_progress_update (Job *job,uint64_t done) +{ + job->progress_current += done; +} + +void job_progress_set_remaining (Job *job,uint64_t remaining) +{ + job->progress_total = job->progress_current + remaining; +} +void job_event_cancelled (Job *job) {notifier_list_notify (&job->on_finalize_cancelled, job);
qemu-img.c +6 -2
@@ -864,9 +864,13 @@ static void run_block_job(BlockJob *job, Error **errp)aio_context_acquire (aio_context);job_ref (&job->job);do { +float progress =0.0f ;aio_poll (aio_context, true); -qemu_progress_print (job->len ? - ((float )job->offset / job->len *100.f ) :0.0f ,0 ); +if (job->job.progress_total) { + progress = (float )job->job.progress_current / + job->job.progress_total *100.f ; + } +qemu_progress_print (progress,0 ); }while (!job_is_ready (&job->job) && !job_is_completed (&job->job));if (!job_is_completed (&job->job)) {
Signature · 1 line
2.13.6
Re: [Qemu-devel] [PATCH 37/42] job: Move progress fields to Job
Quoted 8 lines · Kevin Wolf
BlockJob has fields .offset and .len, which are actually misnomers today because they are no longer tied to block device sizes, but just progress counters. As such they make a lot of sense in generic Jobs.
This patch moves the fields to Job and renames them to .progress_current and .progress_total to describe their function better.
Signed-off-by Kevin Wolf <[email protected]>
Quoted 11 lines
--- include/block/blockjob.h | 25 ------------------------- include/qemu/job.h | 27 +++++++++++++++++++++++++++ block/backup.c | 8 ++++---- block/commit.c | 4 ++-- block/mirror.c | 4 ++-- block/stream.c | 4 ++-- blockjob.c | 26 ++++++++------------------ job.c | 10 ++++++++++ qemu-img.c | 8 ++++++-- 9 files changed, 61 insertions(+), 55 deletions(-)
Re: [Qemu-devel] [PATCH 37/42] job: Move progress fields to Job
In reply to Kevin WolfQuoted 29 lines · Kevin Wolf
BlockJob has fields .offset and .len, which are actually misnomers today because they are no longer tied to block device sizes, but just progress counters. As such they make a lot of sense in generic Jobs.
This patch moves the fields to Job and renames them to .progress_current and .progress_total to describe their function better.
Signed-off-by Kevin Wolf <[email protected]>---
+++
b/include/qemu/job.hinclude/qemu/job.h +10
-0@@ -114,6 +114,16 @@ typedef struct Job {/** True if this job should automatically dismiss itself */ bool auto_dismiss; +/** +* Current progress. The unit is arbitrary as long as the ratio between +* progress_current and progress_total represents the estimated percentage +* of work already done. +*/ +int64_t progress_current; + +/** Estimated progress_current value at the completion of the job */ +int64_t progress_total; +/** ret code passed to block_job_completed. */ int ret;
include/qemu/job.h void job_ref(Job *job); · line 321
Quoted 8 lines
include/qemu/job.h +5
-0@@ -304,6 +314,23 @@ void job_ref(Job *job);*/ void job_unref (Job *job); +/** +* @job: The job that has made progress +* @done: How much progress the job made +* +* Updates the progress counter of the job.
Which progress counter? It might be worth calling out that this specifically updates the progress_current counter. Also, it might be worth mentioning that the value is added to the current value of progress_counter, rather than directly assigned.
include/qemu/job.h void job_ref(Job *job); · line 332
Quoted 11 lines
include/qemu/job.h +11
-0+*/ +void job_progress_update (Job *job,uint64_t done); + +/** +* @job: The job whose expected progress end value is set +* @remaining: Expected end value of the progress counter of the job +* +* Sets the expected end value of the progress counter of a job so that a +* completion percentage can be calculated when the progress is updated. +*/ +void job_progress_set_remaining (Job *job,uint64_t remaining);
The name 'remaining' sounds like this is a delta (that progress_total is computed by summing the current progress_current + remaining) rather than directly the new value of progress_total. Is that intended?
job.c void job_unref(Job *job) · line 374
Quoted 13 lines
job.c +9
-0+++ b/job.c@@ -364,6 +364,16 @@ void job_unref(Job *job) } } +void job_progress_update (Job *job,uint64_t done) +{ + job->progress_current += done; +} + +void job_progress_set_remaining (Job *job,uint64_t remaining) +{ + job->progress_total = job->progress_current + remaining;
Okay, so implementation-wise, both functions ARE taking a delta, rather than directly assigning the internal field. The delta can only be positive, but job->progress_total can still shrink over time as needed.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org