add byte-based block_status driver callbacks
v8 cover add byte-based block_status driver callbacks
There are patches floating around to add NBD_CMD_BLOCK_STATUS, but NBD wants to report status on byte granularity (even if the reporting will probably be naturally aligned to sectors or even much higher levels). I've therefore started the task of converting our block status code to report at a byte granularity rather than sectors.
These patches have been around for a while, but it's time to finish it now that 2.12 is open for patches.
Based-on: <[email protected]> (Kevin's [PULL 00/55] Block layer patches)
The overall conversion currently looks like:
part 1: bdrv_is_allocated (merged, commit 51b0a488, 2.10) part 2: dirty-bitmap (merged, commit ca759622, 2.11) part 3: bdrv_get_block_status (merged, commit f0a9c18f, 2.11) part 4: .bdrv_co_block_status (this series, v7 was here [1])
[1] https://lists.gnu.org/archive/html/qemu-devel/2018-01/msg00954.html
Available as a tag at:
git fetch git://repo.or.cz/qemu/ericb.git nbd-byte-callback-v8
Since v7:
- rebase to master (more iscsi context changes, nvme driver is new)
- add a few more R-bys
Eric Blake (21):
block: Add .bdrv_co_block_status() callback
nvme: Drop pointless .bdrv_co_get_block_status()
block: Switch passthrough drivers to .bdrv_co_block_status()
file-posix: Switch to .bdrv_co_block_status()
gluster: Switch to .bdrv_co_block_status()
iscsi: Switch cluster_sectors to byte-based
iscsi: Switch iscsi_allocmap_update() to byte-based
iscsi: Switch to .bdrv_co_block_status()
null: Switch to .bdrv_co_block_status()
parallels: Switch to .bdrv_co_block_status()
qcow: Switch to .bdrv_co_block_status()
qcow2: Switch to .bdrv_co_block_status()
qed: Switch to .bdrv_co_block_status()
raw: Switch to .bdrv_co_block_status() sheepdog: Switch to .bdrv_co_block_status() vdi: Avoid bitrot of debugging code vdi: Switch to .bdrv_co_block_status() vmdk: Switch to .bdrv_co_block_status() vpc: Switch to .bdrv_co_block_status() vvfat: Switch to .bdrv_co_block_status() block: Drop unused .bdrv_co_get_block_status()
include/block/block.h | 14 ++--- include/block/block_int.h | 51 +++++++++------ block/io.c | 86 +++++++++++-------------- block/blkdebug.c | 20 +++--- block/commit.c | 2 +- block/file-posix.c | 62 +++++++++--------- block/gluster.c | 70 ++++++++++----------- block/iscsi.c | 157 ++++++++++++++++++++++++---------------------- block/mirror.c | 2 +- block/null.c | 23 +++---- block/nvme.c | 14 ----- block/parallels.c | 22 ++++--- block/qcow.c | 27 ++++---- block/qcow2.c | 24 +++---- block/qed.c | 84 +++++++++---------------- block/raw-format.c | 16 ++--- block/sheepdog.c | 26 ++++---- block/throttle.c | 2 +- block/vdi.c | 45 +++++++------ block/vmdk.c | 38 +++++------ block/vpc.c | 45 ++++++------- block/vvfat.c | 16 +++-- 22 files changed, 404 insertions(+), 442 deletions(-)
Signature · 1 line
2.14.3
PATCH v8 1/21 block: Add .bdrv_co_block_status() callback
We are gradually moving away from sector-based interfaces, towards byte-based. Now that the block layer exposes byte-based allocation, it's time to tackle the drivers. Add a new callback that operates on as small as byte boundaries. Subsequent patches will then update individual drivers, then finally remove .bdrv_co_get_block_status().
The new code also passes through the 'want_zero' hint, which will allow subsequent patches to further optimize callers that only care about how much of the image is allocated (want_zero is false), rather than full details about runs of zeroes and which offsets the allocation actually maps to (want_zero is true). As part of this effort, fix another part of the documentation: the claim in commit 4c41cb4 that BDRV_BLOCK_ALLOCATED is short for 'DATA || ZERO' is a lie at the block layer (see commit e88ae2264), even though it is how the bit is computed from the driver layer. After all, there are intentionally cases where we return ZERO but not ALLOCATED at the block layer, when we know that a read sees zero because the backing file is too short. Note that the driver interface is thus slightly different than the public interface with regards to which bits will be set, and what guarantees are provided on input.
We also add an assertion that any driver using the new callback will make progress (the only time pnum will be 0 is if the block layer already handled an out-of-bounds request, or if there is an error); the old driver interface did not provide this guarantee, which could lead to some inf-loops in drastic corner-case failures.
---
v7: add R-b
v6: drop now-useless rounding of mid-sector end-of-file hole [Kevin], better documentation of 'want_zero' [Kevin]
v5: rebase to master, typo fix, document more block layer guarantees
v4: rebase to master
v3: no change
v2: improve alignment handling, ensure all iotests still pass
--- include/block/block.h | 14 +++++++------- include/block/block_int.h | 20 +++++++++++++++----- block/io.c | 28 +++++++++++++++++++--------- 3 files changed, 41 insertions(+), 21 deletions(-)
include/block/block.h +7 -7
@@ -115,19 +115,19 @@ typedef struct HDGeometry {* BDRV_BLOCK_ZERO: offset reads as zero * BDRV_BLOCK_OFFSET_VALID: an associated offset exists for accessing raw data * BDRV_BLOCK_ALLOCATED: the content of the block is determined by this -* layer (short for DATA || ZERO), set by block layer -* BDRV_BLOCK_EOF: the returned pnum covers through end of file for this layer +* layer rather than any backing, set by block layer +* BDRV_BLOCK_EOF: the returned pnum covers through end of file for this +* layer, set by block layer * * Internal flag: * BDRV_BLOCK_RAW: for use by passthrough drivers, such as raw, to request * that the block layer recompute the answer from the returned * BDS; must be accompanied by just BDRV_BLOCK_OFFSET_VALID. * -* If BDRV_BLOCK_OFFSET_VALID is set, bits 9-62 (BDRV_BLOCK_OFFSET_MASK) of -* the return value (old interface) or the entire map parameter (new -* interface) represent the offset in the returned BDS that is allocated for -* the corresponding raw data. However, whether that offset actually -* contains data also depends on BDRV_BLOCK_DATA, as follows: +* If BDRV_BLOCK_OFFSET_VALID is set, the map parameter represents the +* host offset within the returned BDS that is allocated for the +* corresponding raw guest data. However, whether that offset +* actually contains data also depends on BDRV_BLOCK_DATA, as follows: * * DATA ZERO OFFSET_VALID * t t t sectors read as zero, returned file is zero at offset
include/block/block_int.h +15 -5
@@ -202,15 +202,25 @@ struct BlockDriver {/* * Building block for bdrv_block_status[_above] and * bdrv_is_allocated[_above]. The driver should answer only -* according to the current layer, and should not set -* BDRV_BLOCK_ALLOCATED, but may set BDRV_BLOCK_RAW. See block.h -* for the meaning of _DATA, _ZERO, and _OFFSET_VALID. The block -* layer guarantees input aligned to request_alignment, as well as -* non-NULL pnum and file. +* according to the current layer, and should only need to set +* BDRV_BLOCK_DATA, BDRV_BLOCK_ZERO, BDRV_BLOCK_OFFSET_VALID, +* and/or BDRV_BLOCK_RAW; if the current layer defers to a backing +* layer, the result should be 0 (and not BDRV_BLOCK_ZERO). See +* block.h for the overall meaning of the bits. As a hint, the +* flag want_zero is true if the caller cares more about precise +* mappings (favor accurate _OFFSET_VALID/_ZERO) or false for +* overall allocation (favor larger *pnum, perhaps by reporting +* _DATA instead of _ZERO). The block layer guarantees input +* clamped to bdrv_getlength() and aligned to request_alignment, +* as well as non-NULL pnum, map, and file; in turn, the driver +* must return an error or set pnum to an aligned non-zero value. */ int64_t coroutine_fn (*bdrv_co_get_block_status)(BlockDriverState *bs,int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file); + int coroutine_fn (*bdrv_co_block_status)(BlockDriverState *bs, +bool want_zero,int64_t offset,int64_t bytes,int64_t *pnum, +int64_t *map,BlockDriverState **file);/* * Invalidate any cached meta-data.
block/io.c +19 -9
@@ -1899,10 +1899,10 @@ int64_t coroutine_fn bdrv_co_get_block_status_from_backing(BlockDriverState *bs,* Drivers not implementing the functionality are assumed to not support * backing files, hence all their sectors are reported as allocated. * -* If 'want_zero' is true, the caller is querying for mapping purposes, -* and the result should include BDRV_BLOCK_OFFSET_VALID and -* BDRV_BLOCK_ZERO where possible; otherwise, the result may omit those -* bits particularly if it allows for a larger value in 'pnum'. +* If 'want_zero' is true, the caller is querying for mapping +* purposes, with a focus on valid BDRV_BLOCK_OFFSET_VALID, _DATA, and +* _ZERO where possible; otherwise, the result favors larger 'pnum', +* with a focus on accurate BDRV_BLOCK_ALLOCATED. * * If 'offset' is beyond the end of the disk image the return value is * BDRV_BLOCK_EOF and 'pnum' is set to 0.
@@ -1959,7 +1959,7 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,/* Must be non-NULL or bdrv_getlength() would have failed */ assert (bs->drv); -if (!bs->drv->bdrv_co_get_block_status) { +if (!bs->drv->bdrv_co_get_block_status && !bs->drv->bdrv_co_block_status) { *pnum = bytes; ret =BDRV_BLOCK_DATA |BDRV_BLOCK_ALLOCATED ;if (offset + bytes == total_size) {
@@ -1976,13 +1976,14 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,bdrv_inc_in_flight (bs);/* Round out to request_alignment boundaries */ -/* TODO: until we have a byte-based driver callback, we also have to -* round out to sectors, even if that is bigger than request_alignment */ - align =MAX (bs->bl.request_alignment,BDRV_SECTOR_SIZE ); + align = bs->bl.request_alignment; +if (bs->drv->bdrv_co_get_block_status && align <BDRV_SECTOR_SIZE ) { + align =BDRV_SECTOR_SIZE ; + } aligned_offset =QEMU_ALIGN_DOWN (offset, align); aligned_bytes =ROUND_UP (offset + bytes, align) - aligned_offset; - { +if (bs->drv->bdrv_co_get_block_status) {int count;/* sectors */ int64_t longret;
@@ -2007,6 +2008,15 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs, } ret = longret & ~BDRV_BLOCK_OFFSET_MASK ; *pnum = count *BDRV_SECTOR_SIZE ; + }else { + ret = bs->drv->bdrv_co_block_status (bs, want_zero, aligned_offset, + aligned_bytes, pnum, &local_map, + &local_file); +if (ret <0 ) { + *pnum =0 ; + goto out; + } +assert (*pnum);/* The block driver must make progress */ }/*
Signature · 1 line
2.14.3
PATCH v8 2/21 nvme: Drop pointless .bdrv_co_get_block_status()
In reply to Eric BlakeCommit bdd6a90 has a bug: drivers should never directly set BDRV_BLOCK_ALLOCATED, but only io.c should do that (as needed). Instead, drivers should report BDRV_BLOCK_DATA if it knows that data comes from this BDS.
But let's look at the bigger picture: semantically, the nvme driver is similar to the nbd, null, and raw drivers (no backing file, all data comes from this BDS). But while two of those other drivers have to supply the callback (null because it can special-case BDRV_BLOCK_ZERO, raw because it can special-case a different offset), in this case the block layer defaults are good enough without the callback at all (similar to nbd).
So, fix the bug by deletion ;)
---
v8: new patch
--- block/nvme.c | 14 -------------- 1 file changed, 14 deletions(-)
block/nvme.c +0 -14
@@ -1068,18 +1068,6 @@ static int nvme_reopen_prepare(BDRVReopenState *reopen_state,return 0 ; } -static int64_t coroutine_fn nvme_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) -{ - *pnum = nb_sectors; - *file = bs; - -return BDRV_BLOCK_ALLOCATED |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); -} -static void nvme_refresh_filename (BlockDriverState *bs,QDict *opts) {QINCREF (opts);
@@ -1179,8 +1167,6 @@ static BlockDriver bdrv_nvme = { .bdrv_co_flush_to_disk = nvme_co_flush, .bdrv_reopen_prepare = nvme_reopen_prepare, - .bdrv_co_get_block_status = nvme_co_get_block_status, - .bdrv_refresh_filename = nvme_refresh_filename, .bdrv_refresh_limits = nvme_refresh_limits,
Signature · 1 line
2.14.3
Re: [Qemu-devel] [PATCH v8 02/21] nvme: Drop pointless .bdrv_co_get_block_status()
Quoted 2 lines · Eric Blake
Commit bdd6a90 has a bug: drivers should never directly set
BDRV_BLOCK_ALLOCATED, but onlyio.cshould do that (as needed).
Doesn't "pointless" in subject hide this is a bugfix?
Quoted 53 lines
Instead, drivers should report
BDRV_BLOCK_DATAif it knows that data comes from this BDS.But let's look at the bigger picture: semantically, the nvme driver is similar to the nbd, null, and raw drivers (no backing file, all data comes from this BDS). But while two of those other drivers have to supply the callback (null because it can special-case
BDRV_BLOCK_ZERO, raw because it can special-case a different offset), in this case the block layer defaults are good enough without the callback at all (similar to nbd).So, fix the bug by deletion ;)
Signed-off-by Eric Blake <[email protected]>---
v8: new patch--- block/nvme.c | 14 -------------- 1 file changed, 14 deletions(-)block/nvme.c +0
-14@@ -1068,18 +1068,6 @@ static int nvme_reopen_prepare(BDRVReopenState *reopen_state,return 0 ; } -static int64_t coroutine_fn nvme_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) -{ - *pnum = nb_sectors; - *file = bs; - -return BDRV_BLOCK_ALLOCATED |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); -} -static void nvme_refresh_filename (BlockDriverState *bs,QDict *opts) {QINCREF (opts);@@ -1179,8 +1167,6 @@ static BlockDriver bdrv_nvme = { .bdrv_co_flush_to_disk = nvme_co_flush, .bdrv_reopen_prepare = nvme_reopen_prepare, - .bdrv_co_get_block_status = nvme_co_get_block_status, - .bdrv_refresh_filename = nvme_refresh_filename, .bdrv_refresh_limits = nvme_refresh_limits,
PATCH v8 3/21 block: Switch passthrough drivers to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the generic helpers, and all passthrough clients (blkdebug, commit, mirror, throttle) accordingly.
---
v6-v7: no change
v5: rebase to master
v4: rebase to interface tweak
v3: rebase to addition of throttle driver
v2: rebase to master, retitle while merging blkdebug, commit, and mirror
--- include/block/block_int.h | 28 ++++++++++++++++------------ block/io.c | 36 ++++++++++++++++++++---------------- block/blkdebug.c | 20 +++++++++++--------- block/commit.c | 2 +- block/mirror.c | 2 +- block/throttle.c | 2 +- 6 files changed, 50 insertions(+), 40 deletions(-)
include/block/block_int.h +16 -12
@@ -1041,23 +1041,27 @@ void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c, uint64_t *nperm, uint64_t *nshared);/* -* Default implementation for drivers to pass bdrv_co_get_block_status() to +* Default implementation for drivers to pass bdrv_co_block_status() to * their file. */ -int64_t coroutine_fn bdrv_co_get_block_status_from_file(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors, -int *pnum, -BlockDriverState **file); +int coroutine_fn bdrv_co_block_status_from_file(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file);/* -* Default implementation for drivers to pass bdrv_co_get_block_status() to +* Default implementation for drivers to pass bdrv_co_block_status() to * their backing file. */ -int64_t coroutine_fn bdrv_co_get_block_status_from_backing(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors, -int *pnum, -BlockDriverState **file); +int coroutine_fn bdrv_co_block_status_from_backing(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file);const char *bdrv_get_parent_name (const BlockDriverState *bs);void blk_dev_change_media_cb (BlockBackend *blk,bool load,Error **errp);bool blk_dev_has_removable_media (BlockBackend *blk);
block/io.c +20 -16
@@ -1868,30 +1868,34 @@ typedef struct BdrvCoBlockStatusData {bool done; } BdrvCoBlockStatusData; -int64_t coroutine_fn bdrv_co_get_block_status_from_file(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors, -int *pnum, -BlockDriverState **file) +int coroutine_fn bdrv_co_block_status_from_file(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file) {assert (bs->file && bs->file->bs); - *pnum = nb_sectors; + *pnum = bytes; + *map = offset; *file = bs->file->bs; -return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); +return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID ; } -int64_t coroutine_fn bdrv_co_get_block_status_from_backing(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors, -int *pnum, -BlockDriverState **file) +int coroutine_fn bdrv_co_block_status_from_backing(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file) {assert (bs->backing && bs->backing->bs); - *pnum = nb_sectors; + *pnum = bytes; + *map = offset; *file = bs->backing->bs; -return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); +return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID ; }/*
block/blkdebug.c +11 -9
@@ -627,15 +627,17 @@ static int coroutine_fn blkdebug_co_pdiscard(BlockDriverState *bs,return bdrv_co_pdiscard (bs->file->bs, offset, bytes); } -static int64_t coroutine_fn blkdebug_co_get_block_status( -BlockDriverState *bs,int64_t sector_num,int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn blkdebug_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file) { -assert (QEMU_IS_ALIGNED (sector_num | nb_sectors, -DIV_ROUND_UP (bs->bl.request_alignment, -BDRV_SECTOR_SIZE ))); -return bdrv_co_get_block_status_from_file (bs, sector_num, nb_sectors, - pnum, file); +assert (QEMU_IS_ALIGNED (offset | bytes, bs->bl.request_alignment)); +return bdrv_co_block_status_from_file (bs, want_zero, offset, bytes, + pnum, map, file); }static void blkdebug_close (BlockDriverState *bs)
@@ -907,7 +909,7 @@ static BlockDriver bdrv_blkdebug = { .bdrv_co_flush_to_disk = blkdebug_co_flush, .bdrv_co_pwrite_zeroes = blkdebug_co_pwrite_zeroes, .bdrv_co_pdiscard = blkdebug_co_pdiscard, - .bdrv_co_get_block_status = blkdebug_co_get_block_status, + .bdrv_co_block_status = blkdebug_co_block_status, .bdrv_debug_event = blkdebug_debug_event, .bdrv_debug_breakpoint = blkdebug_debug_breakpoint,
block/commit.c +1 -1
@@ -265,7 +265,7 @@ static void bdrv_commit_top_child_perm(BlockDriverState *bs, BdrvChild *c,static BlockDriver bdrv_commit_top = { .format_name ="commit_top" , .bdrv_co_preadv = bdrv_commit_top_preadv, - .bdrv_co_get_block_status = bdrv_co_get_block_status_from_backing, + .bdrv_co_block_status = bdrv_co_block_status_from_backing, .bdrv_refresh_filename = bdrv_commit_top_refresh_filename, .bdrv_close = bdrv_commit_top_close, .bdrv_child_perm = bdrv_commit_top_child_perm,
block/mirror.c +1 -1
@@ -1094,7 +1094,7 @@ static BlockDriver bdrv_mirror_top = { .bdrv_co_pwrite_zeroes = bdrv_mirror_top_pwrite_zeroes, .bdrv_co_pdiscard = bdrv_mirror_top_pdiscard, .bdrv_co_flush = bdrv_mirror_top_flush, - .bdrv_co_get_block_status = bdrv_co_get_block_status_from_backing, + .bdrv_co_block_status = bdrv_co_block_status_from_backing, .bdrv_refresh_filename = bdrv_mirror_top_refresh_filename, .bdrv_close = bdrv_mirror_top_close, .bdrv_child_perm = bdrv_mirror_top_child_perm,
block/throttle.c +1 -1
@@ -240,7 +240,7 @@ static BlockDriver bdrv_throttle = { .bdrv_reopen_prepare = throttle_reopen_prepare, .bdrv_reopen_commit = throttle_reopen_commit, .bdrv_reopen_abort = throttle_reopen_abort, - .bdrv_co_get_block_status = bdrv_co_get_block_status_from_file, + .bdrv_co_block_status = bdrv_co_block_status_from_file, .bdrv_co_drain_begin = throttle_co_drain_begin, .bdrv_co_drain_end = throttle_co_drain_end,
Signature · 1 line
2.14.3
PATCH v8 4/21 file-posix: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the file protocol driver accordingly.
In want_zero mode, we continue to report fine-grained hole information (the caller wants as much mapping detail as possible); but when not in that mode, the caller prefers larger *pnum and merely cares about what offsets are allocated at this layer, rather than where the holes live. Since holes still read as zeroes at this layer (rather than deferring to a backing layer), we can take the shortcut of skipping lseek(), and merely state that all bytes are allocated.
We can also drop redundant bounds checks that are already guaranteed by the block layer.
---
v6-v7: no change
v5: drop redundant code
v4: tweak commit message [Fam], rebase to interface tweak
v3: no change
v2: tweak comment, add mapping support
--- block/file-posix.c | 62 +++++++++++++++++++++++++----------------------------- 1 file changed, 29 insertions(+), 33 deletions(-)
block/file-posix.c +29 -33
@@ -2131,25 +2131,24 @@ static int find_allocation(BlockDriverState *bs, off_t start, }/* -* Returns the allocation status of the specified sectors. +* Returns the allocation status of the specified offset. * -* If 'sector_num' is beyond the end of the disk image the return value is 0 -* and 'pnum' is set to 0. +* The block layer guarantees 'offset' and 'bytes' are within bounds. * -* 'pnum' is set to the number of sectors (including and immediately following -* the specified sector) that are known to be in the same +* 'pnum' is set to the number of bytes (including and immediately following +* the specified offset) that are known to be in the same * allocated/unallocated state. * -* 'nb_sectors' is the max value 'pnum' should be set to. If nb_sectors goes -* beyond the end of the disk image it will be clamped. +* 'bytes' is the max value 'pnum' should be set to. */ -static int64_t coroutine_fn raw_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn raw_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes,int64_t *pnum, +int64_t *map, +BlockDriverState **file) { -off_t start, data =0 , hole =0 ; -int64_t total_size; +off_t data =0 , hole =0 ;int ret; ret =fd_open (bs);
@@ -2157,39 +2156,36 @@ static int64_t coroutine_fn raw_co_get_block_status(BlockDriverState *bs,return ret; } - start = sector_num *BDRV_SECTOR_SIZE ; - total_size =bdrv_getlength (bs); -if (total_size <0 ) { -return total_size; - }else if (start >= total_size) { - *pnum =0 ; -return 0 ; - }else if (start + nb_sectors *BDRV_SECTOR_SIZE > total_size) { - nb_sectors =DIV_ROUND_UP (total_size - start,BDRV_SECTOR_SIZE ); +if (!want_zero) { + *pnum = bytes; + *map = offset; + *file = bs; +return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; } - ret =find_allocation (bs, start, &data, &hole); + ret =find_allocation (bs, offset, &data, &hole);if (ret == -ENXIO ) {/* Trailing hole */ - *pnum = nb_sectors; + *pnum = bytes; ret =BDRV_BLOCK_ZERO ; }else if (ret <0 ) {/* No info available, so pretend there are no holes */ - *pnum = nb_sectors; + *pnum = bytes; ret =BDRV_BLOCK_DATA ; - }else if (data == start) { -/* On a data extent, compute sectors to the end of the extent, + }else if (data == offset) { +/* On a data extent, compute bytes to the end of the extent, * possibly including a partial sector at EOF. */ - *pnum =MIN (nb_sectors,DIV_ROUND_UP (hole - start,BDRV_SECTOR_SIZE )); + *pnum =MIN (bytes, hole - offset); ret =BDRV_BLOCK_DATA ; }else { -/* On a hole, compute sectors to the beginning of the next extent. */ -assert (hole == start); - *pnum =MIN (nb_sectors, (data - start) /BDRV_SECTOR_SIZE ); +/* On a hole, compute bytes to the beginning of the next extent. */ +assert (hole == offset); + *pnum =MIN (bytes, data - offset); ret =BDRV_BLOCK_ZERO ; } + *map = offset; *file = bs; -return ret |BDRV_BLOCK_OFFSET_VALID | start; +return ret |BDRV_BLOCK_OFFSET_VALID ; } static coroutine_fn BlockAIOCB *raw_aio_pdiscard(BlockDriverState *bs,
@@ -2282,7 +2278,7 @@ BlockDriver bdrv_file = { .bdrv_close = raw_close, .bdrv_create = raw_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, - .bdrv_co_get_block_status = raw_co_get_block_status, + .bdrv_co_block_status = raw_co_block_status, .bdrv_co_pwrite_zeroes = raw_co_pwrite_zeroes, .bdrv_co_preadv = raw_co_preadv,
Signature · 1 line
2.14.3
PATCH v8 5/21 gluster: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the gluster driver accordingly.
In want_zero mode, we continue to report fine-grained hole information (the caller wants as much mapping detail as possible); but when not in that mode, the caller prefers larger *pnum and merely cares about what offsets are allocated at this layer, rather than where the holes live. Since holes still read as zeroes at this layer (rather than deferring to a backing layer), we can take the shortcut of skipping find_allocation(), and merely state that all bytes are allocated.
We can also drop redundant bounds checks that are already guaranteed by the block layer.
---
v6-v7: no change
v5: drop redundant code
v4: tweak commit message [Fam], rebase to interface tweak
v3: no change
v2: tweak comments [Prasanna], add mapping, drop R-b
--- block/gluster.c | 70 ++++++++++++++++++++++++++++----------------------------- 1 file changed, 34 insertions(+), 36 deletions(-)
block/gluster.c +34 -36
@@ -1362,68 +1362,66 @@ exit: }/* -* Returns the allocation status of the specified sectors. +* Returns the allocation status of the specified offset. * -* If 'sector_num' is beyond the end of the disk image the return value is 0 -* and 'pnum' is set to 0. +* The block layer guarantees 'offset' and 'bytes' are within bounds. * -* 'pnum' is set to the number of sectors (including and immediately following -* the specified sector) that are known to be in the same +* 'pnum' is set to the number of bytes (including and immediately following +* the specified offset) that are known to be in the same * allocated/unallocated state. * -* 'nb_sectors' is the max value 'pnum' should be set to. If nb_sectors goes -* beyond the end of the disk image it will be clamped. +* 'bytes' is the max value 'pnum' should be set to. * -* (Based on raw_co_get_block_status() from file-posix.c.) +* (Based on raw_co_block_status() from file-posix.c.) */ -static int64_t coroutine_fn qemu_gluster_co_get_block_status( -BlockDriverState *bs,int64_t sector_num,int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn qemu_gluster_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file) {BDRVGlusterState *s = bs->opaque; -off_t start, data =0 , hole =0 ; -int64_t total_size; +off_t data =0 , hole =0 ;int ret = -EINVAL ;if (!s->fd) {return ret; } - start = sector_num *BDRV_SECTOR_SIZE ; - total_size =bdrv_getlength (bs); -if (total_size <0 ) { -return total_size; - }else if (start >= total_size) { - *pnum =0 ; -return 0 ; - }else if (start + nb_sectors *BDRV_SECTOR_SIZE > total_size) { - nb_sectors =DIV_ROUND_UP (total_size - start,BDRV_SECTOR_SIZE ); +if (!want_zero) { + *pnum = bytes; + *map = offset; + *file = bs; +return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; } - ret =find_allocation (bs, start, &data, &hole); + ret =find_allocation (bs, offset, &data, &hole);if (ret == -ENXIO ) {/* Trailing hole */ - *pnum = nb_sectors; + *pnum = bytes; ret =BDRV_BLOCK_ZERO ; }else if (ret <0 ) {/* No info available, so pretend there are no holes */ - *pnum = nb_sectors; + *pnum = bytes; ret =BDRV_BLOCK_DATA ; - }else if (data == start) { -/* On a data extent, compute sectors to the end of the extent, + }else if (data == offset) { +/* On a data extent, compute bytes to the end of the extent, * possibly including a partial sector at EOF. */ - *pnum =MIN (nb_sectors,DIV_ROUND_UP (hole - start,BDRV_SECTOR_SIZE )); + *pnum =MIN (bytes, hole - offset); ret =BDRV_BLOCK_DATA ; }else { -/* On a hole, compute sectors to the beginning of the next extent. */ -assert (hole == start); - *pnum =MIN (nb_sectors, (data - start) /BDRV_SECTOR_SIZE ); +/* On a hole, compute bytes to the beginning of the next extent. */ +assert (hole == offset); + *pnum =MIN (bytes, data - offset); ret =BDRV_BLOCK_ZERO ; } + *map = offset; *file = bs; -return ret |BDRV_BLOCK_OFFSET_VALID | start; +return ret |BDRV_BLOCK_OFFSET_VALID ; }
@@ -1451,7 +1449,7 @@ static BlockDriver bdrv_gluster = {#ifdef CONFIG_GLUSTERFS_ZEROFILL .bdrv_co_pwrite_zeroes = qemu_gluster_co_pwrite_zeroes, #endif - .bdrv_co_get_block_status = qemu_gluster_co_get_block_status, + .bdrv_co_block_status = qemu_gluster_co_block_status, .create_opts = &qemu_gluster_create_opts, };
@@ -1479,7 +1477,7 @@ static BlockDriver bdrv_gluster_tcp = {#ifdef CONFIG_GLUSTERFS_ZEROFILL .bdrv_co_pwrite_zeroes = qemu_gluster_co_pwrite_zeroes, #endif - .bdrv_co_get_block_status = qemu_gluster_co_get_block_status, + .bdrv_co_block_status = qemu_gluster_co_block_status, .create_opts = &qemu_gluster_create_opts, };
@@ -1507,7 +1505,7 @@ static BlockDriver bdrv_gluster_unix = {#ifdef CONFIG_GLUSTERFS_ZEROFILL .bdrv_co_pwrite_zeroes = qemu_gluster_co_pwrite_zeroes, #endif - .bdrv_co_get_block_status = qemu_gluster_co_get_block_status, + .bdrv_co_block_status = qemu_gluster_co_block_status, .create_opts = &qemu_gluster_create_opts, };
@@ -1541,7 +1539,7 @@ static BlockDriver bdrv_gluster_rdma = {#ifdef CONFIG_GLUSTERFS_ZEROFILL .bdrv_co_pwrite_zeroes = qemu_gluster_co_pwrite_zeroes, #endif - .bdrv_co_get_block_status = qemu_gluster_co_get_block_status, + .bdrv_co_block_status = qemu_gluster_co_block_status, .create_opts = &qemu_gluster_create_opts, };
Signature · 1 line
2.14.3
PATCH v8 6/21 iscsi: Switch cluster_sectors to byte-based
In reply to Eric BlakeWe are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert all uses of the cluster size in sectors, along with adding assertions that we are not dividing by zero.
Improve some comment grammar while in the area.
---
v8: rebase to master
v2-v7: no change
--- block/iscsi.c | 56 +++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 21 deletions(-)
block/iscsi.c +35 -21
@@ -86,7 +86,7 @@ typedef struct IscsiLun {unsigned long *allocmap;unsigned long *allocmap_valid;long allocmap_size; -int cluster_sectors; +int cluster_size;bool use_16_for_rw;bool write_protected;bool lbpme;
@@ -430,9 +430,10 @@ static int iscsi_allocmap_init(IscsiLun *iscsilun, int open_flags) {iscsi_allocmap_free (iscsilun); +assert (iscsilun->cluster_size); iscsilun->allocmap_size = -DIV_ROUND_UP (sector_lun2qemu (iscsilun->num_blocks, iscsilun), - iscsilun->cluster_sectors); +DIV_ROUND_UP (iscsilun->num_blocks * iscsilun->block_size, + iscsilun->cluster_size); iscsilun->allocmap =bitmap_try_new (iscsilun->allocmap_size);if (!iscsilun->allocmap) {
@@ -440,7 +441,7 @@ static int iscsi_allocmap_init(IscsiLun *iscsilun, int open_flags) }if (open_flags &BDRV_O_NOCACHE ) { -/* in case that cache.direct = on all allocmap entries are +/* when cache.direct = on all allocmap entries are * treated as invalid to force a relookup of the block * status on every read request */ return 0 ;
@@ -461,17 +462,19 @@ iscsi_allocmap_update(IscsiLun *iscsilun, int64_t sector_num, int nb_sectors, bool allocated, bool valid) { int64_t cl_num_expanded, nb_cls_expanded, cl_num_shrunk, nb_cls_shrunk; +int cluster_sectors = iscsilun->cluster_size >>BDRV_SECTOR_BITS ;if (iscsilun->allocmap ==NULL ) {return ; }/* expand to entirely contain all affected clusters */ - cl_num_expanded = sector_num / iscsilun->cluster_sectors; +assert (cluster_sectors); + cl_num_expanded = sector_num / cluster_sectors; nb_cls_expanded =DIV_ROUND_UP (sector_num + nb_sectors, - iscsilun->cluster_sectors) - cl_num_expanded; + cluster_sectors) - cl_num_expanded;/* shrink to touch only completely contained clusters */ - cl_num_shrunk =DIV_ROUND_UP (sector_num, iscsilun->cluster_sectors); - nb_cls_shrunk = (sector_num + nb_sectors) / iscsilun->cluster_sectors + cl_num_shrunk =DIV_ROUND_UP (sector_num, cluster_sectors); + nb_cls_shrunk = (sector_num + nb_sectors) / cluster_sectors - cl_num_shrunk;if (allocated) {bitmap_set (iscsilun->allocmap, cl_num_expanded, nb_cls_expanded);
@@ -535,9 +538,12 @@ iscsi_allocmap_is_allocated(IscsiLun *iscsilun, int64_t sector_num,if (iscsilun->allocmap ==NULL ) {return true; } - size =DIV_ROUND_UP (sector_num + nb_sectors, iscsilun->cluster_sectors); +assert (iscsilun->cluster_size); + size =DIV_ROUND_UP (sector_num + nb_sectors, + iscsilun->cluster_size >>BDRV_SECTOR_BITS );return !(find_next_bit (iscsilun->allocmap, size, - sector_num / iscsilun->cluster_sectors) == size); + sector_num *BDRV_SECTOR_SIZE / + iscsilun->cluster_size) == size); } static inline bool iscsi_allocmap_is_valid(IscsiLun *iscsilun,
@@ -547,9 +553,12 @@ static inline bool iscsi_allocmap_is_valid(IscsiLun *iscsilun,if (iscsilun->allocmap_valid ==NULL ) {return false; } - size =DIV_ROUND_UP (sector_num + nb_sectors, iscsilun->cluster_sectors); +assert (iscsilun->cluster_size); + size =DIV_ROUND_UP (sector_num + nb_sectors, + iscsilun->cluster_size >>BDRV_SECTOR_BITS );return (find_next_zero_bit (iscsilun->allocmap_valid, size, - sector_num / iscsilun->cluster_sectors) == size); + sector_num *BDRV_SECTOR_SIZE / + iscsilun->cluster_size) == size); }static int coroutine_fn
@@ -793,16 +802,21 @@ static int coroutine_fn iscsi_co_readv(BlockDriverState *bs,BlockDriverState *file;/* check the block status from the beginning of the cluster * containing the start sector */ -int64_t ret =iscsi_co_get_block_status (bs, - sector_num - sector_num % iscsilun->cluster_sectors, -BDRV_REQUEST_MAX_SECTORS , &pnum, &file); +int cluster_sectors = iscsilun->cluster_size >>BDRV_SECTOR_BITS ; +int head; +int64_t ret; + +assert (cluster_sectors); + head = sector_num % cluster_sectors; + ret =iscsi_co_get_block_status (bs, sector_num - head, +BDRV_REQUEST_MAX_SECTORS , &pnum, + &file);if (ret <0 ) {return ret; }/* if the whole request falls into an unallocated area we can avoid -* to read and directly return zeroes instead */ -if (ret &BDRV_BLOCK_ZERO && - pnum >= nb_sectors + sector_num % iscsilun->cluster_sectors) { +* reading and directly return zeroes instead */ +if (ret &BDRV_BLOCK_ZERO && pnum >= nb_sectors + head) {qemu_iovec_memset (iov,0 ,0x00 , iov->size);return 0 ; }
@@ -1953,8 +1967,8 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags,* reasonable size */ if (iscsilun->bl.opt_unmap_gran * iscsilun->block_size >=4 *1024 && iscsilun->bl.opt_unmap_gran * iscsilun->block_size <=16 *1024 *1024 ) { - iscsilun->cluster_sectors = (iscsilun->bl.opt_unmap_gran * - iscsilun->block_size) >>BDRV_SECTOR_BITS ; + iscsilun->cluster_size = iscsilun->bl.opt_unmap_gran * + iscsilun->block_size;if (iscsilun->lbprz) { ret =iscsi_allocmap_init (iscsilun, bs->open_flags); }
@@ -2163,7 +2177,7 @@ static int iscsi_get_info(BlockDriverState *bs, BlockDriverInfo *bdi) {IscsiLun *iscsilun = bs->opaque; bdi->unallocated_blocks_are_zero = iscsilun->lbprz; - bdi->cluster_size = iscsilun->cluster_sectors *BDRV_SECTOR_SIZE ; + bdi->cluster_size = iscsilun->cluster_size;return 0 ; }
Signature · 1 line
2.14.3
PATCH v8 7/21 iscsi: Switch iscsi_allocmap_update() to byte-based
In reply to Eric BlakeWe are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert all uses of the allocmap (no semantic change). Callers that already had bytes available are simpler, and callers that now scale to bytes will be easier to switch to byte-based in the future.
---
v7: rebase to master, simple enough to keep ack
v3-v6: no change
v2: rebase to count/bytes rename
--- block/iscsi.c | 90 +++++++++++++++++++++++++++++------------------------------ 1 file changed, 44 insertions(+), 46 deletions(-)
block/iscsi.c +44 -46
@@ -458,24 +458,22 @@ static int iscsi_allocmap_init(IscsiLun *iscsilun, int open_flags) }static void -iscsi_allocmap_update (IscsiLun *iscsilun,int64_t sector_num, -int nb_sectors,bool allocated,bool valid) +iscsi_allocmap_update (IscsiLun *iscsilun,int64_t offset, +int64_t bytes,bool allocated,bool valid) {int64_t cl_num_expanded, nb_cls_expanded, cl_num_shrunk, nb_cls_shrunk; -int cluster_sectors = iscsilun->cluster_size >>BDRV_SECTOR_BITS ;if (iscsilun->allocmap ==NULL ) {return ; }/* expand to entirely contain all affected clusters */ -assert (cluster_sectors); - cl_num_expanded = sector_num / cluster_sectors; - nb_cls_expanded =DIV_ROUND_UP (sector_num + nb_sectors, - cluster_sectors) - cl_num_expanded; +assert (iscsilun->cluster_size); + cl_num_expanded = offset / iscsilun->cluster_size; + nb_cls_expanded =DIV_ROUND_UP (offset + bytes, + iscsilun->cluster_size) - cl_num_expanded;/* shrink to touch only completely contained clusters */ - cl_num_shrunk =DIV_ROUND_UP (sector_num, cluster_sectors); - nb_cls_shrunk = (sector_num + nb_sectors) / cluster_sectors - - cl_num_shrunk; + cl_num_shrunk =DIV_ROUND_UP (offset, iscsilun->cluster_size); + nb_cls_shrunk = (offset + bytes) / iscsilun->cluster_size - cl_num_shrunk;if (allocated) {bitmap_set (iscsilun->allocmap, cl_num_expanded, nb_cls_expanded); }else {
@@ -498,26 +496,26 @@ iscsi_allocmap_update(IscsiLun *iscsilun, int64_t sector_num, }static void -iscsi_allocmap_set_allocated (IscsiLun *iscsilun,int64_t sector_num, -int nb_sectors) +iscsi_allocmap_set_allocated (IscsiLun *iscsilun,int64_t offset, +int64_t bytes) { -iscsi_allocmap_update (iscsilun, sector_num, nb_sectors, true, true); +iscsi_allocmap_update (iscsilun, offset, bytes, true, true); }static void -iscsi_allocmap_set_unallocated (IscsiLun *iscsilun,int64_t sector_num, -int nb_sectors) +iscsi_allocmap_set_unallocated (IscsiLun *iscsilun,int64_t offset, +int64_t bytes) {/* Note: if cache.direct=on the fifth argument to iscsi_allocmap_update * is ignored, so this will in effect be an iscsi_allocmap_set_invalid. */ -iscsi_allocmap_update (iscsilun, sector_num, nb_sectors, false, true); +iscsi_allocmap_update (iscsilun, offset, bytes, false, true); } -static void iscsi_allocmap_set_invalid (IscsiLun *iscsilun,int64_t sector_num, -int nb_sectors) +static void iscsi_allocmap_set_invalid (IscsiLun *iscsilun,int64_t offset, +int64_t bytes) { -iscsi_allocmap_update (iscsilun, sector_num, nb_sectors, false, false); +iscsi_allocmap_update (iscsilun, offset, bytes, false, false); }static void iscsi_allocmap_invalidate (IscsiLun *iscsilun)
@@ -531,34 +529,30 @@ static void iscsi_allocmap_invalidate(IscsiLun *iscsilun) }static inline bool -iscsi_allocmap_is_allocated (IscsiLun *iscsilun,int64_t sector_num, -int nb_sectors) +iscsi_allocmap_is_allocated (IscsiLun *iscsilun,int64_t offset, +int64_t bytes) {unsigned long size;if (iscsilun->allocmap ==NULL ) {return true; }assert (iscsilun->cluster_size); - size =DIV_ROUND_UP (sector_num + nb_sectors, - iscsilun->cluster_size >>BDRV_SECTOR_BITS ); + size =DIV_ROUND_UP (offset + bytes, iscsilun->cluster_size);return !(find_next_bit (iscsilun->allocmap, size, - sector_num *BDRV_SECTOR_SIZE / - iscsilun->cluster_size) == size); + offset / iscsilun->cluster_size) == size); }static inline bool iscsi_allocmap_is_valid (IscsiLun *iscsilun, -int64_t sector_num,int nb_sectors) +int64_t offset,int64_t bytes) {unsigned long size;if (iscsilun->allocmap_valid ==NULL ) {return false; }assert (iscsilun->cluster_size); - size =DIV_ROUND_UP (sector_num + nb_sectors, - iscsilun->cluster_size >>BDRV_SECTOR_BITS ); + size =DIV_ROUND_UP (offset + bytes, iscsilun->cluster_size);return (find_next_zero_bit (iscsilun->allocmap_valid, size, - sector_num *BDRV_SECTOR_SIZE / - iscsilun->cluster_size) == size); + offset / iscsilun->cluster_size) == size); }static int coroutine_fn
@@ -640,14 +634,16 @@ retry: }if (iTask.status !=SCSI_STATUS_GOOD ) { -iscsi_allocmap_set_invalid (iscsilun, sector_num, nb_sectors); +iscsi_allocmap_set_invalid (iscsilun, sector_num *BDRV_SECTOR_SIZE , + nb_sectors *BDRV_SECTOR_SIZE );error_report ("iSCSI WRITE10/16 failed at lba %" PRIu64": %s" , lba, iTask.err_str); r = iTask.err_code; goto out_unlock; } -iscsi_allocmap_set_allocated (iscsilun, sector_num, nb_sectors); +iscsi_allocmap_set_allocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , + nb_sectors *BDRV_SECTOR_SIZE ); out_unlock:qemu_mutex_unlock (&iscsilun->mutex);
@@ -747,9 +743,11 @@ retry: }if (ret &BDRV_BLOCK_ZERO ) { -iscsi_allocmap_set_unallocated (iscsilun, sector_num, *pnum); +iscsi_allocmap_set_unallocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , + *pnum *BDRV_SECTOR_SIZE ); }else { -iscsi_allocmap_set_allocated (iscsilun, sector_num, *pnum); +iscsi_allocmap_set_allocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , + *pnum *BDRV_SECTOR_SIZE ); }if (*pnum > nb_sectors) {
@@ -789,15 +787,19 @@ static int coroutine_fn iscsi_co_readv(BlockDriverState *bs,/* if cache.direct is off and we have a valid entry in our allocation map * we can skip checking the block status and directly return zeroes if * the request falls within an unallocated area */ -if (iscsi_allocmap_is_valid (iscsilun, sector_num, nb_sectors) && - !iscsi_allocmap_is_allocated (iscsilun, sector_num, nb_sectors)) { +if (iscsi_allocmap_is_valid (iscsilun, sector_num *BDRV_SECTOR_SIZE , + nb_sectors *BDRV_SECTOR_SIZE ) && + !iscsi_allocmap_is_allocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , + nb_sectors *BDRV_SECTOR_SIZE )) {qemu_iovec_memset (iov,0 ,0x00 , iov->size);return 0 ; }if (nb_sectors >=ISCSI_CHECKALLOC_THRES && - !iscsi_allocmap_is_valid (iscsilun, sector_num, nb_sectors) && - !iscsi_allocmap_is_allocated (iscsilun, sector_num, nb_sectors)) { + !iscsi_allocmap_is_valid (iscsilun, sector_num *BDRV_SECTOR_SIZE , + nb_sectors *BDRV_SECTOR_SIZE ) && + !iscsi_allocmap_is_allocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , + nb_sectors *BDRV_SECTOR_SIZE )) {int pnum;BlockDriverState *file;/* check the block status from the beginning of the cluster
@@ -1160,8 +1162,7 @@ retry: goto retry; } -iscsi_allocmap_set_invalid (iscsilun, offset >>BDRV_SECTOR_BITS , - bytes >>BDRV_SECTOR_BITS ); +iscsi_allocmap_set_invalid (iscsilun, offset, bytes);if (iTask.status ==SCSI_STATUS_CHECK_CONDITION ) {/* the target might fail with a check condition if it
@@ -1274,8 +1275,7 @@ retry: }if (iTask.status !=SCSI_STATUS_GOOD ) { -iscsi_allocmap_set_invalid (iscsilun, offset >>BDRV_SECTOR_BITS , - bytes >>BDRV_SECTOR_BITS ); +iscsi_allocmap_set_invalid (iscsilun, offset, bytes);error_report ("iSCSI WRITESAME10/16 failed at lba %" PRIu64": %s" , lba, iTask.err_str); r = iTask.err_code;
@@ -1283,11 +1283,9 @@ retry: }if (flags &BDRV_REQ_MAY_UNMAP ) { -iscsi_allocmap_set_invalid (iscsilun, offset >>BDRV_SECTOR_BITS , - bytes >>BDRV_SECTOR_BITS ); +iscsi_allocmap_set_invalid (iscsilun, offset, bytes); }else { -iscsi_allocmap_set_allocated (iscsilun, offset >>BDRV_SECTOR_BITS , - bytes >>BDRV_SECTOR_BITS ); +iscsi_allocmap_set_allocated (iscsilun, offset, bytes); } out_unlock:
Signature · 1 line
2.14.3
PATCH v8 8/21 iscsi: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the iscsi driver accordingly. In this case, it is handy to teach iscsi_co_block_status() to handle a NULL map and file parameter, even though the block layer passes non-NULL values, because we also call the function directly. For now, there are no optimizations done based on the want_zero flag.
We can also make the simplification of asserting that the block layer passed in aligned values.
---
v8: rebase to master
v7: rebase to master
v6: no change
v5: assert rather than check for alignment
v4: rebase to interface tweaks
v3: no change
v2: rebase to mapping parameter
--- block/iscsi.c | 67 ++++++++++++++++++++++++++++------------------------------- 1 file changed, 32 insertions(+), 35 deletions(-)
block/iscsi.c +32 -35
@@ -653,36 +653,36 @@ out_unlock: -static int64_t coroutine_fn iscsi_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn iscsi_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *pnum, +int64_t *map, +BlockDriverState **file) {IscsiLun *iscsilun = bs->opaque;struct scsi_get_lba_status *lbas =NULL ;struct scsi_lba_status_descriptor *lbasd =NULL ;struct IscsiTask iTask;uint64_t lba; -int64_t ret; +int ret;iscsi_co_init_iscsitask (iscsilun, &iTask); -if (!is_sector_request_lun_aligned (sector_num, nb_sectors, iscsilun)) { - ret = -EINVAL ; - goto out; - } +assert (QEMU_IS_ALIGNED (offset | bytes, iscsilun->block_size));/* default to all sectors allocated */ - ret =BDRV_BLOCK_DATA ; - ret |= (sector_num <<BDRV_SECTOR_BITS ) |BDRV_BLOCK_OFFSET_VALID ; - *pnum = nb_sectors; + ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; +if (map) { + *map = offset; + } + *pnum = bytes;/* LUN does not support logical block provisioning */ if (!iscsilun->lbpme) { goto out; } - lba =sector_qemu2lun (sector_num, iscsilun); + lba = offset / iscsilun->block_size;qemu_mutex_lock (&iscsilun->mutex); retry:
@@ -727,12 +727,12 @@ retry: lbasd = &lbas->descriptors[0 ]; -if (sector_qemu2lun (sector_num, iscsilun) != lbasd->lba) { +if (lba != lbasd->lba) { ret = -EIO ; goto out_unlock; } - *pnum =sector_lun2qemu (lbasd->num_blocks, iscsilun); + *pnum = lbasd->num_blocks * iscsilun->block_size;if (lbasd->provisioning ==SCSI_PROVISIONING_TYPE_DEALLOCATED || lbasd->provisioning ==SCSI_PROVISIONING_TYPE_ANCHORED ) {
@@ -743,15 +743,13 @@ retry: }if (ret &BDRV_BLOCK_ZERO ) { -iscsi_allocmap_set_unallocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , - *pnum *BDRV_SECTOR_SIZE ); +iscsi_allocmap_set_unallocated (iscsilun, offset, *pnum); }else { -iscsi_allocmap_set_allocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , - *pnum *BDRV_SECTOR_SIZE ); +iscsi_allocmap_set_allocated (iscsilun, offset, *pnum); } -if (*pnum > nb_sectors) { - *pnum = nb_sectors; +if (*pnum > bytes) { + *pnum = bytes; } out_unlock:qemu_mutex_unlock (&iscsilun->mutex);
@@ -760,7 +758,7 @@ out:if (iTask.task !=NULL ) {scsi_free_scsi_task (iTask.task); } -if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID ) { +if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID && file) { *file = bs; }return ret;
@@ -800,25 +798,24 @@ static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, nb_sectors * BDRV_SECTOR_SIZE) && !iscsi_allocmap_is_allocated(iscsilun, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE)) { -int pnum; -BlockDriverState *file; +int64_t pnum;/* check the block status from the beginning of the cluster * containing the start sector */ -int cluster_sectors = iscsilun->cluster_size >>BDRV_SECTOR_BITS ; -int head; -int64_t ret; +int64_t head; +int ret; -assert (cluster_sectors); - head = sector_num % cluster_sectors; - ret =iscsi_co_get_block_status (bs, sector_num - head, -BDRV_REQUEST_MAX_SECTORS , &pnum, - &file); +assert (iscsilun->cluster_size); + head = (sector_num *BDRV_SECTOR_SIZE ) % iscsilun->cluster_size; + ret =iscsi_co_block_status (bs, false, + sector_num *BDRV_SECTOR_SIZE - head, +BDRV_REQUEST_MAX_BYTES , &pnum,NULL ,NULL );if (ret <0 ) {return ret; }/* if the whole request falls into an unallocated area we can avoid * reading and directly return zeroes instead */ -if (ret &BDRV_BLOCK_ZERO && pnum >= nb_sectors + head) { +if (ret &BDRV_BLOCK_ZERO && + pnum >= nb_sectors *BDRV_SECTOR_SIZE + head) {qemu_iovec_memset (iov,0 ,0x00 , iov->size);return 0 ; }
@@ -2218,7 +2215,7 @@ static BlockDriver bdrv_iscsi = { .bdrv_truncate = iscsi_truncate, .bdrv_refresh_limits = iscsi_refresh_limits, - .bdrv_co_get_block_status = iscsi_co_get_block_status, + .bdrv_co_block_status = iscsi_co_block_status, .bdrv_co_pdiscard = iscsi_co_pdiscard, .bdrv_co_pwrite_zeroes = iscsi_co_pwrite_zeroes, .bdrv_co_readv = iscsi_co_readv,
@@ -2253,7 +2250,7 @@ static BlockDriver bdrv_iser = { .bdrv_truncate = iscsi_truncate, .bdrv_refresh_limits = iscsi_refresh_limits, - .bdrv_co_get_block_status = iscsi_co_get_block_status, + .bdrv_co_block_status = iscsi_co_block_status, .bdrv_co_pdiscard = iscsi_co_pdiscard, .bdrv_co_pwrite_zeroes = iscsi_co_pwrite_zeroes, .bdrv_co_readv = iscsi_co_readv,
Signature · 1 line
2.14.3
Re: [Qemu-devel] [PATCH v8 08/21] iscsi: Switch to .bdrv_co_block_status()
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 57 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the iscsi driver accordingly. In this case, it is handy to teach
iscsi_co_block_status()to handle a NULL map and file parameter, even though the block layer passes non-NULL values, because we also call the function directly. For now, there are no optimizations done based on thewant_zeroflag.We can also make the simplification of asserting that the block layer passed in aligned values.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Fam Zheng <[email protected]>---
v8: rebase to master
v7: rebase to master
v6: no change
v5: assert rather than check for alignment
v4: rebase to interface tweaks
v3: no change
v2: rebase to mapping parameter--- block/iscsi.c | 67 ++++++++++++++++++++++++++++------------------------------- 1 file changed, 32 insertions(+), 35 deletions(-)block/iscsi.c +6
-9@@ -653,36 +653,36 @@ out_unlock: -static int64_t coroutine_fn iscsi_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn iscsi_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *pnum, +int64_t *map, +BlockDriverState **file) {IscsiLun *iscsilun = bs->opaque;struct scsi_get_lba_status *lbas =NULL ;struct scsi_lba_status_descriptor *lbasd =NULL ;struct IscsiTask iTask;uint64_t lba; -int64_t ret; +int ret;iscsi_co_init_iscsitask (iscsilun, &iTask); -if (!is_sector_request_lun_aligned (sector_num, nb_sectors, iscsilun)) { - ret = -EINVAL ; - goto out; - }
block/iscsi.c out_unlock: · line 677
Quoted 10 lines
block/iscsi.c +5
-3+assert (QEMU_IS_ALIGNED (offset | bytes, iscsilun->block_size));/* default to all sectors allocated */ - ret =BDRV_BLOCK_DATA ; - ret |= (sector_num <<BDRV_SECTOR_BITS ) |BDRV_BLOCK_OFFSET_VALID ; - *pnum = nb_sectors; + ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; +if (map) { + *map = offset; + }
Can map ever be NULL? You didn't have that check in other drivers.
Quoted 48 lines
block/iscsi.c +8
-9+ *pnum = bytes;/* LUN does not support logical block provisioning */ if (!iscsilun->lbpme) { goto out; } - lba =sector_qemu2lun (sector_num, iscsilun); + lba = offset / iscsilun->block_size;qemu_mutex_lock (&iscsilun->mutex); retry:@@ -727,12 +727,12 @@ retry: lbasd = &lbas->descriptors[0 ]; -if (sector_qemu2lun (sector_num, iscsilun) != lbasd->lba) { +if (lba != lbasd->lba) { ret = -EIO ; goto out_unlock; } - *pnum =sector_lun2qemu (lbasd->num_blocks, iscsilun); + *pnum = lbasd->num_blocks * iscsilun->block_size;if (lbasd->provisioning ==SCSI_PROVISIONING_TYPE_DEALLOCATED || lbasd->provisioning ==SCSI_PROVISIONING_TYPE_ANCHORED ) {@@ -743,15 +743,13 @@ retry: }if (ret &BDRV_BLOCK_ZERO ) { -iscsi_allocmap_set_unallocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , - *pnum *BDRV_SECTOR_SIZE ); +iscsi_allocmap_set_unallocated (iscsilun, offset, *pnum); }else { -iscsi_allocmap_set_allocated (iscsilun, sector_num *BDRV_SECTOR_SIZE , - *pnum *BDRV_SECTOR_SIZE ); +iscsi_allocmap_set_allocated (iscsilun, offset, *pnum); } -if (*pnum > nb_sectors) { - *pnum = nb_sectors; +if (*pnum > bytes) { + *pnum = bytes; } out_unlock:qemu_mutex_unlock (&iscsilun->mutex);
block/iscsi.c out: · line 763
Quoted 7 lines
block/iscsi.c +1
-1@@ -760,7 +758,7 @@ out:if (iTask.task !=NULL ) {scsi_free_scsi_task (iTask.task); } -if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID ) { +if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID && file) { *file = bs; }
Can file ever be NULL?
block/iscsi.c static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, · line 798
Quoted 3 lines
block/iscsi.c +0
-0return ret;@@ -800,25 +798,24 @@ static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, nb_sectors * BDRV_SECTOR_SIZE) &&
No iscsi_co_preadv() yet... :-(
Quoted 13 lines
block/iscsi.c +3
-5!iscsi_allocmap_is_allocated(iscsilun, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE)) { -int pnum; -BlockDriverState *file; +int64_t pnum;/* check the block status from the beginning of the cluster * containing the start sector */ -int cluster_sectors = iscsilun->cluster_size >>BDRV_SECTOR_BITS ; -int head; -int64_t ret; +int64_t head; +int ret;
block/iscsi.c static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, · line 811
Quoted 10 lines
block/iscsi.c +5
-5-assert (cluster_sectors); - head = sector_num % cluster_sectors; - ret =iscsi_co_get_block_status (bs, sector_num - head, -BDRV_REQUEST_MAX_SECTORS , &pnum, - &file); +assert (iscsilun->cluster_size); + head = (sector_num *BDRV_SECTOR_SIZE ) % iscsilun->cluster_size; + ret =iscsi_co_block_status (bs, false, + sector_num *BDRV_SECTOR_SIZE - head, +BDRV_REQUEST_MAX_BYTES , &pnum,NULL ,NULL );
It doesn't make a difference with your current implementation because it ignores want_zero, but consistent with your approach that want_zero=false returns just that everyhting is allocated for drivers without support for backing files, I think you want want_zero=true here.
block/iscsi.c static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, · line 821
Quoted 11 lines
block/iscsi.c +2
-1if (ret <0 ) {return ret; }/* if the whole request falls into an unallocated area we can avoid * reading and directly return zeroes instead */ -if (ret &BDRV_BLOCK_ZERO && pnum >= nb_sectors + head) { +if (ret &BDRV_BLOCK_ZERO && + pnum >= nb_sectors *BDRV_SECTOR_SIZE + head) {qemu_iovec_memset (iov,0 ,0x00 , iov->size);return 0 ; }
Kevin
Re: [Qemu-devel] [PATCH v8 08/21] iscsi: Switch to .bdrv_co_block_status()
Quoted 7 lines · Kevin Wolf
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 6 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the iscsi driver accordingly. In this case, it is handy to teach
iscsi_co_block_status()to handle a NULL map and file parameter, even though the block layer passes non-NULL values, because we also call the function directly. For now, there are no optimizations done based on thewant_zeroflag.
[1]
Quoted 17 lines
Quoted 9 lines
We can also make the simplification of asserting that the block layer passed in aligned values.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Fam Zheng <[email protected]>/* default to all sectors allocated */block/iscsi.c out_unlock: · line 677
Quoted 7 lines
block/iscsi.c +4
-3- ret =BDRV_BLOCK_DATA ; - ret |= (sector_num <<BDRV_SECTOR_BITS ) |BDRV_BLOCK_OFFSET_VALID ; - *pnum = nb_sectors; + ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; +if (map) { + *map = offset; + }
Quoted 1 line
Can map ever be NULL? You didn't have that check in other drivers.
The documentation in block_int.h states that io.c never passes NULL for map. However, see my commit message [1], and the code below [2], for why THIS driver has to check for NULL.
Quoted 10 lines
block/iscsi.c out: · line 763
Quoted 8 lines
block/iscsi.c +1
-1@@ -760,7 +758,7 @@ out:if (iTask.task !=NULL ) {scsi_free_scsi_task (iTask.task); } -if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID ) { +if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID && file) { *file = bs; }Can file ever be NULL?
Ditto.
Quoted 6 lines
block/iscsi.c static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, · line 798
Quoted 4 lines
block/iscsi.c +0
-0return ret;@@ -800,25 +798,24 @@ static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, nb_sectors * BDRV_SECTOR_SIZE) &&No
iscsi_co_preadv()yet... :-(
Yeah, but that's for another day.
Quoted 14 lines
Quoted 14 lines
block/iscsi.c +3
-5!iscsi_allocmap_is_allocated(iscsilun, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE)) { -int pnum; -BlockDriverState *file; +int64_t pnum;/* check the block status from the beginning of the cluster * containing the start sector */ -int cluster_sectors = iscsilun->cluster_size >>BDRV_SECTOR_BITS ; -int head; -int64_t ret; +int64_t head; +int ret;
Quoted 11 lines
block/iscsi.c static int coroutine_fn iscsi_co_readv(BlockDriverState *bs, · line 811
Quoted 10 lines
block/iscsi.c +5
-5-assert (cluster_sectors); - head = sector_num % cluster_sectors; - ret =iscsi_co_get_block_status (bs, sector_num - head, -BDRV_REQUEST_MAX_SECTORS , &pnum, - &file); +assert (iscsilun->cluster_size); + head = (sector_num *BDRV_SECTOR_SIZE ) % iscsilun->cluster_size; + ret =iscsi_co_block_status (bs, false, + sector_num *BDRV_SECTOR_SIZE - head, +BDRV_REQUEST_MAX_BYTES , &pnum,NULL ,NULL );
[2] This is the reason that THIS driver has to check for NULL, even though the block layer never passes NULL.
Quoted 4 lines
It doesn't make a difference with your current implementation because it ignores
want_zero, but consistent with your approach that want_zero=false returns just that everyhting is allocated for drivers without support for backing files, I think you want want_zero=true here.
Makes sense. If that's the only tweak, can you make it while taking the series, or will I need to respin?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH v8 9/21 null: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the null driver accordingly.
---
v6-v7: no change
v5: minor fix to type of 'ret'
v4: rebase to interface tweak
v3: no change
v2: rebase to mapping parameter
--- block/null.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
block/null.c +12 -11
@@ -223,22 +223,23 @@ static int null_reopen_prepare(BDRVReopenState *reopen_state,return 0 ; } -static int64_t coroutine_fn null_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn null_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *pnum, +int64_t *map, +BlockDriverState **file) {BDRVNullState *s = bs->opaque; -off_t start = sector_num *BDRV_SECTOR_SIZE ; +int ret =BDRV_BLOCK_OFFSET_VALID ; - *pnum = nb_sectors; + *pnum = bytes; + *map = offset; *file = bs;if (s->read_zeroes) { -return BDRV_BLOCK_OFFSET_VALID | start |BDRV_BLOCK_ZERO ; - }else { -return BDRV_BLOCK_OFFSET_VALID | start; + ret |=BDRV_BLOCK_ZERO ; } +return ret; }static void null_refresh_filename (BlockDriverState *bs,QDict *opts)
@@ -270,7 +271,7 @@ static BlockDriver bdrv_null_co = { .bdrv_co_flush_to_disk = null_co_flush, .bdrv_reopen_prepare = null_reopen_prepare, - .bdrv_co_get_block_status = null_co_get_block_status, + .bdrv_co_block_status = null_co_block_status, .bdrv_refresh_filename = null_refresh_filename, };
@@ -290,7 +291,7 @@ static BlockDriver bdrv_null_aio = { .bdrv_aio_flush = null_aio_flush, .bdrv_reopen_prepare = null_reopen_prepare, - .bdrv_co_get_block_status = null_co_get_block_status, + .bdrv_co_block_status = null_co_block_status, .bdrv_refresh_filename = null_refresh_filename, };
Signature · 1 line
2.14.3
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 42 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the null driver accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>---
v6-v7: no change
v5: minor fix to type of 'ret'
v4: rebase to interface tweak
v3: no change
v2: rebase to mapping parameter--- block/null.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)block/null.c +8
-6@@ -223,22 +223,23 @@ static int null_reopen_prepare(BDRVReopenState *reopen_state,return 0 ; } -static int64_t coroutine_fn null_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, -BlockDriverState **file) +static int coroutine_fn null_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *pnum, +int64_t *map, +BlockDriverState **file) {BDRVNullState *s = bs->opaque; -off_t start = sector_num *BDRV_SECTOR_SIZE ; +int ret =BDRV_BLOCK_OFFSET_VALID ; - *pnum = nb_sectors; + *pnum = bytes; + *map = offset;
block/null.c static int null_reopen_prepare(BDRVReopenState *reopen_state, · line 243
Quoted 10 lines
block/null.c +2
-3*file = bs;if (s->read_zeroes) { -return BDRV_BLOCK_OFFSET_VALID | start |BDRV_BLOCK_ZERO ; - }else { -return BDRV_BLOCK_OFFSET_VALID | start; + ret |=BDRV_BLOCK_ZERO ; } +return ret; }
Preexisting, but I think this return value is wrong. OFFSET_VALID without DATA is to documented to have the following semantics:
* DATA ZERO OFFSET_VALID
* f t t sectors preallocated, read as zero, returned file not * necessarily zero at offset * f f t sectors preallocated but read from backing_hd, * returned file contains garbage at offset
I'm not sure what OFFSET_VALID is even supposed to mean for null.
Or in fact, what it is supposed to mean for any protocol driver, because normally it just means I can use this offset for accessing bs->file. But protocol drivers don't have a bs->file, so it's interesting to see that they still all set this flag.
OFFSET_VALID | DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), but OFFSET_VALID without DATA probably isn't.
Kevin
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Quoted 21 lines · Kevin Wolf
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 9 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the null driver accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>if (s->read_zeroes) {block/null.c static int null_reopen_prepare(BDRVReopenState *reopen_state, · line 243
Quoted 7 lines
block/null.c +2
-3-return BDRV_BLOCK_OFFSET_VALID | start |BDRV_BLOCK_ZERO ; - }else { -return BDRV_BLOCK_OFFSET_VALID | start; + ret |=BDRV_BLOCK_ZERO ; } +return ret; }Preexisting, but I think this return value is wrong.
OFFSET_VALIDwithout DATA is to documented to have the following semantics:
Quoted 7 lines
* DATA ZERO
OFFSET_VALID* f t t sectors preallocated, read as zero, returned file not * necessarily zero at offset * f f t sectors preallocated but read from backing_hd, * returned file contains garbage at offsetI'm not sure what
OFFSET_VALIDis even supposed to mean for null.
Yeah, and I was even thinking about that a bit yesterday when figuring out what to do with nvme. It does highlight the fact that you get garbage when reading from the null driver (unless the zero option was enabled, then ZERO is set and you know you read zeros instead) - but there no pointer that is preallocated (whether it contains garbage or otherwise) that you can actually dereference to read what the guest would see.
Quoted 11 lines
Or in fact, what it is supposed to mean for any protocol driver, because normally it just means I can use this offset for accessing
bs->file. But protocol drivers don't have abs->file, so it's interesting to see that they still all set this flag.
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.
Hmm, you're probably right. Maybe that means I should tweak the documentation to be more explicit: for a format driver, OFFSET_VALID can always be used (and *file will be set to the underlying protocol driver); but for a protocol driver, OFFSET_VALID only makes sense if *file is the BDS itself and there is an actual buffer to read (that is, the protocol driver must also be returning DATA and/or ZERO). Or maybe we can indeed state that protocol drivers always set *file to NULL (there is no further backing file to reference), and thus never need to return OFFSET_VALID (but I'm not sure whether that will accidentally propagate back up the call stack and negatively affect status queries of format drivers).
Since it is pre-existing, should I respin to address the issue in a separate patch, or should that be a followup after this series?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Am 14.02.2018 um 15:44 hat Eric Blake geschrieben:
Quoted 60 lines · Kevin Wolf
Quoted 21 lines
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 9 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the null driver accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>if (s->read_zeroes) {block/null.c static int null_reopen_prepare(BDRVReopenState *reopen_state, · line 243
Quoted 7 lines
block/null.c +2
-3-return BDRV_BLOCK_OFFSET_VALID | start |BDRV_BLOCK_ZERO ; - }else { -return BDRV_BLOCK_OFFSET_VALID | start; + ret |=BDRV_BLOCK_ZERO ; } +return ret; }Preexisting, but I think this return value is wrong.
OFFSET_VALIDwithout DATA is to documented to have the following semantics:Quoted 7 lines
* DATA ZERO
OFFSET_VALID* f t t sectors preallocated, read as zero, returned file not * necessarily zero at offset * f f t sectors preallocated but read from backing_hd, * returned file contains garbage at offsetI'm not sure what
OFFSET_VALIDis even supposed to mean for null.Yeah, and I was even thinking about that a bit yesterday when figuring out what to do with nvme. It does highlight the fact that you get garbage when reading from the null driver (unless the zero option was enabled, then ZERO is set and you know you read zeros instead) - but there no pointer that is preallocated (whether it contains garbage or otherwise) that you can actually dereference to read what the guest would see.
Quoted 11 lines
Or in fact, what it is supposed to mean for any protocol driver, because normally it just means I can use this offset for accessing
bs->file. But protocol drivers don't have abs->file, so it's interesting to see that they still all set this flag.
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.Hmm, you're probably right. Maybe that means I should tweak the documentation to be more explicit: for a format driver,
OFFSET_VALIDcan always be used (and *file will be set to the underlying protocol driver); but for a protocol driver,OFFSET_VALIDonly makes sense if *file is the BDS itself and there is an actual buffer to read (that is, the protocol driver must also be returning DATA and/or ZERO). Or maybe we can indeed state that protocol drivers always set *file to NULL (there is no further backing file to reference), and thus never need to returnOFFSET_VALID(but I'm not sure whether that will accidentally propagate back up the call stack and negatively affect status queries of format drivers).
Quoted 2 lines
Since it is pre-existing, should I respin to address the issue in a separate patch, or should that be a followup after this series?
It's a more fundamental question that shouldn't hold up this series. I just wanted to raise it while I was looking at it. So yes, a followup is fine.
Kevin
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
In reply to Kevin WolfQuoted 14 lines · Kevin Wolf
Review
Quoted 10 lines
(quoted diff) +3
-3+static int coroutine_fn null_co_block_status(BlockDriverState *bs, if (s->read_zeroes) { - return BDRV_BLOCK_OFFSET_VALID | start | BDRV_BLOCK_ZERO; - } else { - return BDRV_BLOCK_OFFSET_VALID | start; + ret |= BDRV_BLOCK_ZERO; } + return ret; }Preexisting, but I think this return value is wrong.
OFFSET_VALIDwithout DATA is to documented to have the following semantics:
Quoted 7 lines
* DATA ZERO
OFFSET_VALID* f t t sectors preallocated, read as zero, returned file not * necessarily zero at offset * f f t sectors preallocated but read from backing_hd, * returned file contains garbage at offsetI'm not sure what
OFFSET_VALIDis even supposed to mean for null.
I'm finally getting around to playing with this.
Quoted 4 lines
Or in fact, what it is supposed to mean for any protocol driver, because
normally it just means I can use this offset for accessingbs->file. But > protocol drivers don't have abs->file, so it's interesting to see that they still all set this flag.
More precisely, it means "I can use this offset for accessing the returned *file". Format and filter drivers set *file = bs->file (ie. their protocol layer), but protocol drivers set *file = bs (ie. themselves). As long as you read it as "the offset is valid in the returned *file", and are careful as to _which_ BDS gets returned in *file*, it can still make sense.
So next I tried playing with a patch, to see how much returning OFFSET_VALID with DATA matters; and it turns out is is easily observable anywhere that the underlying protocol bleeds through to the format layer (particularly the raw format driver):
$ echo abc > tmp $ truncate --size=10M tmp
pre-patch:
$ ./qemu-img map --output=json tmp
[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": 0},
{ "start": 4096, "length": 10481664, "depth": 0, "zero": true, "data": false, "offset": 4096}]turn off OFFSET_VALID at the protocol layer:
block/file-posix.c +2 -6
@@ -2158,9 +2158,7 @@ static int coroutine_fn raw_co_block_status(BlockDriverState *bs,if (!want_zero) { *pnum = bytes; - *map = offset; - *file = bs; -return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; +return BDRV_BLOCK_DATA ; } ret =find_allocation (bs, offset, &data, &hole);
@@ -2183,9 +2181,7 @@ static int coroutine_fn raw_co_block_status(BlockDriverState *bs, *pnum =MIN (bytes, data - offset); ret =BDRV_BLOCK_ZERO ; } - *map = offset; - *file = bs; -return ret |BDRV_BLOCK_OFFSET_VALID ; +return ret; } static coroutine_fn BlockAIOCB *raw_aio_pdiscard(BlockDriverState *bs,
post-patch:
$ ./qemu-img map --output=json tmp
[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true},
{ "start": 4096, "length": 10481664, "depth": 0, "zero": true, "data": false}]Quoted 6 lines
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.
So OFFSET_VALID | DATA for a protocol BDS is not just convenient, but necessary to avoid breaking qemu-img map output. But you are also right that OFFSET_VALID without data makes little sense at a protocol layer. So with that in mind, I'm auditing all of the protocol layers to make sure OFFSET_VALID ends up as something sane.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben:
Quoted 11 lines
Quoted 5 lines
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.So
OFFSET_VALID| DATA for a protocol BDS is not just convenient, but necessary to avoid breaking qemu-img map output. But you are also right thatOFFSET_VALIDwithout data makes little sense at a protocol layer. So with that in mind, I'm auditing all of the protocol layers to make sureOFFSET_VALIDends up as something sane.
That's one way to look at it.
The other way is that qemu-img map shouldn't ask the protocol layer for its offset because it already knows the offset (it is what it passes as a parameter to bdrv_co_block_status).
Anyway, it's probably not worth changing the interface, we should just make sure that the return values of the individual drivers are consistent.
Kevin
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Quoted 15 lines · Kevin Wolf
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben:
Quoted 11 lines
Quoted 5 lines
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.So
OFFSET_VALID| DATA for a protocol BDS is not just convenient, but necessary to avoid breaking qemu-img map output. But you are also right thatOFFSET_VALIDwithout data makes little sense at a protocol layer. So with that in mind, I'm auditing all of the protocol layers to make sureOFFSET_VALIDends up as something sane.That's one way to look at it.
Quoted 7 lines
The other way is that qemu-img map shouldn't ask the protocol layer for its offset because it already knows the offset (it is what it passes as a parameter to
bdrv_co_block_status).Anyway, it's probably not worth changing the interface, we should just make sure that the return values of the individual drivers are consistent.
Yet another inconsistency, and it's making me scratch my head today.
By the way, in my byte-based stuff that is now pending on your tree, I tried hard to NOT change semantics or the set of flags returned by a given driver, and we agreed that's why you'd accept the series as-is and make me do this followup exercise. But it's looking like my followups may end up touching a lot of the same drivers again, now that I'm looking at what the semantics SHOULD be (and whatever I do end up tweaking, I will at least make sure that iotests is still happy with it).
First, let's read what states the NBD spec is proposing:
Quoted 6 lines
It defines the following flags for the flags field:
NBD_STATE_HOLE(bit 0): if set, the block represents a hole (and future writes to that area may cause fragmentation or encounter an ENOSPC error); if clear, the block is allocated or the server could not otherwise determine its status. Note that the use ofNBD_CMD_TRIMis related to this status, but that the server MAY report a hole even whereNBD_CMD_TRIMhas not been requested, and also that a server MAY report that the block is allocated even whereNBD_CMD_TRIMhas been requested.NBD_STATE_ZERO(bit 1): if set, the block contents read as all zeroes; if clear, the block contents are not known. Note that the use ofNBD_CMD_WRITE_ZEROESis related to this status, but that the server MAY report zeroes even whereNBD_CMD_WRITE_ZEROEShas not been requested, and also that a server MAY report unknown content even whereNBD_CMD_WRITE_ZEROEShas been requested.It is not an error for a server to report that a region of the export has both
NBD_STATE_HOLEset andNBD_STATE_ZEROclear. The contents of such an area are undefined, and a client reading such an area should make no assumption as to its contents or stability.
So here's how Vladimir proposed implementing it in his series (written before my byte-based block status stuff went in to your tree):
https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg04038.html
Server side (3/9):
+ int ret = bdrv_block_status_above(bs, NULL, offset, tail_bytes, &num,
+ NULL, NULL);
+ if (ret < 0) {
+ return ret;
+ }
+
+ flags = (ret & BDRV_BLOCK_ALLOCATED ? 0 : NBD_STATE_HOLE) |
+ (ret & BDRV_BLOCK_ZERO ? NBD_STATE_ZERO : 0);
Client side (6/9):
+ *pnum = extent.length >> BDRV_SECTOR_BITS;
+ return (extent.flags & NBD_STATE_HOLE ? 0 : BDRV_BLOCK_DATA) |
+ (extent.flags & NBD_STATE_ZERO ? BDRV_BLOCK_ZERO : 0);
Does anything there strike you as odd? In isolation, they seemed fine to me, but side-by-side, I'm scratching my head: the server queries the block layer, and turns BDRV_BLOCK_ALLOCATED into !NBD_STATE_HOLE; the client side then takes the NBD protocol and tries to turn it back into information to feed the block layer, where !NBD_STATE_HOLE now feeds BDRV_BLOCK_DATA. Why the different choice of bits?
Part of the story is that right now, we document that ONLY the block layer sets _ALLOCATED, in io.c, as a result of the driver layer returning HOLE || ZERO (there are cases where the block layer can return ZERO but not ALLOCATED, because the driver layer returned 0 but the block layer still knows that area reads as zero). So Victor's patch matches the fact that the driver shouldn't set ALLOCATED. Still, if we are tying ALLOCATED to whether there is a hole, then that seems like information we should be getting from the driver, not something synthesized after we've left the driver!
Then there's the question of file-posix.c: what should it return for a hole, ZERO|OFFSET_VALID or DATA|ZERO|OFFSET_VALID? The wording in block.h implies that if DATA is not set, then the area reads as zero to the guest, but may have indeterminate value on the underlying file - but we KNOW that a hole in a POSIX file reads as 0 rather than having indeterminate value, and returning DATA fits the current documentation (but doing so bleeds through to at least 'qemu-img map --output=json' for the raw format). I think we're overloading too many things into DATA (which layer of the chain feeds what the guest sees, and do we have a hole or is storage allocated for the data). The only uses of BDRV_BLOCK_ALLOCATED are in the computation of bdrv_is_allocated(), in qcow2 measure, and in qemu-img compare, which all really do care about the semantics of "does THIS layer provide the guest image, or do I defer to a backing layer". But the question NBD wants answered is "do I know whether there is a hole in the storage" There are also relatively few clients of BDRV_BLOCK_DATA (mirror.c, qemu-img, bdrv_co_block_status_above), and I wonder if some of them are more worried about BDRV_BLOCK_ALLOCATED instead.
I'm thinking of revamping things to still keep four bits, but with new names and semantics as follows:
BDRV_BLOCK_LOCAL - the guest gets this portion of the file from this BDS, rather than the backing chain - makes sense for format drivers, pointless for protocol drivers BDRV_BLOCK_ZERO - this portion of the file reads as zeroesBDRV_BLOCK_ALLOC - this portion of the file has reserved disk spaceBDRV_BLOCK_OFFSET_VALID - offset for accessing raw data
For format drivers:
L Z A O read as zero, returned file is zero at offset L - A O read as valid from file at offset L Z - O read as zero, but returned file has hole at offset L - - O preallocated at offset but reads as garbage - bug? L Z A - read as zero, but from unknown offset with storage L - A - read as valid, but from unknown offset (including compressed, encrypted) L Z - - read as zero, but from unknown offset with hole L - - - preallocated but no offset known - bug? - Z A O read defers to backing layer, but protocol layer contains allocated zeros at offset - - A O read defers to backing layer, but preallocated at offset - Z - O bug - - - O bug - Z A - bug - - A - bug - Z - - bug - - - - read defers to backing layer
For protocol drivers:
- Z A O read as zero, offset is allocated - - A O read as data, offset is allocated - Z - O read as zero, offset is hole - - - O bug? - Z A - read as zero, but from unknown offset with storage - - A - read as valid, but from unknown offset - Z - - read as zero, but from unknown offset with hole - - - - can't access this portion of file
With the new bit definitions, any driver that returns RAW (necessarily with OFFSET_VALID) will have the block layer set LOCAL in addition to whatever the next layer returns (turning the protocol driver's response into the correct format layer response). Protocol drivers can omit the callback and get the sane default of '- - A O' mapped in place (or would that be better as '- - A -'?). file-posix.c would return either '- - A O' (after SEEK_DATA) or '- Z - O' (after SEEK_HOLE). NBD would map ZERO to NBD_STATE_ZERO, and ALLOC to !NBD_STATE_HOLE, in both server (block-layer-to-NBD-protocol) and client (NBD-protocol-to-block-layer). Format drivers would set LOCAL themselves (rather than the block layer synthesizing it).
bdrv_is_allocated will still let clients learn which layers are local without grabbing full mapping information, but is tied to the BDRV_BLOCK_LOCAL bit. Optimizations made during mirror based on whether and qemu-img compare previously based on BDRV_BLOCK_ALLOCATED are now based on BDRV_BLOCK_LOCAL, those based on BDRV_BLOCK_DATA are now based on BDRV_BLOCK_ALLOC.
Thoughts?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Am 24.02.2018 um 00:38 hat Eric Blake geschrieben:
Quoted 26 lines · Kevin Wolf
Quoted 15 lines
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben:
Quoted 11 lines
Quoted 5 lines
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.So
OFFSET_VALID| DATA for a protocol BDS is not just convenient, but necessary to avoid breaking qemu-img map output. But you are also right thatOFFSET_VALIDwithout data makes little sense at a protocol layer. So with that in mind, I'm auditing all of the protocol layers to make sureOFFSET_VALIDends up as something sane.That's one way to look at it.
Quoted 7 lines
The other way is that qemu-img map shouldn't ask the protocol layer for its offset because it already knows the offset (it is what it passes as a parameter to
bdrv_co_block_status).Anyway, it's probably not worth changing the interface, we should just make sure that the return values of the individual drivers are consistent.
Yet another inconsistency, and it's making me scratch my head today.
Quoted 7 lines
By the way, in my byte-based stuff that is now pending on your tree, I tried hard to NOT change semantics or the set of flags returned by a given driver, and we agreed that's why you'd accept the series as-is and make me do this followup exercise. But it's looking like my followups may end up touching a lot of the same drivers again, now that I'm looking at what the semantics SHOULD be (and whatever I do end up tweaking, I will at least make sure that iotests is still happy with it).
Hm, that's unfortunate, but I don't think we should hold up your first series just so we can touch the drivers only once.
Quoted 31 lines
First, let's read what states the NBD spec is proposing:
Quoted 6 lines
It defines the following flags for the flags field:
NBD_STATE_HOLE(bit 0): if set, the block represents a hole (and future writes to that area may cause fragmentation or encounter an ENOSPC error); if clear, the block is allocated or the server could not otherwise determine its status. Note that the use ofNBD_CMD_TRIMis related to this status, but that the server MAY report a hole even whereNBD_CMD_TRIMhas not been requested, and also that a server MAY report that the block is allocated even whereNBD_CMD_TRIMhas been requested.NBD_STATE_ZERO(bit 1): if set, the block contents read as all zeroes; if clear, the block contents are not known. Note that the use ofNBD_CMD_WRITE_ZEROESis related to this status, but that the server MAY report zeroes even whereNBD_CMD_WRITE_ZEROEShas not been requested, and also that a server MAY report unknown content even whereNBD_CMD_WRITE_ZEROEShas been requested.It is not an error for a server to report that a region of the export has both
NBD_STATE_HOLEset andNBD_STATE_ZEROclear. The contents of such an area are undefined, and a client reading such an area should make no assumption as to its contents or stability.So here's how Vladimir proposed implementing it in his series (written before my byte-based block status stuff went in to your tree):
https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg04038.htmlServer side (3/9):
+ int ret = bdrv_block_status_above(bs, NULL, offset,
tail_bytes,
&num,(quoted diff) +7
-0+ NULL, NULL); + if (ret < 0) { + return ret; + } + + flags = (ret & BDRV_BLOCK_ALLOCATED ? 0 : NBD_STATE_HOLE) | + (ret & BDRV_BLOCK_ZERO ? NBD_STATE_ZERO : 0);Client side (6/9):
(quoted diff) +3
-0+ *pnum = extent.length >> BDRV_SECTOR_BITS; + return (extent.flags & NBD_STATE_HOLE ? 0 : BDRV_BLOCK_DATA) | + (extent.flags & NBD_STATE_ZERO ? BDRV_BLOCK_ZERO : 0);
Quoted 1 line
Does anything there strike you as odd?
Two things I noticed while reading the above:
1. NBD doesn't consider backing files, so the definition of holes becomes ambiguous. Is a hole any block that isn't allocated in the top layer (may cause fragmentation or encounter an ENOSPC error) or is it any block that isn't allocated anywhere in the whole backing chain (may read as non-zero)?
Considering that there is a separate NBD_STATE_ZERO and nothing forbids a state of NBD_STATE_HOLE without NBD_STATE_ZERO, maybe the former is more useful. The code you quote implements the latter.
Maybe if we go with the former, we should add a note to the NBD spec that explictly says that NBD_STATE_HOLE doesn't imply any specific content that is returned on reads.
2. Using BDRV_BLOCK_ALLOCATED to determine NBD_STATE_HOLE seems wrong. A (not preallocated) zero cluster in qcow2 returns BDRV_BLOCK_ALLOCATED (because we don't fall through to the backing file) even though I think it's a hole. BDRV_BLOCK_DATA should be used there (which makes it consistent with the other direction).
Quoted 6 lines
In isolation, they seemed fine to
me, but side-by-side, I'm scratching my head: the server queries the block layer, and turnsBDRV_BLOCK_ALLOCATEDinto !NBD_STATE_HOLE; the client side then takes the NBD protocol and tries to turn it back into information to feed the block layer, where !NBD_STATE_HOLE now feedsBDRV_BLOCK_DATA. Why the different choice of bits?
Which is actually consistent in the end, becaue BDRV_BLOCK_DATA implies BDRV_BLOCK_ALLOCATED.
Essentially, assuming a simple backing chain 'base <- overlay', we got these combinations to represent in NBD (with my suggestion of the flags to use):
1. Cluster allocated in overlay
a. non-zero data 0 b. explicit zeroes 0 or ZERO 2. Cluster marked zero in overlay HOLE | ZERO 3. Cluster preallocated/zero in overlay ZERO 4. Cluster unallocated in overlay a. Cluster allocated in base (non-zero) HOLE b. Cluster allocated in base (zero) HOLE or HOLE | ZERO c. Cluster marked zero in base HOLE | ZERO d. Cluster preallocated/zero in base HOLE | ZERO e. Cluster unallocated in base HOLE | ZERO
Instead of 'base' you can read 'anywhere in the backing chain' and the flags should stay the same.
So !BDRV_BLOCK_ALLOCATED (i.e. falling through to the backing file) does indeed imply NBD_STATE_HOLE, but so does case 2, which is just !DATA.
Quoted 8 lines
Part of the story is that right now, we document that ONLY the block layer sets
_ALLOCATED, inio.c, as a result of the driver layer returning HOLE || ZERO (there are cases where the block layer can return ZERO but not ALLOCATED, because the driver layer returned 0 but the block layer still knows that area reads as zero). So Victor's patch matches the fact that the driver shouldn't set ALLOCATED. Still, if we are tying ALLOCATED to whether there is a hole, then that seems like information we should be getting from the driver, not something synthesized after we've left the driver!
Yes, I'm getting this impression, too. If your documentation says something like "not allocated or unknown offset" (for !OFFSET_VALID), you should probably be using one bit more to distinguish these cases.
Quoted 7 lines
Then there's the question of
file-posix.c: what should it return for a hole, ZERO|OFFSET_VALID or DATA|ZERO|OFFSET_VALID? The wording inblock.himplies that if DATA is not set, then the area reads as zero to the guest, but may have indeterminate value on the underlying file - but we KNOW that a hole in a POSIX file reads as 0 rather than having indeterminate value, and returning DATA fits the current documentation (but doing so bleeds through to at least 'qemu-img map --output=json' for the raw format).
The "underlying file" for the file-posix layer (i.e. the filesystem) is a block device. A hole in a file is defined by not mapping to anywhere on the block device, so DATA should not be set.
DATA | ZERO would mean that the block is actually allocated on the block device, but it still reads as zero.
The thing that is inconsistent here is OFFSET_VALID and the offset returned because the protocol layer refers to itself there instead of referring to the "underlying file". If done consistently, it would have to return the offset on the block device (which is useless information in QEMU, so I suggested not to set OFFSET_VALID there at all - but we decided that that's too much hassle for no practical benefit).
Quoted 3 lines
I think we're overloading too many things into DATA (which layer of the chain feeds what the guest sees, and do we have a hole or is storage allocated for the data).
As I understand it, DATA should only be about holes (in the sense of not being mapped to anywhere in bs->file or any other child apart from the backing file).
Documentation does define OFFSET_VALID without DATA, though, as preallocation. Maybe preallocation would better be expressed as DATA, but without ALLOCATED.
Quoted 9 lines
The only uses of
BDRV_BLOCK_ALLOCATEDare in the computation ofbdrv_is_allocated(), in qcow2 measure, and in qemu-img compare, which all really do care about the semantics of "does THIS layer provide the guest image, or do I defer to a backing layer". But the question NBD wants answered is "do I know whether there is a hole in the storage" There are also relatively few clients ofBDRV_BLOCK_DATA(mirror.c, qemu-img,bdrv_co_block_status_above), and I wonder if some of them are more worried aboutBDRV_BLOCK_ALLOCATEDinstead.
Quoted 6 lines
I'm thinking of revamping things to still keep four bits, but with new names and semantics as follows:
BDRV_BLOCK_LOCAL- the guest gets this portion of the file from this BDS, rather than the backing chain - makes sense for format drivers, pointless for protocol drivers
This is the old BDRV_BLOCK_ALLOCATED.
Data almost never comes from the qcow2 layer, so what this really means is that data doesn't come from bs->backing.
Quoted 1 line
BDRV_BLOCK_ZERO- this portion of the file reads as zeroes
Same as before.
Quoted 1 line
BDRV_BLOCK_ALLOC- this portion of the file has reserved disk space
I think this is essentially what I believe BDRV_BLOCK_DATA should have been. "Disk space" isn't clearly defined, but "there is a mapping to a child node (except bs->backing)" seems to be close enough to what you have in mind.
Quoted 1 line
BDRV_BLOCK_OFFSET_VALID- offset for accessing raw data
Same as before.
As I understand it, you're just renaming the existing flags. I'm not sure if this is a good idea, especially with ALLOC(ATED), which changes the meaning. This is pretty confusing. I suggest MAPPED as an alternative.
Quoted 2 lines
For format drivers:
L Z A O read as zero, returned file is zero at offset
This is not what your definition above said. ZERO is about reading from the node itself rather than reading from bs->file.
I interpret this as: Read as zero, space is preallocated in the image file, content in the image file is undefined.
Quoted 2 lines
L - A O read as valid from file at offset
L Z - O read as zero, but returned file has hole at offset
Read as zero, no mapping into bs->file, but the offset in bs->file is valid. Doesn't make sense, OFFSET_VALID (O) should always imply MAPPED (A).
Quoted 1 line
L - - O preallocated at offset but reads as garbage - bug?
Again O without A - yes, a bug.
Quoted 1 line
L Z A - read as zero, but from unknown offset with storage
And the space is preallocated in a non-backing child, though not necessarily zeroed.
Quoted 4 lines
L - A - read as valid, but from unknown offset (including compressed,
encrypted)
L Z - - read as zero, but from unknown offset with hole
L - - - preallocated but no offset known - bug?
No, not preallocated, because MAPPED isn't set.
This is a block that isn't mapped to a block in any child node and doesn't read as zero. It might be the appropriate response for the null driver with read-zeroes=off.
Quoted 2 lines
- Z A O read defers to backing layer, but protocol layer contains
allocated zeros at offset
No. Space is preallocated for this block, but read defers to the backing layer and we know that the backing layer provides zeros.
This is not something that a driver should return, but
it's a valid return value from bdrv_co_block_status(). One example for this is reading from an offset that is higher than the length of the backing file.
Quoted 1 line
- - A O read defers to backing layer, but preallocated at offset
Yes, this one is preallocation, finally.
Quoted 3 lines
- Z - O bug - - - O bug - Z A - bug
Same as '- Z A O' except that the offset can't be directly accessed in the child node (e.g. because this is an encrypted image).
Quoted 1 line
- - A - bug
Preallocated, but reads from backing file. Offset can't be directly accessed in the child node.
Quoted 1 line
- Z - - bug
Read defers to backing layer and we know it will read zeros. Like for '- Z A O', this isn't something that a driver should return, but makes sense as a return value of bdrv_co_block_status().
Quoted 11 lines
- - - - read defers to backing layer
For protocol drivers:
- Z A O read as zero, offset is allocated - - A O read as data, offset is allocated - Z - O read as zero, offset is hole - - - O bug? - Z A - read as zero, but from unknown offset with storage - - A - read as valid, but from unknown offset - Z - - read as zero, but from unknown offset with hole - - - - can't access this portion of file
Why don't you set LOCAL? It's usually true for protocol drivers that they don't get their data from a backing file (though in theory you could imagine a protocol driver with backing file support).
As discussed before, OFFSET_VALID doesn't really make sense here because we don't return offsets of the image file on the block device, but we only decided to keep it because of convenience. But if we change everything, then this should be changed, too.
While the offset still refers to the same node for protocol drivers, "unknown offset" doesn't make any sense. There is no mapping involved that could be unknown. We really only have four cases for protocols.
ZERO and MAPPED make sense this way.
Not sure about 0 (or only OFFSET_VALID), could this ever be valid? You say "can't access this portion of file", but where would this happen?
Kevin
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Quoted 106 lines · Kevin Wolf
Am 24.02.2018 um 00:38 hat Eric Blake geschrieben:
Quoted 23 lines · Kevin Wolf
Quoted 13 lines
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben:
Quoted 10 lines
Quoted 5 lines
OFFSET_VALID| DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it would be pointless to actually follow the file pointer), butOFFSET_VALIDwithout DATA probably isn't.So
OFFSET_VALID| DATA for a protocol BDS is not just convenient, but necessary to avoid breaking qemu-img map output. But you are also right thatOFFSET_VALIDwithout data makes little sense at a protocol layer. So with that in mind, I'm auditing all of the protocol layers to make sureOFFSET_VALIDends up as something sane.That's one way to look at it.
Quoted 7 lines
The other way is that qemu-img map shouldn't ask the protocol layer for its offset because it already knows the offset (it is what it passes as a parameter to
bdrv_co_block_status).Anyway, it's probably not worth changing the interface, we should just make sure that the return values of the individual drivers are consistent.
Yet another inconsistency, and it's making me scratch my head today.
Quoted 7 lines
By the way, in my byte-based stuff that is now pending on your tree, I tried hard to NOT change semantics or the set of flags returned by a given driver, and we agreed that's why you'd accept the series as-is and make me do this followup exercise. But it's looking like my followups may end up touching a lot of the same drivers again, now that I'm looking at what the semantics SHOULD be (and whatever I do end up tweaking, I will at least make sure that iotests is still happy with it).
Hm, that's unfortunate, but I don't think we should hold up your first series just so we can touch the drivers only once.
Quoted 30 lines
First, let's read what states the NBD spec is proposing:
Quoted 6 lines
It defines the following flags for the flags field:
NBD_STATE_HOLE(bit 0): if set, the block represents a hole (and future writes to that area may cause fragmentation or encounter an ENOSPC error); if clear, the block is allocated or the server could not otherwise determine its status. Note that the use ofNBD_CMD_TRIMis related to this status, but that the server MAY report a hole even whereNBD_CMD_TRIMhas not been requested, and also that a server MAY report that the block is allocated even whereNBD_CMD_TRIMhas been requested.NBD_STATE_ZERO(bit 1): if set, the block contents read as all zeroes; if clear, the block contents are not known. Note that the use ofNBD_CMD_WRITE_ZEROESis related to this status, but that the server MAY report zeroes even whereNBD_CMD_WRITE_ZEROEShas not been requested, and also that a server MAY report unknown content even whereNBD_CMD_WRITE_ZEROEShas been requested.It is not an error for a server to report that a region of the export has both
NBD_STATE_HOLEset andNBD_STATE_ZEROclear. The contents of such an area are undefined, and a client reading such an area should make no assumption as to its contents or stability.So here's how Vladimir proposed implementing it in his series (written before my byte-based block status stuff went in to your tree):
https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg04038.htmlServer side (3/9):
+ int ret = bdrv_block_status_above(bs, NULL, offset,
tail_bytes,
&num,(quoted diff) +7
-0+ NULL, NULL); + if (ret < 0) { + return ret; + } + + flags = (ret & BDRV_BLOCK_ALLOCATED ? 0 : NBD_STATE_HOLE) | + (ret & BDRV_BLOCK_ZERO ? NBD_STATE_ZERO : 0);Client side (6/9):
(quoted diff) +3
-0+ *pnum = extent.length >> BDRV_SECTOR_BITS; + return (extent.flags & NBD_STATE_HOLE ? 0 : BDRV_BLOCK_DATA) | + (extent.flags & NBD_STATE_ZERO ? BDRV_BLOCK_ZERO : 0);Quoted 1 line
Does anything there strike you as odd?
Two things I noticed while reading the above:
1. NBD doesn't consider backing files, so the definition of holes becomes ambiguous. Is a hole any block that isn't allocated in the top layer (may cause fragmentation or encounter an ENOSPC error) or is it any block that isn't allocated anywhere in the whole backing chain (may read as non-zero)?
Considering that there is a separate
NBD_STATE_ZEROand nothing forbids a state ofNBD_STATE_HOLEwithoutNBD_STATE_ZERO, maybe the former is more useful. The code you quote implements the latter.Maybe if we go with the former, we should add a note to the NBD spec that explictly says that
NBD_STATE_HOLEdoesn't imply any specific content that is returned on reads.2. Using
BDRV_BLOCK_ALLOCATEDto determineNBD_STATE_HOLEseems wrong. A (not preallocated) zero cluster in qcow2 returnsBDRV_BLOCK_ALLOCATED(because we don't fall through to the backing file) even though I think it's a hole.BDRV_BLOCK_DATAshould be used there (which makes it consistent with the other direction).Quoted 6 lines
In isolation, they seemed fine to
me, but side-by-side, I'm scratching my head: the server queries the block layer, and turnsBDRV_BLOCK_ALLOCATEDinto !NBD_STATE_HOLE; the client side then takes the NBD protocol and tries to turn it back into information to feed the block layer, where !NBD_STATE_HOLE now feedsBDRV_BLOCK_DATA. Why the different choice of bits?Which is actually consistent in the end, becaue
BDRV_BLOCK_DATAimpliesBDRV_BLOCK_ALLOCATED.Essentially, assuming a simple backing chain 'base <- overlay', we got these combinations to represent in NBD (with my suggestion of the flags to use):
1. Cluster allocated in overlay
a. non-zero data 0
b. explicit zeroes 0 or ZERO
2. Cluster marked zero in overlay HOLE | ZERO
3. Cluster preallocated/zero in overlay ZERO
4. Cluster unallocated in overlay
Quoted 8 lines
a. Cluster allocated in base (non-zero) HOLE b. Cluster allocated in base (zero) HOLE or HOLE | ZERO c. Cluster marked zero in base HOLE | ZERO d. Cluster preallocated/zero in base HOLE | ZERO e. Cluster unallocated in base HOLE | ZEROInstead of 'base' you can read 'anywhere in the backing chain' and the flags should stay the same.
I think only "anywhere in the backing chain" is valid here. Otherwise, semantics of bdrv_is_allocated would differ for NBD and for not-NBD. I think, if bdrv_is_allocated returns false, it means that we can skip this region in copying process, am I right?
Quoted 5 lines
So !BDRV_BLOCK_ALLOCATED (i.e. falling through to the backing file) does indeed imply
NBD_STATE_HOLE, but so does case 2, which is just !DATA.
Signature · 2 lines
Best regards, Vladimir
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Am 01.03.2018 um 08:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 17 lines
Quoted 16 lines · Kevin Wolf
Essentially, assuming a simple backing chain 'base <- overlay', we got these combinations to represent in NBD (with my suggestion of the flags to use):
1. Cluster allocated in overlay
a. non-zero data 0
b. explicit zeroes 0 or ZERO
2. Cluster marked zero in overlay HOLE | ZERO
3. Cluster preallocated/zero in overlay ZERO
4. Cluster unallocated in overlaya. Cluster allocated in base (non-zero) HOLE b. Cluster allocated in base (zero) HOLE or HOLE | ZERO c. Cluster marked zero in base HOLE | ZERO d. Cluster preallocated/zero in base HOLE | ZERO e. Cluster unallocated in base HOLE | ZERO
Quoted 5 lines
Quoted 2 lines
Instead of 'base' you can read 'anywhere in the backing chain' and the flags should stay the same.
I think only "anywhere in the backing chain" is valid here. Otherwise, semantics of
bdrv_is_allocatedwould differ for NBD and for not-NBD.
This was meant as a mapping from cases to flags, not the other way round, so really doesn't say anything about the cases where the block is allocated further down the chain.
But yes, it shouldn't make a difference where in the backing chain a block is allocated, so these cases are the same as 4.
Quoted 2 lines
I think, if
bdrv_is_allocatedreturns false, it means that we can skip this region in copying process, am I right?
-ENOCONTEXT? Which copying process?
There are cases where you want to copy such regions, and other cases where you want to skip them. It depends on the use case. For example, 'qemu-img convert' skips them with -B (because the backing file is reused), but not without -B (which creates a full copy).
Kevin
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Quoted 31 lines · Kevin Wolf
Am 01.03.2018 um 08:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 17 lines
Quoted 16 lines · Kevin Wolf
Essentially, assuming a simple backing chain 'base <- overlay', we got these combinations to represent in NBD (with my suggestion of the flags to use):
1. Cluster allocated in overlay
a. non-zero data 0
b. explicit zeroes 0 or ZERO
2. Cluster marked zero in overlay HOLE | ZERO
3. Cluster preallocated/zero in overlay ZERO
4. Cluster unallocated in overlaya. Cluster allocated in base (non-zero) HOLE b. Cluster allocated in base (zero) HOLE or HOLE | ZERO c. Cluster marked zero in base HOLE | ZERO d. Cluster preallocated/zero in base HOLE | ZERO e. Cluster unallocated in base HOLE | ZEROQuoted 4 lines
Quoted 2 lines
Instead of 'base' you can read 'anywhere in the backing chain' and the flags should stay the same.
I think only "anywhere in the backing chain" is valid here. Otherwise, semantics of
bdrv_is_allocatedwould differ for NBD and for not-NBD.This was meant as a mapping from cases to flags, not the other way round, so really doesn't say anything about the cases where the block is allocated further down the chain.
But yes, it shouldn't make a difference where in the backing chain a block is allocated, so these cases are the same as 4.
Quoted 2 lines
I think, if
bdrv_is_allocatedreturns false, it means that we can skip this region in copying process, am I right?
Quoted 8 lines
-ENOCONTEXT? Which copying process?There are cases where you want to copy such regions, and other cases where you want to skip them. It depends on the use case. For example, 'qemu-img convert' skips them with -B (because the backing file is reused), but not without -B (which creates a full copy).
Kevin
Hm, I thought that bdrv_is_allocated loops through backings, but it doesn't, sorry.
Signature · 2 lines
Best regards, Vladimir
Re: [Qemu-devel] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()
Am 01.03.2018 um 10:57 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 39 lines
Quoted 38 lines · Kevin Wolf
Am 01.03.2018 um 08:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 17 lines
Quoted 16 lines · Kevin Wolf
Essentially, assuming a simple backing chain 'base <- overlay', we got these combinations to represent in NBD (with my suggestion of the flags to use):
1. Cluster allocated in overlay
a. non-zero data 0
b. explicit zeroes 0 or ZERO
2. Cluster marked zero in overlay HOLE | ZERO
3. Cluster preallocated/zero in overlay ZERO
4. Cluster unallocated in overlaya. Cluster allocated in base (non-zero) HOLE b. Cluster allocated in base (zero) HOLE or HOLE | ZERO c. Cluster marked zero in base HOLE | ZERO d. Cluster preallocated/zero in base HOLE | ZERO e. Cluster unallocated in base HOLE | ZEROQuoted 4 lines
Quoted 2 lines
Instead of 'base' you can read 'anywhere in the backing chain' and the flags should stay the same.
I think only "anywhere in the backing chain" is valid here. Otherwise, semantics of
bdrv_is_allocatedwould differ for NBD and for not-NBD.This was meant as a mapping from cases to flags, not the other way round, so really doesn't say anything about the cases where the block is allocated further down the chain.
But yes, it shouldn't make a difference where in the backing chain a block is allocated, so these cases are the same as 4.
Quoted 2 lines
I think, if
bdrv_is_allocatedreturns false, it means that we can skip this region in copying process, am I right?
-ENOCONTEXT? Which copying process?There are cases where you want to copy such regions, and other cases where you want to skip them. It depends on the use case. For example, 'qemu-img convert' skips them with -B (because the backing file is reused), but not without -B (which creates a full copy).
Quoted 4 lines
Quoted 1 line
Kevin
Hm, I thought that
bdrv_is_allocatedloops through backings, but it doesn't, sorry.
That would be bdrv_is_allocated_above() with a NULL base.
Kevin
PATCH v8 10/21 parallels: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the parallels driver accordingly. Note that the internal function block_status() is still sector-based, because it is still in use by other sector-based functions; but that's okay because request_alignment is 512 as a result of those functions. For now, no optimizations are added based on the mapping hint.
---
v7: fix bug in *map [Vladimir]
v6: no change
v5: fix pnum when return is 0
v4: rebase to interface tweak, R-b dropped
v3: no change
v2: rebase to mapping parameter; it is ignored, so R-b kept
--- block/parallels.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)
block/parallels.c +15 -7
@@ -261,23 +261,31 @@ static coroutine_fn int parallels_co_flush_to_os(BlockDriverState *bs) } -static int64_t coroutine_fn parallels_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn parallels_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset, +int64_t bytes, +int64_t *pnum, +int64_t *map, +BlockDriverState **file) {BDRVParallelsState *s = bs->opaque; -int64_t offset; +int count; +assert (QEMU_IS_ALIGNED (offset | bytes,BDRV_SECTOR_SIZE ));qemu_co_mutex_lock (&s->lock); - offset =block_status (s, sector_num, nb_sectors, pnum); + offset =block_status (s, offset >>BDRV_SECTOR_BITS , + bytes >>BDRV_SECTOR_BITS , &count);qemu_co_mutex_unlock (&s->lock); + *pnum = count *BDRV_SECTOR_SIZE ;if (offset <0 ) {return 0 ; } + *map = offset *BDRV_SECTOR_SIZE ; *file = bs->file->bs; -return (offset <<BDRV_SECTOR_BITS ) | -BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; +return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; } static coroutine_fn int parallels_co_writev(BlockDriverState *bs,
@@ -782,7 +790,7 @@ static BlockDriver bdrv_parallels = { .bdrv_open = parallels_open, .bdrv_close = parallels_close, .bdrv_child_perm = bdrv_format_default_perms, - .bdrv_co_get_block_status = parallels_co_get_block_status, + .bdrv_co_block_status = parallels_co_block_status, .bdrv_has_zero_init = bdrv_has_zero_init_1, .bdrv_co_flush_to_os = parallels_co_flush_to_os, .bdrv_co_readv = parallels_co_readv,
Signature · 1 line
2.14.3
PATCH v8 11/21 qcow: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the qcow driver accordingly. There is no intent to optimize based on the want_zero flag for this format.
---
v5-v7: no change
v4: rebase to interface tweak
v3: rebase to master
v2: rebase to mapping flag
--- block/qcow.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-)
block/qcow.c +16 -11
@@ -524,23 +524,28 @@ static int get_cluster_offset(BlockDriverState *bs,return 1 ; } -static int64_t coroutine_fn qcow_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn qcow_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset,int64_t bytes, +int64_t *pnum,int64_t *map, +BlockDriverState **file) {BDRVQcowState *s = bs->opaque; -int index_in_cluster, n, ret; +int index_in_cluster, ret; +int64_t n;uint64_t cluster_offset;qemu_co_mutex_lock (&s->lock); - ret =get_cluster_offset (bs, sector_num <<9 ,0 ,0 ,0 ,0 , &cluster_offset); + ret =get_cluster_offset (bs, offset,0 ,0 ,0 ,0 , &cluster_offset);qemu_co_mutex_unlock (&s->lock);if (ret <0 ) {return ret; } - index_in_cluster = sector_num & (s->cluster_sectors -1 ); - n = s->cluster_sectors - index_in_cluster; -if (n > nb_sectors) - n = nb_sectors; + index_in_cluster = offset & (s->cluster_size -1 ); + n = s->cluster_size - index_in_cluster; +if (n > bytes) { + n = bytes; + } *pnum = n;if (!cluster_offset) {return 0 ;
@@ -548,9 +553,9 @@ static int64_t coroutine_fn qcow_co_get_block_status(BlockDriverState *bs,if ((cluster_offset &QCOW_OFLAG_COMPRESSED ) || s->crypto) {return BDRV_BLOCK_DATA ; } - cluster_offset |= (index_in_cluster <<BDRV_SECTOR_BITS ); + *map = cluster_offset | index_in_cluster; *file = bs->file->bs; -return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | cluster_offset; +return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; } static int decompress_buffer(uint8_t *out_buf, int out_buf_size,
@@ -1128,7 +1133,7 @@ static BlockDriver bdrv_qcow = { .bdrv_co_readv = qcow_co_readv, .bdrv_co_writev = qcow_co_writev, - .bdrv_co_get_block_status = qcow_co_get_block_status, + .bdrv_co_block_status = qcow_co_block_status, .bdrv_make_empty = qcow_make_empty, .bdrv_co_pwritev_compressed = qcow_co_pwritev_compressed,
Signature · 1 line
2.14.3
PATCH v8 12/21 qcow2: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the qcow2 driver accordingly.
For now, we are ignoring the 'want_zero' hint. However, it should be relatively straightforward to honor the hint as a way to return larger *pnum values when we have consecutive clusters with the same data/zero status but which differ only in having non-consecutive mappings.
---
v5-v7: no change
v4: update to interface tweak
v3: no change
v2: rebase to mapping flag
--- block/qcow2.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-)
block/qcow2.c +13 -11
@@ -1670,32 +1670,34 @@ static void qcow2_join_options(QDict *options, QDict *old_options) } } -static int64_t coroutine_fn qcow2_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset,int64_t count, +int64_t *pnum,int64_t *map, +BlockDriverState **file) {BDRVQcow2State *s = bs->opaque;uint64_t cluster_offset;int index_in_cluster, ret;unsigned int bytes; -int64_t status =0 ; +int status =0 ; - bytes =MIN (INT_MAX , nb_sectors *BDRV_SECTOR_SIZE ); + bytes =MIN (INT_MAX , count);qemu_co_mutex_lock (&s->lock); - ret =qcow2_get_cluster_offset (bs, sector_num <<BDRV_SECTOR_BITS , &bytes, - &cluster_offset); + ret =qcow2_get_cluster_offset (bs, offset, &bytes, &cluster_offset);qemu_co_mutex_unlock (&s->lock);if (ret <0 ) {return ret; } - *pnum = bytes >>BDRV_SECTOR_BITS ; + *pnum = bytes;if (cluster_offset !=0 && ret !=QCOW2_CLUSTER_COMPRESSED && !s->crypto) { - index_in_cluster = sector_num & (s->cluster_sectors -1 ); - cluster_offset |= (index_in_cluster <<BDRV_SECTOR_BITS ); + index_in_cluster = offset & (s->cluster_size -1 ); + *map = cluster_offset | index_in_cluster; *file = bs->file->bs; - status |=BDRV_BLOCK_OFFSET_VALID | cluster_offset; + status |=BDRV_BLOCK_OFFSET_VALID ; }if (ret ==QCOW2_CLUSTER_ZERO_PLAIN || ret ==QCOW2_CLUSTER_ZERO_ALLOC ) { status |=BDRV_BLOCK_ZERO ;
@@ -4352,7 +4354,7 @@ BlockDriver bdrv_qcow2 = { .bdrv_child_perm = bdrv_format_default_perms, .bdrv_create = qcow2_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, - .bdrv_co_get_block_status = qcow2_co_get_block_status, + .bdrv_co_block_status = qcow2_co_block_status, .bdrv_co_preadv = qcow2_co_preadv, .bdrv_co_pwritev = qcow2_co_pwritev,
Signature · 1 line
2.14.3
PATCH v8 13/21 qed: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the qed driver accordingly, taking the opportunity to inline qed_is_allocated_cb() into its lone caller (the callback used to be important, until we switched qed to coroutines). There is no intent to optimize based on the want_zero flag for this format.
---
v6-v7: no change
v5: initialize len before qed_find_cluster() [Vladimir]
v4: rebase to interface change, inline pointless callback
v3: no change
v2: rebase to mapping flag, fix mask in qed_is_allocated_cb
--- block/qed.c | 84 +++++++++++++++++++++---------------------------------------- 1 file changed, 28 insertions(+), 56 deletions(-)
block/qed.c +28 -56
@@ -688,74 +688,46 @@ finish:return ret; } -typedef struct { -BlockDriverState *bs; -Coroutine *co; -uint64_t pos; -int64_t status; -int *pnum; -BlockDriverState **file; -}QEDIsAllocatedCB ; - -/* Called with table_lock held. */ -static void qed_is_allocated_cb (void *opaque,int ret,uint64_t offset,size_t len) -{ -QEDIsAllocatedCB *cb = opaque; -BDRVQEDState *s = cb->bs->opaque; - *cb->pnum = len /BDRV_SECTOR_SIZE ; -switch (ret) { -case QED_CLUSTER_FOUND : - offset |=qed_offset_into_cluster (s, cb->pos); - cb->status =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | offset; - *cb->file = cb->bs->file->bs; -break ; -case QED_CLUSTER_ZERO : - cb->status =BDRV_BLOCK_ZERO ; -break ; -case QED_CLUSTER_L2 : -case QED_CLUSTER_L1 : - cb->status =0 ; -break ; -default : -assert (ret <0 ); - cb->status = ret; -break ; - } - -if (cb->co) { -aio_co_wake (cb->co); - } -} - -static int64_t coroutine_fn bdrv_qed_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, +static int coroutine_fn bdrv_qed_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t pos,int64_t bytes, +int64_t *pnum,int64_t *map,BlockDriverState **file) {BDRVQEDState *s = bs->opaque; -size_t len = (size_t )nb_sectors *BDRV_SECTOR_SIZE ; -QEDIsAllocatedCB cb = { - .bs = bs, - .pos = (uint64_t )sector_num *BDRV_SECTOR_SIZE , - .status =BDRV_BLOCK_OFFSET_MASK , - .pnum = pnum, - .file = file, - }; +size_t len =MIN (bytes,SIZE_MAX ); +int status;QEDRequest request = { .l2_table =NULL };uint64_t offset;int ret;qemu_co_mutex_lock (&s->table_lock); - ret =qed_find_cluster (s, &request, cb.pos, &len, &offset); -qed_is_allocated_cb (&cb, ret, offset, len); + ret =qed_find_cluster (s, &request, pos, &len, &offset); -/* The callback was invoked immediately */ -assert (cb.status !=BDRV_BLOCK_OFFSET_MASK ); + *pnum = len; +switch (ret) { +case QED_CLUSTER_FOUND : + *map = offset |qed_offset_into_cluster (s, pos); + status =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; + *file = bs->file->bs; +break ; +case QED_CLUSTER_ZERO : + status =BDRV_BLOCK_ZERO ; +break ; +case QED_CLUSTER_L2 : +case QED_CLUSTER_L1 : + status =0 ; +break ; +default : +assert (ret <0 ); + status = ret; +break ; + }qed_unref_l2_cache_entry (request.l2_table);qemu_co_mutex_unlock (&s->table_lock); -return cb.status; +return status; }static BDRVQEDState *acb_to_s (QEDAIOCB *acb)
@@ -1594,7 +1566,7 @@ static BlockDriver bdrv_qed = { .bdrv_child_perm = bdrv_format_default_perms, .bdrv_create = bdrv_qed_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, - .bdrv_co_get_block_status = bdrv_qed_co_get_block_status, + .bdrv_co_block_status = bdrv_qed_co_block_status, .bdrv_co_readv = bdrv_qed_co_readv, .bdrv_co_writev = bdrv_qed_co_writev, .bdrv_co_pwrite_zeroes = bdrv_qed_co_pwrite_zeroes,
Signature · 1 line
2.14.3
PATCH v8 14/21 raw: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the raw driver accordingly.
---
v5-v7: no change
v4: rebase to interface tweak
v3: no change
v2: rebase to mapping
--- block/raw-format.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
block/raw-format.c +8 -8
@@ -250,17 +250,17 @@ fail:return ret; } -static int64_t coroutine_fn raw_co_get_block_status(BlockDriverState *bs, -int64_t sector_num, -int nb_sectors,int *pnum, +static int coroutine_fn raw_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *pnum, +int64_t *map,BlockDriverState **file) {BDRVRawState *s = bs->opaque; - *pnum = nb_sectors; + *pnum = bytes; *file = bs->file->bs; - sector_num += s->offset /BDRV_SECTOR_SIZE ; -return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); + *map = offset + s->offset; +return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID ; } static int coroutine_fn raw_co_pwrite_zeroes(BlockDriverState *bs,
@@ -496,7 +496,7 @@ BlockDriver bdrv_raw = { .bdrv_co_pwritev = &raw_co_pwritev, .bdrv_co_pwrite_zeroes = &raw_co_pwrite_zeroes, .bdrv_co_pdiscard = &raw_co_pdiscard, - .bdrv_co_get_block_status = &raw_co_get_block_status, + .bdrv_co_block_status = &raw_co_block_status, .bdrv_truncate = &raw_truncate, .bdrv_getlength = &raw_getlength, .has_variable_length = true,
Signature · 1 line
2.14.3
PATCH v8 15/21 sheepdog: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the sheepdog driver accordingly.
---
v7: rebase to minor spacing changes in master
v5-v6: no change
v4: update to interface tweak
v3: no change
v2: rebase to mapping flag
--- block/sheepdog.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
block/sheepdog.c +13 -13
@@ -3004,19 +3004,19 @@ static coroutine_fn int sd_co_pdiscard(BlockDriverState *bs, int64_t offset,return acb.ret; } -static coroutine_fn int64_t -sd_co_get_block_status (BlockDriverState *bs,int64_t sector_num,int nb_sectors, -int *pnum,BlockDriverState **file) +static coroutine_fn int +sd_co_block_status (BlockDriverState *bs,bool want_zero,int64_t offset, +int64_t bytes,int64_t *pnum,int64_t *map, +BlockDriverState **file) {BDRVSheepdogState *s = bs->opaque;SheepdogInode *inode = &s->inode;uint32_t object_size = (UINT32_C (1 ) << inode->block_size_shift); -uint64_t offset = sector_num *BDRV_SECTOR_SIZE ;unsigned long start = offset / object_size, - end =DIV_ROUND_UP ((sector_num + nb_sectors) * -BDRV_SECTOR_SIZE , object_size); + end =DIV_ROUND_UP (offset + bytes, object_size);unsigned long idx; -int64_t ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | offset; + *map = offset; +int ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ;for (idx = start; idx < end; idx++) {if (inode->data_vdi_id[idx] ==0 ) {
@@ -3033,9 +3033,9 @@ sd_co_get_block_status(BlockDriverState *bs, int64_t sector_num, int nb_sectors, } } - *pnum = (idx - start) * object_size /BDRV_SECTOR_SIZE ; -if (*pnum > nb_sectors) { - *pnum = nb_sectors; + *pnum = (idx - start) * object_size; +if (*pnum > bytes) { + *pnum = bytes; }if (ret >0 && ret &BDRV_BLOCK_OFFSET_VALID ) { *file = bs;
@@ -3113,7 +3113,7 @@ static BlockDriver bdrv_sheepdog = { .bdrv_co_writev = sd_co_writev, .bdrv_co_flush_to_disk = sd_co_flush_to_disk, .bdrv_co_pdiscard = sd_co_pdiscard, - .bdrv_co_get_block_status = sd_co_get_block_status, + .bdrv_co_block_status = sd_co_block_status, .bdrv_snapshot_create = sd_snapshot_create, .bdrv_snapshot_goto = sd_snapshot_goto,
@@ -3149,7 +3149,7 @@ static BlockDriver bdrv_sheepdog_tcp = { .bdrv_co_writev = sd_co_writev, .bdrv_co_flush_to_disk = sd_co_flush_to_disk, .bdrv_co_pdiscard = sd_co_pdiscard, - .bdrv_co_get_block_status = sd_co_get_block_status, + .bdrv_co_block_status = sd_co_block_status, .bdrv_snapshot_create = sd_snapshot_create, .bdrv_snapshot_goto = sd_snapshot_goto,
@@ -3185,7 +3185,7 @@ static BlockDriver bdrv_sheepdog_unix = { .bdrv_co_writev = sd_co_writev, .bdrv_co_flush_to_disk = sd_co_flush_to_disk, .bdrv_co_pdiscard = sd_co_pdiscard, - .bdrv_co_get_block_status = sd_co_get_block_status, + .bdrv_co_block_status = sd_co_block_status, .bdrv_snapshot_create = sd_snapshot_create, .bdrv_snapshot_goto = sd_snapshot_goto,
Signature · 1 line
2.14.3
PATCH v8 16/21 vdi: Avoid bitrot of debugging code
In reply to Eric BlakeRework the debug define so that we always get -Wformat checking, even when debugging is disabled.
---
v2-v7: no change
--- block/vdi.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
block/vdi.c +9 -3
@@ -87,12 +87,18 @@#define DEFAULT_CLUSTER_SIZE (1 * MiB)#if defined(CONFIG_VDI_DEBUG ) -#define logout (fmt, ...) \ - fprintf(stderr, "vdi\t%-24s" fmt, __func__, ##__VA_ARGS__) +#define VDI_DEBUG 1#else -#define logout (fmt, ...) ((void)0) +#define VDI_DEBUG 0#endif +#define logout (fmt, ...) \ + do { \ + if (VDI_DEBUG) { \ + fprintf(stderr, "vdi\t%-24s" fmt, __func__, ##__VA_ARGS__); \ + } \ + } while (0) +/* Image signature. */ #define VDI_SIGNATURE 0xbeda107f
Signature · 1 line
2.14.3
PATCH v8 17/21 vdi: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the vdi driver accordingly. Note that the TODO is already covered (the block layer guarantees bounds of its requests), and that we can remove the now-unused s->block_sectors.
---
v6-v7: no change
v5: fix pnum when offset rounded down to block_size [Vladimir]
v4: rebase to interface tweak
v3: no change
v2: rebase to mapping flag
--- block/vdi.c | 33 +++++++++++++-------------------- 1 file changed, 13 insertions(+), 20 deletions(-)
block/vdi.c +13 -20
@@ -172,8 +172,6 @@ typedef struct {uint32_t *bmap;/* Size of block (bytes). */ uint32_t block_size; -/* Size of block (sectors). */ -uint32_t block_sectors;/* First sector of block map. */ uint32_t bmap_sector;/* VDI header (converted to host endianness). */
@@ -463,7 +461,6 @@ static int vdi_open(BlockDriverState *bs, QDict *options, int flags, bs->total_sectors = header.disk_size /SECTOR_SIZE ; s->block_size = header.block_size; - s->block_sectors = header.block_size /SECTOR_SIZE ; s->bmap_sector = header.offset_bmap /SECTOR_SIZE ; s->header = header;
@@ -509,33 +506,29 @@ static int vdi_reopen_prepare(BDRVReopenState *state,return 0 ; } -static int64_t coroutine_fn vdi_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn vdi_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset,int64_t bytes, +int64_t *pnum,int64_t *map, +BlockDriverState **file) { -/* TODO: Check for too large sector_num (in bdrv_is_allocated or here). */ BDRVVdiState *s = (BDRVVdiState *)bs->opaque; -size_t bmap_index = sector_num / s->block_sectors; -size_t sector_in_block = sector_num % s->block_sectors; -int n_sectors = s->block_sectors - sector_in_block; +size_t bmap_index = offset / s->block_size; +size_t index_in_block = offset % s->block_size;uint32_t bmap_entry =le32_to_cpu (s->bmap[bmap_index]); -uint64_t offset;int result; -logout ("%p, %" PRId64", %d, %p\n" , bs, sector_num, nb_sectors, pnum); -if (n_sectors > nb_sectors) { - n_sectors = nb_sectors; - } - *pnum = n_sectors; +logout ("%p, %" PRId64", %" PRId64", %p\n" , bs, offset, bytes, pnum); + *pnum =MIN (s->block_size - index_in_block, bytes); result =VDI_IS_ALLOCATED (bmap_entry);if (!result) {return 0 ; } - offset = s->header.offset_data + - (uint64_t )bmap_entry * s->block_size + - sector_in_block *SECTOR_SIZE ; + *map = s->header.offset_data + (uint64_t )bmap_entry * s->block_size + + index_in_block; *file = bs->file->bs; -return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | offset; +return BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ; }static int coroutine_fn
@@ -903,7 +896,7 @@ static BlockDriver bdrv_vdi = { .bdrv_child_perm = bdrv_format_default_perms, .bdrv_create = vdi_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, - .bdrv_co_get_block_status = vdi_co_get_block_status, + .bdrv_co_block_status = vdi_co_block_status, .bdrv_make_empty = vdi_make_empty, .bdrv_co_preadv = vdi_co_preadv,
Signature · 1 line
2.14.3
PATCH v8 18/21 vmdk: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the vmdk driver accordingly. Drop the now-unused vmdk_find_index_in_cluster().
Also, fix a pre-existing bug: if find_extent() fails (unlikely, since the block layer did a bounds check), then we must return a failure, rather than 0.
---
v6-v7: no change
v5: drop dead code [Vladimir], return error on find_extent() failure
v4: rebase to interface tweak
v3: no change
v2: rebase to mapping flag
--- block/vmdk.c | 38 ++++++++++++++------------------------ 1 file changed, 14 insertions(+), 24 deletions(-)
block/vmdk.c +14 -24
@@ -1304,33 +1304,27 @@ static inline uint64_t vmdk_find_offset_in_cluster(VmdkExtent *extent,return extent_relative_offset % cluster_size; } -static inline uint64_t vmdk_find_index_in_cluster (VmdkExtent *extent, -int64_t sector_num) -{ -uint64_t offset; - offset =vmdk_find_offset_in_cluster (extent, sector_num *BDRV_SECTOR_SIZE ); -return offset /BDRV_SECTOR_SIZE ; -} - -static int64_t coroutine_fn vmdk_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn vmdk_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset,int64_t bytes, +int64_t *pnum,int64_t *map, +BlockDriverState **file) {BDRVVmdkState *s = bs->opaque;int64_t index_in_cluster, n, ret; -uint64_t offset; +uint64_t cluster_offset;VmdkExtent *extent; - extent =find_extent (s, sector_num,NULL ); + extent =find_extent (s, offset >>BDRV_SECTOR_BITS ,NULL );if (!extent) { -return 0 ; +return -EIO ; }qemu_co_mutex_lock (&s->lock); - ret =get_cluster_offset (bs, extent,NULL , - sector_num *512 , false, &offset, + ret =get_cluster_offset (bs, extent,NULL , offset, false, &cluster_offset,0 ,0 );qemu_co_mutex_unlock (&s->lock); - index_in_cluster =vmdk_find_index_in_cluster (extent, sector_num); + index_in_cluster =vmdk_find_offset_in_cluster (extent, offset);switch (ret) {case VMDK_ERROR : ret = -EIO ;
@@ -1345,18 +1339,14 @@ static int64_t coroutine_fn vmdk_co_get_block_status(BlockDriverState *bs, ret =BDRV_BLOCK_DATA ;if (!extent->compressed) { ret |=BDRV_BLOCK_OFFSET_VALID ; - ret |= (offset + (index_in_cluster <<BDRV_SECTOR_BITS )) - &BDRV_BLOCK_OFFSET_MASK ; + *map = cluster_offset + index_in_cluster; } *file = extent->file->bs;break ; } - n = extent->cluster_sectors - index_in_cluster; -if (n > nb_sectors) { - n = nb_sectors; - } - *pnum = n; + n = extent->cluster_sectors *BDRV_SECTOR_SIZE - index_in_cluster; + *pnum =MIN (n, bytes);return ret; }
@@ -2410,7 +2400,7 @@ static BlockDriver bdrv_vmdk = { .bdrv_close = vmdk_close, .bdrv_create = vmdk_create, .bdrv_co_flush_to_disk = vmdk_co_flush, - .bdrv_co_get_block_status = vmdk_co_get_block_status, + .bdrv_co_block_status = vmdk_co_block_status, .bdrv_get_allocated_file_size = vmdk_get_allocated_file_size, .bdrv_has_zero_init = vmdk_has_zero_init, .bdrv_get_specific_info = vmdk_get_specific_info,
Signature · 1 line
2.14.3
PATCH v8 19/21 vpc: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the vpc driver accordingly.
---
v7: tweak commit message and type of 'n' [Fam]
v6: no change
v5: fix incorrect rounding in 'map' and bad loop condition [Vladimir]
v4: rebase to interface tweak
v3: rebase to master
v2: drop get_sector_offset() [Kevin], rebase to mapping flag
--- block/vpc.c | 45 +++++++++++++++++++++++---------------------- 1 file changed, 23 insertions(+), 22 deletions(-)
block/vpc.c +23 -22
@@ -706,53 +706,54 @@ fail:return ret; } -static int64_t coroutine_fn vpc_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn vpc_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset,int64_t bytes, +int64_t *pnum,int64_t *map, +BlockDriverState **file) {BDRVVPCState *s = bs->opaque;VHDFooter *footer = (VHDFooter *) s->footer_buf; -int64_t start, offset; +int64_t image_offset;bool allocated; -int64_t ret; -int n; +int ret; +int64_t n;if (be32_to_cpu (footer->type) ==VHD_FIXED ) { - *pnum = nb_sectors; + *pnum = bytes; + *map = offset; *file = bs->file->bs; -return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); +return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID ; }qemu_co_mutex_lock (&s->lock); - offset =get_image_offset (bs, sector_num <<BDRV_SECTOR_BITS , false,NULL ); - start = offset; - allocated = (offset !=-1 ); + image_offset =get_image_offset (bs, offset, false,NULL ); + allocated = (image_offset !=-1 ); *pnum =0 ; ret =0 ;do {/* All sectors in a block are contiguous (without using the bitmap) */ - n =ROUND_UP (sector_num +1 , s->block_size /BDRV_SECTOR_SIZE ) - - sector_num; - n =MIN (n, nb_sectors); + n =ROUND_UP (offset +1 , s->block_size) - offset; + n =MIN (n, bytes); *pnum += n; - sector_num += n; - nb_sectors -= n; + offset += n; + bytes -= n;/* *pnum can't be greater than one block for allocated * sectors since there is always a bitmap in between. */ if (allocated) { *file = bs->file->bs; - ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | start; + *map = image_offset; + ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID ;break ; } -if (nb_sectors ==0 ) { +if (bytes ==0 ) {break ; } - offset =get_image_offset (bs, sector_num <<BDRV_SECTOR_BITS , false, -NULL ); - }while (offset ==-1 ); + image_offset =get_image_offset (bs, offset, false,NULL ); + }while (image_offset ==-1 );qemu_co_mutex_unlock (&s->lock);return ret;
@@ -1098,7 +1099,7 @@ static BlockDriver bdrv_vpc = { .bdrv_co_preadv = vpc_co_preadv, .bdrv_co_pwritev = vpc_co_pwritev, - .bdrv_co_get_block_status = vpc_co_get_block_status, + .bdrv_co_block_status = vpc_co_block_status, .bdrv_get_info = vpc_get_info,
Signature · 1 line
2.14.3
Re: [Qemu-devel] [PATCH v8 19/21] vpc: Switch to .bdrv_co_block_status()
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 73 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the vpc driver accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>---
v7: tweak commit message and type of 'n' [Fam]
v6: no change
v5: fix incorrect rounding in 'map' and bad loop condition [Vladimir]
v4: rebase to interface tweak
v3: rebase to master
v2: dropget_sector_offset()[Kevin], rebase to mapping flag--- block/vpc.c | 45 +++++++++++++++++++++++---------------------- 1 file changed, 23 insertions(+), 22 deletions(-)block/vpc.c +15
-14@@ -706,53 +706,54 @@ fail:return ret; } -static int64_t coroutine_fn vpc_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum,BlockDriverState **file) +static int coroutine_fn vpc_co_block_status(BlockDriverState *bs, +bool want_zero, +int64_t offset,int64_t bytes, +int64_t *pnum,int64_t *map, +BlockDriverState **file) {BDRVVPCState *s = bs->opaque;VHDFooter *footer = (VHDFooter *) s->footer_buf; -int64_t start, offset; +int64_t image_offset;bool allocated; -int64_t ret; -int n; +int ret; +int64_t n;if (be32_to_cpu (footer->type) ==VHD_FIXED ) { - *pnum = nb_sectors; + *pnum = bytes; + *map = offset; *file = bs->file->bs; -return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID | - (sector_num <<BDRV_SECTOR_BITS ); +return BDRV_BLOCK_RAW |BDRV_BLOCK_OFFSET_VALID ; }qemu_co_mutex_lock (&s->lock); - offset =get_image_offset (bs, sector_num <<BDRV_SECTOR_BITS , false,NULL ); - start = offset; - allocated = (offset !=-1 ); + image_offset =get_image_offset (bs, offset, false,NULL ); + allocated = (image_offset !=-1 ); *pnum =0 ; ret =0 ; do { /* All sectors in a block are contiguous (without using the bitmap) */ - n = ROUND_UP(sector_num + 1, s->block_size / BDRV_SECTOR_SIZE) - - sector_num; - n = MIN(n, nb_sectors); + n = ROUND_UP(offset + 1, s->block_size) - offset; + n = MIN(n, bytes); *pnum += n;
block/vpc.c fail: · line 748
Quoted 10 lines
block/vpc.c +3
-3- sector_num += n; - nb_sectors -= n; + offset += n; + bytes -= n;/* *pnum can't be greater than one block for allocated * sectors since there is always a bitmap in between. */ if (allocated) { *file = bs->file->bs; - ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | start; + *map = image_offset;
This does work, but only because the loop isn't actually looping for allocated == true. In the old code, it was obvious that start was assigned only once and never changed during the loop, but image_offset changes in each loop iteration.
It would probably be cleaner and more obviously correct to move the assignment of *map to before the loop.
Kevin
Re: [Qemu-devel] [PATCH v8 19/21] vpc: Switch to .bdrv_co_block_status()
Quoted 34 lines · Kevin Wolf
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 22 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the vpc driver accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>---
block/vpc.c +3
-3+ allocated = (image_offset !=-1 ); *pnum =0 ; ret =0 ; do { /* All sectors in a block are contiguous (without using the bitmap) */ - n = ROUND_UP(sector_num + 1, s->block_size / BDRV_SECTOR_SIZE) - - sector_num; - n = MIN(n, nb_sectors); + n = ROUND_UP(offset + 1, s->block_size) - offset; + n = MIN(n, bytes); *pnum += n;block/vpc.c fail: · line 748
Quoted 10 lines
block/vpc.c +3
-3- sector_num += n; - nb_sectors -= n; + offset += n; + bytes -= n;/* *pnum can't be greater than one block for allocated * sectors since there is always a bitmap in between. */ if (allocated) { *file = bs->file->bs; - ret =BDRV_BLOCK_DATA |BDRV_BLOCK_OFFSET_VALID | start; + *map = image_offset;
Quoted 7 lines
This does work, but only because the loop isn't actually looping for allocated == true. In the old code, it was obvious that start was assigned only once and never changed during the loop, but
image_offsetchanges in each loop iteration.It would probably be cleaner and more obviously correct to move the assignment of *map to before the loop.
Yes, that would be a bit nicer.
Quoted 3 lines
Kevin
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH v8 20/21 vvfat: Switch to .bdrv_co_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Update the vvfat driver accordingly. Note that we can rely on the block driver having already clamped limits to our block size, and simplify accordingly.
---
v5-v7: no change
v4: rebase to interface tweak
v3: no change
v2: rebase to earlier changes, simplify
--- block/vvfat.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-)
block/vvfat.c +7 -9
@@ -3088,15 +3088,13 @@ vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,return ret; } -static int64_t coroutine_fn vvfat_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *n,BlockDriverState **file) +static int coroutine_fn vvfat_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *n, +int64_t *map, +BlockDriverState **file) { - *n = bs->total_sectors - sector_num; -if (*n > nb_sectors) { - *n = nb_sectors; - }else if (*n <0 ) { -return 0 ; - } + *n = bytes;return BDRV_BLOCK_DATA ; }
@@ -3257,7 +3255,7 @@ static BlockDriver bdrv_vvfat = { .bdrv_co_preadv = vvfat_co_preadv, .bdrv_co_pwritev = vvfat_co_pwritev, - .bdrv_co_get_block_status = vvfat_co_get_block_status, + .bdrv_co_block_status = vvfat_co_block_status, };static void bdrv_vvfat_init (void )
Signature · 1 line
2.14.3
Re: [Qemu-devel] [PATCH v8 20/21] vvfat: Switch to .bdrv_co_block_status()
Am 13.02.2018 um 21:27 hat Eric Blake geschrieben:
Quoted 33 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the vvfat driver accordingly. Note that we can rely on the block driver having already clamped limits to our block size, and simplify accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>---
v5-v7: no change
v4: rebase to interface tweak
v3: no change
v2: rebase to earlier changes, simplify--- block/vvfat.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-)block/vvfat.c +5
-2@@ -3088,15 +3088,13 @@ vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,return ret; } -static int64_t coroutine_fn vvfat_co_get_block_status(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *n,BlockDriverState **file) +static int coroutine_fn vvfat_co_block_status(BlockDriverState *bs, +bool want_zero,int64_t offset, +int64_t bytes,int64_t *n, +int64_t *map, +BlockDriverState **file)
block/vvfat.c vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, · line 3099
Quoted 10 lines
block/vvfat.c +1
-6{ - *n = bs->total_sectors - sector_num; -if (*n > nb_sectors) { - *n = nb_sectors; - }else if (*n <0 ) { -return 0 ; - } + *n = bytes;return BDRV_BLOCK_DATA ; }
Preexisting, but this is inconsistent with other protocol drivers as far as OFFSET_VALID is concerned (as I hinted in another mail, I like it better this way, but it's still inconsistent).
Do we actually need any callback here or could the solution be to simply remove it like with nvme?
Kevin
Re: [Qemu-devel] [PATCH v8 20/21] vvfat: Switch to .bdrv_co_block_status()
Quoted 21 lines · Kevin Wolf
Am 13.02.2018 um 21:27 hat Eric Blake geschrieben:
Quoted 10 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the vvfat driver accordingly. Note that we can rely on the block driver having already clamped limits to our block size, and simplify accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>{block/vvfat.c vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, · line 3099
Quoted 9 lines
block/vvfat.c +1
-6- *n = bs->total_sectors - sector_num; -if (*n > nb_sectors) { - *n = nb_sectors; - }else if (*n <0 ) { -return 0 ; - } + *n = bytes;return BDRV_BLOCK_DATA ; }
Quoted 6 lines
Preexisting, but this is inconsistent with other protocol drivers as far as
OFFSET_VALIDis concerned (as I hinted in another mail, I like it better this way, but it's still inconsistent).Do we actually need any callback here or could the solution be to simply remove it like with nvme?
Does that mean io.c's defaults for protocol drivers is wrong? It defaults to setting OFFSET_VALID and *map on all protocol drivers without a callback (at least nvme, nbd); I didn't delete this callback because I noticed the difference in return value, and couldn't justify whether it was intentional. Also, vvfat is weird - it is somewhat of a format driver, rather than just a protocol; even though it sets .protocol_name. It may be possible for vvfat to actually set OFFSET_VALID to particular offsets within the host file that correspond to what the guest would read, where it is not a simple 1:1 mapping, precisely because it is imposing format on the host file. However, vvfat is one of those things that I try to avoid as much as possible, because it is just so weird.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v8 20/21] vvfat: Switch to .bdrv_co_block_status()
Am 14.02.2018 um 15:50 hat Eric Blake geschrieben:
Quoted 29 lines · Kevin Wolf
Quoted 21 lines
Am 13.02.2018 um 21:27 hat Eric Blake geschrieben:
Quoted 10 lines
We are gradually moving away from sector-based interfaces, towards byte-based. Update the vvfat driver accordingly. Note that we can rely on the block driver having already clamped limits to our block size, and simplify accordingly.
Signed-off-by Eric Blake <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Reviewed-by Fam Zheng <[email protected]>{block/vvfat.c vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, · line 3099
Quoted 9 lines
block/vvfat.c +1
-6- *n = bs->total_sectors - sector_num; -if (*n > nb_sectors) { - *n = nb_sectors; - }else if (*n <0 ) { -return 0 ; - } + *n = bytes;return BDRV_BLOCK_DATA ; }Quoted 6 lines
Preexisting, but this is inconsistent with other protocol drivers as far as
OFFSET_VALIDis concerned (as I hinted in another mail, I like it better this way, but it's still inconsistent).Do we actually need any callback here or could the solution be to simply remove it like with nvme?
Quoted 11 lines
Does that mean io.c's defaults for protocol drivers is wrong? It defaults to setting
OFFSET_VALIDand *map on all protocol drivers without a callback (at least nvme, nbd); I didn't delete this callback because I noticed the difference in return value, and couldn't justify whether it was intentional. Also, vvfat is weird - it is somewhat of a format driver, rather than just a protocol; even though it sets .protocol_name. It may be possible for vvfat to actually setOFFSET_VALIDto particular offsets within the host file that correspond to what the guest would read, where it is not a simple 1:1 mapping, precisely because it is imposing format on the host file. However, vvfat is one of those things that I try to avoid as much as possible, because it is just so weird.
As I said in my reply to the null block driver, OFFSET_VALID doesn't really make sense for protocol drivers anyway. Making use of it with vvfat isn't any more practical than directly accessing the undefined data of the null driver.
I think the unwritten rule at the moment is that protocols should always set OFFSET_VALID and *file = bs (even though it doesn't make sense). So with the current interface, I'd consider deleting the callback a vvfat fix.
I also think that we should possibly look into changing the interface so that protocols don't set OFFSET_VALID and *file, but then the default handling would change too, and deleting the callback in vvfat would still be right.
As this is preexisting, I'm okay with just merging the series as it is, and then we can handle this while dealing with the more fundamental question what protocol drivers should return in general.
Kevin
PATCH v8 21/21 block: Drop unused .bdrv_co_get_block_status()
In reply to Eric BlakeWe are gradually moving away from sector-based interfaces, towards byte-based. Now that all drivers have been updated to provide the byte-based .bdrv_co_block_status(), we can delete the sector-based interface.
---
v7: no change
v6: rebase to changes in patch 1, drop R-b
v5: rebase to master
v4: rebase to interface tweak
v3: no change
v2: rebase to earlier changes
--- include/block/block_int.h | 3 --- block/io.c | 50 ++++++++++------------------------------------- 2 files changed, 10 insertions(+), 43 deletions(-)
include/block/block_int.h +0 -3
@@ -215,9 +215,6 @@ struct BlockDriver {* as well as non-NULL pnum, map, and file; in turn, the driver * must return an error or set pnum to an aligned non-zero value. */ - int64_t coroutine_fn (*bdrv_co_get_block_status)(BlockDriverState *bs, -int64_t sector_num,int nb_sectors,int *pnum, -BlockDriverState **file); int coroutine_fn (*bdrv_co_block_status)(BlockDriverState *bs,bool want_zero,int64_t offset,int64_t bytes,int64_t *pnum,int64_t *map,BlockDriverState **file);
block/io.c +10 -40
70 lines not on this page · raw message
Signature · 1 line
2.14.3
Am 13.02.2018 um 21:26 hat Eric Blake geschrieben:
Quoted 9 lines
There are patches floating around to add
NBD_CMD_BLOCK_STATUS, but NBD wants to report status on byte granularity (even if the reporting will probably be naturally aligned to sectors or even much higher levels). I've therefore started the task of converting our block status code to report at a byte granularity rather than sectors.These patches have been around for a while, but it's time to finish it now that 2.12 is open for patches.
Thanks, touched up patch 8 and applied to the block branch.
Kevin