xen-block: fix sector size confusion
v2 cover xen-block: fix sector size confusion
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
Paul Durrant (2):
xen-block: scale sector based quantities correctly
xen-block: always report 'sectors' in terms of 512-byte units
hw/block/dataplane/xen-block.c | 28 +++++++++++++--------------- hw/block/xen-block.c | 2 +- hw/block/xen_blkif.h | 2 ++ 3 files changed, 16 insertions(+), 16 deletions(-)
---
v2:
- Split up previous single patch
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 1/2 xen-block: scale sector based quantities correctly
The Xen blkif protocol requires that sector based quantities should be interpreted strictly as multiples of 512 bytes. Specifically:
"first_sect and last_sect in blkif_request_segment, as well as sector_number in blkif_request, are always expressed in 512-byte units."
This patch modifies the xen-block code accordingly.
---
Cc: Stefan Hajnoczi <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Anthony Perard <[email protected]>
Cc: Kevin Wolf <[email protected]>
Cc: Max Reitz <[email protected]>
--- hw/block/dataplane/xen-block.c | 28 +++++++++++++--------------- hw/block/xen_blkif.h | 2 ++ 2 files changed, 15 insertions(+), 15 deletions(-)
hw/block/dataplane/xen-block.c +13 -15
@@ -49,7 +49,6 @@ struct XenBlockDataPlane {unsigned int *ring_ref;unsigned int nr_ring_ref;void *sring; -int64_t file_blk;int protocol;blkif_back_rings_t rings;int more_work;
@@ -168,7 +167,7 @@ static int xen_block_parse_request(XenBlockRequest *request) goto err; } - request->start = request->req.sector_number * dataplane->file_blk; + request->start = request->req.sector_number *XEN_BLKIF_SECTOR_SIZE ;for (i =0 ; i < request->req.nr_segments; i++) {if (i ==BLKIF_MAX_SEGMENTS_PER_REQUEST ) {error_report ("error: nr_segments too big" );
@@ -178,14 +177,14 @@ static int xen_block_parse_request(XenBlockRequest *request)error_report ("error: first > last sector" ); goto err; } -if (request->req.seg[i].last_sect * dataplane->file_blk >= +if (request->req.seg[i].last_sect *XEN_BLKIF_SECTOR_SIZE >=XC_PAGE_SIZE ) {error_report ("error: page crossing" ); goto err; } len = (request->req.seg[i].last_sect - - request->req.seg[i].first_sect +1 ) * dataplane->file_blk; + request->req.seg[i].first_sect +1 ) *XEN_BLKIF_SECTOR_SIZE ; request->size += len; }if (request->start + request->size >blk_getlength (dataplane->blk)) {
@@ -205,7 +204,6 @@ static int xen_block_copy_request(XenBlockRequest *request)XenDevice *xendev = dataplane->xendev;XenDeviceGrantCopySegment segs[BLKIF_MAX_SEGMENTS_PER_REQUEST ];int i, count; -int64_t file_blk = dataplane->file_blk;bool to_domain = (request->req.operation ==BLKIF_OP_READ );void *virt = request->buf;Error *local_err =NULL ;
@@ -220,16 +218,17 @@ static int xen_block_copy_request(XenBlockRequest *request)if (to_domain) { segs[i].dest.foreign.ref = request->req.seg[i].gref; segs[i].dest.foreign.offset = request->req.seg[i].first_sect * - file_blk; +XEN_BLKIF_SECTOR_SIZE ; segs[i].source.virt = virt; }else { segs[i].source.foreign.ref = request->req.seg[i].gref; segs[i].source.foreign.offset = request->req.seg[i].first_sect * - file_blk; +XEN_BLKIF_SECTOR_SIZE ; segs[i].dest.virt = virt; } segs[i].len = (request->req.seg[i].last_sect - - request->req.seg[i].first_sect +1 ) * file_blk; + request->req.seg[i].first_sect +1 ) * +XEN_BLKIF_SECTOR_SIZE ; virt += segs[i].len; }
@@ -331,22 +330,22 @@ static bool xen_block_split_discard(XenBlockRequest *request,XenBlockDataPlane *dataplane = request->dataplane;int64_t byte_offset;int byte_chunk; -uint64_t byte_remaining, limit; +uint64_t byte_remaining;uint64_t sec_start = sector_number;uint64_t sec_count = nr_sectors;/* Wrap around, or overflowing byte limit? */ if (sec_start + sec_count < sec_count || - sec_start + sec_count >INT64_MAX / dataplane->file_blk) { + sec_start + sec_count >INT64_MAX /XEN_BLKIF_SECTOR_SIZE ) {return false; } - limit =BDRV_REQUEST_MAX_SECTORS * dataplane->file_blk; - byte_offset = sec_start * dataplane->file_blk; - byte_remaining = sec_count * dataplane->file_blk; + byte_offset = sec_start *XEN_BLKIF_SECTOR_SIZE ; + byte_remaining = sec_count *XEN_BLKIF_SECTOR_SIZE ; do { - byte_chunk = byte_remaining > limit ? limit : byte_remaining; + byte_chunk = byte_remaining > BDRV_REQUEST_MAX_BYTES ? + BDRV_REQUEST_MAX_BYTES : byte_remaining; request->aio_inflight++; blk_aio_pdiscard(dataplane->blk, byte_offset, byte_chunk, xen_block_complete_aio, request);
@@ -632,7 +631,6 @@ XenBlockDataPlane *xen_block_dataplane_create(XenDevice *xendev,XenBlockDataPlane *dataplane =g_new0 (XenBlockDataPlane,1 ); dataplane->xendev = xendev; - dataplane->file_blk = conf->logical_block_size; dataplane->blk = conf->blk;QLIST_INIT (&dataplane->inflight);
hw/block/xen_blkif.h +2 -0
@@ -143,4 +143,6 @@ static inline void blkif_get_x86_64_req(blkif_request_t *dst, } } +#define XEN_BLKIF_SECTOR_SIZE 512 +#endif /* XEN_BLKIF_H */
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 2/2 xen-block: always report 'sectors' in terms of 512-byte units
In reply to Paul DurrantThe mail thread at [1] clarifies that the Xen blkif protocol requires that 'sectors' value reported in xenstore is strictly in terms of 512-byte units and is not dependent on the logical sector size reported in 'sector-size'.
[1] https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg01600.html
---
Cc: Stefano Stabellini <[email protected]>
Cc: Anthony Perard <[email protected]>
Cc: Kevin Wolf <[email protected]>
Cc: Max Reitz <[email protected]>
--- hw/block/xen-block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
hw/block/xen-block.c +1 -1
@@ -149,7 +149,7 @@ static void xen_block_set_size(XenBlockDevice *blockdev)const char *type =object_get_typename (OBJECT (blockdev));XenBlockVdev *vdev = &blockdev->props.vdev;BlockConf *conf = &blockdev->props.conf; -int64_t sectors =blk_getlength (conf->blk) / conf->logical_block_size; +int64_t sectors =blk_getlength (conf->blk) /XEN_BLKIF_SECTOR_SIZE ;XenDevice *xendev =XEN_DEVICE (blockdev);trace_xen_block_size (type, vdev->disk, vdev->partition, sectors);
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 6 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 15 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony Perard <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionQuoted 5 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
Quoted 4 lines
Quoted 1 line
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 11 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.
Yes, that's how we are going to fix things.
Quoted 4 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec. I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed as far as patch #1 goes.
Anthony, thoughts?
Paul
Quoted 2 lines
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 37 lines · Paul Durrant
Quoted 15 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony Perard <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionQuoted 5 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
Quoted 4 lines
Quoted 1 line
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 11 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Yes, that's how we are going to fix things.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec. I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed as far as patch #1 goes.
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Signature · 6 lines
Anthony PERARD _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 48 lines · Anthony PERARD
Quoted 32 lines · Paul Durrant
Quoted 10 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony Perard <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionOn 27/03/2019 17:32, Paul Durrant wrote:
Quoted 8 lines
Quoted 6 lines
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 10 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Yes, that's how we are going to fix things.
Quoted 7 lines
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec. I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed as far as patch #1 goes.
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
Quoted 7 lines
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Hmm - I hadn't realised this is an unreleased issue in qemu.
So, Qemu used to unconditionally set sector-size=512, and your work to qdev-ify everything introduced a change which has identified a spec/protocol issue?
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Am 28.03.2019 um 12:46 hat Andrew Cooper geschrieben:
Quoted 57 lines · Anthony PERARD
Quoted 48 lines · Paul Durrant
Quoted 32 lines
Quoted 10 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony Perard <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionOn 27/03/2019 17:32, Paul Durrant wrote:
Quoted 8 lines
Quoted 6 lines
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 10 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Yes, that's how we are going to fix things.
Quoted 7 lines
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec. I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed as far as patch #1 goes.
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
Quoted 7 lines
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Quoted 5 lines
Hmm - I hadn't realised this is an unreleased issue in qemu.
So, Qemu used to unconditionally set sector-size=512, and your work to qdev-ify everything introduced a change which has identified a spec/protocol issue?
The old implementation has the sector size hardcoded:
#define BLOCK_SIZE 512
Whereas the qdevified version uses DEFINE_BLOCK_PROPERTIES(), which includes user-visible options for logical/physical_block_size.
So before, you couldn't even define a different sector size and the question whether 512 or the sector size should be used didn't make a difference anyway.
Quoted 6 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
The closest thing to returning to the old behaviour would be erroring out during device initialisation if logical_block_size != 512.
Kevin
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 70 lines
-----Original Message-----
From: Andrew Cooper
Sent: 28 March 2019 11:46
To: Anthony Perard <[email protected]>; Paul Durrant <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionQuoted 8 lines · Anthony PERARD
Quoted 7 lines · Paul Durrant
Quoted 7 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony PerardQuoted 23 lines
Quoted 23 lines
Quoted 11 lines
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusion
Quoted 6 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 10 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Quoted 6 lines
Quoted 6 lines
Yes, that's how we are going to fix things.
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as
sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec.
Quoted 1 line
Quoted 1 line
I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed
as far as patch #1 goes.
Quoted 10 lines
Quoted 2 lines
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
Quoted 7 lines
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Quoted 5 lines
Hmm - I hadn't realised this is an unreleased issue in qemu.
So, Qemu used to unconditionally set sector-size=512, and your work to qdev-ify everything introduced a change which has identified a spec/protocol issue?
Basically, yes. I had not realized at the time how bad the spec. is. I was referring to prevailing implementations, which seemed to use sector-size as the multiple and, in the Windows case, appear to cope with a logical sector size other than 512. Given what I know now, fixing sector-size to be 512 seems like the only way to avoid regressions.
Quoted 7 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
Ok, I'll expand the commit comment and state that the spec. will be revised to allow greater logical block sizes.
Paul
Quoted 2 lines
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 70 lines
-----Original Message-----
From: Andrew Cooper
Sent: 28 March 2019 11:46
To: Anthony Perard <[email protected]>; Paul Durrant <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionQuoted 8 lines · Anthony PERARD
Quoted 7 lines · Paul Durrant
Quoted 7 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony PerardQuoted 23 lines
Quoted 23 lines
Quoted 11 lines
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusion
Quoted 6 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 10 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Quoted 6 lines
Quoted 6 lines
Yes, that's how we are going to fix things.
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as
sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec.
Quoted 1 line
Quoted 1 line
I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed
as far as patch #1 goes.
Quoted 10 lines
Quoted 2 lines
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
Quoted 7 lines
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Quoted 5 lines
Hmm - I hadn't realised this is an unreleased issue in qemu.
So, Qemu used to unconditionally set sector-size=512, and your work to qdev-ify everything introduced a change which has identified a spec/protocol issue?
Basically, yes. I had not realized at the time how bad the spec. is. I was referring to prevailing implementations, which seemed to use sector-size as the multiple and, in the Windows case, appear to cope with a logical sector size other than 512. Given what I know now, fixing sector-size to be 512 seems like the only way to avoid regressions.
Quoted 7 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
Ok, I'll expand the commit comment and state that the spec. will be revised to allow greater logical block sizes.
Paul
Quoted 2 lines
~Andrew
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 96 lines
-----Original Message-----
From: Kevin Wolf [mailto:[email protected]]
Sent: 28 March 2019 11:56
To: Andrew Cooper <[email protected]>
Cc: Anthony Perard <[email protected]>; Paul Durrant <[email protected]>; xen- [email protected]; [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionAm 28.03.2019 um 12:46 hat Andrew Cooper geschrieben:
Quoted 6 lines · Anthony PERARD
Quoted 5 lines · Paul Durrant
Quoted 4 lines
Quoted 4 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; qemu-Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
[email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony PerardQuoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 11 lines
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusion
Quoted 6 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 10 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Yes, that's how we are going to fix things.
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as
sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec.
Quoted 1 line
Quoted 1 line
Quoted 1 line
I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed
as far as patch #1 goes.
Quoted 18 lines
Quoted 10 lines
Quoted 2 lines
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
Quoted 7 lines
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Quoted 5 lines
Hmm - I hadn't realised this is an unreleased issue in qemu.
So, Qemu used to unconditionally set sector-size=512, and your work to qdev-ify everything introduced a change which has identified a spec/protocol issue?
The old implementation has the sector size hardcoded:
#define BLOCK_SIZE 512Whereas the qdevified version uses
DEFINE_BLOCK_PROPERTIES(), which includes user-visible options for logical/physical_block_size.So before, you couldn't even define a different sector size and the question whether 512 or the sector size should be used didn't make a difference anyway.
Quoted 6 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
Quoted 2 lines
The closest thing to returning to the old behaviour would be erroring out during device initialisation if
logical_block_size!= 512.
One thing I've not figured out... If I create a blockdev in QEMU that is pointing at a real device with a logical_block_size of 4k, will the QEMU block layer perform the necessary read-modify-write cycles for accesses < 4k? IOW would it be safe to always advertise a size of 512 to a frontend?
The problem with erroring out during device init is that it does not give us a way of fixing things in future, as the frontend has not started at that time and thus we'd have no idea whether it could use whatever protocol fix we come up with. I think the only thing the backend could do is refuse to connect to an old frontend if logical_block_size != 512.
Paul
Quoted 2 lines
Kevin
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Am 01.04.2019 um 11:01 hat Paul Durrant geschrieben:
Quoted 96 lines
Quoted 96 lines
-----Original Message-----
From: Kevin Wolf [mailto:[email protected]]
Sent: 28 March 2019 11:56
To: Andrew Cooper <[email protected]>
Cc: Anthony Perard <[email protected]>; Paul Durrant <[email protected]>; xen- [email protected]; [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusionAm 28.03.2019 um 12:46 hat Andrew Cooper geschrieben:
Quoted 6 lines · Anthony PERARD
Quoted 5 lines · Paul Durrant
Quoted 4 lines
Quoted 4 lines
-----Original Message-----
From: Andrew Cooper
Sent: 27 March 2019 18:20
To: Paul Durrant <[email protected]>; [email protected]; qemu-Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
[email protected]
Cc: Kevin Wolf <[email protected]>; Stefano Stabellini <[email protected]>; Max Reitz <[email protected]>; Stefan Hajnoczi <[email protected]>; Anthony PerardQuoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 11 lines
Subject: Re: [Xen-devel] [PATCH v2 0/2] xen-block: fix sector size confusion
Quoted 6 lines · Paul Durrant
The Xen blkif protocol is confusing but discussion with the maintainer has clarified that sector based quantities in requests and the 'sectors' value advertized in xenstore should always be in terms of 512-byte units and not the advertised logical 'sector-size' value.
This series fixes xen-block to adhere to the spec.
I thought we agreed that hardcoding things to 512 bytes was the wrong thing to do.
To some extent we decided it was the *only* thing to do.
Quoted 10 lines
I was expecting something like:
1) Clarify the spec with the intended meaning, (which is what some implementations actually use already) and wont cripple 4k datapaths.
2) Introduce a compatibility key for "I don't rely on sector-size being 512", which fixed implementations should advertise.
3) Specify that because of bugs in the spec which got out into the wild, drivers which don't find the key being advertised by the other end should emulate sector-size=512 for compatibility with broken implementations.Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Yes, that's how we are going to fix things.
Quoted 3 lines
Whatever the eventual way out, the first thing which needs to happen is an update to the spec, before actions are taken to alter existing implementations.
Well the implementation is currently wrong w.r.t. the spec and these patches fix that. As long as
sector-size remains at 512 then no existing frontend should break, so I guess you could argue that patch #2 should also make sure that sector-size is also 512... but that is not yet in the spec.
Quoted 1 line
Quoted 1 line
Quoted 1 line
I guess I'm ok to defer patch #2 until a revised spec. is agreed, but the ship has already sailed
as far as patch #1 goes.
Quoted 18 lines
Quoted 10 lines
Quoted 2 lines
Anthony, thoughts?
So QEMU used to always set "sector-size" to 512, and used that for request. The new implementation (not released yet) doesn't do that anymore, and may set "sector-size" to a different value and used that for requests.
patch #1 is one way to fix the requests (and avoid regression) and more clearly spell out the weird thing about the spec.
Quoted 7 lines
I also think patch #2 is too soon and should point to a commit in xen.git instead of a thread on xen-devel.
In the meantime, we should probably set "sector-size" to 512, like QEMU used to do anyway, with a comment about the fact that different implementations uses sector-size differently and a value of 512 would work fine.
Quoted 5 lines
Hmm - I hadn't realised this is an unreleased issue in qemu.
So, Qemu used to unconditionally set sector-size=512, and your work to qdev-ify everything introduced a change which has identified a spec/protocol issue?
The old implementation has the sector size hardcoded:
#define BLOCK_SIZE 512Whereas the qdevified version uses
DEFINE_BLOCK_PROPERTIES(), which includes user-visible options for logical/physical_block_size.So before, you couldn't even define a different sector size and the question whether 512 or the sector size should be used didn't make a difference anyway.
Quoted 6 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
Quoted 8 lines
Quoted 2 lines
The closest thing to returning to the old behaviour would be erroring out during device initialisation if
logical_block_size!= 512.One thing I've not figured out... If I create a blockdev in QEMU that is pointing at a real device with a
logical_block_sizeof 4k, will the QEMU block layer perform the necessary read-modify-write cycles for accesses < 4k? IOW would it be safe to always advertise a size of 512 to a frontend?
Yes, for 512 accesses on native 4k disks with O_DIRECT, the QEMU block layer performs the necessary RMW. Of course, it still comes with a performance penalty, so you want to avoid such setups, but they do work.
Quoted 6 lines
The problem with erroring out during device init is that it does not give us a way of fixing things in future, as the frontend has not started at that time and thus we'd have no idea whether it could use whatever protocol fix we come up with. I think the only thing the backend could do is refuse to connect to an old frontend if
logical_block_size!= 512.
I was just thinking of getting back to the old state, with a quick fix (by making the problematic new setting inaccessible) for the bug in 4.0 that could possible be merged today or tomorrow for rc2.
What you need to do for actually supporting 4k disks in the long term (QEMU 4.1 or later) depends on what the drivers look like currently and is a separate discussion.
Kevin
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 1 line
-----Original Message-----
[snip]
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
The old implementation has the sector size hardcoded:
#define BLOCK_SIZE 512Whereas the qdevified version uses
DEFINE_BLOCK_PROPERTIES(), which includes user-visible options for logical/physical_block_size.So before, you couldn't even define a different sector size and the question whether 512 or the sector size should be used didn't make a difference anyway.
Quoted 6 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
Quoted 8 lines
Quoted 2 lines
The closest thing to returning to the old behaviour would be erroring out during device initialisation if
logical_block_size!= 512.One thing I've not figured out... If I create a blockdev in QEMU that is pointing at a real device with a
logical_block_sizeof 4k, will the QEMU block layer perform the necessary read-modify-write cycles for accesses < 4k? IOW would it be safe to always advertise a size of 512 to a frontend?
Quoted 4 lines
Yes, for 512 accesses on native 4k disks with
O_DIRECT, the QEMU block layer performs the necessary RMW. Of course, it still comes with a performance penalty, so you want to avoid such setups, but they do work.
Ok, that's good. Thanks.
Quoted 11 lines
Quoted 6 lines
The problem with erroring out during device init is that it does not give us a way of fixing things in future, as the frontend has not started at that time and thus we'd have no idea whether it could use whatever protocol fix we come up with. I think the only thing the backend could do is refuse to connect to an old frontend if
logical_block_size!= 512.I was just thinking of getting back to the old state, with a quick fix (by making the problematic new setting inaccessible) for the bug in 4.0 that could possible be merged today or tomorrow for rc2.
Ok, I see what you mean. I'll modify and resubmit patch #1 today.
Quoted 4 lines
What you need to do for actually supporting 4k disks in the long term (QEMU 4.1 or later) depends on what the drivers look like currently and is a separate discussion.
Yes, agreed.
Paul
Quoted 1 line
Kevin
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 1 line
-----Original Message-----
[snip]
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
The old implementation has the sector size hardcoded:
#define BLOCK_SIZE 512Whereas the qdevified version uses
DEFINE_BLOCK_PROPERTIES(), which includes user-visible options for logical/physical_block_size.So before, you couldn't even define a different sector size and the question whether 512 or the sector size should be used didn't make a difference anyway.
Quoted 6 lines
If so, then I think it is fine for this series to state (much more clearly than it does) that it is returning qemu's behaviour to match the currently released version, because we've discovered an issue in the spec/protocol, and that we will subsequently work address the issue in the spec and provide a forwards path which doesn't involve nailing our feet to the floor.
Quoted 8 lines
Quoted 2 lines
The closest thing to returning to the old behaviour would be erroring out during device initialisation if
logical_block_size!= 512.One thing I've not figured out... If I create a blockdev in QEMU that is pointing at a real device with a
logical_block_sizeof 4k, will the QEMU block layer perform the necessary read-modify-write cycles for accesses < 4k? IOW would it be safe to always advertise a size of 512 to a frontend?
Quoted 4 lines
Yes, for 512 accesses on native 4k disks with
O_DIRECT, the QEMU block layer performs the necessary RMW. Of course, it still comes with a performance penalty, so you want to avoid such setups, but they do work.
Ok, that's good. Thanks.
Quoted 11 lines
Quoted 6 lines
The problem with erroring out during device init is that it does not give us a way of fixing things in future, as the frontend has not started at that time and thus we'd have no idea whether it could use whatever protocol fix we come up with. I think the only thing the backend could do is refuse to connect to an old frontend if
logical_block_size!= 512.I was just thinking of getting back to the old state, with a quick fix (by making the problematic new setting inaccessible) for the bug in 4.0 that could possible be merged today or tomorrow for rc2.
Ok, I see what you mean. I'll modify and resubmit patch #1 today.
Quoted 4 lines
What you need to do for actually supporting 4k disks in the long term (QEMU 4.1 or later) depends on what the drivers look like currently and is a separate discussion.
Yes, agreed.
Paul
Quoted 1 line
Kevin
Den 01.04.2019 11:34, skrev Kevin Wolf:
Quoted 4 lines
Yes, for 512 accesses on native 4k disks with
O_DIRECT, the QEMU block layer performs the necessary RMW. Of course, it still comes with a performance penalty, so you want to avoid such setups, but they do work.
I suspect that the approximately 1/10 -th disk-speed I see (e.g 28389 vs 258719 K/sec sequential disk Output) in domU compared to dom0 must be due to this. I have dom0 and domU drives as partitions on an md raid6. The performance is sub-optimal to put it mildly. Here is one example bonnie++ run on a guest:
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 3 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
gt-credit2-on-p 16G 1062 90 28389 2 20233 2 3950 82 199633 10 1195 17
Latency 19003us 140ms 652ms 20811us 602ms 33189us
Version 1.97 ------Sequential Create------ --------Random Create--------
gt-credit2-on-pt -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
127 34445 26 +++++ +++ 42424 27 67418 51 +++++ +++ 52590 35
Latency 12979us 329us 2903us 304us 38us 668us 1.97,1.97,gt-credit2-on-pt,3,1550928114,16G,,1062,90,28389,2,20233,2,3950,82,199633,10,1195,17,127,,,,,34445,26,+++++,+++,42424,27,67418,51,+++++,+++,52590,35,19003us,140ms,652ms,20811us,602ms,33189us,12979us,329us,2903us,304us,38us,668us
Same on dom0:
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 3 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
XEN-4.12-DOM0 16G 475 91 258719 20 151508 19 585 99 324725 22 392.3 14
Latency 16093us 1097ms 406ms 18136us 160ms 155ms
Version 1.97 ------Sequential Create------ --------Random Create--------
XEN-4.12-DOM0 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
8 6563 11 +++++ +++ 13522 20 10969 18 +++++ +++ 9560 13
Latency 104us 68us 18179us 120us 20us 63us 1.97,1.97,XEN-4.12-DOM0,3,1547961396,16G,,475,91,258719,20,151508,19,585,99,324725,22,392.3,14,8,,,,,6563,11,+++++,+++,13522,20,10969,18,+++++,+++,9560,13,16093us,1097ms,406ms,18136us,160ms,155ms,104us,68us,18179us,120us,20us,63us
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel