virtio-blk: discard and write zeroes clarifications
v2 cover virtio-blk: discard and write zeroes clarifications
Several clarifications have been requested for the VIRTIO 1.1 virtio-blk discard and write zeroes commands. This series takes care of them.
Stefan Hajnoczi (4):
virtio-blk: document data[] size constraints
virtio-blk: move virtio_blk_discard_write_zeroes definition
virtio-blk: describe write zeroes unmap semantics
virtio-blk: avoid inconsistent "DISCARD" term
content.tex | 47 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 15 deletions(-)
Signature · 6 lines
2.20.1 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v2 1/4 virtio-blk: document data[] size constraints
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct virtio_blk_discard_write_zeroes elements into data[], so it must be a multiple of the struct size.
The uint8_t data[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since struct virtio_blk_discard_write_data is only 16 bytes long.
Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
content.tex +13 -1
@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };
@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while
@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request. +The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. + If the VIRTIO_BLK_F_CONFIG_WCE feature is negotiated, the driver MAY switch to writethrough or writeback mode by writing respectively 0 and 1 to the \field{writeback} field. After writing a 0 to \field{writeback},
Signature · 6 lines
2.20.1 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[virtio-dev] Re: [PATCH v2 1/4] virtio-blk: document data[] size constraints
Quoted 53 lines · Stefan Hajnoczi
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct
virtio_blk_discard_write_zeroeselements into data[], so it must be a multiple of the struct size.The
uint8_tdata[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since structvirtio_blk_discard_write_datais only 16 bytes long.Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
Cc Michael S. Tsirkin <[email protected]>Cc Changpeng Liu <[email protected]>Cc Stefano Garzarella <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)content.tex +6
-1@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while
content.tex A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. · line 4011
Quoted 10 lines
content.tex +7
-0@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request. +The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. +
So a single request can discard/write multiple ranges? It might be a good idea to make this explicit.
Also is this capability useful/used? And what's the value of status in case some of the requests fail?
Quoted 5 lines
content.tex +0
-0If the VIRTIO_BLK_F_CONFIG_WCE feature is negotiated, the driver MAY switch to writethrough or writeback mode by writing respectively 0 and 1 to the \field{writeback} field. After writing a 0 to \field{writeback},Signature · 1 line
2.20.1
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [virtio-dev] Re: [PATCH v2 1/4] virtio-blk: document data[] size constraints
Quoted 57 lines · Michael S. Tsirkin
Quoted 56 lines · Stefan Hajnoczi
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct
virtio_blk_discard_write_zeroeselements into data[], so it must be a multiple of the struct size.The
uint8_tdata[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since structvirtio_blk_discard_write_datais only 16 bytes long.Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
Cc Michael S. Tsirkin <[email protected]>Cc Changpeng Liu <[email protected]>Cc Stefano Garzarella <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)content.tex +6
-1@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request.
Quoted 14 lines
content.tex A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. · line 4011
Quoted 7 lines
content.tex +7
-0+The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. +So a single request can discard/write multiple ranges? It might be a good idea to make this explicit.
Also is this capability useful/used? And what's the value of status in case some of the requests fail?
What happened with this comment? I don't see a follow-up nor a resolution elsewhere, just the opening of issue #32 for voting. Please clarify.
Jan
Quoted 10 lines
Quoted 5 lines
content.tex +0
-0If the VIRTIO_BLK_F_CONFIG_WCE feature is negotiated, the driver MAY switch to writethrough or writeback mode by writing respectively 0 and 1 to the \field{writeback} field. After writing a 0 to \field{writeback},Signature · 1 line
2.20.1---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Signature · 6 lines
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [virtio-dev] Re: [PATCH v2 1/4] virtio-blk: document data[] size constraints
Quoted 55 lines · Jan Kiszka
Quoted 54 lines · Michael S. Tsirkin
Quoted 53 lines · Stefan Hajnoczi
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct
virtio_blk_discard_write_zeroeselements into data[], so it must be a multiple of the struct size.The
uint8_tdata[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since structvirtio_blk_discard_write_datais only 16 bytes long.Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
Cc Michael S. Tsirkin <[email protected]>Cc Changpeng Liu <[email protected]>Cc Stefano Garzarella <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)content.tex +6
-1@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request.
Quoted 8 lines
Quoted 8 lines
content.tex A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. · line 4011
Quoted 7 lines
content.tex +7
-0+The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. +
I'm not the original spec author. Feel free to correct me if this is wrong:
Quoted 3 lines
Quoted 3 lines
So a single request can discard/write multiple ranges? It might be a good idea to make this explicit. Also is this capability useful/used?
The multiple segments feature was included because the underlying storage might support it. Currently we don't expect much use but future hardware may rely on it more heavily (e.g. for performance).
Quoted 2 lines
Quoted 2 lines
And what's the value of status
in case some of the requests fail?
A failure for any segment causes the entire request to fail with no information about which segments completed or failed.
Quoted 2 lines
What happened with this comment? I don't see a follow-up nor a resolution elsewhere, just the opening of issue #32 for voting. Please clarify.
With #32 applied the spec says:
"max_discard_seg can be read to determine the [...] maximum number of discard segments for the block driver to use"
and
"The length of \field{data} MUST be a multiple of the size of struct virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and VIRTIO_BLK_T_WRITE_ZEROES requests."
This is not very explicit but it means multiple struct virtio_blk_discard_write_zeroes can be included in a request, up to max_discard_seg.
I think two things are appropriate:
1. A driver normative statement saying up to max_discard_seg/max_write_zeroes_seg structs may be included in a request
2. A general description that says DISCARD/WRITE_ZEROES requests may have more than 1 "segment" (struct virtio_blk_discard_write_zeroes)
Does that sound good?
Stefan
Re: [virtio-dev] Re: [PATCH v2 1/4] virtio-blk: document data[] size constraints
Quoted 76 lines · Stefan Hajnoczi
Quoted 55 lines · Jan Kiszka
Quoted 54 lines · Michael S. Tsirkin
Quoted 53 lines · Stefan Hajnoczi
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct
virtio_blk_discard_write_zeroeselements into data[], so it must be a multiple of the struct size.The
uint8_tdata[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since structvirtio_blk_discard_write_datais only 16 bytes long.Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
Cc Michael S. Tsirkin <[email protected]>Cc Changpeng Liu <[email protected]>Cc Stefano Garzarella <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)content.tex +6
-1@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request.Quoted 8 lines
Quoted 8 lines
content.tex A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. · line 4011
Quoted 7 lines
content.tex +7
-0+The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. +I'm not the original spec author. Feel free to correct me if this is wrong:
Quoted 3 lines
Quoted 3 lines
So a single request can discard/write multiple ranges? It might be a good idea to make this explicit. Also is this capability useful/used?
The multiple segments feature was included because the underlying storage might support it. Currently we don't expect much use but future hardware may rely on it more heavily (e.g. for performance).
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
And what's the value of status
in case some of the requests fail?A failure for any segment causes the entire request to fail with no information about which segments completed or failed.
And what's the status?
Quoted 24 lines
Quoted 2 lines
What happened with this comment? I don't see a follow-up nor a resolution elsewhere, just the opening of issue #32 for voting. Please clarify.
With #32 applied the spec says:
"
max_discard_segcan be read to determine the [...] maximum number of discard segments for the block driver to use"and
"The length of \field{data} MUST be a multiple of the size of struct
virtio_blk_discard_write_zeroesforVIRTIO_BLK_T_DISCARDandVIRTIO_BLK_T_WRITE_ZEROESrequests."This is not very explicit but it means multiple struct
virtio_blk_discard_write_zeroescan be included in a request, up tomax_discard_seg.I think two things are appropriate:
1. A driver normative statement saying up to max_discard_seg/max_write_zeroes_seg structs may be included in a request
2. A general description that says DISCARD/WRITE_ZEROES requests may have more than 1 "segment" (structvirtio_blk_discard_write_zeroes)
Quoted 3 lines
Does that sound good?
Stefan
Also pls include explanation about failure mode.
Signature · 5 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [virtio-dev] Re: [PATCH v2 1/4] virtio-blk: document data[] size constraints
In reply to Stefan HajnocziQuoted 106 lines · Stefan Hajnoczi
Quoted 55 lines · Jan Kiszka
Quoted 54 lines · Michael S. Tsirkin
Quoted 53 lines · Stefan Hajnoczi
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct
virtio_blk_discard_write_zeroeselements into data[], so it must be a multiple of the struct size.The
uint8_tdata[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since structvirtio_blk_discard_write_datais only 16 bytes long.Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
Cc Michael S. Tsirkin <[email protected]>Cc Changpeng Liu <[email protected]>Cc Stefano Garzarella <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)content.tex +6
-1@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request.Quoted 8 lines
Quoted 8 lines
content.tex A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. · line 4011
Quoted 7 lines
content.tex +7
-0+The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. +I'm not the original spec author. Feel free to correct me if this is wrong:
Quoted 3 lines
Quoted 3 lines
So a single request can discard/write multiple ranges? It might be a good idea to make this explicit. Also is this capability useful/used?
The multiple segments feature was included because the underlying storage might support it. Currently we don't expect much use but future hardware may rely on it more heavily (e.g. for performance).
Quoted 2 lines
Quoted 2 lines
And what's the value of status
in case some of the requests fail?A failure for any segment causes the entire request to fail with no information about which segments completed or failed.
Quoted 2 lines
What happened with this comment? I don't see a follow-up nor a resolution elsewhere, just the opening of issue #32 for voting. Please clarify.
With #32 applied the spec says:
"
max_discard_segcan be read to determine the [...] maximum number of discard segments for the block driver to use"and
"The length of \field{data} MUST be a multiple of the size of struct
virtio_blk_discard_write_zeroesforVIRTIO_BLK_T_DISCARDandVIRTIO_BLK_T_WRITE_ZEROESrequests."This is not very explicit but it means multiple struct
virtio_blk_discard_write_zeroescan be included in a request, up tomax_discard_seg.I think two things are appropriate:
1. A driver normative statement saying up to max_discard_seg/max_write_zeroes_seg structs may be included in a request
2. A general description that says DISCARD/WRITE_ZEROES requests may have more than 1 "segment" (structvirtio_blk_discard_write_zeroes)
Quoted 3 lines
Does that sound good?
Stefan
Jan could you confirm please? Let's create an issue and close this.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [virtio-dev] Re: [PATCH v2 1/4] virtio-blk: document data[] size constraints
Quoted 107 lines · Michael S. Tsirkin
Quoted 106 lines · Stefan Hajnoczi
Quoted 55 lines · Jan Kiszka
Quoted 54 lines · Michael S. Tsirkin
Quoted 53 lines · Stefan Hajnoczi
The struct virtio_blk_req->data[] field is a multiple of 512 bytes long for read and write requests. Flush requests don't use data[] at all.
The new discard and write zeroes requests being introduced in VIRTIO 1.1 put struct
virtio_blk_discard_write_zeroeselements into data[], so it must be a multiple of the struct size.The
uint8_tdata[][512] pseudo-code makes it look like discard and write zeroes requests must pad to 512 bytes. This wastes memory since structvirtio_blk_discard_write_datais only 16 bytes long.Furthermore, all known implementations wishing to take advantage of this upcoming VIRTIO 1.1 feature do not use 512-byte padding (Linux virtio_blk.ko, QEMU virtio-blk device emulation, the SPDK virtio-blk driver, and the SPDK vhost-user-blk device backend).
This patch documents the data[] size constraints clearly in the driver normative section. This is clearer than the current pseudo-code.
Cc Michael S. Tsirkin <[email protected]>Cc Changpeng Liu <[email protected]>Cc Stefano Garzarella <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)content.tex +6
-1@@ -3941,7 +3941,7 @@ struct virtio_blk_req { le32 type; le32 reserved; le64 sector; - u8 data[][512]; + u8 data[]; u8 status; };@@ -3971,6 +3971,11 @@ The \field{sector} number indicates the offset (multiplied by 512) where the read or write is to occur. This field is unused and set to 0 for commands other than read or write. +VIRTIO_BLK_T_IN requests populate \field{data} with the contents of sectors +read from the block device (in multiples of 512 bytes). VIRTIO_BLK_T_OUT +requests write the contents of \field{data} to the block device (in multiples +of 512 bytes). + The \field{data} used for discard or write zeroes command is described by one or more virtio_blk_discard_write_zeroes structs. \field{sector} indicates the starting offset (in 512-byte units) of the segment, while@@ -3997,6 +4002,13 @@ A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. A driver MUST set \field{sector} to 0 for a VIRTIO_BLK_T_FLUSH request. A driver SHOULD NOT include any data in a VIRTIO_BLK_T_FLUSH request.Quoted 8 lines
Quoted 8 lines
content.tex A driver SHOULD accept the VIRTIO_BLK_F_RO feature if offered. · line 4011
Quoted 7 lines
content.tex +7
-0+The length of \field{data} MUST be a multiple of 512 bytes for VIRTIO_BLK_T_IN +and VIRTIO_BLK_T_OUT requests. + +The length of \field{data} MUST be a multiple of the size of struct +virtio_blk_discard_write_zeroes for VIRTIO_BLK_T_DISCARD and +VIRTIO_BLK_T_WRITE_ZEROES requests. +I'm not the original spec author. Feel free to correct me if this is wrong:
Quoted 3 lines
Quoted 3 lines
So a single request can discard/write multiple ranges? It might be a good idea to make this explicit. Also is this capability useful/used?
The multiple segments feature was included because the underlying storage might support it. Currently we don't expect much use but future hardware may rely on it more heavily (e.g. for performance).
Quoted 2 lines
Quoted 2 lines
And what's the value of status
in case some of the requests fail?A failure for any segment causes the entire request to fail with no information about which segments completed or failed.
Quoted 2 lines
What happened with this comment? I don't see a follow-up nor a resolution elsewhere, just the opening of issue #32 for voting. Please clarify.
With #32 applied the spec says:
"
max_discard_segcan be read to determine the [...] maximum number of discard segments for the block driver to use"and
"The length of \field{data} MUST be a multiple of the size of struct
virtio_blk_discard_write_zeroesforVIRTIO_BLK_T_DISCARDandVIRTIO_BLK_T_WRITE_ZEROESrequests."This is not very explicit but it means multiple struct
virtio_blk_discard_write_zeroescan be included in a request, up tomax_discard_seg.I think two things are appropriate:
1. A driver normative statement saying up to max_discard_seg/max_write_zeroes_seg structs may be included in a request
2. A general description that says DISCARD/WRITE_ZEROES requests may have more than 1 "segment" (structvirtio_blk_discard_write_zeroes)
Quoted 5 lines
Quoted 3 lines
Does that sound good?
Stefan
Jan could you confirm please? Let's create an issue and close this.
I have sent a new revision of the patches with the clarifications we've discussed here.
Stefan
PATCH v2 2/4 virtio-blk: move virtio_blk_discard_write_zeroes definition
In reply to Stefan Hajnoczistruct virtio_blk_discard_write_zeroes is defined alongside struct virtio_blk_req but only discussed later in the text. Move it to where it belongs.
--- content.tex | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-)
content.tex +16 -13
@@ -3944,15 +3944,6 @@ struct virtio_blk_req { u8 data[]; u8 status; }; - -struct virtio_blk_discard_write_zeroes { - le64 sector; - le32 num_sectors; - struct { - le32 unmap:1; - le32 reserved:31; - } flags; -}; \end{lstlisting} The type of the request is either a read (VIRTIO_BLK_T_IN), a write
@@ -3977,10 +3968,22 @@ requests write the contents of \field{data} to the block device (in multiples of 512 bytes). The \field{data} used for discard or write zeroes command is described -by one or more virtio_blk_discard_write_zeroes structs. \field{sector} -indicates the starting offset (in 512-byte units) of the segment, while -\field{num_sectors} indicates the number of sectors in each discarded -range. \field{unmap} is only used for write zeroes command. +by one or more virtio_blk_discard_write_zeroes structs: + +\begin{lstlisting} +struct virtio_blk_discard_write_zeroes { + le64 sector; + le32 num_sectors; + struct { + le32 unmap:1; + le32 reserved:31; + } flags; +}; +\end{lstlisting} + +\field{sector} indicates the starting offset (in 512-byte units) of the +segment, while \field{num_sectors} indicates the number of sectors in each +discarded range. \field{unmap} is only used for write zeroes command. The final \field{status} byte is written by the device: either VIRTIO_BLK_S_OK for success, VIRTIO_BLK_S_IOERR for device or driver
Signature · 6 lines
2.20.1 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v2 3/4 virtio-blk: describe write zeroes unmap semantics
In reply to Stefan HajnocziExplain the meaning of the unmap flag. The details are already covered in the device normative section but mentioning it here makes the text easier to understand.
--- content.tex | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
content.tex +3 -1
@@ -3983,7 +3983,9 @@ struct virtio_blk_discard_write_zeroes { \field{sector} indicates the starting offset (in 512-byte units) of the segment, while \field{num_sectors} indicates the number of sectors in each -discarded range. \field{unmap} is only used for write zeroes command. +discarded range. \field{unmap} is only used in write zeroes commands and allows +the device to discard the specified range, provided that following reads return +zeroes. The final \field{status} byte is written by the device: either VIRTIO_BLK_S_OK for success, VIRTIO_BLK_S_IOERR for device or driver
Signature · 6 lines
2.20.1 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
PATCH v2 4/4 virtio-blk: avoid inconsistent "DISCARD" term
In reply to Stefan Hajnoczi"discard" (lowercase) is used throughout the text. Remove a lone instance of "DISCARD" (uppercase).
--- content.tex | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
content.tex +1 -1
@@ -4040,7 +4040,7 @@ sectors in the device backend storage. For write zeroes commands, if the \field{unmap} is set, the device MAY deallocate the specified range of sectors in the device backend storage, -as if the DISCARD command had been sent. After a write zeroes command +as if the discard command had been sent. After a write zeroes command is completed, reads of the specified ranges of sectors MUST return zeroes. This is true independent of whether \field{unmap} was set or clear.
Signature · 6 lines
2.20.1 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]