Add VIRTIO_RING_F_INDIRECT_SIZE and queue_indirect_size
cover Add VIRTIO_RING_F_INDIRECT_SIZE and queue_indirect_size
This is a follow-up of:
[0/2]: https://lists.oasis-open.org/archives/virtio-comment/202111/msg00089.html
[1/2]: https://lists.oasis-open.org/archives/virtio-comment/202112/msg00005.html
[2/2]: https://lists.oasis-open.org/archives/virtio-comment/202112/msg00006.html
These two patches introduce a more fine graded control over the maximum length of indirect descriptor tables.
Associated Github Task:
https://github.com/oasis-tcs/virtio-spec/issues/122
Changes:
* VIRTIO_RING_F_LARGE_INDIRECT_DESC -> VIRTIO_RING_F_INDIRECT_SIZE
[patch 1], [patch 2]
* queue_indirect_size field: 16 bit -> 32 bit [patch 2]
* Semantic change: "per indirect descriptor table" -> "per vring slot".
[patch 2]
* Revised first paragraph of VIRTIO_RING_F_INDIRECT_SIZE(40) to be more clear that it is not only about exceeding the queue size
("This feature indicates ... "). [patch 1]
* Drop mentioning "revision 3". [patch 2]
* revised commit log [patch 1]
* Grammar: "... but does not limit the actual bulk data size ...". [patch 1]
* Grammar: "... if the device ...". [patch 1]
* queue_indirect_size field may only be used if VIRTIO_RING_F_INDIRECT_SIZE was used. [patch 2]
* Drop usage of SHOULD/MUST in common description. [patch 2]
* Grammar: had been -> was [patch 2]
* Add requirements for field 'queue_indirect_size' to driver normative section. [patch 2]
Christian Schoenebeck (2):
Add VIRTIO_RING_F_INDIRECT_SIZE
Add common configuration field "queue_indirect_size"
content.tex | 44 ++++++++++++++++++++++++++++++++++++++++++++ split-ring.tex | 5 ++++- 2 files changed, 48 insertions(+), 1 deletion(-)
Signature · 1 line
2.20.1
PATCH 1/2 Add VIRTIO_RING_F_INDIRECT_SIZE
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
--- content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)
content.tex +21 -0
@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} transport specific. For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}. + \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that the + maximum amount of descriptors in indirect descriptor tables is independent + from the Queue Size. + + If this feature bit is not negotiated, then a driver MUST NOT create a + descriptor chain longer than the Queue Size, which then also applies to + indirect descriptor tables as in \ref{sec:Basic Facilities of a Virtio + Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}. Which means without this feature, the Queue Size limits + both the maximum amount of slots in the vring, as well as the actual + bulk data size being transmitted per vring slot. + + With this feature enabled, the Queue Size only limits the maximum amount + of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulk data + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table. + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
split-ring.tex +1 -1
@@ -269,7 +269,7 @@ \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi one table per descriptor). A driver MUST NOT create a descriptor chain longer than the Queue Size of -the device. +the device unless VIRTIO_RING_F_INDIRECT_SIZE has been negotiated. A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in \field{flags}.
Signature · 1 line
2.20.1
[virtio-comment] Re: [PATCH 1/2] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 15 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Signed-off-by Christian Schoenebeck <[email protected]>--- content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)content.tex +0
-0
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6700
Quoted 8 lines
content.tex +5
-0@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} transport specific. For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}. + \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that the + maximum amount of descriptors in indirect descriptor tables is independent + from the Queue Size. + + If this feature bit is not negotiated, then a driver MUST NOT create a
"MUST NOT" does only belong into normative sections... maybe you can reword the last sentence in this paragraph a bit and move the "MUST NOT" part into a normative section?
Quoted 24 lines
content.tex +16
-0+ descriptor chain longer than the Queue Size, which then also applies to + indirect descriptor tables as in \ref{sec:Basic Facilities of a Virtio + Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}. Which means without this feature, the Queue Size limits + both the maximum amount of slots in the vring, as well as the actual + bulk data size being transmitted per vring slot. + + With this feature enabled, the Queue Size only limits the maximum amount + of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulk data + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table. + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}split-ring.tex +0
-0
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 272
Quoted 5 lines
split-ring.tex +1
-1@@ -269,7 +269,7 @@ \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi one table per descriptor). A driver MUST NOT create a descriptor chain longer than the Queue Size of -the device. +the device unless VIRTIO_RING_F_INDIRECT_SIZE has been negotiated.
This might already be enough of a normative statement, though.
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 275
Quoted 3 lines
split-ring.tex +0
-0A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in \field{flags}.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH 1/2] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 23 lines · On Dienstag, 14. Dezember 2021 17:59:51 CET Cornelia Huck wrote:
Quoted 22 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)content.tex +0
-0@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:ReservedFeature Bits}>
transport specific.
For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}.>
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that the
Quoted 8 lines
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6700
Quoted 4 lines
+ maximum amount of descriptors in indirect descriptor tables is independent + from the Queue Size.
content.tex +2
-0+ + If this feature bit is not negotiated, then a driver MUST NOT create a"MUST NOT" does only belong into normative sections... maybe you can reword the last sentence in this paragraph a bit and move the "MUST NOT" part into a normative section?
I lack creativity in this case. Suggestions appreciated. On doubt I would just delete the entire paragraph here ("If this feature ... transmitted per vring slot.").
Quoted 31 lines
Quoted 31 lines
content.tex +8
-0+ descriptor chain longer than the Queue Size, which then also applies to + indirect descriptor tables as in \ref{sec:Basic Facilities of a Virtio + Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}. Which means without this feature, the Queue Size limits + both the maximum amount of slots in the vring, as well as the actual + bulk data size being transmitted per vring slot. + + With this feature enabled, the Queue Size only limits the maximumamount
content.tex +3
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulkdata + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table.
content.tex +1
-0+ \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}split-ring.tex +0
-0@@ -269,7 +269,7 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi>
one table per descriptor).
Quoted 9 lines
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 272
Quoted 4 lines
A driver MUST NOT create a descriptor chain longer than the Queue Size of
split-ring.tex +1
-1-the device. +the device unless VIRTIO_RING_F_INDIRECT_SIZE has been negotiated.This might already be enough of a normative statement, though.
Quoted 2 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin \field{flags}.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] Re: [PATCH 1/2] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 33 lines · Christian Schoenebeck
Quoted 23 lines · On Dienstag, 14. Dezember 2021 17:59:51 CET Cornelia Huck wrote:
Quoted 22 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)content.tex +0
-0@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:ReservedFeature Bits}>
transport specific.
For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}.>
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that theQuoted 8 lines
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6700
Quoted 4 lines
+ maximum amount of descriptors in indirect descriptor tables is independent + from the Queue Size.
content.tex +2
-0+ + If this feature bit is not negotiated, then a driver MUST NOT create a"MUST NOT" does only belong into normative sections... maybe you can reword the last sentence in this paragraph a bit and move the "MUST NOT" part into a normative section?
Quoted 3 lines
I lack creativity in this case. Suggestions appreciated. On doubt I would just delete the entire paragraph here ("If this feature ... transmitted per vring slot.").
Maybe
"Without this feature bit, the queue size limits the length of the descriptor chain, including indirect descriptor tables, i.e. both the maximum amound of slots in the vring and the actual bulk data size transmitted per vring slot."
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
content.tex +8
-0+ descriptor chain longer than the Queue Size, which then also applies to + indirect descriptor tables as in \ref{sec:Basic Facilities of a Virtio + Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}. Which means without this feature, the Queue Size limits + both the maximum amount of slots in the vring, as well as the actual + bulk data size being transmitted per vring slot. + + With this feature enabled, the Queue Size only limits the maximumamount
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
content.tex +3
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulkdata + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table.
I think the second paragraph can stay like this.
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
content.tex +1
-0+ \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}split-ring.tex +0
-0@@ -269,7 +269,7 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi>
one table per descriptor).A driver MUST NOT create a descriptor chain longer than the Queue Size of
Quoted 7 lines
Quoted 7 lines
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 272
Quoted 2 lines
split-ring.tex +1
-1-the device. +the device unless VIRTIO_RING_F_INDIRECT_SIZE has been negotiated.This might already be enough of a normative statement, though.
Quoted 2 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin \field{flags}.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [PATCH 1/2] Add VIRTIO_RING_F_INDIRECT_SIZE
In reply to Christian SchoenebeckQuoted 15 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Signed-off-by Christian Schoenebeck <[email protected]>--- content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)content.tex +0
-0
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6696
Quoted 4 lines
content.tex +1
-0@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} transport specific. For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}. + \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that the
Another feature bit was added in the meantime:
\item[VIRTIO_F_RING_RESET(40)] This feature indicates
^^
Please use 41 and update the text that says:
\item[24 to 40] Feature bits reserved for extensions to the queue and feature negotiation mechanisms
\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
[virtio-comment] Re: [PATCH 1/2] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 24 lines · Stefan Hajnoczi
Quoted 15 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Signed-off-by Christian Schoenebeck <[email protected]>--- content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)content.tex +0
-0content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6696
Quoted 4 lines
content.tex +1
-0@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} transport specific. For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}. + \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that theAnother feature bit was added in the meantime:
\item[VIRTIO_F_RING_RESET(40)] This feature indicates
Quoted 8 lines
^^Please use 41 and update the text that says:
\item[24 to 40] Feature bits reserved for extensions to the queue and feature negotiation mechanisms
\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
Indeed... I'm wondering whether we could automate updating this with some clever variables?
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [PATCH 1/2] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 8 lines · On Montag, 24. Januar 2022 14:14:07 CET Cornelia Huck wrote:
Quoted 7 lines · Stefan Hajnoczi
Quoted 6 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Signed-off-by Christian Schoenebeck <[email protected]>---
Adding Dominique Martinet on CC from now on, for being informed about progress on this virtio issue for Linux kernel side.
Quoted 21 lines
Quoted 21 lines
Quoted 10 lines
content.tex | 21 +++++++++++++++++++++ split-ring.tex | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-)content.tex +0
-0@@ -6693,6 +6693,27 @@ \chapter{Reserved Feature Bits}\label{sec:ReservedQuoted 7 lines
Feature Bits}>>
transport specific.
For more details about driver notifications over PCI see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}.>>
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(40)] This feature indicates that theAnother feature bit was added in the meantime:
\item[VIRTIO_F_RING_RESET(40)] This feature indicates
Quoted 8 lines
Quoted 8 lines
^^Please use 41 and update the text that says:
\item[24 to 40] Feature bits reserved for extensions to the queue and feature negotiation mechanisms\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
Sure, I'll take care of that. As I am always rebasing to latest git version of the spec, I guess I would have realized it anyway.
Quoted 2 lines
Indeed... I'm wondering whether we could automate updating this with some clever variables?
Well, for LaTeX there is
\newcommand{\newCommandName}{1234}and
\def\variable{1234}Both solutions have pros and cons. But not sure I would want to rack my brain on that. I just manually update the feature bit number for now :)
Best regards,
Christian Schoenebeck
PATCH 2/2 Add common configuration field "queue_indirect_size"
In reply to Christian SchoenebeckThis new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
--- content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)
content.tex +24 -1
@@ -859,6 +859,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ + le32 queue_indirect_size; /* read-write */ }; \end{lstlisting}
@@ -938,6 +939,16 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport may benefit from providing another value, for example an internal virtqueue identifier, or an internal offset related to the virtqueue number. \end{note} + +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptors per + vring slot as in \ref{sec:Basic Facilities of a Virtio Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect Descriptors} if + and only if the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated. + + The device specifies its maximum supported number of descriptors per + vring slot. If the driver requires fewer descriptors, it writes its + lower value to inform the device of the reduced resource requirements. \end{description} \devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
@@ -1003,6 +1014,12 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport The driver MUST NOT write a 0 to \field{queue_enable}. +The driver SHOULD write to \field{queue_indirect_size} if its maximum number of +descriptors per vring slot is lower than that reported by the device. + +The driver MUST NOT write a higher value to \field{queue_indirect_size} than the +one it reads from the device. + \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability} The notification location is found using the VIRTIO_PCI_CAP_NOTIFY_CFG
@@ -6712,7 +6729,13 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} being transferred per vring slot, but also avoids complicated synchronization mechanisms if the device only supports a very small amount of vring slots. Due to the 16-bit size of a descriptor's "next" field there is still an absolute - limit of $2^{16}$ descriptors per indirect descriptor table. + limit of $2^{16}$ descriptors per indirect descriptor table. However the + actual maximum amount supported by either device or driver might be less, + and therefore the common configuration field "queue_indirect_size" MUST exist + if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to subsequently + negotiate the actual amount of maximum indirect descriptors supported + by both sides, as described in \ref{sec:Virtio Transport Options / Virtio + Over PCI Bus / PCI Device Layout / Common configuration structure layout}. \end{description}
split-ring.tex +3 -0
@@ -270,6 +270,9 @@ \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi A driver MUST NOT create a descriptor chain longer than the Queue Size of the device unless VIRTIO_RING_F_INDIRECT_SIZE has been negotiated. +Furthermore if VIRTIO_RING_F_INDIRECT_SIZE has been negotiated then the number +of descriptors per vring slot MUST NOT exceed the value negotiated by common +configuration field "queue_indirect_size". A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in \field{flags}.
Signature · 1 line
2.20.1
[virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 15 lines · Christian Schoenebeck
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>--- content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0
content.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 862
Quoted 4 lines
content.tex +1
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ + le32 queue_indirect_size; /* read-write */
[Note for below: this "common" configuration structure layout is actually PCI-specific, it is only common between the different device types.]
Quoted 21 lines
content.tex +10
-0}; \end{lstlisting}@@ -938,6 +939,16 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport may benefit from providing another value, for example an internal virtqueue identifier, or an internal offset related to the virtqueue number. \end{note} + +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptors per + vring slot as in \ref{sec:Basic Facilities of a Virtio Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect Descriptors} if + and only if the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated. + + The device specifies its maximum supported number of descriptors per + vring slot. If the driver requires fewer descriptors, it writes its + lower value to inform the device of the reduced resource requirements. \end{description} \devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
content.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 1018
Quoted 5 lines
content.tex +2
-0@@ -1003,6 +1014,12 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport The driver MUST NOT write a 0 to \field{queue_enable}. +The driver SHOULD write to \field{queue_indirect_size} if its maximum number of +descriptors per vring slot is lower than that reported by the device.
Maybe
"If the driver's maximum number of descriptors per vring slot is lower than the maximum value reported by the device, it SHOULD write that number to \field{queue_indirect_size}."
?
Quoted 27 lines
content.tex +11
-1+ +The driver MUST NOT write a higher value to \field{queue_indirect_size} than the +one it reads from the device. + \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability} The notification location is found using the VIRTIO_PCI_CAP_NOTIFY_CFG@@ -6712,7 +6729,13 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} being transferred per vring slot, but also avoids complicated synchronization mechanisms if the device only supports a very small amount of vring slots. Due to the 16-bit size of a descriptor's "next" field there is still an absolute - limit of $2^{16}$ descriptors per indirect descriptor table. + limit of $2^{16}$ descriptors per indirect descriptor table. However the + actual maximum amount supported by either device or driver might be less, + and therefore the common configuration field "queue_indirect_size" MUST exist + if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to subsequently + negotiate the actual amount of maximum indirect descriptors supported + by both sides, as described in \ref{sec:Virtio Transport Options / Virtio + Over PCI Bus / PCI Device Layout / Common configuration structure layout}. \end{description}split-ring.tex +0
-0
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 275
Quoted 6 lines
split-ring.tex +3
-0@@ -270,6 +270,9 @@ \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi A driver MUST NOT create a descriptor chain longer than the Queue Size of the device unless VIRTIO_RING_F_INDIRECT_SIZE has been negotiated. +Furthermore if VIRTIO_RING_F_INDIRECT_SIZE has been negotiated then the number +of descriptors per vring slot MUST NOT exceed the value negotiated by common +configuration field "queue_indirect_size".
As mentioned above, the "common configuration layout" is actually PCI-specific; other transports will use different mechanism. Maybe it would make sense to reword the whole paragraph and add it in patch 1?
"If VIRTIO_RING_F_INDIRECT_SIZE has not been negotiated, the driver MUST NOT create a descriptor chain longer than the Queue Size of the device.
If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of descriptors per vring slot MUST NOT exceed the negotiated Queue Indirect Size."
I also wonder whether we need a device normative statement as well, something like:
"With VIRTIO_RING_F_INDIRECT_SIZE, the device MUST provide the maximum Queue Indirect Size for the number of descriptors per vring slot. It MUST allow the driver to set a lower value."
Maybe we also need to mention that the mechanism is transport specific?
Also, this is only for split ring; does packed ring need any updates?
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 278
Quoted 3 lines
split-ring.tex +0
-0A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in \field{flags}.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 22 lines · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Quoted 21 lines · Christian Schoenebeck
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */
Quoted 5 lines
Quoted 1 line
+ le32
queue_indirect_size; /* read-write */[Note for below: this "common" configuration structure layout is actually PCI-specific, it is only common between the different device types.]
True
Quoted 36 lines
Quoted 31 lines
};
\end{lstlisting}content.tex +0
-0@@ -938,6 +939,16 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
may benefit from providing another value, for example an internal virtqueue
identifier, or an internal offset related to the virtqueue number.
\end{note}content.tex +3
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptorsper + vring slot as in \ref{sec:Basic Facilities of a Virtio
Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect
Descriptors} if + and only if theVIRTIO_RING_F_INDIRECT_SIZE
feature has been negotiated. +
+ The device specifies its maximum supported number of descriptors
per + vring slot. If the driver requires fewer descriptors, it
writes its + lower value to inform the device of the reduced
resource requirements.>
\end{description}\devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}>
content.tex +0
-0@@ -1003,6 +1014,12 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
The driver MUST NOT write a 0 to \field{queue_enable}.Quoted 3 lines
+The driver SHOULD write to \field{queue_indirect_size} if its maximum number of +descriptors per vring slot is lower than that reported by the device.
Maybe
Quoted 5 lines
"If the driver's maximum number of descriptors per vring slot is lower than the maximum value reported by the device, it SHOULD write that number to \field{queue_indirect_size}."
?
I actually just used Stefan's wording here and extended it with semantically required components, i.e. "vring slot" and "\field{queue_indirect_size}":
https://lists.oasis-open.org/archives/virtio-comment/202112/msg00006.html
For me both versions are fine.
Quoted 47 lines
Quoted 39 lines
content.tex +2
-0+ +The driver MUST NOT write a higher value to \field{queue_indirect_size}than the +one it reads from the device.
content.tex +1
-0+ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability} The notification location is found using the VIRTIO_PCI_CAP_NOTIFY_CFG@@ -6712,7 +6729,13 @@ \chapter{Reserved Feature Bits}\label{sec:ReservedFeature Bits}>
being transferred per vring slot, but also avoids complicated synchronization mechanisms if the device only supports a very small amount of vring slots. Due to the 16-bit size of a descriptor's "next" field there is still an absolute>content.tex +1
-1- limit of $2^{16}$ descriptors per indirect descriptor table. + limit of $2^{16}$ descriptors per indirect descriptor table. Howeverthe
+ actual maximum amount supported by either device or driver might be less, + and therefore the common configuration field "queue_indirect_size" MUST exist + ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequentlycontent.tex +2
-0+ negotiate the actual amount of maximum indirect descriptors supported + by both sides, as described in \ref{sec:Virtio Transport Options /Virtio + Over PCI Bus / PCI Device Layout / Common configuration structure layout}.>
\end{description}split-ring.tex +0
-0@@ -270,6 +270,9 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi>
A driver MUST NOT create a descriptor chain longer than the Queue Size of the device unlessVIRTIO_RING_F_INDIRECT_SIZEhas been negotiated.Quoted 3 lines
+Furthermore if
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated then the number +of descriptors per vring slot MUST NOT exceed the value negotiated by common +configuration field "queue_indirect_size".As mentioned above, the "common configuration layout" is actually PCI-specific; other transports will use different mechanism. Maybe it would make sense to reword the whole paragraph and add it in patch 1?
Quoted 6 lines
"If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUST NOT create a descriptor chain longer than the Queue Size of the device.If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the number of descriptors per vring slot MUST NOT exceed the negotiated Queue Indirect Size."
That simplifies the statements, yes.
About the term "Queue Indirect Size": I understand your point about the field being PCI specific, but for somebody who just reads the virtio spec for the first time, how would you know what "Queue Indirect Size" means?
Quoted 6 lines
I also wonder whether we need a device normative statement as well, something like:
"With
VIRTIO_RING_F_INDIRECT_SIZE, the device MUST provide the maximum Queue Indirect Size for the number of descriptors per vring slot. It MUST allow the driver to set a lower value."
Makes sense.
Quoted 1 line
Maybe we also need to mention that the mechanism is transport specific?
I wouldn't do that. Instead this would probably be extended for other transports in future appropriately.
Quoted 1 line
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so I could not say reliably all the parts that shall be updated for the packed ring. There are some obvious parts like:
2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport- specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue size."
However the question is, would anybody want large descriptor chains with the packaged ring in the first place? If I understand it correctly, the benefits of the packed ring over the split ring only manifest for devices that interchange a very large number of rather small bulk data (e.g. network devices), no?
Quoted 2 lines
Quoted 2 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin \field{flags}.
[virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 71 lines · Christian Schoenebeck
Quoted 22 lines · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Quoted 21 lines · Christian Schoenebeck
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */Quoted 5 lines
Quoted 1 line
+ le32
queue_indirect_size; /* read-write */[Note for below: this "common" configuration structure layout is actually PCI-specific, it is only common between the different device types.]
True
Quoted 40 lines
Quoted 31 lines
};
\end{lstlisting}content.tex +0
-0@@ -938,6 +939,16 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
may benefit from providing another value, for example an internal virtqueue
identifier, or an internal offset related to the virtqueue number.
\end{note}content.tex +3
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptorsper + vring slot as in \ref{sec:Basic Facilities of a Virtio
Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect
Descriptors} if + and only if theVIRTIO_RING_F_INDIRECT_SIZE
feature has been negotiated. +
+ The device specifies its maximum supported number of descriptors
per + vring slot. If the driver requires fewer descriptors, it
writes its + lower value to inform the device of the reduced
resource requirements.>
\end{description}\devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}>
content.tex +0
-0@@ -1003,6 +1014,12 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
The driver MUST NOT write a 0 to \field{queue_enable}.Quoted 3 lines
+The driver SHOULD write to \field{queue_indirect_size} if its maximum number of +descriptors per vring slot is lower than that reported by the device.
Maybe
"If the driver's maximum number of descriptors per vring slot is lower than the maximum value reported by the device, it SHOULD write that number to \field{queue_indirect_size}."
Quoted 7 lines
Quoted 1 line
?
I actually just used Stefan's wording here and extended it with semantically required components, i.e. "vring slot" and "\field{queue_indirect_size}":
https://lists.oasis-open.org/archives/virtio-comment/202112/msg00006.htmlFor me both versions are fine.
No strong opinion here, feel free to keep it like this if you prefer.
Quoted 24 lines
Quoted 17 lines
Quoted 9 lines
split-ring.tex +0
-0@@ -270,6 +270,9 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi>
A driver MUST NOT create a descriptor chain longer than the Queue Size of the device unlessVIRTIO_RING_F_INDIRECT_SIZEhas been negotiated.Quoted 3 lines
+Furthermore if
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated then the number +of descriptors per vring slot MUST NOT exceed the value negotiated by common +configuration field "queue_indirect_size".As mentioned above, the "common configuration layout" is actually PCI-specific; other transports will use different mechanism. Maybe it would make sense to reword the whole paragraph and add it in patch 1?
Quoted 6 lines
"If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUST NOT create a descriptor chain longer than the Queue Size of the device.If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the number of descriptors per vring slot MUST NOT exceed the negotiated Queue Indirect Size."
Quoted 5 lines
That simplifies the statements, yes.
About the term "Queue Indirect Size": I understand your point about the field being PCI specific, but for somebody who just reads the virtio spec for the first time, how would you know what "Queue Indirect Size" means?
If we follow my suggestion below, it is described in the device normative statement. (Not sure if we actually do the same for Queue Size already? That's a similar case.)
Quoted 8 lines
Quoted 7 lines
I also wonder whether we need a device normative statement as well, something like:
"With
VIRTIO_RING_F_INDIRECT_SIZE, the device MUST provide the maximum Queue Indirect Size for the number of descriptors per vring slot. It MUST allow the driver to set a lower value."
Quoted 6 lines
Makes sense.
Quoted 1 line
Maybe we also need to mention that the mechanism is transport specific?
I wouldn't do that. Instead this would probably be extended for other transports in future appropriately.
The mechanics, not providing the value... something like
"The maximum Queue Indirect Size is provided by the device in a transport-specific way."
(There should already be some precedence for other values.)
Quoted 13 lines
Quoted 2 lines
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so I could not say reliably all the parts that shall be updated for the packed ring. There are some obvious parts like:
2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport- specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue size."
Quoted 5 lines
However the question is, would anybody want large descriptor chains with the packaged ring in the first place? If I understand it correctly, the benefits of the packed ring over the split ring only manifest for devices that interchange a very large number of rather small bulk data (e.g. network devices), no?
If we think that the feature does not make sense for packed ring, they should probably conflict with each other. Otherwise, I think we need at least a statement that the higher limit does not take effect for packed ring, or touch all the places where it would be relevant.
What do others think?
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 3 lines · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
On Tue, Dec 14 2021, Christian Schoenebeck <[email protected]>
Quoted 169 lines · wrote:
Quoted 80 lines
Quoted 22 lines
Quoted 22 lines
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */Quoted 5 lines
Quoted 1 line
+ le32
queue_indirect_size; /* read-write */[Note for below: this "common" configuration structure layout is actually PCI-specific, it is only common between the different device types.]
True
Quoted 50 lines
Quoted 37 lines
};
\end{lstlisting}content.tex +0
-0@@ -938,6 +939,16 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
may benefit from providing another value, for example an internal
virtqueue
identifier, or an internal offset related to the virtqueue number.
\end{note}content.tex +3
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount ofdescriptors
per + vring slot as in \ref{sec:Basic Facilities of a Virtio
Device / + Virtqueues / The Virtqueue Descriptor Table /
Indirect
Descriptors} if + and only if theVIRTIO_RING_F_INDIRECT_SIZE
feature has been negotiated. +
+ The device specifies its maximum supported number of
descriptors
per + vring slot. If the driver requires fewer descriptors, it
writes its + lower value to inform the device of the reduced
resource requirements.>\end{description}
\devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}>
content.tex +0
-0@@ -1003,6 +1014,12 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
Quoted 6 lines
The driver MUST NOT write a 0 to \field{queue_enable}.
+The driver SHOULD write to \field{queue_indirect_size} if its maximum number of +descriptors per vring slot is lower than that reported by the
device.Maybe
"If the driver's maximum number of descriptors per vring slot is lower than the maximum value reported by the device, it SHOULD write that number to \field{queue_indirect_size}."
Quoted 8 lines
Quoted 1 line
?
I actually just used Stefan's wording here and extended it with semantically required components, i.e. "vring slot" and "\field{queue_indirect_size}":
https://lists.oasis-open.org/archives/virtio-comment/202112/msg00006.htmlFor me both versions are fine.
No strong opinion here, feel free to keep it like this if you prefer.
Quoted 28 lines
Quoted 21 lines
Quoted 8 lines
split-ring.tex +0
-0@@ -270,6 +270,9 @@ \subsubsection{IndirectDescriptors}\label{sec:Basic
Facilities of a Virtio Devi>Quoted 8 lines
A driver MUST NOT create a descriptor chain longer than the Queue Size of
the device unlessVIRTIO_RING_F_INDIRECT_SIZEhas been negotiated.+Furthermore if
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated then the
number +of descriptors per vring slot MUST NOT exceed the value negotiated by common +configuration field "queue_indirect_size".As mentioned above, the "common configuration layout" is actually PCI-specific; other transports will use different mechanism. Maybe it would make sense to reword the whole paragraph and add it in patch 1?
Quoted 6 lines
"If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUST NOT create a descriptor chain longer than the Queue Size of the device.If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the number of descriptors per vring slot MUST NOT exceed the negotiated Queue Indirect Size."Quoted 5 lines
That simplifies the statements, yes.
About the term "Queue Indirect Size": I understand your point about the field being PCI specific, but for somebody who just reads the virtio spec for the first time, how would you know what "Queue Indirect Size" means?
If we follow my suggestion below, it is described in the device normative statement. (Not sure if we actually do the same for Queue Size already? That's a similar case.)
Quoted 7 lines
Quoted 6 lines
I also wonder whether we need a device normative statement as well, something like:
"With
VIRTIO_RING_F_INDIRECT_SIZE, the device MUST provide the maximum Queue Indirect Size for the number of descriptors per vring slot. It MUST allow the driver to set a lower value."Quoted 6 lines
Makes sense.
Quoted 1 line
Maybe we also need to mention that the mechanism is transport specific?
I wouldn't do that. Instead this would probably be extended for other transports in future appropriately.
The mechanics, not providing the value... something like
"The maximum Queue Indirect Size is provided by the device in a transport-specific way."
(There should already be some precedence for other values.)
Quoted 13 lines
Quoted 1 line
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so I could not say reliably all the parts that shall be updated for the packed ring. There are some obvious parts like:
2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport-
specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue size."Quoted 5 lines
However the question is, would anybody want large descriptor chains with the packaged ring in the first place? If I understand it correctly, the benefits of the packed ring over the split ring only manifest for devices that interchange a very large number of rather small bulk data (e.g. network devices), no?
Quoted 6 lines
If we think that the feature does not make sense for packed ring, they should probably conflict with each other. Otherwise, I think we need at least a statement that the higher limit does not take effect for packed ring, or touch all the places where it would be relevant.
What do others think?
It would indeed be very useful if other people express their opinion about this issue (packed ring scenario) as well before I continue on this issue.
Probably the fact that my patches never made it through to the list were not necessarily supporting this. Should I contact somebody regarding this ML issue? Do members of the other ML also read this virtio-comment list?
I tried to compensate the current situation by updating the corresponding issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122
If nobody replies early January, I would suggest to continue by ignoring the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things, because this feature is negotiated per queue, not for the entire device.
Best regards,
Christian Schoenebeck
[virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 27 lines · Christian Schoenebeck
Quoted 26 lines · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
Quoted 14 lines · Christian Schoenebeck
Quoted 1 line · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so I could not say reliably all the parts that shall be updated for the packed ring. There are some obvious parts like:
2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport-
specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue size."Quoted 5 lines
However the question is, would anybody want large descriptor chains with the packaged ring in the first place? If I understand it correctly, the benefits of the packed ring over the split ring only manifest for devices that interchange a very large number of rather small bulk data (e.g. network devices), no?
If we think that the feature does not make sense for packed ring, they should probably conflict with each other. Otherwise, I think we need at least a statement that the higher limit does not take effect for packed ring, or touch all the places where it would be relevant.
Quoted 8 lines
Quoted 1 line
What do others think?
It would indeed be very useful if other people express their opinion about this issue (packed ring scenario) as well before I continue on this issue.
Probably the fact that my patches never made it through to the list were not necessarily supporting this. Should I contact somebody regarding this ML issue? Do members of the other ML also read this virtio-comment list?
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
Quoted 4 lines
I tried to compensate the current situation by updating the corresponding issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122
Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 5 lines
If nobody replies early January, I would suggest to continue by ignoring the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things, because this feature is negotiated per queue, not for the entire device.
The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we think it is useful
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 3 lines · On Montag, 3. Januar 2022 14:21:13 CET Cornelia Huck wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
On Wed, Dec 15 2021, Christian Schoenebeck <[email protected]>
Quoted 32 lines · wrote:
Quoted 32 lines
Quoted 31 lines
Quoted 17 lines · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Quoted 1 line
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so I
could not say reliably all the parts that shall be updated for the packed
ring. There are some obvious parts like:2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport-
specific and/or device-specific value. If not limited, the maximum number
of descriptors in a list is the virt queue size."Quoted 7 lines
However the question is, would anybody want large descriptor chains with
the packaged ring in the first place? If I understand it correctly, the benefits of the packed ring over the split ring only manifest for devices
that interchange a very large number of rather small bulk data (e.g. network devices), no?If we think that the feature does not make sense for packed ring, they should probably conflict with each other. Otherwise, I think we need at least a statement that the higher limit does not take effect for packed ring, or touch all the places where it would be relevant.
What do others think?
Quoted 8 lines
Quoted 6 lines
It would indeed be very useful if other people express their opinion about this issue (packed ring scenario) as well before I continue on this issue.
Probably the fact that my patches never made it through to the list were not necessarily supporting this. Should I contact somebody regarding this ML issue? Do members of the other ML also read this virtio-comment list?
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Only my emails with patches are refused by the list. All my other emails are accepted. So not sure if the cause is really DKIM or something else. Maybe the admins can suggest a workaround for me.
Quoted 3 lines
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
What I thought was wether subscribers of virtio-dev would typically read virtio-comment as well. Because AFAICS people who more frequently deal with virtio for their companies rather seem to post to virtio-dev.
Quoted 16 lines
Quoted 3 lines
I tried to compensate the current situation by updating the corresponding issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 4 lines
If nobody replies early January, I would suggest to continue by ignoring the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things, because this feature is negotiated per queue, not for the entire device.
Quoted 6 lines
The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we think it is useful
Mja, I have to correct myself on that: I wrote in my previous email that this was negotiated per queue. That's false of course as all virtio features are currently negotiated for the entire device.
So you are right, if this new indirect size feature was negotiated then it would apply to both a) split rings and b) packed rings a device might have. Which is unfortunate.
Stefan, you are aware about this circumstance as well, right? Because I remember we originally had a discussion on qemu-devel where you wanted to have this configurable per queue, not per device.
Best regards,
Christian Schoenebeck
[virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 85 lines · Christian Schoenebeck
Quoted 3 lines · On Montag, 3. Januar 2022 14:21:13 CET Cornelia Huck wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
On Wed, Dec 15 2021, Christian Schoenebeck <[email protected]>
Quoted 32 lines · wrote:
Quoted 32 lines
Quoted 31 lines
Quoted 17 lines · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Quoted 1 line
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so I
could not say reliably all the parts that shall be updated for the packed
ring. There are some obvious parts like:2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport-
specific and/or device-specific value. If not limited, the maximum number
of descriptors in a list is the virt queue size."Quoted 7 lines
However the question is, would anybody want large descriptor chains with
the packaged ring in the first place? If I understand it correctly, the benefits of the packed ring over the split ring only manifest for devices
that interchange a very large number of rather small bulk data (e.g. network devices), no?If we think that the feature does not make sense for packed ring, they should probably conflict with each other. Otherwise, I think we need at least a statement that the higher limit does not take effect for packed ring, or touch all the places where it would be relevant.
What do others think?
Quoted 8 lines
Quoted 6 lines
It would indeed be very useful if other people express their opinion about this issue (packed ring scenario) as well before I continue on this issue.
Probably the fact that my patches never made it through to the list were not necessarily supporting this. Should I contact somebody regarding this ML issue? Do members of the other ML also read this virtio-comment list?
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Only my emails with patches are refused by the list. All my other emails are accepted. So not sure if the cause is really DKIM or something else. Maybe the admins can suggest a workaround for me.
Quoted 3 lines
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
What I thought was wether subscribers of virtio-dev would typically read virtio-comment as well. Because AFAICS people who more frequently deal with virtio for their companies rather seem to post to virtio-dev.
Quoted 16 lines
Quoted 3 lines
I tried to compensate the current situation by updating the corresponding issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 4 lines
If nobody replies early January, I would suggest to continue by ignoring the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things, because this feature is negotiated per queue, not for the entire device.
Quoted 6 lines
The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we think it is usefulMja, I have to correct myself on that: I wrote in my previous email that this was negotiated per queue. That's false of course as all virtio features are currently negotiated for the entire device.
Quoted 7 lines
So you are right, if this new indirect size feature was negotiated then it would apply to both a) split rings and b) packed rings a device might have. Which is unfortunate.
Stefan, you are aware about this circumstance as well, right? Because I remember we originally had a discussion on qemu-devel where you wanted to have this configurable per queue, not per device.
Regarding packed virtqueues, there are multiple reasons for using them. Recently someone asked about reducing complexity in VIRTIO implementations and one of the suggestions that Michael Tsirkin and I both made independently was to use the packed layout instead of split layout. I don't think we should assume packed virtqueues are only used in scenarios that don't need INDIRECT_SIZE.
Extending INDIRECT_SIZE to packed virtqueues looks straightforward and does not require many changes to the Packed Virtqueues section.
The reason I pushed for per-virtqueue Indirect Size values is because multi-virtqueue devices have specific purposes for each virtqueue. The Queue Size field is per-virtqueue already across all transports. This way a device with control virtqueue can allocate fewer resources (smaller Queue Size and Indirect Size) to it than to the data path virtqueues that carry I/O data buffers.
I don't think it's necessary to support INDIRECT_SIZE on only a subset of a device's virtqueues. If the device doesn't want to "enable" INDIRECT_SIZE on a specific virtqueue it should just report Queue Size as the Indirect Size value?
Stefan
Re: [virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 3 lines · On Montag, 24. Januar 2022 14:39:28 CET Stefan Hajnoczi wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Montag, 3. Januar 2022 14:21:13 CET Cornelia Huck wrote:
On Wed, Dec 29 2021, Christian Schoenebeck <[email protected]>
Quoted 118 lines · wrote:
Quoted 102 lines
Quoted 2 lines
Quoted 2 lines · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
Quoted 1 line
On Wed, Dec 15 2021, Christian Schoenebeck <[email protected]>
Quoted 51 lines · wrote:
Quoted 44 lines
Quoted 37 lines
Quoted 19 lines · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Quoted 2 lines
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so
I
could not say reliably all the parts that shall be updated for the packed
ring. There are some obvious parts like:2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport-
specific and/or device-specific value. If not limited, the maximum number
of descriptors in a list is the virt queue size."Quoted 9 lines
However the question is, would anybody want large descriptor chains with
the packaged ring in the first place? If I understand it correctly, the
benefits of the packed ring over the split ring only manifest for devices
that interchange a very large number of rather small bulk data (e.g.
network devices), no?If we think that the feature does not make sense for packed ring, they
should probably conflict with each other. Otherwise, I think we need at
least a statement that the higher limit does not take effect for packed
ring, or touch all the places where it would be relevant.Quoted 1 line
What do others think?
It would indeed be very useful if other people express their opinion about
this issue (packed ring scenario) as well before I continue on this issue.Quoted 6 lines
Probably the fact that my patches never made it through to the list were
not necessarily supporting this. Should I contact somebody regarding this
ML issue? Do members of the other ML also read this virtio-comment list?Quoted 2 lines
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Only my emails with patches are refused by the list. All my other emails are accepted. So not sure if the cause is really DKIM or something else. Maybe the admins can suggest a workaround for me.
Quoted 3 lines
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
What I thought was wether subscribers of virtio-dev would typically read virtio-comment as well. Because AFAICS people who more frequently deal with
virtio for their companies rather seem to post to virtio-dev.Quoted 19 lines
Quoted 4 lines
I tried to compensate the current situation by updating the corresponding
issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 6 lines
If nobody replies early January, I would suggest to continue by ignoring
the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things, because this feature is negotiated per queue, not for the entire device.Quoted 7 lines
The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we thinkit is useful
Mja, I have to correct myself on that: I wrote in my previous email that this was negotiated per queue. That's false of course as all virtio features are currently negotiated for the entire device.
Quoted 8 lines
So you are right, if this new indirect size feature was negotiated then it would apply to both a) split rings and b) packed rings a device might have.
Which is unfortunate.Stefan, you are aware about this circumstance as well, right? Because I remember we originally had a discussion on qemu-devel where you wanted to have this configurable per queue, not per device.
Regarding packed virtqueues, there are multiple reasons for using them. Recently someone asked about reducing complexity in VIRTIO implementations and one of the suggestions that Michael Tsirkin and I both made independently was to use the packed layout instead of split layout. I don't think we should assume packed virtqueues are only used in scenarios that don't need
INDIRECT_SIZE.
Quoted 2 lines
Extending
INDIRECT_SIZEto packed virtqueues looks straightforward and does not require many changes to the Packed Virtqueues section.
Yes I agree, it does make sense to apply this new INDIRECT_SIZE feature to packed queues as well.
Quoted 11 lines
The reason I pushed for per-virtqueue Indirect Size values is because multi-virtqueue devices have specific purposes for each virtqueue. The Queue Size field is per-virtqueue already across all transports. This way a device with control virtqueue can allocate fewer resources (smaller Queue Size and Indirect Size) to it than to the data path virtqueues that carry I/O data buffers.
I don't think it's necessary to support
INDIRECT_SIZEon only a subset of a device's virtqueues. If the device doesn't want to "enable"INDIRECT_SIZEon a specific virtqueue it should just report Queue Size as the Indirect Size value?
Or another idea: what about adding this new indirect size field to the struct that holds the queue size field already, then this would become transport- independent (i.e. the discussed PCI config field would be unnecessary), and it would be a feature configurable per queue. Or is that struct inappropriate for feature negotiation purposes?
Best regards,
Christian Schoenebeck
Re: [virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 141 lines · Christian Schoenebeck
Quoted 3 lines · On Montag, 24. Januar 2022 14:39:28 CET Stefan Hajnoczi wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Montag, 3. Januar 2022 14:21:13 CET Cornelia Huck wrote:
On Wed, Dec 29 2021, Christian Schoenebeck <[email protected]>
Quoted 118 lines · wrote:
Quoted 102 lines
Quoted 2 lines
Quoted 2 lines · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
Quoted 1 line
On Wed, Dec 15 2021, Christian Schoenebeck <[email protected]>
Quoted 51 lines · wrote:
Quoted 44 lines
Quoted 37 lines
Quoted 19 lines · On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck wrote:
Quoted 2 lines
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so
I
could not say reliably all the parts that shall be updated for the packed
ring. There are some obvious parts like:2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a transport-
specific and/or device-specific value. If not limited, the maximum number
of descriptors in a list is the virt queue size."Quoted 9 lines
However the question is, would anybody want large descriptor chains with
the packaged ring in the first place? If I understand it correctly, the
benefits of the packed ring over the split ring only manifest for devices
that interchange a very large number of rather small bulk data (e.g.
network devices), no?If we think that the feature does not make sense for packed ring, they
should probably conflict with each other. Otherwise, I think we need at
least a statement that the higher limit does not take effect for packed
ring, or touch all the places where it would be relevant.Quoted 1 line
What do others think?
It would indeed be very useful if other people express their opinion about
this issue (packed ring scenario) as well before I continue on this issue.Quoted 6 lines
Probably the fact that my patches never made it through to the list were
not necessarily supporting this. Should I contact somebody regarding this
ML issue? Do members of the other ML also read this virtio-comment list?Quoted 2 lines
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Only my emails with patches are refused by the list. All my other emails are accepted. So not sure if the cause is really DKIM or something else. Maybe the admins can suggest a workaround for me.
Quoted 3 lines
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
What I thought was wether subscribers of virtio-dev would typically read virtio-comment as well. Because AFAICS people who more frequently deal with
virtio for their companies rather seem to post to virtio-dev.Quoted 19 lines
Quoted 4 lines
I tried to compensate the current situation by updating the corresponding
issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 6 lines
If nobody replies early January, I would suggest to continue by ignoring
the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things, because this feature is negotiated per queue, not for the entire device.Quoted 7 lines
The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we thinkit is useful
Mja, I have to correct myself on that: I wrote in my previous email that this was negotiated per queue. That's false of course as all virtio features are currently negotiated for the entire device.
Quoted 8 lines
So you are right, if this new indirect size feature was negotiated then it would apply to both a) split rings and b) packed rings a device might have.
Which is unfortunate.Stefan, you are aware about this circumstance as well, right? Because I remember we originally had a discussion on qemu-devel where you wanted to have this configurable per queue, not per device.
Regarding packed virtqueues, there are multiple reasons for using them. Recently someone asked about reducing complexity in VIRTIO implementations and one of the suggestions that Michael Tsirkin and I both made independently was to use the packed layout instead of split layout. I don't think we should assume packed virtqueues are only used in scenarios that don't need
INDIRECT_SIZE.Quoted 2 lines
Extending
INDIRECT_SIZEto packed virtqueues looks straightforward and does not require many changes to the Packed Virtqueues section.Yes I agree, it does make sense to apply this new
INDIRECT_SIZEfeature to packed queues as well.Quoted 11 lines
The reason I pushed for per-virtqueue Indirect Size values is because multi-virtqueue devices have specific purposes for each virtqueue. The Queue Size field is per-virtqueue already across all transports. This way a device with control virtqueue can allocate fewer resources (smaller Queue Size and Indirect Size) to it than to the data path virtqueues that carry I/O data buffers.
I don't think it's necessary to support
INDIRECT_SIZEon only a subset of a device's virtqueues. If the device doesn't want to "enable"INDIRECT_SIZEon a specific virtqueue it should just report Queue Size as the Indirect Size value?
Quoted 5 lines
Or another idea: what about adding this new indirect size field to the struct that holds the queue size field already, then this would become transport- independent (i.e. the discussed PCI config field would be unnecessary), and it would be a feature configurable per queue. Or is that struct inappropriate for feature negotiation purposes?
There is no transport-independent struct that contains Queue Size. It is handled by struct virtio_pci_common_cfg->queue_size, the MMIO QueueNum register, and the CCW struct vq_config_block->max_num and struct vq_info_block->num fields.
Stefan
Re: [virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 13 lines · On Montag, 24. Januar 2022 17:41:09 CET Stefan Hajnoczi wrote:
Quoted 12 lines · Christian Schoenebeck
Quoted 3 lines · On Montag, 24. Januar 2022 14:39:28 CET Stefan Hajnoczi wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Montag, 3. Januar 2022 14:21:13 CET Cornelia Huck wrote:
On Wed, Dec 29 2021, Christian Schoenebeck <[email protected]>
Quoted 6 lines · wrote:
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
Quoted 2 lines
On Wed, Dec 15 2021, Christian Schoenebeck
<[email protected]>Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck
Quoted 158 lines · wrote:
Quoted 152 lines
Quoted 134 lines
Quoted 121 lines
Quoted 61 lines
Quoted 52 lines
Quoted 43 lines
Quoted 21 lines
Quoted 2 lines
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so
I
could not say reliably all the parts that shall be updated for the
packed
ring. There are some obvious parts like:2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a
transport-
specific and/or device-specific value. If not limited, the maximum
number
of descriptors in a list is the virt queue size."Quoted 12 lines
However the question is, would anybody want large descriptor chains
with
the packaged ring in the first place? If I understand it correctly,
the
benefits of the packed ring over the split ring only manifest for
devices
that interchange a very large number of rather small bulk data (e.g.
network devices), no?If we think that the feature does not make sense for packed ring, they
should probably conflict with each other. Otherwise, I think we need
at
least a statement that the higher limit does not take effect for packed
ring, or touch all the places where it would be relevant.Quoted 1 line
What do others think?
It would indeed be very useful if other people express their opinion
about
this issue (packed ring scenario) as well before I continue on this
issue.Quoted 8 lines
Probably the fact that my patches never made it through to the list
were
not necessarily supporting this. Should I contact somebody regarding
this
ML issue? Do members of the other ML also read this virtio-comment list?Quoted 2 lines
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Only my emails with patches are refused by the list. All my other emails
are accepted. So not sure if the cause is really DKIM or something else.
Maybe the admins can suggest a workaround for me.Quoted 3 lines
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
What I thought was wether subscribers of virtio-dev would typically read
virtio-comment as well. Because AFAICS people who more frequently deal with
virtio for their companies rather seem to post to virtio-dev.Quoted 28 lines
Quoted 4 lines
I tried to compensate the current situation by updating the corresponding
issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 7 lines
If nobody replies early January, I would suggest to continue by ignoring
the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things,
because this feature is negotiated per queue, not for the entire device.The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add
statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we thinkQuoted 1 line
it is useful
Mja, I have to correct myself on that: I wrote in my previous email that
this was negotiated per queue. That's false of course as all virtio features are currently negotiated for the entire device.So you are right, if this new indirect size feature was negotiated then it
would apply to both a) split rings and b) packed rings a device might have.
Which is unfortunate.Quoted 5 lines
Stefan, you are aware about this circumstance as well, right? Because I
remember we originally had a discussion on qemu-devel where you wanted to
have this configurable per queue, not per device.Regarding packed virtqueues, there are multiple reasons for using them. Recently someone asked about reducing complexity in VIRTIO implementations and one of the suggestions that Michael Tsirkin and I both made independently was to use the packed layout instead of split layout. I don't think we should assume packed virtqueues are only used in scenarios that don't need
INDIRECT_SIZE.Quoted 2 lines
Extending
INDIRECT_SIZEto packed virtqueues looks straightforward and does not require many changes to the Packed Virtqueues section.Yes I agree, it does make sense to apply this new
INDIRECT_SIZEfeature to packed queues as well.Quoted 11 lines
The reason I pushed for per-virtqueue Indirect Size values is because multi-virtqueue devices have specific purposes for each virtqueue. The Queue Size field is per-virtqueue already across all transports. This way a device with control virtqueue can allocate fewer resources (smaller Queue Size and Indirect Size) to it than to the data path virtqueues that carry I/O data buffers.
I don't think it's necessary to support
INDIRECT_SIZEon only a subset of a device's virtqueues. If the device doesn't want to "enable"INDIRECT_SIZEon a specific virtqueue it should just report Queue Size as the Indirect Size value?Quoted 5 lines
Or another idea: what about adding this new indirect size field to the struct that holds the queue size field already, then this would become transport- independent (i.e. the discussed PCI config field would be unnecessary), and it would be a feature configurable per queue. Or is that struct inappropriate for feature negotiation purposes?
Quoted 4 lines
There is no transport-independent struct that contains Queue Size. It is handled by struct
virtio_pci_common_cfg->queue_size, the MMIO QueueNum register, and the CCW structvq_config_block->max_numand structvq_info_block->numfields.
Aaah, I just realized I was completely confused about
struct virtio_pci_common_cfg: so far I assumed this were all entirely device global settings.
The respective queue is first selected via 'queue_select' field and then the subsequent fields reflect the selected queue. Now I got it.
Ok fine, then the current draft actually already negotiated the indirect size for each queue independently already, without me even being aware. ;-)
I guess I know all I need then for the next round of patches. Maybe I also come along for changes for MMIO and CCW side, we'll see.
Thanks!
Best regards,
Christian Schoenebeck
Re: [virtio-comment] Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 182 lines · Christian Schoenebeck
Quoted 13 lines · On Montag, 24. Januar 2022 17:41:09 CET Stefan Hajnoczi wrote:
Quoted 12 lines · Christian Schoenebeck
Quoted 3 lines · On Montag, 24. Januar 2022 14:39:28 CET Stefan Hajnoczi wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Montag, 3. Januar 2022 14:21:13 CET Cornelia Huck wrote:
On Wed, Dec 29 2021, Christian Schoenebeck <[email protected]>
Quoted 6 lines · wrote:
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines · On Donnerstag, 23. Dezember 2021 12:03:50 CET Cornelia Huck wrote:
Quoted 2 lines
On Wed, Dec 15 2021, Christian Schoenebeck
<[email protected]>Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Dienstag, 14. Dezember 2021 18:20:28 CET Cornelia Huck
Quoted 158 lines · wrote:
Quoted 152 lines
Quoted 134 lines
Quoted 121 lines
Quoted 61 lines
Quoted 52 lines
Quoted 43 lines
Quoted 21 lines
Quoted 2 lines
Also, this is only for split ring; does packed ring need any updates?
I have not reviewed the packed ring as much as I did the split ring, so
I
could not say reliably all the parts that shall be updated for the
packed
ring. There are some obvious parts like:2.7.5 Scatter-Gather Support
"The device limits the number of descriptors in a list through a
transport-
specific and/or device-specific value. If not limited, the maximum
number
of descriptors in a list is the virt queue size."Quoted 12 lines
However the question is, would anybody want large descriptor chains
with
the packaged ring in the first place? If I understand it correctly,
the
benefits of the packed ring over the split ring only manifest for
devices
that interchange a very large number of rather small bulk data (e.g.
network devices), no?If we think that the feature does not make sense for packed ring, they
should probably conflict with each other. Otherwise, I think we need
at
least a statement that the higher limit does not take effect for packed
ring, or touch all the places where it would be relevant.Quoted 1 line
What do others think?
It would indeed be very useful if other people express their opinion
about
this issue (packed ring scenario) as well before I continue on this
issue.Quoted 8 lines
Probably the fact that my patches never made it through to the list
were
not necessarily supporting this. Should I contact somebody regarding
this
ML issue? Do members of the other ML also read this virtio-comment list?Quoted 2 lines
Yes, this situation is very unsatisfactory :( (I have contacted the people running this list, but there have not yet been any fixes...)
Only my emails with patches are refused by the list. All my other emails
are accepted. So not sure if the cause is really DKIM or something else.
Maybe the admins can suggest a workaround for me.Quoted 3 lines
Not sure which other lists would be appropriate to cc: -- maybe [email protected], but that one also suffers from DKIM issues :(
What I thought was wether subscribers of virtio-dev would typically read
virtio-comment as well. Because AFAICS people who more frequently deal with
virtio for their companies rather seem to post to virtio-dev.Quoted 28 lines
Quoted 4 lines
I tried to compensate the current situation by updating the corresponding
issue description on Github in a very defailed and verbose way:
https://github.com/oasis-tcs/virtio-spec/issues/122Thanks. Hopefully me quoting this makes it more visible (I tried to quote more than I usually would in my other replies already...)
Just to feature it more prominently for people who collapse quotes:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 7 lines
If nobody replies early January, I would suggest to continue by ignoring
the packed ring. Because if somebody wants this for packed ring in future, this can still be added to the specs without breaking things,
because this feature is negotiated per queue, not for the entire device.The problem is that we need to specify what is supposed to happen if packed ring *and* this feature are negotiated. If we do not want to add
statements for the packed ring case, my suggestion would be
- make packed ring and this feature mutually exclusive
- add a new feature bit that works with packed ring later, if we thinkQuoted 1 line
it is useful
Mja, I have to correct myself on that: I wrote in my previous email that
this was negotiated per queue. That's false of course as all virtio features are currently negotiated for the entire device.So you are right, if this new indirect size feature was negotiated then it
would apply to both a) split rings and b) packed rings a device might have.
Which is unfortunate.Quoted 5 lines
Stefan, you are aware about this circumstance as well, right? Because I
remember we originally had a discussion on qemu-devel where you wanted to
have this configurable per queue, not per device.Regarding packed virtqueues, there are multiple reasons for using them. Recently someone asked about reducing complexity in VIRTIO implementations and one of the suggestions that Michael Tsirkin and I both made independently was to use the packed layout instead of split layout. I don't think we should assume packed virtqueues are only used in scenarios that don't need
INDIRECT_SIZE.Quoted 2 lines
Extending
INDIRECT_SIZEto packed virtqueues looks straightforward and does not require many changes to the Packed Virtqueues section.Yes I agree, it does make sense to apply this new
INDIRECT_SIZEfeature to packed queues as well.Quoted 11 lines
The reason I pushed for per-virtqueue Indirect Size values is because multi-virtqueue devices have specific purposes for each virtqueue. The Queue Size field is per-virtqueue already across all transports. This way a device with control virtqueue can allocate fewer resources (smaller Queue Size and Indirect Size) to it than to the data path virtqueues that carry I/O data buffers.
I don't think it's necessary to support
INDIRECT_SIZEon only a subset of a device's virtqueues. If the device doesn't want to "enable"INDIRECT_SIZEon a specific virtqueue it should just report Queue Size as the Indirect Size value?Quoted 5 lines
Or another idea: what about adding this new indirect size field to the struct that holds the queue size field already, then this would become transport- independent (i.e. the discussed PCI config field would be unnecessary), and it would be a feature configurable per queue. Or is that struct inappropriate for feature negotiation purposes?
Quoted 4 lines
There is no transport-independent struct that contains Queue Size. It is handled by struct
virtio_pci_common_cfg->queue_size, the MMIO QueueNum register, and the CCW structvq_config_block->max_numand structvq_info_block->numfields.Aaah, I just realized I was completely confused about
structvirtio_pci_common_cfg: so far I assumed this were all entirely device global settings.
Quoted 8 lines
The respective queue is first selected via '
queue_select' field and then the subsequent fields reflect the selected queue. Now I got it.Ok fine, then the current draft actually already negotiated the indirect size for each queue independently already, without me even being aware. ;-)
I guess I know all I need then for the next round of patches. Maybe I also come along for changes for MMIO and CCW side, we'll see.
Awesome!
Stefan
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
In reply to Christian SchoenebeckQuoted 15 lines · Christian Schoenebeck
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>--- content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0
content.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 862
Quoted 4 lines
content.tex +1
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ + le32 queue_indirect_size; /* read-write */
I see no other unaligned fields in this struct, so I think a le16 padding field is needed for safety:
le64 queue_device; /* read-write */
le16 queue_notify_data; /* read-only for driver */
+ le16 reserved; /* no access */
+ le32 queue_indirect_size; /* read-write */Quoted 7 lines
content.tex +0
-0}; \end{lstlisting}@@ -938,6 +939,16 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport may benefit from providing another value, for example an internal virtqueue identifier, or an internal offset related to the virtqueue number. \end{note}
content.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 951
Quoted 10 lines
content.tex +10
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptors per + vring slot as in \ref{sec:Basic Facilities of a Virtio Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect Descriptors} if + and only if the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated. + + The device specifies its maximum supported number of descriptors per + vring slot. If the driver requires fewer descriptors, it writes its + lower value to inform the device of the reduced resource requirements.
"vring slot" is a little vague, it means "indirect descriptor table"? The two paragraphs could use "maximum number of descriptors per indirect descriptor table" instead of referring to vring slots to imply indirect descriptor tables.
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 8 lines · On Montag, 24. Januar 2022 14:53:45 CET Stefan Hajnoczi wrote:
Quoted 7 lines · Christian Schoenebeck
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>---
Quoted 5 lines
content.tex
Quoted 5 lines
content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0
Stefan, while preparing the next version of patches for this issue (#122), I noticed that I missed your email here ...
Quoted 14 lines
Quoted 10 lines
index 0aa4842..e3cfcae 100644
content.tex +0
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */+ le32
queue_indirect_size; /* read-write */I see no other unaligned fields in this struct, so I think a le16 padding field is needed for safety:
Quoted 5 lines
le64 queue_device; /* read-write */
le16 queue_notify_data; /* read-only for driver */
+ le16 reserved; /* no access */
+ le32 queue_indirect_size; /* read-write */That's no longer needed as there's now the new 'queue_reset' field, i.e.:
content.tex +1 -0
@@ -902,6 +902,7 @@ \subsubsection{Common configuration structure layout} le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ le16 queue_reset; /* read-write */ + le32 queue_indirect_size; /* read-write */ }; \end{lstlisting}
Quoted 11 lines
Quoted 11 lines
};
\end{lstlisting}content.tex +0
-0@@ -938,6 +939,16 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
may benefit from providing another value, for example an internal virtqueue
identifier, or an internal offset related to the virtqueue number.
\end{note}
Quoted 15 lines
Quoted 11 lines
content.tex +3
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptorsper + vring slot as in \ref{sec:Basic Facilities of a Virtio
Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect
Descriptors} if + and only if theVIRTIO_RING_F_INDIRECT_SIZE
feature has been negotiated. +
+ The device specifies its maximum supported number of descriptors
per + vring slot. If the driver requires fewer descriptors, it
writes its + lower value to inform the device of the reduced
resource requirements."vring slot" is a little vague, it means "indirect descriptor table"? The two paragraphs could use "maximum number of descriptors per indirect descriptor table" instead of referring to vring slots to imply indirect descriptor tables.
The intended phrasing was intended to reflect that "Queue Indirect Size" is meant to be the *sum* of all indirect descriptors:
https://lists.oasis-open.org/archives/virtio-comment/202112/msg00008.html
Just to avoid that I am missing something here; as of now, there can only be two indirect tables per round-trip message, right?
Best regards,
Christian Schoenebeck
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 80 lines · Christian Schoenebeck
Quoted 8 lines · On Montag, 24. Januar 2022 14:53:45 CET Stefan Hajnoczi wrote:
Quoted 7 lines · Christian Schoenebeck
This new common configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>---
Quoted 5 lines
content.tex
Quoted 5 lines
content.tex | 25 ++++++++++++++++++++++++- split-ring.tex | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)content.tex +0
-0Stefan, while preparing the next version of patches for this issue (#122), I noticed that I missed your email here ...
Quoted 14 lines
Quoted 10 lines
index 0aa4842..e3cfcae 100644
content.tex +0
-0@@ -859,6 +859,7 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
le64 queue_driver; /* read-write */ le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */+ le32
queue_indirect_size; /* read-write */I see no other unaligned fields in this struct, so I think a le16 padding field is needed for safety:
Quoted 5 lines
le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ + le16 reserved; /* no access */ + le32 queue_indirect_size; /* read-write */That's no longer needed as there's now the new '
queue_reset' field, i.e.:content.tex +0
-0@@ -902,6 +902,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport
le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ le16 queue_reset; /* read-write */ + le32 queue_indirect_size; /* read-write */};
\end{lstlisting}Quoted 11 lines
Quoted 11 lines
};
\end{lstlisting}content.tex +0
-0@@ -938,6 +939,16 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
may benefit from providing another value, for example an internal virtqueue
identifier, or an internal offset related to the virtqueue number.
\end{note}Quoted 15 lines
Quoted 11 lines
content.tex +3
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount of descriptorsper + vring slot as in \ref{sec:Basic Facilities of a Virtio
Device / + Virtqueues / The Virtqueue Descriptor Table / Indirect
Descriptors} if + and only if theVIRTIO_RING_F_INDIRECT_SIZE
feature has been negotiated. +
+ The device specifies its maximum supported number of descriptors
per + vring slot. If the driver requires fewer descriptors, it
writes its + lower value to inform the device of the reduced
resource requirements."vring slot" is a little vague, it means "indirect descriptor table"? The two paragraphs could use "maximum number of descriptors per indirect descriptor table" instead of referring to vring slots to imply indirect descriptor tables.
Quoted 6 lines
The intended phrasing was intended to reflect that "Queue Indirect Size" is meant to be the *sum* of all indirect descriptors:
https://lists.oasis-open.org/archives/virtio-comment/202112/msg00008.htmlJust to avoid that I am missing something here; as of now, there can only be two indirect tables per round-trip message, right?
No, just one indirect descriptor table. An indirect descriptor table may contain both IN and OUT descriptors so it can handle a round-trip message.
2.6.5.3.1 Driver Requirements: Indirect Descriptors says:
A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in flags.
It's unclear whether this statement is referring to 1) one descriptor,
2) to all descriptors in a buffer, or 3) to all descriptors ever made available by the driver. QEMU's hw/virtio.c:virtqueue_pop() shows that the interpretation is #2. So if a buffer uses an indirect descriptor table then it has exactly 1 descriptor in the virtqueue's descriptor table.
Stefan
Re: [PATCH 2/2] Add common configuration field "queue_indirect_size"
Quoted 38 lines · On Montag, 21. Februar 2022 11:33:45 CET Stefan Hajnoczi wrote:
Quoted 37 lines · Christian Schoenebeck
Quoted 22 lines · On Montag, 24. Januar 2022 14:53:45 CET Stefan Hajnoczi wrote:
Quoted 21 lines · Christian Schoenebeck
(quoted diff) +0
-0@@ -938,6 +939,16 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
may benefit from providing another value, for example an internal
virtqueue
identifier, or an internal offset related to the virtqueue number.
\end{note}(quoted diff) +3
-0+ +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum amount ofdescriptors
per + vring slot as in \ref{sec:Basic Facilities of a Virtio
Device / + Virtqueues / The Virtqueue Descriptor Table /
Indirect
Descriptors} if + and only if theVIRTIO_RING_F_INDIRECT_SIZE
feature has been negotiated. +
+ The device specifies its maximum supported number of
descriptorsQuoted 8 lines
Quoted 3 lines
per + vring slot. If the driver requires fewer descriptors, it
writes its + lower value to inform the device of the reduced
resource requirements."vring slot" is a little vague, it means "indirect descriptor table"? The two paragraphs could use "maximum number of descriptors per indirect descriptor table" instead of referring to vring slots to imply indirect descriptor tables.
The intended phrasing was intended to reflect that "Queue Indirect Size" is
meant to be the *sum* of all indirect descriptors:
https://lists.oasis-open.org/archives/virtio-comment/202112/msg00008.html
Quoted 6 lines
Quoted 2 lines
Just to avoid that I am missing something here; as of now, there can only be two indirect tables per round-trip message, right?
No, just one indirect descriptor table. An indirect descriptor table may contain both IN and OUT descriptors so it can handle a round-trip message.
Ok, you are right. I thought it would pick (per round-trip message) 2 pre- allocated descriptors (directly from the queue), and that those 2 descriptors would then point to a separate indirect table each.
But it is like you said: just one pre-allocated descriptor, pointing to exactly one indirect table, and inside that indirect table there are first the indirect descriptors (pages) for device<-driver request message, followed by indirect descriptors (pages) for device->driver response message. And this order (request first, response second) is implied by the QEMU code as well.
The names for "in" and "out" in the code are a bit confusing BTW.
Quoted 4 lines
2.6.5.3.1 Driver Requirements: Indirect Descriptors says:
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin
flags.
Yes, I was aware about that and that should have made it clear to me already. I had a calculation error while debugging the code which mislead me.
Quoted 8 lines
It's unclear whether this statement is referring to 1) one descriptor,
2) to all descriptors in a buffer, or 3) to all descriptors ever made available by the driver. QEMU's hw/virtio.c:virtqueue_pop() shows that the interpretation is #2. So if a buffer uses an indirect descriptor table then it has exactly 1 descriptor in the virtqueue's descriptor table.Stefan
Yes, it is 2).
Ok, I'll change the phrasing as you suggested to "per indirect descriptor table" and will send a new version this week.
Thanks!
Best regards,
Christian Schoenebeck
Quoted 7 lines · Christian Schoenebeck
This is a follow-up of:
[0/2]: https://lists.oasis-open.org/archives/virtio-comment/202111/msg00089.html
[1/2]: https://lists.oasis-open.org/archives/virtio-comment/202112/msg00005.html
[2/2]: https://lists.oasis-open.org/archives/virtio-comment/202112/msg00006.htmlThese two patches introduce a more fine graded control over the maximum length of indirect descriptor tables.
Sorry for the long delay, I am very behind on reviews!
Stefan