Add VIRTIO_RING_F_INDIRECT_SIZE and queue_indirect_size
v3 cover Add VIRTIO_RING_F_INDIRECT_SIZE and queue_indirect_size
This is a follow-up of:
https://lists.oasis-open.org/archives/virtio-comment/202202/msg00059.html
These 4 patches introduce a more fine graded control over the maximum length of indirect descriptor tables.
Associated Github Task:
https://github.com/oasis-tcs/virtio-spec/issues/122
v2 -> v3:
* s/bus specific/transport specific/ [patch 1]
* CCW: Add new field 'indirect_num' both to to struct vq_config_block and struct vq_info_block. [patch 4]
* CCW: Bump virtio-ccw revision to 3 and make existence of new field 'indirect_num' dependent on revision 3. [patch 4]
* CCW: Rephrase description of new field 'indirect_num'. [patch 4]
Christian Schoenebeck (4):
Add VIRTIO_RING_F_INDIRECT_SIZE
Add PCI configuration field "queue_indirect_size" Add MMIO configuration register "QueueIndirectNum" Add CCW configuration field "indirect_num"
content.tex | 86 ++++++++++++++++++++++++++++++++++++++++++++++--- packed-ring.tex | 2 +- split-ring.tex | 8 +++-- 3 files changed, 88 insertions(+), 8 deletions(-)
Signature · 1 line
2.30.2
PATCH v3 1/4 Add VIRTIO_RING_F_INDIRECT_SIZE
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
--- content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)
content.tex +30 -2
@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B \begin{description} \item[0 to 23, and 50 to 127] Feature bits for the specific device type -\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and feature negotiation mechanisms -\item[41 to 49, and 128 and above] Feature bits reserved for future extensions. +\item[42 to 49, and 128 and above] Feature bits reserved for future extensions. \end{description} \begin{note}
@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport present either a value of 0 or a power of 2 in \field{queue_size}. +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the device MUST provide the +Queue Indirect Size supported by device, which is a transport specific +configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout} The driver MUST NOT write to \field{device_feature}, \field{num_queues}, \field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
@@ -6847,6 +6851,30 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}. + \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of the descriptor + chain, including indirect descriptor tables as in \ref{sec:Basic Facilities of + a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}, i.e. both the maximum amount of slots in the vring and the + actual bulk data size transmitted per vring slot. + + With this feature enabled, the Queue Size only limits the maximum amount + of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulk data + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table. However the + actual maximum amount supported by either device or driver might be less, + and therefore the bus specific Queue Indirect Size value MUST additionally + be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to subsequently + negotiate the actual amount of maximum indirect descriptors supported + by both sides. + \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
packed-ring.tex +1 -1
@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated. \subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +6 -2
@@ -268,8 +268,12 @@ \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi set the VIRTQ_DESC_F_INDIRECT flag within an indirect descriptor (ie. only one table per descriptor). -A driver MUST NOT create a descriptor chain longer than the Queue Size of -the device. +If VIRTIO_RING_F_INDIRECT_SIZE has not been negotiated, the driver MUST +NOT create a descriptor chain longer than the Queue Size of the device. + +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed the negotiated +Queue Indirect Size. A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in \field{flags}.
Signature · 1 line
2.30.2
[virtio-comment] Re: [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 20 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0
(...)
content.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 1055
Quoted 6 lines
content.tex +2
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport present either a value of 0 or a power of 2 in \field{queue_size}. +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the device MUST provide the +Queue Indirect Size supported by device, which is a transport specific
"supported by the device", or maybe "it supports"?
content.tex \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport · line 1056
Quoted 1 line
content.tex +1
-0+configuration. It MUST allow the driver to set a lower value.
Maybe "It MUST allow the driver to specify a lower maximum size." ?
Quoted 9 lines
content.tex +2
-0+ \drivernormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout} The driver MUST NOT write to \field{device_feature}, \field{num_queues}, \field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.@@ -6847,6 +6851,30 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}. + \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6864
Quoted 10 lines
content.tex +10
-0+ Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of the descriptor + chain, including indirect descriptor tables as in \ref{sec:Basic Facilities of + a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}, i.e. both the maximum amount of slots in the vring and the + actual bulk data size transmitted per vring slot. + + With this feature enabled, the Queue Size only limits the maximum amount
s/enabled/negotiated/ ?
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6876
Quoted 12 lines
content.tex +12
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulk data + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table. However the + actual maximum amount supported by either device or driver might be less, + and therefore the bus specific Queue Indirect Size value MUST additionally + be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to subsequently + negotiate the actual amount of maximum indirect descriptors supported + by both sides.
No 'MUST' in non-normative sections, please. Maybe make that
"and therefore the device and the driver additionally need to negotiate the transport specific Queue Indirect Size value if VIRTIO_RING_F_INDIRECT_SIZE was negotiated in order to determine the actual amount of maximum indirect descriptors supported by both."
I'm not sure whether we would actually need some normative statements in the sections below, but probably not.
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6880
Quoted 4 lines
content.tex +1
-0+ \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 30 lines · On Donnerstag, 17. März 2022 14:40:27 CET Cornelia Huck wrote:
Quoted 21 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0(...)
Quoted 5 lines
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
present either a value of 0 or a power of 2 in
\field{queue_size}.
Quoted 4 lines
Quoted 3 lines
+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide the +Queue Indirect Size supported by device, which is a transport specific"supported by the device", or maybe "it supports"?
Article "the" is missing here, yes. Both "the device" or "it" would be fine.
Quoted 3 lines
Quoted 1 line
+configuration. It MUST allow the driver to set a lower value.
Maybe "It MUST allow the driver to specify a lower maximum size." ?
That exact phrase was actually suggested by you (2021-12-14 18:20). I have no strong opinion on that. I find the existing "to set a lower value" clear enough and a less complicated wording though.
Quoted 21 lines
Quoted 21 lines
content.tex +1
-0+ \drivernormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout} The driver MUST NOT write to \field{device_feature}, \field{num_queues}, \field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.>@@ -6847,6 +6851,30 @@ \chapter{Reserved Feature Bits}\label{sec:ReservedFeature Bits}>
that the driver can reset a queue individually.
See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}.>content.tex +5
-0+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of thedescriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic
Quoted 8 lines
Quoted 6 lines
Facilities of + a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}, i.e. both the maximum amount of slots in the vring and the + actual bulk data size transmitted per vring slot.
content.tex +2
-0+ + With this feature enabled, the Queue Size only limits the maximumamount
s/enabled/negotiated/ ?
Also no strong opinion on that. "negotiated" makes it a bit more clear, yes.
Quoted 14 lines
Quoted 5 lines
content.tex +3
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulkdata + being transferred per vring slot, but also avoids complicated synchronization + mechanisms if the device only supports a very small
Quoted 8 lines
amount of vring slots. Due + to the 16-bit size of a descriptor's "next" field there is still an absolute + limit of $2^{16}$ descriptors per indirect descriptor table. However the + actual maximum amount supported by either device or driver might be less, + and therefore the bus specific Queue Indirect Size value MUST additionally + be negotiated if
VIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequently + negotiate the actual amount of maximum indirect descriptors supported + by both sides.
Quoted 6 lines
No 'MUST' in non-normative sections, please. Maybe make that
"and therefore the device and the driver additionally need to negotiate the transport specific Queue Indirect Size value if
VIRTIO_RING_F_INDIRECT_SIZEwas negotiated in order to determine the actual amount of maximum indirect descriptors supported by both."
Yes, that still slipped through, sorry. That change looks fine to me.
Quoted 2 lines
I'm not sure whether we would actually need some normative statements in the sections below, but probably not.
Like what would you potentially miss here?
Quoted 5 lines
Quoted 5 lines
+
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
[virtio-comment] Re: [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 31 lines · Christian Schoenebeck
Quoted 30 lines · On Donnerstag, 17. März 2022 14:40:27 CET Cornelia Huck wrote:
Quoted 21 lines · Christian Schoenebeck
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0(...)
Quoted 5 lines
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport>
present either a value of 0 or a power of 2 in
\field{queue_size}.
Quoted 6 lines
Quoted 4 lines
Quoted 3 lines
+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide the +Queue Indirect Size supported by device, which is a transport specific"supported by the device", or maybe "it supports"?
Article "the" is missing here, yes. Both "the device" or "it" would be fine.
Yes; just use whichever you prefer :)
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
+configuration. It MUST allow the driver to set a lower value.
Maybe "It MUST allow the driver to specify a lower maximum size." ?
That exact phrase was actually suggested by you (2021-12-14 18:20). I have no strong opinion on that. I find the existing "to set a lower value" clear enough and a less complicated wording though.
Heh :) I do not really have a strong opinion here (hence the "maybe".)
Quoted 4 lines
Quoted 2 lines
I'm not sure whether we would actually need some normative statements in the sections below, but probably not.
Like what would you potentially miss here?
That we MUST also negotiate the value if we negotiate the feature. But as I wrote, we probably don't need to be that explicit.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
In reply to Christian SchoenebeckQuoted 3 lines · On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck <[email protected]>…
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
if we are extending these limits, I suggest reusing the feature flag to also add a limit on total s/g list size. making it separate from queue size was requested a while ago.
Quoted 64 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of aVirtio Device / Feature B
\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type-\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms-\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.
\end{description}\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport
present either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide thecontent.tex +3
-0+Queue Indirect Size supported by device, which is a transport specific +configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
content.tex +6
-0+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of the descriptor + chain, including indirect descriptor tables as in \ref{sec:BasicFacilities of
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6862
Quoted 3 lines
content.tex +3
-0+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors}, i.e. both the maximum amount of slots in the vring and the + actual bulk data size transmitted per vring slot.
spect does not call these slots elsewhere.
+
Quoted 19 lines
content.tex +4
-0+ With this feature enabled, the Queue Size only limits the maximum amount + of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulkdata
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an absolutecontent.tex +2
-0+ limit of $2^{16}$ descriptors per indirect descriptor table. However the + actual maximum amount supported by either device or driver might beless,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequently
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6876
Quoted 2 lines
content.tex +2
-0+ negotiate the actual amount of maximum indirect descriptors supported + by both sides.
still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 26 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated. \subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi
set theVIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie. only one table per descriptor).
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 272
Quoted 2 lines
split-ring.tex +0
-2-A driver MUST NOT create a descriptor chain longer than the Queue Size of -the device.
+If VIRTIO_RING_F_INDIRECT_SIZE has not been negotiated, the driver MUST
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 276
Quoted 6 lines
split-ring.tex +5
-0+NOT create a descriptor chain longer than the Queue Size of the device. + +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed the negotiated +Queue Indirect Size.
it is not negotiated is it?
Quoted 28 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin \field{flags}.
--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 9 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck <[email protected]>
Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
if we are extending these limits, I suggest reusing the feature flag to also add a limit on total s/g list size. making it separate from queue size was requested a while ago.
What do you mean with "total s/g list size"? The maximum bulk data size per message? Sum of both in and out s/g lists' bulk data or only for one of them? Or maximum size of exactly only one memory segment?
And are you suggesting this should become part of this series already?
Quoted 67 lines
Quoted 67 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of aVirtio Device / Feature B
\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type-\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport
present either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide thecontent.tex +3
-0+Queue Indirect Size supported by device, which is a transport specific +configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
Quoted 15 lines
Quoted 5 lines
content.tex +5
-0+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of theQuoted 8 lines
descriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic Facilities of
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring and the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find the latter too vague in this context. A "buffer" can be a memory segment, a set of memory segments and what not. "vring slot" OTOH makes it clear that it is about exactly one, atomic pointer (hence with fixed size) in a Ring Buffer, as depicted in the ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of messages that can be emplaced into a Ring Buffer, independent of any "bulk data buffer size".
Quoted 68 lines
+
Quoted 20 lines
+ With this feature enabled, the Queue Size only limits the maximum amount
content.tex +3
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulkdata
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table. However the
+ actual maximum amount supported by either device or driver might be less,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequentlycontent.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6876
Quoted 2 lines
content.tex +2
-0+ negotiate the actual amount of maximum indirect descriptors supported + by both sides.still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 31 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated.\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi
set the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie. only
one table per descriptor).split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 272
Quoted 2 lines
split-ring.tex +0
-2-A driver MUST NOT create a descriptor chain longer than the Queue Size of -the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUST
Quoted 7 lines
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 276
Quoted 5 lines
split-ring.tex +5
-0+NOT create a descriptor chain longer than the Queue Size of the device. + +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed the negotiated +Queue Indirect Size.it is not negotiated is it?
What makes you think it is not negotiated?
Quoted 27 lines
Quoted 27 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin \field{flags}.--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 7 lines · Christian Schoenebeck
Quoted 6 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck <[email protected]>
Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Quoted 8 lines
Quoted 3 lines
if we are extending these limits, I suggest reusing the feature flag to also add a limit on total s/g list size. making it separate from queue size was requested a while ago.
What do you mean with "total s/g list size"? The maximum bulk data size per message?
Sum of both in and out s/g lists' bulk data or only for one of them? Or maximum size of exactly only one memory segment?
I don't really know what does "bulk data size" mean. Suggest we use terminology from the spec. A buffer includes a group of direct and/or indirect descriptors, in turn indirect descriptors point to direct descriptors.
What has been requested a while is ability to limit per vq the
# of direct descriptors in a buffer.
Since IIUC what you want to do is allow more descriptors than VQ size, then one way to achieve that is just to have a per VQ limit on descriptor size and have that limit > VQ size.
Another thing related is that people wanted to block indirect descriptors for some VQs. Not yet sure how to combine that with this proposal, worth thinking about.
Quoted 2 lines
And are you suggesting this should become part of this series already?
yes since it's touching mostly same areas in the spec.
Quoted 201 lines
Quoted 67 lines
Quoted 67 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of aVirtio Device / Feature B
\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type-\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport
present either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide thecontent.tex +3
-0+Queue Indirect Size supported by device, which is a transport specific +configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
Quoted 15 lines
Quoted 5 lines
content.tex +5
-0+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of theQuoted 8 lines
descriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic Facilities of
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring and the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find the latter too vague in this context. A "buffer" can be a memory segment, a set of memory segments and what not. "vring slot" OTOH makes it clear that it is about exactly one, atomic pointer (hence with fixed size) in a Ring Buffer, as depicted in the ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of messages that can be emplaced into a Ring Buffer, independent of any "bulk data buffer size".
Quoted 68 lines
+
Quoted 20 lines
+ With this feature enabled, the Queue Size only limits the maximum amount
content.tex +3
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decoupling these + two configuration parameters this way not only allows much larger bulkdata
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table. However the
+ actual maximum amount supported by either device or driver might be less,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequentlycontent.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6876
Quoted 2 lines
content.tex +2
-0+ negotiate the actual amount of maximum indirect descriptors supported + by both sides.still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 31 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated.\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{Indirect Descriptors}\label{sec:BasicFacilities of a Virtio Devi
set the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie. only
one table per descriptor).split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 272
Quoted 2 lines
split-ring.tex +0
-2-A driver MUST NOT create a descriptor chain longer than the Queue Size of -the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUSTQuoted 7 lines
split-ring.tex \subsubsection{Indirect Descriptors}\label{sec:Basic Facilities of a Virtio Devi · line 276
Quoted 5 lines
split-ring.tex +5
-0+NOT create a descriptor chain longer than the Queue Size of the device. + +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed the negotiated +Queue Indirect Size.it is not negotiated is it?
What makes you think it is not negotiated?
Quoted 27 lines
Quoted 27 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin \field{flags}.--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 20 lines · On Sonntag, 20. März 2022 13:31:51 CET Michael S. Tsirkin wrote:
Quoted 13 lines · Christian Schoenebeck
Quoted 11 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck
<[email protected]>Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
if we are extending these limits, I suggest reusing the feature flag to also add a limit on total s/g list size. making it separate from queue size
was requested a while ago.Quoted 5 lines
What do you mean with "total s/g list size"? The maximum bulk data size per
message?
Sum of both in and out s/g lists' bulk data or only for one of them? Or maximum size of exactly only one memory segment?
Quoted 4 lines
I don't really know what does "bulk data size" mean. Suggest we use terminology from the spec. A buffer includes a group of direct and/or indirect descriptors, in turn indirect descriptors point to direct descriptors.
I already described why I think it makes sense not calling it "buffer" in this particular context. So I am against changing this to "buffer". If other people support your position then I'll change it though of course.
About "bulk data size": "Bulk data" is the user data actually being used on application/driver level, i.e. above virtio level, and "Bulk data size" the size of that data. See ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
The terminology "bulk data" is already used in the spec already BTW.
Quoted 2 lines
What has been requested a while is ability to limit per vq the
# of direct descriptors in a buffer.
So in cases where indirect descriptors are *not* used. This series is about indirect descriptors only though.
Quoted 3 lines
Since IIUC what you want to do is allow more descriptors than VQ size, then one way to achieve that is just to have a per VQ limit on descriptor size and have that limit > VQ size.
Sorry, I can't follow you. What do you mean with "descriptor size"? For me a descriptor has a predefined constant struct size. You mean the size of one memory segment referenced by one indirect descriptor? And why would it be better than what this series suggests?
Quoted 3 lines
Another thing related is that people wanted to block indirect descriptors for some VQs. Not yet sure how to combine that with this proposal, worth thinking about.
This series allows both, increasing *and* decreasing the number of indirect descriptors per VQ already.
Quoted 3 lines
Quoted 1 line
And are you suggesting this should become part of this series already?
yes since it's touching mostly same areas in the spec.
:/ Please note that I sent the first draft on this issue already in November last year, and have not seen any response from your side so far. I actually assumed we were already at a point where it was just about precise wording et al., not restarting to redesign everything again from scratch now.
Quoted 188 lines
Quoted 188 lines
Quoted 83 lines
Quoted 83 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilitiesof a
Virtio Device / Feature B\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type-\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport
present either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide the
+Queue Indirect Size supported by device, which is a transport specificcontent.tex +2
-0+configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout
/ Common configuration structure layout}The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved FeatureBits}\label{sec:Reserved
Feature Bits}that the driver can reset a queue individually.
See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the
+ Queue Indirect Size, i.e. the maximum amount of descriptors in indirectcontent.tex +3
-0+ descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of thedescriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic Facilities ofQuoted 8 lines
Quoted 6 lines
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring and
the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find the latter too vague in this context. A "buffer" can be a memory segment, a set of memory segments and what not. "vring slot" OTOH makes it clear that it is about exactly one, atomic pointer (hence with fixed size) in a Ring Buffer, as depicted in the ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of messages that can be emplaced into a Ring Buffer, independent of any "bulk data buffer size".
Quoted 83 lines
+
Quoted 18 lines
+ With this feature enabled, the Queue Size only limits the maximum amount
content.tex +2
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decouplingthese
+ two configuration parameters this way not only allows much larger bulk
data
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table. However
the
+ actual maximum amount supported by either device or driver might beQuoted 8 lines
less,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequently
+ negotiate the actual amount of maximum indirect descriptors supported
+ by both sides.still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 30 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has beennegotiated.
\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{IndirectDescriptors}\label{sec:Basic
Facilities of a Virtio DeviQuoted 7 lines
set the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie. only
one table per descriptor).-A driver MUST NOT create a descriptor chain longer than the Queue Size of
-the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUSTQuoted 7 lines
+NOT create a descriptor chain longer than the Queue Size of the device.
split-ring.tex +3
-0+ +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed thenegotiated
+Queue Indirect Size.it is not negotiated is it?
Quoted 1 line
Quoted 1 line
What makes you think it is not negotiated?
Also see my previous question here ^
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXT
in \field{flags}.--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License:
https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 21 lines · Christian Schoenebeck
Quoted 20 lines · On Sonntag, 20. März 2022 13:31:51 CET Michael S. Tsirkin wrote:
Quoted 18 lines · Christian Schoenebeck
Quoted 11 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck
<[email protected]>Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
if we are extending these limits, I suggest reusing the feature flag to also add a limit on total s/g list size. making it separate from queue size
was requested a while ago.What do you mean with "total s/g list size"? The maximum bulk data size per
message?
Sum of both in and out s/g lists' bulk data or only for one of them? Or maximum size of exactly only one memory segment?
Quoted 7 lines
Quoted 4 lines
I don't really know what does "bulk data size" mean. Suggest we use terminology from the spec. A buffer includes a group of direct and/or indirect descriptors, in turn indirect descriptors point to direct descriptors.
I already described why I think it makes sense not calling it "buffer" in this particular context. So I am against changing this to "buffer".
Well spec just defines what buffers are.
If you are using a different term then you need to define it in the spec.
Quoted 9 lines
If other people
support your position then I'll change it though of course.About "bulk data size": "Bulk data" is the user data actually being used on application/driver level, i.e. above virtio level, and "Bulk data size" the size of that data. See ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122The terminology "bulk data" is already used in the spec already BTW.
It does not refer to anything specific though, just generally to vqs for passing lots of data as opposed to config space used to pass small amount of data.
Quoted 5 lines
Quoted 2 lines
What has been requested a while is ability to limit per vq the
# of direct descriptors in a buffer.So in cases where indirect descriptors are *not* used. This series is about indirect descriptors only though.
No, in all cases.
Quoted 9 lines
Quoted 3 lines
Since IIUC what you want to do is allow more descriptors than VQ size, then one way to achieve that is just to have a per VQ limit on descriptor size and have that limit > VQ size.
Sorry, I can't follow you. What do you mean with "descriptor size"? For me a
descriptor has a predefined constant struct size. You mean the size of one memory segment referenced by one indirect descriptor? And why would it be better than what this series suggests?
My bad. What I meant is a "per VQ limit on # of direct descriptors per buffer".
Quoted 7 lines
Quoted 4 lines
Another thing related is that people wanted to block indirect descriptors for some VQs. Not yet sure how to combine that with this proposal, worth thinking about.
This series allows both, increasing *and* decreasing the number of indirect descriptors per VQ already.
I don't see how you block indirect descritors for a queue with this. Did I miss it?
Also I think you missed the fact that
a direct descriptor can point to an indirect one, the result is that max # of descriptors in a buffer is then:
queue size - 1 + indirect table size
I don't see how your proposal limits the # of descriptors below queue size since guest is never forced to use indirect.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
And are you suggesting this should become part of this series already?
yes since it's touching mostly same areas in the spec.
:/ Please note that I sent the first draft on this issue already in November last year, and have not seen any response from your side so far. I actually assumed we were already at a point where it was just about precise wording et al., not restarting to redesign everything again from scratch now.
Sorry about that.
Quoted 188 lines
Quoted 188 lines
Quoted 188 lines
Quoted 83 lines
Quoted 83 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:Basic Facilitiesof a
Virtio Device / Feature B\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device type-\item[24 to 40] Feature bits reserved for extensions to the queue and +\item[24 to 41] Feature bits reserved for extensions to the queue and
feature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport
present either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST provide the
+Queue Indirect Size supported by device, which is a transport specificcontent.tex +2
-0+configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout
/ Common configuration structure layout}The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved FeatureBits}\label{sec:Reserved
Feature Bits}that the driver can reset a queue individually.
See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the
+ Queue Indirect Size, i.e. the maximum amount of descriptors in indirectcontent.tex +3
-0+ descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of thedescriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic Facilities ofQuoted 8 lines
Quoted 6 lines
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring and
the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find the latter too vague in this context. A "buffer" can be a memory segment, a set of memory segments and what not. "vring slot" OTOH makes it clear that it is about exactly one, atomic pointer (hence with fixed size) in a Ring Buffer, as depicted in the ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of messages that can be emplaced into a Ring Buffer, independent of any "bulk data buffer size".
Quoted 83 lines
+
Quoted 18 lines
+ With this feature enabled, the Queue Size only limits the maximum amount
content.tex +2
-0+ of slots in the vring, but does not limit the actual bulk data size + being transmitted when indirect descriptors are used. Decouplingthese
+ two configuration parameters this way not only allows much larger bulk
data
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table. However
the
+ actual maximum amount supported by either device or driver might beQuoted 8 lines
less,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequently
+ negotiate the actual amount of maximum indirect descriptors supported
+ by both sides.still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 30 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has beennegotiated.
\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{IndirectDescriptors}\label{sec:Basic
Facilities of a Virtio DeviQuoted 7 lines
set the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie. only
one table per descriptor).-A driver MUST NOT create a descriptor chain longer than the Queue Size of
-the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUSTQuoted 7 lines
+NOT create a descriptor chain longer than the Queue Size of the device.
split-ring.tex +3
-0+ +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed thenegotiated
+Queue Indirect Size.it is not negotiated is it?
Quoted 3 lines
Quoted 1 line
Quoted 1 line
What makes you think it is not negotiated?
Also see my previous question here ^
Sorry, what I mean is that you don't define what does negotiation involve. I think you mean this:
The driver SHOULD write to \field{queue_indirect_size} if its maximum number of descriptors per vring slot is lower than that reported by the device.
but driver can just read the value and that's it - and then the value that is set by device applies, right?
If you are going to use terms such as negotiated you need to define what they mean. In this case I would just say something like
"the value of Queue Indirect Size".
Quoted 29 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXT
in \field{flags}.--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License:
https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 30 lines · On Sonntag, 20. März 2022 14:55:59 CET Michael S. Tsirkin wrote:
Quoted 29 lines · Christian Schoenebeck
Quoted 27 lines · On Sonntag, 20. März 2022 13:31:51 CET Michael S. Tsirkin wrote:
Quoted 15 lines · Christian Schoenebeck
Quoted 7 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck
<[email protected]>Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Quoted 6 lines
if we are extending these limits, I suggest reusing the feature flag to
also add a limit on total s/g list size. making it separate from queue
size
was requested a while ago.Quoted 6 lines
What do you mean with "total s/g list size"? The maximum bulk data size
per
message?
Sum of both in and out s/g lists' bulk data or only for one of them? Or maximum size of exactly only one memory segment?I don't really know what does "bulk data size" mean. Suggest we use terminology from the spec. A buffer includes a group of direct and/or indirect descriptors, in turn indirect descriptors point to direct descriptors.
Quoted 6 lines
Quoted 2 lines
I already described why I think it makes sense not calling it "buffer" in this particular context. So I am against changing this to "buffer".
Well spec just defines what buffers are.
If you are using a different term then you need to define it in the spec.
Sorry, but that sounds like nitpicking to me. From split-ring.tex:
"When the driver wants to send a buffer to the device, it fills in a *slot* in the descriptor table (or chains several together)"
So a "slot in the descriptor table" does not need further specification, but the term "vring slot" does?
Quoted 10 lines
Quoted 10 lines
If other people
support your position then I'll change it though of course.About "bulk data size": "Bulk data" is the user data actually being used on
application/driver level, i.e. above virtio level, and "Bulk data size" the
size of that data. See ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 5 lines
Quoted 1 line
The terminology "bulk data" is already used in the spec already BTW.
It does not refer to anything specific though, just generally to vqs for passing lots of data as opposed to config space used to pass small amount of data.
Which is telling pretty much precise enough what it is, at least IMO.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
What has been requested a while is ability to limit per vq the
# of direct descriptors in a buffer.So in cases where indirect descriptors are *not* used. This series is about
indirect descriptors only though.No, in all cases.
?
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since IIUC what you want to do is allow more descriptors than VQ size, then one way to achieve that is just to have a per VQ limit on descriptor size and have that limit > VQ size.
Quoted 8 lines
Quoted 5 lines
Sorry, I can't follow you. What do you mean with "descriptor size"? For me a
descriptor has a predefined constant struct size. You mean the size of one memory segment referenced by one indirect descriptor? And why would it be better than what this series suggests?My bad. What I meant is a "per VQ limit on # of direct descriptors per buffer".
Which is out of the scope of what this series was about.
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines
Another thing related is that people wanted to block indirect descriptors for some VQs. Not yet sure how to combine that with this proposal, worth thinking about.
This series allows both, increasing *and* decreasing the number of indirect
descriptors per VQ already.I don't see how you block indirect descritors for a queue with this. Did I miss it?
By setting the proposed "Queue Indirect Size" value to zero?
Quoted 9 lines
Also I think you missed the fact that
a direct descriptor can point to an indirect one, the result is that max # of descriptors in a buffer is then:queue size - 1 + indirect table size
I don't see how your proposal limits the # of descriptors below queue size since guest is never forced to use indirect.
No, I didn't miss that. The suggested changes were about the amount of *indirect* descriptors, not about the amount of *direct* descriptors. The amount of direct descriptors was still limited to the "Queue Size".
I am aware that QEMU currently has a limit per "buffer" which adds the amount of direct descriptors *and* the amount of indirect ones together for that limit (which I also mentioned from the Github issue summary BTW). Which is a device specific implementation feature of QEMU and would not stop QEMU though to handle this correctly by reducing the negotiated "Queue Indirect Size" value appropriately.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
And are you suggesting this should become part of this series already?
yes since it's touching mostly same areas in the spec.
:
:/ Please note that I sent the first draft on this issue already in :November
Quoted 5 lines
Quoted 4 lines
last year, and have not seen any response from your side so far. I actually
assumed we were already at a point where it was just about precise wording et al., not restarting to redesign everything again from scratch now.Sorry about that.
I'm pretty sure you are.
Quoted 224 lines
Quoted 213 lines
Quoted 213 lines
Quoted 213 lines
Quoted 95 lines
Quoted 95 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:BasicFacilities
of a
Virtio Device / Feature B\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device
type-\item[24 to 40] Feature bits reserved for extensions to the queue and
+\item[24 to 41] Feature bits reserved for extensions to the queue andfeature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future
extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future
extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configurationstructure
layout}\label{sec:Virtio Transportpresent either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST
provide the
+Queue Indirect Size supported by device, which is a transport specificcontent.tex +2
-0+configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device
Layout
/ Common configuration structure layout}The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved FeatureBits}\label{sec:Reserved
Feature Bits}that the driver can reset a queue individually.
See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that
the
+ Queue Indirect Size, i.e. the maximum amount of descriptors in indirectcontent.tex +3
-0+ descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of thedescriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic
Facilities of
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table /Quoted 8 lines
Quoted 6 lines
Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring
and
the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find the
latter too vague in this context. A "buffer" can be a memory segment, a
set of memory segments and what not. "vring slot" OTOH makes it clear that it is about exactly one, atomic pointer (hence with fixed size) in a
Ring Buffer, as depicted in the ASCII illustration here:https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of messages that can be emplaced into a Ring Buffer, independent of any "bulk data buffer size".
Quoted 93 lines
+
Quoted 25 lines
+ With this feature enabled, the Queue Size only limits the maximum
amount
+ of slots in the vring, but does not limit the actual bulk data size
+ being transmitted when indirect descriptors are used. Decoupling
these
+ two configuration parameters this way not only allows much larger
bulk
data
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring
slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an
absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table. However
the
+ actual maximum amount supported by either device or driver might beQuoted 8 lines
less,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequently
+ negotiate the actual amount of maximum indirect descriptors supported
+ by both sides.still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 31 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature
Bits}packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has beennegotiated.
\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{IndirectDescriptors}\label{sec:Basic
Facilities of a Virtio DeviQuoted 8 lines
set the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie.
only
one table per descriptor).-A driver MUST NOT create a descriptor chain longer than the Queue Size of
-the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUSTQuoted 7 lines
+NOT create a descriptor chain longer than the Queue Size of the device.
split-ring.tex +3
-0+ +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed thenegotiated
+Queue Indirect Size.it is not negotiated is it?
Quoted 3 lines
Quoted 1 line
Quoted 1 line
What makes you think it is not negotiated?
Also see my previous question here ^
Sorry, what I mean is that you don't define what does negotiation involve. I think you mean this:
The driver SHOULD write to \field{queue_indirect_size} if its maximum number of descriptors per vring slot is lower than that reported by the device.
Quoted 6 lines
but driver can just read the value and that's it - and then the value that is set by device applies, right?
If you are going to use terms such as negotiated you need to define what they mean. In this case I would just say something like
"the value of Queue Indirect Size".
Which makes me wonder why you just didn't say that in the first place? And I don't agree that it wasn't defined, because I actually think I did:
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size."
Or is that definition of the new term "Queue Indirect Size" not clear enough to you?
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXT
in \field{flags}.--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected] List archive:
https://lists.oasis-open.org/archives/virtio-comment/
Feedback License:
https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 38 lines · Christian Schoenebeck
Quoted 30 lines · On Sonntag, 20. März 2022 14:55:59 CET Michael S. Tsirkin wrote:
Quoted 29 lines · Christian Schoenebeck
Quoted 27 lines · On Sonntag, 20. März 2022 13:31:51 CET Michael S. Tsirkin wrote:
Quoted 15 lines · Christian Schoenebeck
Quoted 7 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck
<[email protected]>Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Quoted 6 lines
if we are extending these limits, I suggest reusing the feature flag to
also add a limit on total s/g list size. making it separate from queue
size
was requested a while ago.Quoted 6 lines
What do you mean with "total s/g list size"? The maximum bulk data size
per
message?
Sum of both in and out s/g lists' bulk data or only for one of them? Or maximum size of exactly only one memory segment?I don't really know what does "bulk data size" mean. Suggest we use terminology from the spec. A buffer includes a group of direct and/or indirect descriptors, in turn indirect descriptors point to direct descriptors.
Quoted 6 lines
Quoted 2 lines
I already described why I think it makes sense not calling it "buffer" in this particular context. So I am against changing this to "buffer".
Well spec just defines what buffers are.
If you are using a different term then you need to define it in the spec.
Quoted 6 lines
Sorry, but that sounds like nitpicking to me. From split-ring.tex:
"When the driver wants to send a buffer to the device, it fills in a *slot* in the descriptor table (or chains several together)" So a "slot in the descriptor table" does not need further specification, but the term "vring slot" does?
One of the reasons is that it's an old text from pre-standard days. It's therefore informal. We are trying to do better and unfortunately this means you might need to clean up some old text to add your feature.
Packed ring has the concept of "elements" maybe fixing up split to use that teminology too can work.
Anothe reason I am not excited about "vring slots" is because it is not concise - it's easy to just say "slots" and introduce confusion.
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
If other people
support your position then I'll change it though of course.About "bulk data size": "Bulk data" is the user data actually being used on
application/driver level, i.e. above virtio level, and "Bulk data size" the
size of that data. See ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
The terminology "bulk data" is already used in the spec already BTW.
It does not refer to anything specific though, just generally to vqs for passing lots of data as opposed to config space used to pass small amount of data.
Which is telling pretty much precise enough what it is, at least IMO.
Well saying generally there's a "bulk of data" just means there's a lot. If you are talking about it's size then I guess you somehow distinguish this data from some other data?
Please look at the Virtqueues chapter and see if any existing terms fit your usage. It's hard enough for people to learn the spec without us changing terminology each release.
Quoted 10 lines
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
What has been requested a while is ability to limit per vq the
# of direct descriptors in a buffer.So in cases where indirect descriptors are *not* used. This series is about
indirect descriptors only though.No, in all cases.
?
what users in the field asked for is ability to increase VQ size to hold more than 1024 buffers. However QEMU can not support requests with more than 1024 elements. Thus the wish to limit buffer size below VQ size. Whether there's an indirect element in the chain does not matter for this use-case.
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
Since IIUC what you want to do is allow more descriptors than VQ size, then one way to achieve that is just to have a per VQ limit on descriptor size and have that limit > VQ size.
Sorry, I can't follow you. What do you mean with "descriptor size"? For me a
descriptor has a predefined constant struct size. You mean the size of one memory segment referenced by one indirect descriptor? And why would it be better than what this series suggests?
Quoted 4 lines
Quoted 2 lines
My bad. What I meant is a "per VQ limit on # of direct descriptors per buffer".
Which is out of the scope of what this series was about.
Maybe the scope is too limited then.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Another thing related is that people wanted to block indirect descriptors for some VQs. Not yet sure how to combine that with this proposal, worth thinking about.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This series allows both, increasing *and* decreasing the number of indirect
descriptors per VQ already.I don't see how you block indirect descritors for a queue with this. Did I miss it?
By setting the proposed "Queue Indirect Size" value to zero?
I guess ... it is better to call this out explicitly in the spec.
Quoted 6 lines
Quoted 6 lines
Also I think you missed the fact that
a direct descriptor can point to an indirect one, the result is that max # of descriptors in a buffer is then:queue size - 1 + indirect table size
Quoted 7 lines
Quoted 3 lines
I don't see how your proposal limits the # of descriptors below queue size since guest is never forced to use indirect.
No, I didn't miss that. The suggested changes were about the amount of *indirect* descriptors, not about the amount of *direct* descriptors. The amount of direct descriptors was still limited to the "Queue Size".
Confused.
The amount of indirect descriptors?
Not the size in each one? But your commit log says
descriptors in indirect descriptor tables
descriptors in the indirect descriptor tables are not indirect descriptors.
At this point I don't understand the motivation for the change.
Quoted 6 lines
I am aware that QEMU currently has a limit per "buffer" which adds the amount of direct descriptors *and* the amount of indirect ones together for that limit (which I also mentioned from the Github issue summary BTW). Which is a device specific implementation feature of QEMU and would not stop QEMU though to handle this correctly by reducing the negotiated "Queue Indirect Size" value appropriately.
Except people want a deeper queue, I can probably find & forward links to mailing list discussions.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
And are you suggesting this should become part of this series already?
yes since it's touching mostly same areas in the spec.
:
:/ Please note that I sent the first draft on this issue already in :November
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
last year, and have not seen any response from your side so far. I actually
assumed we were already at a point where it was just about precise wording et al., not restarting to redesign everything again from scratch now.Sorry about that.
I'm pretty sure you are.
Yes, I wish we could move faster. If you are asking for advice about avoiding such situations in the future then that would be to iterate faster. It's just on v3 since november, and you did get comments on the previous revisions.
Quoted 238 lines
Quoted 224 lines
Quoted 213 lines
Quoted 213 lines
Quoted 213 lines
Quoted 95 lines
Quoted 95 lines
The new term "Queue Indirect Size" is introduced for this purpose, which is a transport specific configuration whose negotiation is further specified for each transport with subsequent patches.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:BasicFacilities
of a
Virtio Device / Feature B\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device
type-\item[24 to 40] Feature bits reserved for extensions to the queue and
+\item[24 to 41] Feature bits reserved for extensions to the queue andfeature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future
extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future
extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configurationstructure
layout}\label{sec:Virtio Transportpresent either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device MUST
provide the
+Queue Indirect Size supported by device, which is a transport specificcontent.tex +2
-0+configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device
Layout
/ Common configuration structure layout}The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved FeatureBits}\label{sec:Reserved
Feature Bits}that the driver can reset a queue individually.
See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /Virtqueue Reset}.
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that
the
+ Queue Indirect Size, i.e. the maximum amount of descriptors in indirectcontent.tex +3
-0+ descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length of thedescriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic
Facilities of
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table /Quoted 8 lines
Quoted 6 lines
Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring
and
the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find the
latter too vague in this context. A "buffer" can be a memory segment, a
set of memory segments and what not. "vring slot" OTOH makes it clear that it is about exactly one, atomic pointer (hence with fixed size) in a
Ring Buffer, as depicted in the ASCII illustration here:https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of messages that can be emplaced into a Ring Buffer, independent of any "bulk data buffer size".
Quoted 93 lines
+
Quoted 25 lines
+ With this feature enabled, the Queue Size only limits the maximum
amount
+ of slots in the vring, but does not limit the actual bulk data size
+ being transmitted when indirect descriptors are used. Decoupling
these
+ two configuration parameters this way not only allows much larger
bulk
data
+ being transferred per vring slot, but also avoids complicated synchronization
+ mechanisms if the device only supports a very small amount of vring
slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an
absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table. However
the
+ actual maximum amount supported by either device or driver might beQuoted 8 lines
less,
+ and therefore the bus specific Queue Indirect Size value MUST additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to subsequently
+ negotiate the actual amount of maximum indirect descriptors supported
+ by both sides.still not sure what exactly is the value. e.g. in a buffer including indirect and direct descriptors.
+
Quoted 31 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature
Bits}packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has beennegotiated.
\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{IndirectDescriptors}\label{sec:Basic
Facilities of a Virtio DeviQuoted 8 lines
set the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor (ie.
only
one table per descriptor).-A driver MUST NOT create a descriptor chain longer than the Queue Size of
-the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver MUSTQuoted 7 lines
+NOT create a descriptor chain longer than the Queue Size of the device.
split-ring.tex +3
-0+ +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, the number of +descriptors per indirect descriptor table MUST NOT exceed thenegotiated
+Queue Indirect Size.it is not negotiated is it?
Quoted 3 lines
Quoted 1 line
Quoted 1 line
What makes you think it is not negotiated?
Also see my previous question here ^
Sorry, what I mean is that you don't define what does negotiation involve. I think you mean this:
The driver SHOULD write to \field{queue_indirect_size} if its maximum number of descriptors per vring slot is lower than that reported by the device.
Quoted 6 lines
but driver can just read the value and that's it - and then the value that is set by device applies, right?
If you are going to use terms such as negotiated you need to define what they mean. In this case I would just say something like
"the value of Queue Indirect Size".Which makes me wonder why you just didn't say that in the first place? And I don't agree that it wasn't defined, because I actually think I did:
(quoted diff) +3
-0+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size."
Quoted 2 lines
Or is that definition of the new term "Queue Indirect Size" not clear enough to you?
Maybe ... but I don't think this will jump out to the reader. I feel we abuse of the word negotiate to the point where reader only has a vague idea what it means. That's why I struggled to give concise comments. Consider:
+ and therefore the transport specific Queue Indirect Size value MUST + additionally be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to + subsequently negotiate the actual amount of maximum indirect descriptors + supported by both sides.
sounds like a tongue-twister to me. Can't we find a term different from "negotiate"?
It's already used for features, please find something else for this. And why is a term even necessary? How is this different from other writeable fields? Consider an example of similar functionality from the spec:
\item[\field{queue_size}]
Queue Size. On reset, specifies the maximum queue size supported by the device. This can be modified by the driver to reduce memory requirements. A 0 means the queue is unavailable.
Hope this helps.
Quoted 30 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
A driver MUST NOT set both
VIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXT
in \field{flags}.--
2.30.2This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected] List help: [email protected] List archive:
https://lists.oasis-open.org/archives/virtio-comment/
Feedback License:
https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines:
https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 7 lines · Michael S. Tsirkin
Quoted 3 lines
No, I didn't miss that. The suggested changes were about the amount of *indirect* descriptors, not about the amount of *direct* descriptors. The amount of direct descriptors was still limited to the "Queue Size".
Confused.
The amount of indirect descriptors?
Not the size in each one?
length, to be exact
Quoted 6 lines
But your commit log says
descriptors in indirect descriptor tables
descriptors in the indirect descriptor tables are not indirect descriptors.At this point I don't understand the motivation for the change.
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
In reply to Michael S. TsirkinQuoted 5 lines · On Sonntag, 20. März 2022 17:06:16 CET Michael S. Tsirkin wrote:
Quoted 4 lines · Christian Schoenebeck
Quoted 3 lines · On Sonntag, 20. März 2022 14:55:59 CET Michael S. Tsirkin wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Sonntag, 20. März 2022 13:31:51 CET Michael S. Tsirkin wrote:
On Sat, Mar 19, 2022 at 01:00:28PM +0100, Christian Schoenebeck
Quoted 50 lines · wrote:
Quoted 38 lines
Quoted 30 lines
Quoted 30 lines
Quoted 29 lines
Quoted 16 lines · On Samstag, 19. März 2022 10:33:49 CET Michael S. Tsirkin wrote:
Quoted 7 lines
On Wed, Mar 16, 2022, 15:47 Christian Schoenebeck
<[email protected]>Quoted 2 lines · wrote:
This new feature flag allows to decouple the maximum amount of descriptors in indirect descriptor tables from the Queue Size.
Quoted 7 lines
if we are extending these limits, I suggest reusing the feature flag
to
also add a limit on total s/g list size. making it separate from queue
size
was requested a while ago.Quoted 7 lines
What do you mean with "total s/g list size"? The maximum bulk data size
per
message?
Sum of both in and out s/g lists' bulk data or only for one of them?
Or maximum size of exactly only one memory segment?I don't really know what does "bulk data size" mean. Suggest we use terminology from the spec. A buffer includes a group of direct and/or
indirect descriptors, in turn indirect descriptors point to direct descriptors.Quoted 7 lines
Quoted 3 lines
I already described why I think it makes sense not calling it "buffer" in
this particular context. So I am against changing this to "buffer".Well spec just defines what buffers are.
If you are using a different term then you need to define it in the spec.Quoted 6 lines
Sorry, but that sounds like nitpicking to me. From split-ring.tex:
"When the driver wants to send a buffer to the device, it fills in a *slot* in the descriptor table (or chains several together)" So a "slot in the descriptor table" does not need further specification, but the term "vring slot" does?
One of the reasons is that it's an old text from pre-standard days. It's therefore informal. We are trying to do better and unfortunately this means you might need to clean up some old text to add your feature.
Quoted 7 lines
Packed ring has the concept of "elements" maybe fixing up split to use that teminology too can work.
Anothe reason I am not excited about "vring slots" is because it is not concise - it's easy to just say "slots" and introduce confusion.
And using the term "buffer" instead of "slot" is better in avoiding confusion. Sorry, but I deeply disagree with you. A "buffer" can mean simply anything, and the only reason why you want to stick to that term is because it is already used in the spec. A "buffer" is still a much more confusing term than "slot", espcially in this context.
Quoted 20 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
If other people
support your position then I'll change it though of course.About "bulk data size": "Bulk data" is the user data actually being used
on
application/driver level, i.e. above virtio level, and "Bulk data size"
the
size of that data. See ASCII illustration here:
https://github.com/oasis-tcs/virtio-spec/issues/122Quoted 7 lines
Quoted 5 lines
Quoted 1 line
The terminology "bulk data" is already used in the spec already BTW.
It does not refer to anything specific though, just generally to vqs for passing lots of data as opposed to config space used to pass small amount of data.
Which is telling pretty much precise enough what it is, at least IMO.
Quoted 7 lines
Well saying generally there's a "bulk of data" just means there's a lot. If you are talking about it's size then I guess you somehow distinguish this data from some other data?
Please look at the Virtqueues chapter and see if any existing terms fit your usage. It's hard enough for people to learn the spec without us changing terminology each release.
Thank you. You are invited to name one that you feel appropriate.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
What has been requested a while is ability to limit per vq the
# of direct descriptors in a buffer.So in cases where indirect descriptors are *not* used. This series is about
indirect descriptors only though.No, in all cases.
Quoted 7 lines
Quoted 1 line
?
what users in the field asked for is ability to increase VQ size to hold more than 1024 buffers. However QEMU can not support requests with more than 1024 elements. Thus the wish to limit buffer size below VQ size. Whether there's an indirect element in the chain does not matter for this use-case.
Yes, and it is a different use case.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 3 lines
Since IIUC what you want to do is allow more descriptors than VQ size, then one way to achieve that is just to have a per VQ limit on descriptor size and have that limit > VQ size.
Sorry, I can't follow you. What do you mean with "descriptor size"? For me a
descriptor has a predefined constant struct size. You mean the size of one
memory segment referenced by one indirect descriptor? And why would it be
better than what this series suggests?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
My bad. What I meant is a "per VQ limit on # of direct descriptors per buffer".
Which is out of the scope of what this series was about.
Maybe the scope is too limited then.
Yes maybe. And maybe you are a little late with realizing that.
Quoted 29 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Another thing related is that people wanted to block indirect descriptors for some VQs. Not yet sure how to combine that with this proposal, worth thinking about.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This series allows both, increasing *and* decreasing the number of indirect
descriptors per VQ already.I don't see how you block indirect descritors for a queue with this. Did I miss it?
By setting the proposed "Queue Indirect Size" value to zero?
I guess ... it is better to call this out explicitly in the spec.
Quoted 6 lines
Quoted 6 lines
Also I think you missed the fact that
a direct descriptor can point to an indirect one, the result is that max # of descriptors in a buffer is then:queue size - 1 + indirect table size
Quoted 7 lines
Quoted 3 lines
I don't see how your proposal limits the # of descriptors below queue size since guest is never forced to use indirect.
No, I didn't miss that. The suggested changes were about the amount of *indirect* descriptors, not about the amount of *direct* descriptors. The amount of direct descriptors was still limited to the "Queue Size".
Quoted 8 lines
Confused.
The amount of indirect descriptors?
Not the size in each one? But your commit log says
descriptors in indirect descriptor tables
descriptors in the indirect descriptor tables are not indirect descriptors.At this point I don't understand the motivation for the change.
You seriously don't understand the motivation, or you rather want to continue with terminology discussions?
Quoted 24 lines
Quoted 6 lines
I am aware that QEMU currently has a limit per "buffer" which adds the amount of direct descriptors *and* the amount of indirect ones together for that limit (which I also mentioned from the Github issue summary BTW). Which is a device specific implementation feature of QEMU and would not stop QEMU though to handle this correctly by reducing the negotiated "Queue Indirect Size" value appropriately.
Except people want a deeper queue, I can probably find & forward links to mailing list discussions.
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 4 lines
Quoted 2 lines
And are you suggesting this should become part of this series already?
yes since it's touching mostly same areas in the spec.
:
:/ Please note that I sent the first draft on this issue already in :Novemberlast year, and have not seen any response from your side so far. I actually
assumed we were already at a point where it was just about precise wording
et al., not restarting to redesign everything again from scratch now.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Sorry about that.
I'm pretty sure you are.
Yes, I wish we could move faster. If you are asking for advice about avoiding such situations in the future then that would be to iterate faster. It's just on v3 since november, and you did get comments on the previous revisions.
And my advice on you to avoid such situations is to react *at* *all* on posted draft suggestions faster, not after 4 months. And this is actually not v3 to be precise, it is v5. The original draft was already extended and renamed consequently. And the reason for delays between iterations was that I sometimes had to wait for a month on response, which I was not blaming up anybody for. But apparently you are turning that on me now.
Quoted 246 lines
Quoted 246 lines
Quoted 246 lines
Quoted 246 lines
Quoted 245 lines
Quoted 245 lines
Quoted 105 lines
Quoted 105 lines
The new term "Queue Indirect Size" is introduced for this purpose,
which is a transport specific configuration whose negotiation is
further specified for each transport with subsequent patches.Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>---
content.tex | 32 ++++++++++++++++++++++++++++++-- packed-ring.tex | 2 +- split-ring.tex | 8 ++++++-- 3 files changed, 37 insertions(+), 5 deletions(-)content.tex +0
-0@@ -99,10 +99,10 @@ \section{Feature Bits}\label{sec:BasicFacilities
of a
Virtio Device / Feature B\begin{description}
\item[0 to 23, and 50 to 127] Feature bits for the specific device
type-\item[24 to 40] Feature bits reserved for extensions to the queue
and
+\item[24 to 41] Feature bits reserved for extensions to the queue
andfeature negotiation mechanisms
-\item[41 to 49, and 128 and above] Feature bits reserved for future
extensions.
+\item[42 to 49, and 128 and above] Feature bits reserved for future
extensions.\end{description}
\begin{note}
content.tex +0
-0@@ -1051,6 +1051,10 @@ \subsubsection{Common configurationstructure
layout}\label{sec:Virtio Transportpresent either a value of 0 or a power of 2 in
\field{queue_size}.+If
VIRTIO_RING_F_INDIRECT_SIZEhas been negotiated, the device
MUST
provide the
+Queue Indirect Size supported by device, which is a transport specificcontent.tex +2
-0+configuration. It MUST allow the driver to set a lower value. + \drivernormative{\paragraph}{Common configuration structurelayout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device
Layout
/ Common configuration structure layout}The driver MUST NOT write to \field{device_feature}, \field{num_queues},
\field{config_generation}, \field{queue_notify_off} or \field{queue_notify_data}.
content.tex +0
-0@@ -6847,6 +6851,30 @@ \chapter{Reserved FeatureBits}\label{sec:Reserved
Feature Bits}that the driver can reset a queue individually. See \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /
Virtqueue Reset}.
+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates
that
the
+ Queue Indirect Size, i.e. the maximum amount of descriptors in
indirectcontent.tex +3
-0+ descriptor tables, is independent from the Queue Size. + + Without this feature, the Queue Size limits the length ofthe
descriptor
+ chain, including indirect descriptor tables as in \ref{sec:Basic
Facilities of
+ a Virtio Device / Virtqueues / The Virtqueue Descriptor Table /Quoted 8 lines
Quoted 6 lines
Indirect
+ Descriptors}, i.e. both the maximum amount of slots in the vring
and
the
+ actual bulk data size transmitted per vring slot.spect does not call these slots elsewhere.
Yes, I intentionally used "vring slot" instead of "buffer" as I find
the
latter too vague in this context. A "buffer" can be a memory segment,
a
set of memory segments and what not. "vring slot" OTOH makes it clear
that it is about exactly one, atomic pointer (hence with fixed size)
in a
Ring Buffer, as depicted in the ASCII illustration here:https://github.com/oasis-tcs/virtio-spec/issues/122
The maximum amount of vring slots is therefore the maximum amount of
messages that can be emplaced into a Ring Buffer, independent of any
"bulk data buffer size".Quoted 97 lines
+
Quoted 31 lines
+ With this feature enabled, the Queue Size only limits the maximum
amount
+ of slots in the vring, but does not limit the actual bulk data
size
+ being transmitted when indirect descriptors are used. Decoupling
these
+ two configuration parameters this way not only allows much larger
bulk
data
+ being transferred per vring slot, but also avoids complicated
synchronization
+ mechanisms if the device only supports a very small amount of
vring
slots. Due
+ to the 16-bit size of a descriptor's "next" field there is still an
absolute
+ limit of $2^{16}$ descriptors per indirect descriptor table.
However
the
+ actual maximum amount supported by either device or driver might be
less,
+ and therefore the bus specific Queue Indirect Size valueQuoted 8 lines
MUST
additionally
+ be negotiated ifVIRTIO_RING_F_INDIRECT_SIZEwas negotiated to
subsequently
+ negotiate the actual amount of maximum indirect descriptors supported
+ by both sides.still not sure what exactly is the value. e.g. in a buffer including
indirect and direct descriptors.+
Quoted 40 lines
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature
Bits}packed-ring.tex +1
-1@@ -195,7 +195,7 @@ \subsection{Scatter-Gather Support} The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virt queue -size. +size unless the VIRTIO_RING_F_INDIRECT_SIZE feature has beennegotiated.
\subsection{Next Flag: Descriptor Chaining} \label{sec:Packed Virtqueues / Next Flag: Descriptor Chaining}
split-ring.tex +0
-0@@ -268,8 +268,12 @@ \subsubsection{IndirectDescriptors}\label{sec:Basic
Facilities of a Virtio Deviset the
VIRTQ_DESC_F_INDIRECTflag within an indirect descriptor
(ie.
only
one table per descriptor).Quoted 4 lines
-A driver MUST NOT create a descriptor chain longer than the Queue
Size of
-the device.+If
VIRTIO_RING_F_INDIRECT_SIZEhas not been negotiated, the driver
MUSTQuoted 11 lines
Quoted 9 lines
+NOT create a descriptor chain longer than the Queue Size of the
device.split-ring.tex +2
-0+ +If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, thenumber of
+descriptors per indirect descriptor table MUST NOT exceed the negotiated
+Queue Indirect Size.it is not negotiated is it?
What makes you think it is not negotiated?
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Also see my previous question here ^
Sorry, what I mean is that you don't define what does negotiation
involve. I think you mean this:
The driver SHOULD write to \field{queue_indirect_size} if its maximum number of descriptors per vring slot is lower than that reported by
the
Quoted 38 lines
Quoted 16 lines
Quoted 8 lines
device.
but driver can just read the value and that's it - and then the value that is set by device applies, right?
If you are going to use terms such as negotiated you need to define what they mean. In this case I would just say something like
"the value of Queue Indirect Size".Which makes me wonder why you just didn't say that in the first place? And I don't agree that it wasn't defined, because I actually think I did:
(quoted diff) +3
-0+ \item[VIRTIO_RING_F_INDIRECT_SIZE(41)] This feature indicates that the + Queue Indirect Size, i.e. the maximum amount of descriptors in indirect + descriptor tables, is independent from the Queue Size."Quoted 2 lines
Or is that definition of the new term "Queue Indirect Size" not clear enough to you?
Maybe ... but I don't think this will jump out to the reader. I feel we abuse of the word negotiate to the point where reader only has a vague idea what it means. That's why I struggled to give concise comments. Consider:
(quoted diff) +2
-0+ and therefore the transport specific Queue Indirect Size value MUST + additionally be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiatedto + subsequently negotiate the actual amount of maximum indirect descriptors + supported by both sides.
sounds like a tongue-twister to me. Can't we find a term different from "negotiate"?
It's already used for features, please find something else for this. And why is a term even necessary? How is this different from other writeable fields? Consider an example of similar functionality from the spec:
Quoted 7 lines
\item[\field{queue_size}]
Queue Size. On reset, specifies the maximum queue size supported by
the device. This can be modified by the driver to reduce memory requirements. A 0 means the queue is unavailable.Hope this helps.
To be honest, I don't feel like discussing precise wordings at this point when you are questioning the proposal on design level already.
Maybe you make some more thorough thoughts on what you actually want this to be on design level before continueing to argue about precise terminology, which you are not using either BTW when you articulating your criticism.
Or even better: come up with your own proposol with the precise wording you feel appropriate.
Best regards,
Christian Schoenebeck
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 9 lines · Christian Schoenebeck
To be honest, I don't feel like discussing precise wordings at this point when you are questioning the proposal on design level already.
Maybe you make some more thorough thoughts on what you actually want this to be on design level before continueing to argue about precise terminology, which you are not using either BTW when you articulating your criticism.
Or even better: come up with your own proposol with the precise wording you feel appropriate.
OK let's go back and agree on what we are trying to achieve. The github issue and the cover letter imply that while indirect descriptors would normally allow huge tables, we artificially limit them to queue size, and you want to be able to relax that.
Fair enough.
However, I feel trying to talk about indirect descriptor is too narrow a use-case, simply because the issue is not indirect at all. Why do we limit number of segments? I think it's really because of backend limitations. And indirect is only used by the frontend. So limiting that is really going about it wrong.
So block for example has seg_max already. What should happen if that exceeds queue size is not defined.
So maybe we can generalize that making it device independent? The litmus paper for this is the block and scsi devices, we should be able to use the new feature as a super-set.
Before we discuss solutions, did I formulate the problem correctly?
Signature · 18 lines
MST This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 9 lines · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
Quoted 8 lines · Christian Schoenebeck
To be honest, I don't feel like discussing precise wordings at this point when you are questioning the proposal on design level already.
Maybe you make some more thorough thoughts on what you actually want this to be on design level before continueing to argue about precise terminology, which you are not using either BTW when you articulating your criticism.
Quoted 8 lines
Quoted 3 lines
Or even better: come up with your own proposol with the precise wording you
feel appropriate.OK let's go back and agree on what we are trying to achieve. The github issue and the cover letter imply that while indirect descriptors would normally allow huge tables, we artificially limit them to queue size, and you want to be able to relax that.
Correct, that's my motivation for all of this.
Quoted 7 lines
Fair enough.
However, I feel trying to talk about indirect descriptor is too narrow a use-case, simply because the issue is not indirect at all. Why do we limit number of segments? I think it's really because of backend limitations. And indirect is only used by the frontend. So limiting that is really going about it wrong.
I am only aware about current implementation situation in QEMU and Linux kernel. As for those two: yes, it is not a limitation on Linux kernel side, but on QEMU side.
As for other implementations: no idea.
Quoted 8 lines
So block for example has
seg_maxalready. What should happen if that exceeds queue size is not defined.So maybe we can generalize that making it device independent? The litmus paper for this is the block and scsi devices, we should be able to use the new feature as a super-set.
Before we discuss solutions, did I formulate the problem correctly?
Keep in mind that I never worked on virtio code or virtio spec before. I just started to review virtio implementation of QEMU and Linux kernel and the virtio spec in November, specifically in context of 9p. I definitely don't know all the other virtioo device classes out there.
In other words: I can't help you on fitting this appropriately into a superset picture.
Best regards,
Christian Schoenebeck
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 14 lines · Christian Schoenebeck
Quoted 13 lines · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
Quoted 11 lines · Christian Schoenebeck
To be honest, I don't feel like discussing precise wordings at this point when you are questioning the proposal on design level already.
Maybe you make some more thorough thoughts on what you actually want this to be on design level before continueing to argue about precise terminology, which you are not using either BTW when you articulating your criticism.
Or even better: come up with your own proposol with the precise wording you
feel appropriate.
Quoted 6 lines
Quoted 4 lines
OK let's go back and agree on what we are trying to achieve. The github issue and the cover letter imply that while indirect descriptors would normally allow huge tables, we artificially limit them to queue size, and you want to be able to relax that.
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
Question:
- I am thinking about bi-directional descriptors such as block and scsi have. it looks like they have a separate limit for read and write parts of the chain. Should we have two limits then? Or should we just make driver use the lower of the two, i.e. the per vq limit applies to the total # of elements in a buffer, and read/write sgs are controlled separately by the per device control?
Stefan, any comments?
Quoted 21 lines
Quoted 7 lines
Fair enough.
However, I feel trying to talk about indirect descriptor is too narrow a use-case, simply because the issue is not indirect at all. Why do we limit number of segments? I think it's really because of backend limitations. And indirect is only used by the frontend. So limiting that is really going about it wrong.
I am only aware about current implementation situation in QEMU and Linux kernel. As for those two: yes, it is not a limitation on Linux kernel side, but on QEMU side.
As for other implementations: no idea.
Quoted 7 lines
So block for example has
seg_maxalready. What should happen if that exceeds queue size is not defined.So maybe we can generalize that making it device independent? The litmus paper for this is the block and scsi devices, we should be able to use the new feature as a super-set.
Quoted 6 lines
Quoted 1 line
Before we discuss solutions, did I formulate the problem correctly?
Keep in mind that I never worked on virtio code or virtio spec before. I just started to review virtio implementation of QEMU and Linux kernel and the virtio spec in November, specifically in context of 9p. I definitely don't know all the other virtioo device classes out there.
I think it's great that we have someone taking care of 9p btw!
Quoted 6 lines
In other words: I can't help you on fitting this appropriately into a superset picture.
Best regards,
Christian Schoenebeck
I think we need some cleanups in the spec to make what you are trying to do possible to do cleanly, specifically move this description:
A buffer consists of zero or more device-readable physically-contiguous elements followed by zero or more physically-contiguous device-writable elements (each buffer has at least one element).
out to the generic part from packed ring part, drop corresponding text from split ring and make it refer to the term "element".
After this, the new field will just be "a number of elements per buffer" (note that indirect descriptors do not themselves describe elements and so won't be included in the math).
Christian, you mentioned you don't like the term buffer generally, changing that can be done before or after this feature but IMHO best not as part of it.
I think it's good in that it will fit better in the superset picture addressing in addition to your requirement also the requirement to have huge rings while limiting descriptors.
Does above sound like it addresses your requirements of having a longer descriptor chain than queue size? If not what is not addressed?
Thanks,
Signature · 1 line
MST
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 23 lines · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
Quoted 22 lines · Christian Schoenebeck
Quoted 20 lines · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
Quoted 10 lines · Christian Schoenebeck
To be honest, I don't feel like discussing precise wordings at this point
when you are questioning the proposal on design level already.Maybe you make some more thorough thoughts on what you actually want this
to be on design level before continueing to argue about precise terminology, which you are not using either BTW when you articulating your criticism.Quoted 4 lines
Or even better: come up with your own proposol with the precise wording
you
feel appropriate.OK let's go back and agree on what we are trying to achieve. The github issue and the cover letter imply that while indirect descriptors would normally allow huge tables, we artificially limit them to queue size, and you want to be able to relax that.
Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size. C. Your use case: forcing chain (within FIFOs) length < queue size.
[1] https://lore.kernel.org/netdev/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
However it would not cover:
D. Your other use case: blocking indirect tables.
E. Potential other people's need: max. chain length (within FIFOs) != max.
indirect table length.
It's not clear to me though why you would want to exclude the descriptor pointing to a table from counting towards that limit.
Instead of yet again mixing different limits again with each other, what about using 3 distinct fields for those limits instead:
1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from FIFOs and indirect tables, including descriptor pointing to a table).
As a side note: the spec currently refers to "table of indirect descriptors", hence my previous assumption that descriptors in such a table should be called "indirect descriptors", whereas you are apparently assuming that an "indirect descriptor" is only that single one pointing to a table and you see the descriptors in the table as "direct" ones I guess.
Quoted 10 lines
Question:
- I am thinking about bi-directional descriptors such as block and scsi have. it looks like they have a separate limit for read and write parts of the chain. Should we have two limits then? Or should we just make driver use the lower of the two, i.e. the per vq limit applies to the total # of elements in a buffer, and read/write sgs are controlled separately by the per device control?Stefan, any comments?
Leaving that question to Stefan. No opinion from my side at this point.
Quoted 22 lines
Quoted 22 lines
Quoted 7 lines
Fair enough.
However, I feel trying to talk about indirect descriptor is too narrow a use-case, simply because the issue is not indirect at all. Why do we limit number of segments? I think it's really because of backend limitations. And indirect is only used by the frontend. So limiting that is really going about it wrong.
I am only aware about current implementation situation in QEMU and Linux kernel. As for those two: yes, it is not a limitation on Linux kernel side,
but on QEMU side.As for other implementations: no idea.
Quoted 7 lines
So block for example has
seg_maxalready. What should happen if that exceeds queue size is not defined.So maybe we can generalize that making it device independent? The litmus paper for this is the block and scsi devices, we should be able to use the new feature as a super-set.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Before we discuss solutions, did I formulate the problem correctly?
Keep in mind that I never worked on virtio code or virtio spec before. I just started to review virtio implementation of QEMU and Linux kernel and the virtio spec in November, specifically in context of 9p. I definitely don't know all the other virtioo device classes out there.
I think it's great that we have someone taking care of 9p btw!
Thanks!
Quoted 16 lines
Quoted 5 lines
In other words: I can't help you on fitting this appropriately into a superset picture.
Best regards,
Christian SchoenebeckI think we need some cleanups in the spec to make what you are trying to do possible to do cleanly, specifically move this description:
A buffer consists of zero or more device-readable physically-contiguous elements followed by zero or more physically-contiguous device-writable elements (each buffer has at least one element).
out to the generic part from packed ring part, drop corresponding text from split ring and make it refer to the term "element".
Quoted 7 lines
After this, the new field will just be "a number of elements per buffer" (note that indirect descriptors do not themselves describe elements and so won't be included in the math).
Christian, you mentioned you don't like the term buffer generally, changing that can be done before or after this feature but IMHO best not as part of it.
For pragmatic reasons I will refrain from questioning any virtio terms in foreseeable future and will just use the ones suggested by people.
Quoted 9 lines
I think it's good in that it will fit better in the superset picture addressing in addition to your requirement also the requirement to have huge rings while limiting descriptors.
Does above sound like it addresses your requirements of having a longer descriptor chain than queue size? If not what is not addressed?
Thanks,
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 38 lines · Christian Schoenebeck
Quoted 23 lines · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
Quoted 22 lines · Christian Schoenebeck
Quoted 20 lines · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
Quoted 10 lines · Christian Schoenebeck
To be honest, I don't feel like discussing precise wordings at this point
when you are questioning the proposal on design level already.Maybe you make some more thorough thoughts on what you actually want this
to be on design level before continueing to argue about precise terminology, which you are not using either BTW when you articulating your criticism.Quoted 4 lines
Or even better: come up with your own proposol with the precise wording
you
feel appropriate.OK let's go back and agree on what we are trying to achieve. The github issue and the cover letter imply that while indirect descriptors would normally allow huge tables, we artificially limit them to queue size, and you want to be able to relax that.
Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size. C. Your use case: forcing chain (within FIFOs) length < queue size.
Quoted 6 lines
[1] https://lore.kernel.org/netdev/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/However it would not cover:
D. Your other use case: blocking indirect tables.
Well, the practical use-case I have in mind is actually for when using mergeable buffers with the network device, in practice there's never a s/g but there's no way for device to know that. So maybe we can declare that indirect must not be used when there's a single descriptor in the table (esp when the new feature is accepted), and then just limiting s/g to 1 will do the trick in this specific case.
Quoted 2 lines
E. Potential other people's need: max. chain length (within FIFOs) != max.
indirect table length.
I don't understand this one, sorry.
Quoted 2 lines
It's not clear to me though why you would want to exclude the descriptor pointing to a table from counting towards that limit.
Because it's a transient thing. It does not need to be processed with a packet, it's just a way of supplying a longer s/g. In other words what devices care about is the s/g list length, bot how the s/g is specified.
I think what QEMU and Linux both do is very typical:
the indirect descriptor is used to fetch the table, but then what QEMU cares about is whether it will overflow 1024 entries when preparing the iovec for linux.
Quoted 7 lines
Instead of yet again mixing different limits again with each other, what about using 3 distinct fields for those limits instead:
1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from FIFOs and indirect tables, including descriptor pointing to a table).
So this doesn't actually address at least what you called "your use-case" above. Well it almost does, but not exactly because of the extra indirect descriptor.
It's not that I'm against this kind of thing but personally I'd like a bit more than a theoretical argument for why all the extra complexity is helpful - remember that all that needs to be coded in the driver, and any mistakes often tend to paint us in the corner because of the need to support existing drivers.
Quoted 5 lines
As a side note: the spec currently refers to "table of indirect descriptors", hence my previous assumption that descriptors in such a table should be called "indirect descriptors", whereas you are apparently assuming that an "indirect descriptor" is only that single one pointing to a table and you see the descriptors in the table as "direct" ones I guess.
Oh. You are right. There's no special name for what I mistakenly called an indirect descriptor. So what I called "direct descriptors" are really "read/write descriptors". It might make sense to add some text calling these something. Scatter/gather descriptors maybe?
And what I called "indirect descriptor" is really just descriptor with INDIRECT flag set. I am not sure we need to call these anything unless we really start counting it.
Quoted 66 lines
Quoted 10 lines
Question:
- I am thinking about bi-directional descriptors such as block and scsi have. it looks like they have a separate limit for read and write parts of the chain. Should we have two limits then? Or should we just make driver use the lower of the two, i.e. the per vq limit applies to the total # of elements in a buffer, and read/write sgs are controlled separately by the per device control?Stefan, any comments?
Leaving that question to Stefan. No opinion from my side at this point.
Quoted 22 lines
Quoted 22 lines
Quoted 7 lines
Fair enough.
However, I feel trying to talk about indirect descriptor is too narrow a use-case, simply because the issue is not indirect at all. Why do we limit number of segments? I think it's really because of backend limitations. And indirect is only used by the frontend. So limiting that is really going about it wrong.
I am only aware about current implementation situation in QEMU and Linux kernel. As for those two: yes, it is not a limitation on Linux kernel side,
but on QEMU side.As for other implementations: no idea.
Quoted 7 lines
So block for example has
seg_maxalready. What should happen if that exceeds queue size is not defined.So maybe we can generalize that making it device independent? The litmus paper for this is the block and scsi devices, we should be able to use the new feature as a super-set.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Before we discuss solutions, did I formulate the problem correctly?
Keep in mind that I never worked on virtio code or virtio spec before. I just started to review virtio implementation of QEMU and Linux kernel and the virtio spec in November, specifically in context of 9p. I definitely don't know all the other virtioo device classes out there.
I think it's great that we have someone taking care of 9p btw!
Thanks!
Quoted 20 lines
Quoted 5 lines
In other words: I can't help you on fitting this appropriately into a superset picture.
Best regards,
Christian SchoenebeckI think we need some cleanups in the spec to make what you are trying to do possible to do cleanly, specifically move this description:
A buffer consists of zero or more device-readable physically-contiguous elements followed by zero or more physically-contiguous device-writable elements (each buffer has at least one element).
out to the generic part from packed ring part, drop corresponding text from split ring and make it refer to the term "element".
After this, the new field will just be "a number of elements per buffer" (note that indirect descriptors do not themselves describe elements and so won't be included in the math).
Quoted 6 lines
Quoted 3 lines
Christian, you mentioned you don't like the term buffer generally, changing that can be done before or after this feature but IMHO best not as part of it.
For pragmatic reasons I will refrain from questioning any virtio terms in foreseeable future and will just use the ones suggested by people.
It's not that the terminology is so great, but we do have used it for a while and by now it's quite entrenched.
Quoted 10 lines
Quoted 9 lines
I think it's good in that it will fit better in the superset picture addressing in addition to your requirement also the requirement to have huge rings while limiting descriptors.
Does above sound like it addresses your requirements of having a longer descriptor chain than queue size? If not what is not addressed?
Thanks,
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 5 lines · On Mittwoch, 23. März 2022 13:35:00 CET Michael S. Tsirkin wrote:
Quoted 4 lines · Christian Schoenebeck
Quoted 3 lines · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
On Sun, Mar 20, 2022 at 06:43:53PM +0100, Christian Schoenebeck
Quoted 38 lines · wrote:
Quoted 22 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
OK let's go back and agree on what we are trying to achieve. The github
issue and the cover letter imply that while indirect descriptors would
normally allow huge tables, we artificially limit them to queue size,
and you want to be able to relax that.Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size. C. Your use case: forcing chain (within FIFOs) length < queue size.
Quoted 8 lines
[1]
https://lore.kernel.org/netdev/[email protected]
om/ [2]
https://lore.kernel.org/all/[email protected]/However it would not cover:
D. Your other use case: blocking indirect tables.
Well, the practical use-case I have in mind is actually for when using mergeable buffers with the network device, in practice there's never a s/g but there's no way for device to know that. So maybe we can declare that indirect must not be used when there's a single descriptor in the table (esp when the new feature is accepted), and then just limiting s/g to 1 will do the trick in this specific case.
Quoted 5 lines
Quoted 3 lines
E. Potential other people's need: max. chain length (within FIFOs) != max.
indirect table length.
I don't understand this one, sorry.
Example: say you would want to allow large indirect tables, but OTOH want to block any chained (pre-allocated) direct descriptors in the vring FIFOs themselfes, e.g.:
Queue Size (max. # "buffers" in FIFO) = 127
Indirect Size (max. # segments in indirect descr. table) = 65535 Direct Size (max. # chained descr. per "buffer" in FIFO) = 1 Total Size (max. # sum of all descr. per "buffer") = 65536
Your suggestion would not cover this example. Why would somebody want that? Because of device/driver simplification (i.e. same motivation you apparently had in mind for blocking indirect tables for network devices) and to prevent one side from starving because other side filled entire FIFO with large bulk data for a single message.
Quoted 9 lines
Quoted 8 lines
Instead of yet again mixing different limits again with each other, what about using 3 distinct fields for those limits instead:
1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from FIFOs>
and indirect tables, including descriptor pointing to a table).
Quoted 3 lines
So this doesn't actually address at least what you called "your use-case" above. Well it almost does, but not exactly because of the extra indirect descriptor.
Of course it would cover it. And of course you would have to either subtract or add one descriptor somewhere, no matter which solution is chosen in the end.
Best regards,
Christian Schoenebeck
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 55 lines · Christian Schoenebeck
Quoted 5 lines · On Mittwoch, 23. März 2022 13:35:00 CET Michael S. Tsirkin wrote:
Quoted 4 lines · Christian Schoenebeck
Quoted 3 lines · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
On Sun, Mar 20, 2022 at 06:43:53PM +0100, Christian Schoenebeck
Quoted 38 lines · wrote:
Quoted 22 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
OK let's go back and agree on what we are trying to achieve. The github
issue and the cover letter imply that while indirect descriptors would
normally allow huge tables, we artificially limit them to queue size,
and you want to be able to relax that.Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size. C. Your use case: forcing chain (within FIFOs) length < queue size.
Quoted 8 lines
[1]
https://lore.kernel.org/netdev/[email protected]
om/ [2]
https://lore.kernel.org/all/[email protected]/However it would not cover:
D. Your other use case: blocking indirect tables.
Well, the practical use-case I have in mind is actually for when using mergeable buffers with the network device, in practice there's never a s/g but there's no way for device to know that. So maybe we can declare that indirect must not be used when there's a single descriptor in the table (esp when the new feature is accepted), and then just limiting s/g to 1 will do the trick in this specific case.
Quoted 5 lines
Quoted 3 lines
E. Potential other people's need: max. chain length (within FIFOs) != max.
indirect table length.
I don't understand this one, sorry.
Example: say you would want to allow large indirect tables, but OTOH want to block any chained (pre-allocated) direct descriptors in the vring FIFOs themselfes, e.g.:
Quoted 7 lines
Queue Size (max. # "buffers" in FIFO) = 127
Indirect Size (max. # segments in indirect descr. table) = 65535 Direct Size (max. # chained descr. per "buffer" in FIFO) = 1 Total Size (max. # sum of all descr. per "buffer") = 65536Your suggestion would not cover this example. Why would somebody want that? Because of device/driver simplification
It does not simplify the driver, does it? It's always an extra resriction on it.
Quoted 2 lines
(i.e. same motivation you apparently
had in mind for blocking indirect tables for network devices)
OK I got this one. Device writers do complain about the complexity. However the tradeoff in limiting what device accepts is ability to migrate to a different device. So there's an advantage to making this a separate feature bit so that it's clear to people it's a low-end device.
Quoted 3 lines
and to prevent
one side from starving because other side filled entire FIFO with large bulk data for a single message.
This last I don't see why would it be the device's problem. If driver will starve the device it can always do so, if it wants to use indirect aggressively to avoid filling up the queue it can do so too.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Instead of yet again mixing different limits again with each other, what about using 3 distinct fields for those limits instead:
1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from FIFOs>
and indirect tables, including descriptor pointing to a table).
Quoted 7 lines
Quoted 3 lines
So this doesn't actually address at least what you called "your use-case" above. Well it almost does, but not exactly because of the extra indirect descriptor.
Of course it would cover it.
And of course you would have to either subtract
or add one descriptor somewhere,
Well, indirect descriptors can be chained with VIRTQ_DESC_F_NEXT right? The number of in the chain is not really predictable, it's not always 1.
Quoted 5 lines
no matter which solution is chosen in the
end.Best regards,
Christian Schoenebeck
Signature · 1 line
MST
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 5 lines · On Donnerstag, 24. März 2022 11:36:50 CET Michael S. Tsirkin wrote:
Quoted 4 lines · Christian Schoenebeck
Quoted 3 lines · On Mittwoch, 23. März 2022 13:35:00 CET Michael S. Tsirkin wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
On Mon, Mar 21, 2022 at 10:23:07AM +0100, Christian Schoenebeck
Quoted 60 lines · wrote:
Quoted 60 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
Quoted 1 line
On Sun, Mar 20, 2022 at 06:43:53PM +0100, Christian Schoenebeck
Quoted 40 lines · wrote:
Quoted 29 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
OK let's go back and agree on what we are trying to achieve. The
github
issue and the cover letter imply that while indirect descriptors would
normally allow huge tables, we artificially limit them to queue size,
and you want to be able to relax that.Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size. C. Your use case: forcing chain (within FIFOs) length < queue size.
[1]
https://lore.kernel.org/netdev/cover.1640870037.git.linux_oss@crudebyte.c
om/ [2]
https://lore.kernel.org/all/[email protected]/Quoted 3 lines
However it would not cover:
D. Your other use case: blocking indirect tables.
Well, the practical use-case I have in mind is actually for when using mergeable buffers with the network device, in practice there's never a s/g but there's no way for device to know that. So maybe we can declare that indirect must not be used when there's a single descriptor in the table (esp when the new feature is accepted), and then just limiting s/g to 1 will do the trick in this specific case.
Quoted 6 lines
Quoted 4 lines
E. Potential other people's need: max. chain length (within FIFOs) != max.
indirect table length.
I don't understand this one, sorry.
Example: say you would want to allow large indirect tables, but OTOH want to block any chained (pre-allocated) direct descriptors in the vring FIFOs>
themselfes, e.g.:
Queue Size (max. # "buffers" in FIFO) = 127
Indirect Size (max. # segments in indirect descr. table) = 65535 Direct Size (max. # chained descr. per "buffer" in FIFO) = 1 Total Size (max. # sum of all descr. per "buffer") = 65536
Quoted 6 lines
Quoted 3 lines
Your suggestion would not cover this example. Why would somebody want that?
Because of device/driver simplificationIt does not simplify the driver, does it? It's always an extra resriction on it.
Why not? If it's the driver that sets Direct Size = 1 then it can ignore handling chained direct descriptors. Sounds like a simplification to me.
Quoted 9 lines
Quoted 2 lines
(i.e. same motivation you apparently
had in mind for blocking indirect tables for network devices)OK I got this one. Device writers do complain about the complexity. However the tradeoff in limiting what device accepts is ability to migrate to a different device. So there's an advantage to making this a separate feature bit so that it's clear to people it's a low-end device.
Quoted 7 lines
Quoted 3 lines
and to prevent
one side from starving because other side filled entire FIFO with large bulk data for a single message.This last I don't see why would it be the device's problem. If driver will starve the device it can always do so, if it wants to use indirect aggressively to avoid filling up the queue it can do so too.
To actually make it possible to enforce such kind of policy: "do not use the FIFO for large bulk data, use indirect space for that instead".
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
Instead of yet again mixing different limits again with each other, what
about using 3 distinct fields for those limits instead:1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from
FIFOs>and indirect tables, including descriptor pointing to a table).
So this doesn't actually address at least what you called "your use-case" above. Well it almost does, but not exactly because of the extra indirect descriptor.
Quoted 7 lines
Quoted 3 lines
Of course it would cover it.
And of course you would have to either subtract
or add one descriptor somewhere,Well, indirect descriptors can be chained with
VIRTQ_DESC_F_NEXTright? The number of in the chain is not really predictable, it's not always 1.
I don't think so:
2.6.5.3.1 Driver Requirements: Indirect Descriptors
..
"A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in flags."
So as far as I can see it, the amount of direct descriptors is currently always exactly one if an indirect table is used.
Best regards,
Christian Schoenebeck
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 71 lines · Christian Schoenebeck
Quoted 5 lines · On Donnerstag, 24. März 2022 11:36:50 CET Michael S. Tsirkin wrote:
Quoted 4 lines · Christian Schoenebeck
Quoted 3 lines · On Mittwoch, 23. März 2022 13:35:00 CET Michael S. Tsirkin wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
On Mon, Mar 21, 2022 at 10:23:07AM +0100, Christian Schoenebeck
Quoted 64 lines · wrote:
Quoted 63 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin wrote:
Quoted 1 line
On Sun, Mar 20, 2022 at 06:43:53PM +0100, Christian Schoenebeck
Quoted 40 lines · wrote:
Quoted 29 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
OK let's go back and agree on what we are trying to achieve. The
github
issue and the cover letter imply that while indirect descriptors would
normally allow huge tables, we artificially limit them to queue size,
and you want to be able to relax that.Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size. C. Your use case: forcing chain (within FIFOs) length < queue size.
[1]
https://lore.kernel.org/netdev/cover.1640870037.git.linux_oss@crudebyte.c
om/ [2]
https://lore.kernel.org/all/[email protected]/Quoted 3 lines
However it would not cover:
D. Your other use case: blocking indirect tables.
Well, the practical use-case I have in mind is actually for when using mergeable buffers with the network device, in practice there's never a s/g but there's no way for device to know that. So maybe we can declare that indirect must not be used when there's a single descriptor in the table (esp when the new feature is accepted), and then just limiting s/g to 1 will do the trick in this specific case.
Quoted 6 lines
Quoted 4 lines
E. Potential other people's need: max. chain length (within FIFOs) != max.
indirect table length.
I don't understand this one, sorry.
Example: say you would want to allow large indirect tables, but OTOH want to block any chained (pre-allocated) direct descriptors in the vring FIFOs>
themselfes, e.g.:
Queue Size (max. # "buffers" in FIFO) = 127
Indirect Size (max. # segments in indirect descr. table) = 65535 Direct Size (max. # chained descr. per "buffer" in FIFO) = 1 Total Size (max. # sum of all descr. per "buffer") = 65536Your suggestion would not cover this example. Why would somebody want that?
Because of device/driver simplification
Quoted 5 lines
Quoted 2 lines
It does not simplify the driver, does it? It's always an extra resriction on it.
Why not? If it's the driver that sets Direct Size = 1 then it can ignore handling chained direct descriptors. Sounds like a simplification to me.
If driver does not want to chain descriptors it does not have to. There's no need to device to forbid it from doing so.
Quoted 13 lines
Quoted 13 lines
Quoted 2 lines
(i.e. same motivation you apparently
had in mind for blocking indirect tables for network devices)OK I got this one. Device writers do complain about the complexity. However the tradeoff in limiting what device accepts is ability to migrate to a different device. So there's an advantage to making this a separate feature bit so that it's clear to people it's a low-end device.
Quoted 3 lines
and to prevent
one side from starving because other side filled entire FIFO with large bulk data for a single message.
Quoted 6 lines
Quoted 3 lines
This last I don't see why would it be the device's problem. If driver will starve the device it can always do so, if it wants to use indirect aggressively to avoid filling up the queue it can do so too.
To actually make it possible to enforce such kind of policy: "do not use the FIFO for large bulk data, use indirect space for that instead".
Why does device want to enforce this policy? Policy is generally something best left to guests they know how device is used.
Quoted 33 lines
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
Instead of yet again mixing different limits again with each other, what
about using 3 distinct fields for those limits instead:1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from
FIFOs>and indirect tables, including descriptor pointing to a table).
So this doesn't actually address at least what you called "your use-case" above. Well it almost does, but not exactly because of the extra indirect descriptor.
Quoted 7 lines
Quoted 3 lines
Of course it would cover it.
And of course you would have to either subtract
or add one descriptor somewhere,Well, indirect descriptors can be chained with
VIRTQ_DESC_F_NEXTright? The number of in the chain is not really predictable, it's not always 1.I don't think so:
2.6.5.3.1 Driver Requirements: Indirect Descriptors
..
"A driver MUST NOT set bothVIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin flags."
So as far as I can see it, the amount of direct descriptors is currently always exactly one if an indirect table is used.
Quoted 3 lines
Best regards,
Christian Schoenebeck
Oh. You are right. Weirdly this text is not in packed-ring.tex - I think this and a bunch of other cases like this are an oversight, however we need to fix them first before adding features that assume they are fixed ...
Signature · 1 line
MST
Re: [virtio-comment] [PATCH v3 1/4] Add VIRTIO_RING_F_INDIRECT_SIZE
Quoted 5 lines · On Donnerstag, 24. März 2022 12:16:51 CET Michael S. Tsirkin wrote:
Quoted 4 lines · Christian Schoenebeck
Quoted 3 lines · On Donnerstag, 24. März 2022 11:36:50 CET Michael S. Tsirkin wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Mittwoch, 23. März 2022 13:35:00 CET Michael S. Tsirkin wrote:
On Wed, Mar 23, 2022 at 11:20:07AM +0100, Christian Schoenebeck
Quoted 5 lines · wrote:
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · On Montag, 21. März 2022 23:13:55 CET Michael S. Tsirkin wrote:
Quoted 1 line
On Mon, Mar 21, 2022 at 10:23:07AM +0100, Christian Schoenebeck
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sonntag, 20. März 2022 22:52:16 CET Michael S. Tsirkin
Quoted 105 lines · wrote:
Quoted 78 lines
Quoted 78 lines
Quoted 77 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Sun, Mar 20, 2022 at 06:43:53PM +0100, Christian Schoenebeck
Quoted 51 lines · wrote:
Quoted 35 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
OK let's go back and agree on what we are trying to achieve. The
github
issue and the cover letter imply that while indirect descriptors
would
normally allow huge tables, we artificially limit them to queue
size,
and you want to be able to relax that.Quoted 7 lines
Quoted 1 line
Correct, that's my motivation for all of this.
Okay. So I think that given this, we can limit the total number of non-indirect descriptors, including non-indirect ones in a chain + all the ones in indirect pointer table if any, and excluding the indirect descriptor itself, and this will address the issue you are describing here, right?
As far as I understand your suggestion, yes, it would cover:
A. My use case [1]: allowing indirect table length > queue size. B. Stefan's use case [2]: forcing indirect table length < queue size.
C. Your use case: forcing chain (within FIFOs) length < queue size.[1]
https://lore.kernel.org/netdev/cover.1640870037.git.linux_oss@crud
ebyte.c
om/ [2]
https://lore.kernel.org/all/[email protected]
in/Quoted 3 lines
However it would not cover:
D. Your other use case: blocking indirect tables.
Well, the practical use-case I have in mind is actually for when using
mergeable buffers with the network device, in practice there's never a
s/g but there's no way for device to know that. So maybe we can declare
that indirect must not be used when there's a single descriptor in the
table (esp when the new feature is accepted), and then just limiting s/g
to 1 will do the trick in this specific case.Quoted 7 lines
Quoted 5 lines
E. Potential other people's need: max. chain length (within FIFOs) !=
max.indirect table length.
I don't understand this one, sorry.
Example: say you would want to allow large indirect tables, but OTOH want
to block any chained (pre-allocated) direct descriptors in the vring FIFOs>themselfes, e.g.:
Queue Size (max. # "buffers" in FIFO) = 127
Indirect Size (max. # segments in indirect descr. table) = 65535 Direct Size (max. # chained descr. per "buffer" in FIFO) = 1 Total Size (max. # sum of all descr. per "buffer") = 65536Your suggestion would not cover this example. Why would somebody want that?
Because of device/driver simplificationQuoted 5 lines
Quoted 2 lines
It does not simplify the driver, does it? It's always an extra resriction on it.
Why not? If it's the driver that sets Direct Size = 1 then it can ignore handling chained direct descriptors. Sounds like a simplification to me.
If driver does not want to chain descriptors it does not have to. There's no need to device to forbid it from doing so.
Quoted 18 lines
Quoted 17 lines
Quoted 2 lines
(i.e. same motivation you apparently
had in mind for blocking indirect tables for network devices)OK I got this one. Device writers do complain about the complexity. However the tradeoff in limiting what device accepts is ability to migrate to a different device. So there's an advantage to making this a separate feature bit so that it's clear to people it's a low-end device.
Quoted 4 lines
and to prevent
one side from starving because other side filled entire FIFO with large
bulk data for a single message.This last I don't see why would it be the device's problem. If driver will starve the device it can always do so, if it wants to use indirect aggressively to avoid filling up the queue it can do so too.
Quoted 5 lines
Quoted 2 lines
To actually make it possible to enforce such kind of policy: "do not use the FIFO for large bulk data, use indirect space for that instead".
Why does device want to enforce this policy? Policy is generally something best left to guests they know how device is used.
I'm just saying. Period. It's not that I care. In the end I leave it to other people to fit this into an appropriate superset picture, caring about future proofness, etc.
As long as "my" use case (indirect size > queue size) will be covered, then I will be fine with any design. #pragmatism
Quoted 32 lines
Quoted 32 lines
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 5 lines
Instead of yet again mixing different limits again with each other,
what
about using 3 distinct fields for those limits instead:Quoted 7 lines
1. max. indirect table length (any descriptor in the table)
2. max. chain length (currently: descriptors within FIFOs)
3. max. total descriptors per buffer (sum of all descriptors, both from
FIFOs>and indirect tables, including descriptor pointing to a table).
So this doesn't actually address at least what you called "your use-case" above. Well it almost does, but not exactly because of the extra indirect descriptor.
Quoted 7 lines
Quoted 3 lines
Of course it would cover it.
And of course you would have to either subtract or add one descriptor somewhere,Well, indirect descriptors can be chained with
VIRTQ_DESC_F_NEXTright? The number of in the chain is not really predictable, it's not always 1.I don't think so:
2.6.5.3.1 Driver Requirements: Indirect Descriptors
..
"A driver MUST NOT set bothVIRTQ_DESC_F_INDIRECTandVIRTQ_DESC_F_NEXTin
flags."
Quoted 7 lines
Quoted 5 lines
So as far as I can see it, the amount of direct descriptors is currently always exactly one if an indirect table is used.
Best regards,
Christian SchoenebeckOh. You are right. Weirdly this text is not in packed-ring.tex - I
As far as I can see it, you both originally added this normative requirement to content.tex with 3b676c2f (Wed Apr 29 11:09:04 2015) and then you moved it to split-ring.tex with 6131f8d9 (Fri Mar 9 23:23:30 2018).
Quoted 3 lines
think this and a bunch of other cases like this are an oversight, however we need to fix them first before adding features that assume they are fixed ...
Makes sense.
Feel free to resume this discussion when you think it is time to.
It would be good to see other people's opinion on this overall issue as well though.
Thanks!
Best regards,
Christian Schoenebeck
PATCH v3 2/4 Add PCI configuration field "queue_indirect_size"
In reply to Christian SchoenebeckThis new PCI configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
--- content.tex | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)
content.tex +22 -4
@@ -902,6 +902,7 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport le64 queue_device; /* read-write */ le16 queue_notify_data; /* read-only for driver */ le16 queue_reset; /* read-write */ + le32 queue_indirect_size; /* read-write */ }; \end{lstlisting}
@@ -987,6 +988,17 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport This field exists only if VIRTIO_F_RING_RESET has been negotiated. (see \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}). +\item[\field{queue_indirect_size}] + This field is used to negotiate the maximum number of descriptors per + indirect descriptor table as in \ref{sec:Basic Facilities of a Virtio + Device / Virtqueues / The Virtqueue Descriptor Table / Indirect + Descriptors} if and only if the VIRTIO_RING_F_INDIRECT_SIZE feature has + been negotiated. + + The device specifies its maximum number of descriptors per indirect + descriptor table. If the driver requires fewer descriptors, it writes + its lower value to inform the device of the reduced resource + requirements. \end{description} \devicenormative{\paragraph}{Common configuration structure layout}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Common configuration structure layout}
@@ -1081,6 +1093,12 @@ \subsubsection{Common configuration structure layout}\label{sec:Virtio Transport values to different values than those that were used before the queue reset. (see \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Reset}). +The driver SHOULD write to \field{queue_indirect_size} if its maximum number of +descriptors per vring slot is lower than that reported by the device. + +The driver MUST NOT write a higher value to \field{queue_indirect_size} than the +one it reads from the device. + \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability} The notification location is found using the VIRTIO_PCI_CAP_NOTIFY_CFG
@@ -6870,10 +6888,10 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} to the 16-bit size of a descriptor's "next" field there is still an absolute limit of $2^{16}$ descriptors per indirect descriptor table. However the actual maximum amount supported by either device or driver might be less, - and therefore the bus specific Queue Indirect Size value MUST additionally - be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to subsequently - negotiate the actual amount of maximum indirect descriptors supported - by both sides. + and therefore the transport specific Queue Indirect Size value MUST + additionally be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to + subsequently negotiate the actual amount of maximum indirect descriptors + supported by both sides. \end{description}
Signature · 1 line
2.30.2
Re: [virtio-comment] [PATCH v3 2/4] Add PCI configuration field "queue_indirect_size"
Quoted 22 lines · On Mittwoch, 16. März 2022 14:50:22 CET Christian Schoenebeck wrote:
This new PCI configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Signed-off-by Christian Schoenebeck <[email protected]>Reviewed-by Stefan Hajnoczi <[email protected]>--- content.tex | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)content.tex +0
-0@@ -902,6 +902,7 @@ \subsubsection{Common configuration structurelayout}\label{sec:Virtio Transport le64
queue_device; /*
read-write */le16 queue_notify_data; /* read-only for driver */ le16 queue_reset; /* read-write */ + le32 queue_indirect_size; /* read-write */};
\end{lstlisting}
[...]
content.tex \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} · line 6894
Quoted 12 lines
content.tex +4
-4@@ -6870,10 +6888,10 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} to the 16-bit size of a descriptor's "next" field there is still an absolute limit of $2^{16}$ descriptors per indirect descriptor table. However the actual maximum amount supported by either device or driver might be less, - and therefore the bus specific Queue Indirect Size value MUST additionally - be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to subsequently - negotiate the actual amount of maximum indirect descriptors supported - by both sides. + and therefore the transport specific Queue Indirect Size value MUST + additionally be negotiated if VIRTIO_RING_F_INDIRECT_SIZE was negotiated to + subsequently negotiate the actual amount of maximum indirect descriptors + supported by both sides.
Oops, I accidentally squashed that change (s/bus/transport/) into this PCI patch instead of patch 1 as it was supposed to.
I'll wait though if there is still something to address on CCW patch 4 before posting a v4.
Best regards,
Christian Schoenebeck
PATCH v3 3/4 Add MMIO configuration register "QueueIndirectNum"
In reply to Christian SchoenebeckThis new MMIO configuration register allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
--- content.tex | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
content.tex +20 -2
@@ -2225,8 +2225,9 @@ \subsection{Legacy interface}\label{sec:Virtio Transport Options / Virtio Over M \hline \mmioreg{QueueSel}{Virtual queue index}{0x030}{W}{% Writing to this register selects the virtual queue that the - following operations on the \field{QueueNumMax}, \field{QueueNum}, \field{QueueAlign} - and \field{QueuePFN} registers apply to. The index + following operations on the \field{QueueNumMax}, \field{QueueNum}, + \field{QueueAlign}, \field{QueuePFN} and \field{QueueIndirectNum} registers + apply to. The index number of the first queue is zero (0x0). . }
@@ -2282,6 +2283,19 @@ \subsection{Legacy interface}\label{sec:Virtio Transport Options / Virtio Over M Also see \ref{sec:General Initialization And Device Operation / Device Initialization}~\nameref{sec:General Initialization And Device Operation / Device Initialization}. } \hline + \mmioreg{QueueIndirectNum}{Virtual queue indirect size}{0x074}{RW}{% + This register is used to negotiate the maximum number of descriptors per + indirect descriptor table as in \ref{sec:Basic Facilities of a Virtio + Device / Virtqueues / The Virtqueue Descriptor Table / Indirect Descriptors} + if and only if the VIRTIO_RING_F_INDIRECT_SIZE feature has been negotiated. + + The device specifies its maximum number of descriptors per indirect + descriptor table. If the driver requires fewer descriptors, it writes + its lower value to inform the device of the reduced resource requirements. + + This applies to the queue selected by writing to \field{QueueSel}. + } + \hline \mmioreg{Config}{Configuration space}{0x100+}{RW}{} \hline \end{longtable}
@@ -2314,6 +2328,10 @@ \subsection{Legacy interface}\label{sec:Virtio Transport Options / Virtio Over M \item Notify the device about the queue size by writing the size to \field{QueueNum}. +\item If VIRTIO_RING_F_INDIRECT_SIZE has been negotiated, read the maximum + length of indirect descriptor tables by reading \field{QueueIndirectNum}. + Write to lower this value if necessary. + \item Notify the device about the used alignment by writing its value in bytes to \field{QueueAlign}.
Signature · 1 line
2.30.2
PATCH v3 4/4 Add CCW configuration field "indirect_num"
In reply to Christian SchoenebeckThis new CCW configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Bump CCW virtio revision to 3 and make the existence of this new field "indirect_num" dependant on revision 3.
--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
content.tex +13 -1
@@ -2520,7 +2520,9 @@ \subsubsection{Setting the Virtio Revision}\label{sec:Virtio Transport Options / \hline 2 & 0 & <empty> & CCW_CMD_READ_STATUS support \\ \hline -3-n & & & reserved for later revisions \\ +3 & 0 & <empty> & Queue Indirect Size (indirect_num) support \\ +\hline +4-n & & & reserved for later revisions \\ \hline \end{tabular}
@@ -2581,12 +2583,17 @@ \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir struct vq_config_block { be16 index; be16 max_num; + be32 indirect_num; /* since virtio-ccw rev. 3 */ }; \end{lstlisting} The requested number of buffers for queue \field{index} is returned in \field{max_num}. +Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue. + Afterwards, CCW_CMD_SET_VQ is issued by the driver to inform the device about the location used for its queue. The transmitted structure is
@@ -2599,6 +2606,7 @@ \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir be16 num; be64 driver; be64 device; + be32 indirect_num; /* since virtio-ccw rev. 3 */ }; \end{lstlisting}
@@ -2607,6 +2615,10 @@ \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir available area and used area for queue \field{index}, respectively. The actual virtqueue size (number of allocated buffers) is transmitted in \field{num}. +Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by driver for this queue. + \devicenormative{\paragraph}{Configuring a Virtqueue}{Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue} \field{res0} is reserved and MUST be ignored by the device.
Signature · 1 line
2.30.2
[virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 12 lines · Christian Schoenebeck
This new CCW configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Bump CCW virtio revision to 3 and make the existence of this new field "
indirect_num" dependant on revision 3.Signed-off-by Christian Schoenebeck <[email protected]>--- content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
(...)
content.tex \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir · line 2586
Quoted 5 lines
content.tex +1
-0@@ -2581,12 +2583,17 @@ \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir struct vq_config_block { be16 index; be16 max_num; + be32 indirect_num; /* since virtio-ccw rev. 3 */
Maybe make it max_indirect_num (to mirror max_num?)
content.tex \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir · line 2595
Quoted 9 lines
content.tex +3
-0}; \end{lstlisting} The requested number of buffers for queue \field{index} is returned in \field{max_num}. +Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue.
It still depends on the feature flag, though.
Maybe
"If revision 2 or lower is set, struct vq_config_block extends up to \field{max_num}.
If revision 3 or higher is set, struct vq_config_block extends to \field{max_indirect_num}. If VIRTIO_RING_F_INDIRECT_SIZE is negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are unpredictable."
Quoted 13 lines
content.tex +2
-0+ Afterwards, CCW_CMD_SET_VQ is issued by the driver to inform the device about the location used for its queue. The transmitted structure is@@ -2599,6 +2606,7 @@ \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir be16 num; be64 driver; be64 device; + be32 indirect_num; /* since virtio-ccw rev. 3 */ }; \end{lstlisting}
content.tex \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir · line 2620
Quoted 6 lines
content.tex +3
-0@@ -2607,6 +2615,10 @@ \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir available area and used area for queue \field{index}, respectively. The actual virtqueue size (number of allocated buffers) is transmitted in \field{num}. +Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by driver for this queue.
Similar to above:
"If revision 2 or lower is set, struct vq_info_block extends up to \field{device}.
If revision 3 or higher is set, struct vq_info_block extends to \field{indirect_num}. If VIRTIO_RING_F_INDIRECT_SIZE is negotiated, the Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the driver for this queue is transmitted in \field{indirect_num}."
content.tex \subsubsection{Configuring a Virtqueue}\label{sec:Virtio Transport Options / Vir · line 2624
Quoted 4 lines
content.tex +1
-0+ \devicenormative{\paragraph}{Configuring a Virtqueue}{Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue} \field{res0} is reserved and MUST be ignored by the device.
We should include a device normative statement:
"The device MUST NOT offer VIRTIO_RING_F_INDIRECT_SIZE, unless at least revision 3 is set, and therefore \field{max_indirect_num} respectively \field{indirect_num} are available."
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 20 lines · On Donnerstag, 17. März 2022 15:12:42 CET Cornelia Huck wrote:
Quoted 12 lines · Christian Schoenebeck
This new CCW configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Bump CCW virtio revision to 3 and make the existence of this new field "
indirect_num" dependant on revision 3.Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)(...)
Quoted 4 lines
(quoted diff) +0
-0@@ -2581,12 +2583,17 @@ \subsubsection{Configuring aVirtqueue}\label{sec:Virtio Transport Options / Vir>
structvq_config_block{
Quoted 6 lines
Quoted 4 lines
be16 index; be16 max_num;+ be32
indirect_num; /* since virtio-ccw rev. 3 */Maybe make it
max_indirect_num(to mirrormax_num?)
Right, that name appears indeed more appropriate here.
Quoted 14 lines
Quoted 9 lines
};
\end{lstlisting}The requested number of buffers for queue \field{index} is returned in \field{max_num}.
+Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue.
It still depends on the feature flag, though.
Maybe
Quoted 8 lines
"If revision 2 or lower is set, struct
vq_config_blockextends up to \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends to \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are unpredictable."
Maybe rather something like this instead:
"Actual size of struct vq_config_block depends on the virtio-ccw revision.
If revision 2 or lower is set, struct vq_config_block extends up to *including* \field{max_num}.
If revision 3 or higher is set, struct vq_config_block extends up to *including* \field{max_indirect_num}. If VIRTIO_RING_F_INDIRECT_SIZE is negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are *undefined*."
Quoted 28 lines
Quoted 18 lines
content.tex +1
-0+ Afterwards, CCW_CMD_SET_VQ is issued by the driver to inform the device about the location used for its queue. The transmitted structure is@@ -2599,6 +2606,7 @@ \subsubsection{Configuring aVirtqueue}\label{sec:Virtio Transport Options / Vir>
be16 num; be64 driver; be64 device;content.tex +1
-0+ be32 indirect_num; /* since virtio-ccw rev. 3 */ }; \end{lstlisting}@@ -2607,6 +2615,10 @@ \subsubsection{Configuring aQuoted 7 lines
Virtqueue}\label{sec:Virtio Transport Options / Vir>
available area and used area for queue \field{index}, respectively. The actual virtqueue size (number of allocated buffers) is transmitted in \field{num}.>
+Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by driver for this queue.Similar to above:
Quoted 8 lines
"If revision 2 or lower is set, struct
vq_info_blockextends up to \field{device}.If revision 3 or higher is set, struct
vq_info_blockextends to \field{indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, the Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the driver for this queue is transmitted in \field{indirect_num}."
Ditto.
Quoted 6 lines
Quoted 6 lines
+
\devicenormative{\paragraph}{Configuring a Virtqueue}{Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue}
Quoted 7 lines
Quoted 1 line
\field{res0} is reserved and MUST be ignored by the device.
We should include a device normative statement:
"The device MUST NOT offer
VIRTIO_RING_F_INDIRECT_SIZE, unless at least revision 3 is set, and therefore \field{max_indirect_num} respectively \field{indirect_num} are available."
Oh, you are suggesting two different names for those two fields? "max_indirect_num" vs. "indirect_num". If yes, why?
Otherwise about adding that normative statement in general here: sure, makes sense.
Best regards,
Christian Schoenebeck
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 57 lines · On Fri, 18 Mar 2022 12:02:31 +0100 Christian Schoenebeck…
Quoted 20 lines · On Donnerstag, 17. März 2022 15:12:42 CET Cornelia Huck wrote:
Quoted 12 lines · Christian Schoenebeck
This new CCW configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Bump CCW virtio revision to 3 and make the existence of this new field "
indirect_num" dependant on revision 3.Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)(...)
Quoted 4 lines
(quoted diff) +0
-0@@ -2581,12 +2583,17 @@ \subsubsection{Configuring aVirtqueue}\label{sec:Virtio Transport Options / Vir>
structvq_config_block{Quoted 6 lines
Quoted 4 lines
be16 index; be16 max_num;+ be32
indirect_num; /* since virtio-ccw rev. 3 */Maybe make it
max_indirect_num(to mirrormax_num?)Right, that name appears indeed more appropriate here.
Quoted 14 lines
Quoted 9 lines
};
\end{lstlisting}The requested number of buffers for queue \field{index} is returned in \field{max_num}.
+Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue.
It still depends on the feature flag, though.
Maybe
Quoted 8 lines
"If revision 2 or lower is set, struct
vq_config_blockextends up to \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends to \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are unpredictable."Maybe rather something like this instead:
"Actual size of struct
vq_config_blockdepends on the virtio-ccw revision.
Quoted 9 lines
If revision 2 or lower is set, struct
vq_config_blockextends up to *including* \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends up to *including* \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are *undefined*."
I agree that the "including" is important, but I'm not sure about the "its contents are undefined". I don't really understand why should we use a plural here. What speaks against specifying that in SHOULD be stored as 0 by the device, and MUST be ignored by the driver?
Currently we say that \field{max_indirect_num} exists like a be32 field even if VIRTIO_RING_F_INDIRECT_SIZE is not negotiated. Which kind of implies that at least type invariants should hold. Of course, there is none here (i.e. every bits value is also a be32 value), but for something like an enum interesting corner cases can pop up.
[..]
Regards,
Halil
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 5 lines · On Freitag, 18. März 2022 17:06:25 CET Halil Pasic wrote:
On Fri, 18 Mar 2022 12:02:31 +0100
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Donnerstag, 17. März 2022 15:12:42 CET Cornelia Huck wrote:
On Wed, Mar 16 2022, Christian Schoenebeck <[email protected]>
Quoted 66 lines · wrote:
Quoted 57 lines
Quoted 18 lines
Quoted 12 lines
This new CCW configuration field allows to negotiate a more fine graded maximum lenght of indirect descriptor chains.
Bump CCW virtio revision to 3 and make the existence of this new field "
indirect_num" dependant on revision 3.Signed-off-by Christian Schoenebeck <[email protected]>---
content.tex | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)(...)
Quoted 3 lines
(quoted diff) +0
-0@@ -2581,12 +2583,17 @@ \subsubsection{Configuring aVirtqueue}\label{sec:Virtio Transport Options / Vir>
Quoted 8 lines
Quoted 6 lines
struct
vq_config_block{be16 index; be16 max_num;+ be32
indirect_num; /* since virtio-ccw rev. 3 */Maybe make it
max_indirect_num(to mirrormax_num?)Right, that name appears indeed more appropriate here.
Quoted 15 lines
Quoted 10 lines
};
\end{lstlisting}The requested number of buffers for queue \field{index} is returned in
\field{max_num}.+Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue.
It still depends on the feature flag, though.
Maybe
Quoted 8 lines
"If revision 2 or lower is set, struct
vq_config_blockextends up to \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends to \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are unpredictable."Maybe rather something like this instead:
"Actual size of struct
vq_config_blockdepends on the virtio-ccw revision.Quoted 8 lines
If revision 2 or lower is set, struct
vq_config_blockextends up to *including* \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends up to *including* \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are *undefined*."
Quoted 4 lines
I agree that the "including" is important, but I'm not sure about the "its contents are undefined". I don't really understand why should we use a plural here. What speaks against specifying that in SHOULD be stored as 0 by the device, and MUST be ignored by the driver?
Both solutions would be viable. Personally I would just use something like "Should be zero" if there is a value in recommending that, but I don't see a value in recommending to set something to zero and at the same time requiring to not access it in the first place.
Quoted 5 lines
Currently we say that \field{max_indirect_num} exists like a be32 field even if
VIRTIO_RING_F_INDIRECT_SIZEis not negotiated. Which kind of implies that at least type invariants should hold. Of course, there is none here (i.e. every bits value is also a be32 value), but for something like an enum interesting corner cases can pop up.
I can't follow you on that one. What has that do with enums in this case?
Anyway, I won't persist on my suggestion to use the (IMO more compact form) "undefined". If you guys prefer the more specific solution "SHOULD be 0 and MUST not be accessed" then I will go that way.
Best regards,
Christian Schoenebeck
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 18 lines · Christian Schoenebeck
Quoted 4 lines · On Freitag, 18. März 2022 17:06:25 CET Halil Pasic wrote:
I agree that the "including" is important, but I'm not sure about the "its contents are undefined". I don't really understand why should we use a plural here. What speaks against specifying that in SHOULD be stored as 0 by the device, and MUST be ignored by the driver?
Both solutions would be viable. Personally I would just use something like "Should be zero" if there is a value in recommending that, but I don't see a value in recommending to set something to zero and at the same time requiring to not access it in the first place.
Quoted 5 lines
Currently we say that \field{max_indirect_num} exists like a be32 field even if
VIRTIO_RING_F_INDIRECT_SIZEis not negotiated. Which kind of implies that at least type invariants should hold. Of course, there is none here (i.e. every bits value is also a be32 value), but for something like an enum interesting corner cases can pop up.
Quoted 5 lines
I can't follow you on that one. What has that do with enums in this case?
Anyway, I won't persist on my suggestion to use the (IMO more compact form) "undefined". If you guys prefer the more specific solution "SHOULD be 0 and MUST not be accessed" then I will go that way.
I'm not sure what mandating 0 and non-access would buy us here... the driver can of course read the field (e.g. when copying the structure wholesale); it just can't make use of the contents when it did not negotiate the feature (but why would it do so in that case anyway?)
Also, I think junk remains junk, whether it is a be32 field or interpreted as an enum. It is simply not valid, even if it might by accident end up to be a defined enum value.
So I think "undefined" should be fine.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 22 lines · On Mon, 21 Mar 2022 17:36:26 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 20 lines · Christian Schoenebeck
Quoted 4 lines · On Freitag, 18. März 2022 17:06:25 CET Halil Pasic wrote:
I agree that the "including" is important, but I'm not sure about the "its contents are undefined". I don't really understand why should we use a plural here. What speaks against specifying that in SHOULD be stored as 0 by the device, and MUST be ignored by the driver?
Both solutions would be viable. Personally I would just use something like "Should be zero" if there is a value in recommending that, but I don't see a value in recommending to set something to zero and at the same time requiring to not access it in the first place.
Quoted 5 lines
Currently we say that \field{max_indirect_num} exists like a be32 field even if
VIRTIO_RING_F_INDIRECT_SIZEis not negotiated. Which kind of implies that at least type invariants should hold. Of course, there is none here (i.e. every bits value is also a be32 value), but for something like an enum interesting corner cases can pop up.I can't follow you on that one. What has that do with enums in this case?
Quoted 8 lines
Quoted 3 lines
Anyway, I won't persist on my suggestion to use the (IMO more compact form) "undefined". If you guys prefer the more specific solution "SHOULD be 0 and MUST not be accessed" then I will go that way.
I'm not sure what mandating 0 and non-access would buy us here... the driver can of course read the field (e.g. when copying the structure wholesale); it just can't make use of the contents when it did not negotiate the feature (but why would it do so in that case anyway?)
My train of thought was that making the device give us a well defined 0 could benefit robustness. The idea was, that even if the driver was buggy, and used the value we would still end up with some sane behavior.
Quoted 4 lines
Also, I think junk remains junk, whether it is a be32 field or interpreted as an enum. It is simply not valid, even if it might by accident end up to be a defined enum value.
What I had in mind is the difference between "trap representation" and "unspecified value" in terms of the C standard. Using a "trap representation" is undefined behavior, while using an "unspecified value" is far less serious. As far as I remember, there are no trap representations for enumerated types in C, so the example ain't perfect. But if some code was to assume that all it can see it the values defined in the enum, strange stuff may happen.
Quoted 3 lines
So I think "undefined" should be fine.
BTW the C standard uses the term "indeterminate value" in this situation.
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 29 lines · Halil Pasic
Quoted 26 lines · On Mon, 21 Mar 2022 17:36:26 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 23 lines · Christian Schoenebeck
Quoted 4 lines · On Freitag, 18. März 2022 17:06:25 CET Halil Pasic wrote:
I agree that the "including" is important, but I'm not sure about the "its contents are undefined". I don't really understand why should we use a plural here. What speaks against specifying that in SHOULD be stored as 0 by the device, and MUST be ignored by the driver?
Both solutions would be viable. Personally I would just use something like "Should be zero" if there is a value in recommending that, but I don't see a value in recommending to set something to zero and at the same time requiring to not access it in the first place.
Quoted 5 lines
Currently we say that \field{max_indirect_num} exists like a be32 field even if
VIRTIO_RING_F_INDIRECT_SIZEis not negotiated. Which kind of implies that at least type invariants should hold. Of course, there is none here (i.e. every bits value is also a be32 value), but for something like an enum interesting corner cases can pop up.I can't follow you on that one. What has that do with enums in this case?
Anyway, I won't persist on my suggestion to use the (IMO more compact form) "undefined". If you guys prefer the more specific solution "SHOULD be 0 and MUST not be accessed" then I will go that way.
Quoted 8 lines
Quoted 4 lines
I'm not sure what mandating 0 and non-access would buy us here... the driver can of course read the field (e.g. when copying the structure wholesale); it just can't make use of the contents when it did not negotiate the feature (but why would it do so in that case anyway?)
My train of thought was that making the device give us a well defined 0 could benefit robustness. The idea was, that even if the driver was buggy, and used the value we would still end up with some sane behavior.
I'm not sure a 0 would lead to sane behaviour in an already buggy driver... operating with a limit of 0 would imply that the driver cannot really do anything, and I'm not sure a driver buggy enough to access the field would heed that. There's nothing wrong with a device using 0 if the feature had not been negotiated, but I don't think it will help much with already buggy drivers.
Quoted 6 lines
Quoted 5 lines
Also, I think junk remains junk, whether it is a be32 field or interpreted as an enum. It is simply not valid, even if it might by accident end up to be a defined enum value.
Quoted 7 lines
What I had in mind is the difference between "trap representation" and "unspecified value" in terms of the C standard. Using a "trap representation" is undefined behavior, while using an "unspecified value" is far less serious. As far as I remember, there are no trap representations for enumerated types in C, so the example ain't perfect. But if some code was to assume that all it can see it the values defined in the enum, strange stuff may happen.
While the struct definitions look suspiciously like C, they are not in fact C :) I don't think the spec defines anything of the above, and I don't think it should.
Quoted 7 lines
Quoted 5 lines
So I think "undefined" should be fine.
BTW the C standard uses the term "indeterminate value" in this situation.
"Indeterminate value" is a bit of a mouthful, though; "undefined" or "unpredictable" from the driver's point of view should already capture it, as the driver is not supposed to do anything with the value anyway.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 40 lines · On Tue, 22 Mar 2022 10:57:00 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 29 lines · Halil Pasic
Quoted 26 lines · On Mon, 21 Mar 2022 17:36:26 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 23 lines · Christian Schoenebeck
Quoted 4 lines · On Freitag, 18. März 2022 17:06:25 CET Halil Pasic wrote:
I agree that the "including" is important, but I'm not sure about the "its contents are undefined". I don't really understand why should we use a plural here. What speaks against specifying that in SHOULD be stored as 0 by the device, and MUST be ignored by the driver?
Both solutions would be viable. Personally I would just use something like "Should be zero" if there is a value in recommending that, but I don't see a value in recommending to set something to zero and at the same time requiring to not access it in the first place.
Quoted 5 lines
Currently we say that \field{max_indirect_num} exists like a be32 field even if
VIRTIO_RING_F_INDIRECT_SIZEis not negotiated. Which kind of implies that at least type invariants should hold. Of course, there is none here (i.e. every bits value is also a be32 value), but for something like an enum interesting corner cases can pop up.I can't follow you on that one. What has that do with enums in this case?
Anyway, I won't persist on my suggestion to use the (IMO more compact form) "undefined". If you guys prefer the more specific solution "SHOULD be 0 and MUST not be accessed" then I will go that way.
Quoted 8 lines
Quoted 4 lines
I'm not sure what mandating 0 and non-access would buy us here... the driver can of course read the field (e.g. when copying the structure wholesale); it just can't make use of the contents when it did not negotiate the feature (but why would it do so in that case anyway?)
My train of thought was that making the device give us a well defined 0 could benefit robustness. The idea was, that even if the driver was buggy, and used the value we would still end up with some sane behavior.
Quoted 7 lines
I'm not sure a 0 would lead to sane behaviour in an already buggy driver... operating with a limit of 0 would imply that the driver cannot really do anything, and I'm not sure a driver buggy enough to access the field would heed that. There's nothing wrong with a device using 0 if the feature had not been negotiated, but I don't think it will help much with already buggy drivers.
I don't consider this awfully important. While I do see some value in devices presenting some saneish value in this situation over presenting junk, I am fine with junk as well. Actually implementations can still do whatever they want.
Quoted 14 lines
Quoted 6 lines
Quoted 5 lines
Also, I think junk remains junk, whether it is a be32 field or interpreted as an enum. It is simply not valid, even if it might by accident end up to be a defined enum value.
Quoted 7 lines
What I had in mind is the difference between "trap representation" and "unspecified value" in terms of the C standard. Using a "trap representation" is undefined behavior, while using an "unspecified value" is far less serious. As far as I remember, there are no trap representations for enumerated types in C, so the example ain't perfect. But if some code was to assume that all it can see it the values defined in the enum, strange stuff may happen.
Quoted 2 lines
While the struct definitions look suspiciously like C, they are not in fact C :)
I'm aware. I actually merely used the C standard lingo, because most of us are familiar with C, and it is easy to read up on the precise meaning.
I pointed out the difference between using an unspecified value and using a trap representation to showcase, that the difference between the two might matter.
Quoted 2 lines
I don't think the spec defines anything of the above, and I don't think it should.
Never stated the spec defines anything of the above.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
So I think "undefined" should be fine.
BTW the C standard uses the term "indeterminate value" in this situation.
"Indeterminate value" is a bit of a mouthful, though; "undefined" or "unpredictable" from the driver's point of view should already capture it, as the driver is not supposed to do anything with the value anyway.
Yes the reader is more than likely to figure out what "undefined value" or "unpredictable value" is supposed to mean in this context from the context. I should really stop splitting hairs. Nevertheless given that revision >= 3 and was VIRTIO_RING_F_INDIRECT_SIZE negotiated, is the value of indirect_max_num predictable by the driver? In my opinion it is not. And neither is the value defined by this spec. The semantic of the value is defined, but the value itself isn't really any more defined than when VIRTIO_RING_F_INDIRECT_SIZE is not negotiated.
We could say that when VIRTIO_RING_F_INDIRECT_SIZE is not negotiated Queue Indirect Size is undefined. But I should really stop splitting hairs. Sorry.
Regards,
Halil
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
In reply to Christian SchoenebeckQuoted 24 lines · Christian Schoenebeck
Quoted 10 lines · On Donnerstag, 17. März 2022 15:12:42 CET Cornelia Huck wrote:
Quoted 3 lines · Christian Schoenebeck
+Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue.
It still depends on the feature flag, though.
Maybe
Quoted 8 lines
"If revision 2 or lower is set, struct
vq_config_blockextends up to \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends to \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are unpredictable."Maybe rather something like this instead:
"Actual size of struct
vq_config_blockdepends on the virtio-ccw revision.
Quoted 8 lines
If revision 2 or lower is set, struct
vq_config_blockextends up to *including* \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends up to *including* \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are *undefined*."
Hm, I was coming from the driver's perspective here. But "undefined" probably captures "junk in there, do not use" better.
Ack on your other points.
Quoted 8 lines
Quoted 5 lines
We should include a device normative statement:
"The device MUST NOT offer
VIRTIO_RING_F_INDIRECT_SIZE, unless at least revision 3 is set, and therefore \field{max_indirect_num} respectively \field{indirect_num} are available."Oh, you are suggesting two different names for those two fields? "
max_indirect_num" vs. "indirect_num". If yes, why?
The "max_indirect_num" goes with the "max_num": this is the maximum value that the device supports (the "maximum maximum", in a way :). "indirect_num" is the actual upper limit while the device is in use.
Quoted 3 lines
Otherwise about adding that normative statement in general here: sure, makes sense.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 3 lines · On Freitag, 18. März 2022 17:10:34 CET Cornelia Huck wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Donnerstag, 17. März 2022 15:12:42 CET Cornelia Huck wrote:
On Wed, Mar 16 2022, Christian Schoenebeck <[email protected]>
Quoted 24 lines · wrote:
Quoted 24 lines
Quoted 8 lines
Quoted 3 lines
+Since revision 3, \field{indirect_num} exists, which is supposed to reflect the +Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) +supported by device for this queue.
It still depends on the feature flag, though.
Maybe
Quoted 8 lines
"If revision 2 or lower is set, struct
vq_config_blockextends up to \field{max_num}.If revision 3 or higher is set, struct
vq_config_blockextends to \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are unpredictable."Maybe rather something like this instead:
"Actual size of struct
vq_config_blockdepends on the virtio-ccw revision.If revision 2 or lower is set, struct
vq_config_blockextends up to *including* \field{max_num}.
Quoted 8 lines
Quoted 5 lines
If revision 3 or higher is set, struct
vq_config_blockextends up to *including* \field{max_indirect_num}. IfVIRTIO_RING_F_INDIRECT_SIZEis negotiated, \field{indirect_max_num} contains the maximum Queue Indirect Size (i.e. the maximum length of indirect descriptor tables) supported by the device for this queue; otherwise, its contents are *undefined*."Hm, I was coming from the driver's perspective here. But "undefined" probably captures "junk in there, do not use" better.
Like written to Halil moments ago, I would prefer "undefined" but I have no strong opininon on it.
Quoted 8 lines
Ack on your other points.
Quoted 6 lines
Quoted 5 lines
We should include a device normative statement:
"The device MUST NOT offer
VIRTIO_RING_F_INDIRECT_SIZE, unless at least revision 3 is set, and therefore \field{max_indirect_num} respectively \field{indirect_num} are available."
Quoted 6 lines
Quoted 2 lines
Oh, you are suggesting two different names for those two fields? "
max_indirect_num" vs. "indirect_num". If yes, why?The "
max_indirect_num" goes with the "max_num": this is the maximum value that the device supports (the "maximum maximum", in a way :). "indirect_num" is the actual upper limit while the device is in use.
Ok, I understand your motivation now, your interpretation aspect for this was:
device's value >= driver's value
Another interpretation aspect would be though from driver PoV: 'what is the maximum bulk size I could send to device?', and in this case you would call both fields 'max_indirect_num'.
Therefore I would prefer using the same name 'max_indirect_num' on both structs, to also make it clear they are about negotiating the same thing.
Quoted 3 lines
Quoted 3 lines
Otherwise about adding that normative statement in general here: sure, makes sense.
Re: [virtio-comment] Re: [PATCH v3 4/4] Add CCW configuration field "indirect_num"
Quoted 11 lines · Christian Schoenebeck
Quoted 7 lines · On Freitag, 18. März 2022 17:10:34 CET Cornelia Huck wrote:
Quoted 2 lines · Christian Schoenebeck
Oh, you are suggesting two different names for those two fields? "
max_indirect_num" vs. "indirect_num". If yes, why?The "
max_indirect_num" goes with the "max_num": this is the maximum value that the device supports (the "maximum maximum", in a way :). "indirect_num" is the actual upper limit while the device is in use.Ok, I understand your motivation now, your interpretation aspect for this was:
Quoted 8 lines
device's value >= driver's value
Another interpretation aspect would be though from driver PoV: 'what is the maximum bulk size I could send to device?', and in this case you would call both fields '
max_indirect_num'.Therefore I would prefer using the same name '
max_indirect_num' on both structs, to also make it clear they are about negotiating the same thing.
So, maybe the 'max' is what is confusing? 'max_indirect_num_limit' vs 'max_indirect_num', perhaps? I'd really like to keep distinct names for the two fields, though, as they are fields in two distinct structures, one used for the device advertising what it supports, and one for the driver advertising what it will actually use. (This is different from PCI or MMIO, where both device and driver interact on the same value.)
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/