Skip to contentSkip to conversation

virtio-pmem: Support describing pmem as shared memory region

PATCH In review [email protected] 5 participants 13 messages Nov 10, 2021 – Nov 24, 2021

lore
Share & tools
lore
#0 lore raw

v5 cover virtio-pmem: Support describing pmem as shared memory region

From: Taylor Stark <[email protected]>

Changes from v4 [1]:
- Clarified which fields of virtio_pmem_config are valid depending on shared memory feature negotiation (suggested by Cornelia Huck).

Changes from v3 [1]:
- Rebased onto master now that the base virtio-pmem spec has been merged.

Changes from v2:
- Incorporated suggestions from Cornelia Huck on rewording driver initialization.

Changes from v1:
- Added in a feature bit (VIRTIO_PMEM_F_SHMEM_REGION) for controlling how the device indicates the guest physical address ranges to the driver. This feature directly affects control flow of the driver, since it seemed weird to have the driver indicate support for shared memory regions, and then needing to include an enum (or similar) informing the driver how the device indicated guest physical address ranges. If devices want to indicate the ranges as guest absolute addresses, they can skip negotiating the feature.
- The linux driver implementation has been updated and tested, but I'm holding off on posting the patches to get some feedback on the new approach.
- Moved some changes to proper subsections (normative subsections).

[1] https://lists.oasis-open.org/archives/virtio-comment/202111/msg00004.html

---

This patch updates the virtio-pmem spec to add support for describing the pmem region as a shared memory region. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

As virtio-pmem hasn't been added to the virtio spec yet (see this issue [1]), this patch is based off the RFC spec [2]. The linux driver implementation has been posted at [3].

[1] https://github.com/oasis-tcs/virtio-spec/issues/78
[2] https://lists.oasis-open.org/archives/virtio-dev/201903/msg00083.html
[3] https://lore.kernel.org/nvdimm/20210715223505.GA29329@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net

Taylor Stark (1):
virtio-pmem: Support describing pmem as shared memory region

 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)
Signature · 18 lines
2.33.0.vfs.0.0


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#1 lore raw

PATCH v5 1/1 virtio-pmem: Support describing pmem as shared memory region

From: Taylor Stark <[email protected]>

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)
conformance.tex +12 -2
index 80547db..d9f2b45 100644
@@ -31,8 +31,9 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \ref{sec:Conformance / Driver Conformance / Sound Driver Conformance},
 \ref{sec:Conformance / Driver Conformance / Memory Driver Conformance},
 \ref{sec:Conformance / Driver Conformance / I2C Adapter Driver Conformance},
-\ref{sec:Conformance / Driver Conformance / SCMI Driver Conformance} or
-\ref{sec:Conformance / Driver Conformance / GPIO Driver Conformance}.
+\ref{sec:Conformance / Driver Conformance / SCMI Driver Conformance},
+\ref{sec:Conformance / Driver Conformance / GPIO Driver Conformance} or
+\ref{sec:Conformance / Driver Conformance / PMEM Driver Conformance}.
 
     \item Clause \ref{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance}.
   \end{itemize}
@@ -314,6 +315,14 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \item \ref{drivernormative:Device Types / GPIO Device / eventq Operation}
 \end{itemize}
 
+\conformance{\subsection}{PMEM Driver Conformance}\label{sec:Conformance / Driver Conformance / PMEM Driver Conformance}
+
+A PMEM driver MUST conform to the following normative statements:
+
+\begin{itemize}
+\item \ref{drivernormative:Device Types / PMEM Device / Device Initialization}
+\end{itemize}
+
 \conformance{\section}{Device Conformance}\label{sec:Conformance / Device Conformance}
 
 A device MUST conform to the following normative statements:
@@ -578,6 +587,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 A PMEM device MUST conform to the following normative statements:
 
 \begin{itemize}
+\item \ref{devicenormative:Device Types / PMEM Device / Device Initialization}
 \item \ref{devicenormative:Device Types / PMEM Device / Device Operation / Virtqueue flush}
 \item \ref{devicenormative:Device Types / PMEM Device / Device Operation / Virtqueue return}
 \end{itemize}
virtio-pmem.tex +33 -7
index 93ab3c1..384eaa3 100644
@@ -24,7 +24,10 @@ \subsection{Virtqueues}\label{sec:Device Types / PMEM Device / Virtqueues}
 
 \subsection{Feature bits}\label{sec:Device Types / PMEM Device / Feature bits}
 
-There are currently no feature bits defined for this device.
+\begin{description}
+\item[VIRTIO_PMEM_F_SHMEM_REGION (0)] The guest physical address range will be
+indicated as a shared memory region.
+\end{description}
 
 \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / Device configuration layout}
 
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.
+
+\drivernormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the driver MUST query
+shared memory ID 0 for the physical address ranges, and MUST NOT use
+\field{start} or \field{stop}.
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the driver MUST read the
+physical address ranges from \field{start} and \field{stop}.
+
 \subsection{Driver Operations}\label{sec:Device Types / PMEM Driver / Driver Operation / Request Queues}
 
 Requests have the following format:
Signature · 18 lines
2.33.0.vfs.0.0


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#2 lore raw
Quoted 8 lines · On Wed, Nov 10 2021, [email protected] wrote:

From: Taylor Stark <[email protected]>

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

Quoted 5 lines
---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)

This looks good to me now.

Could you please open an issue at
https://github.com/oasis-tcs/virtio-spec/issues to get this included? If nobody has any further comments, I think we could start the voting process soon.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#3 lore raw
Quoted 16 lines · Cornelia Huck
Quoted 8 lines · On Wed, Nov 10 2021, [email protected] wrote:

From: Taylor Stark <[email protected]>

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

Quoted 5 lines
---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)
Quoted 6 lines

This looks good to me now.

Could you please open an issue at
https://github.com/oasis-tcs/virtio-spec/issues to get this included? If nobody has any further comments, I think we could start the voting process soon.

Sounds good, thanks Cornelia!

#4 lore raw
In reply to Cornelia Huck
Quoted 10 lines · Cornelia Huck
Quoted 8 lines · On Wed, Nov 10 2021, [email protected] wrote:

From: Taylor Stark <[email protected]>

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

Quoted 7 lines
Quoted 5 lines
---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)

This looks good to me now.

I noticed the patch uses "guest absolute address" instead of "physical address". This terminology was fixed in the original pmem patch and I guess the shared memory patch wasn't updated. Please don't use "guest".

Stefan

#5 lore raw
Quoted 17 lines · Stefan Hajnoczi
Quoted 16 lines · Cornelia Huck
Quoted 13 lines · On Wed, Nov 10 2021, [email protected] wrote:

From: Taylor Stark <[email protected]>

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)
Quoted 5 lines
Quoted 1 line

This looks good to me now.

I noticed the patch uses "guest absolute address" instead of "physical address". This terminology was fixed in the original pmem patch and I guess the shared memory patch wasn't updated. Please don't use "guest".

So s/guest absolute/physical/ ? That can be fixed as an editorial update on top.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#6 lore raw
Quoted 19 lines · Cornelia Huck
Quoted 17 lines · Stefan Hajnoczi
Quoted 16 lines · Cornelia Huck
Quoted 13 lines · On Wed, Nov 10 2021, [email protected] wrote:

From: Taylor Stark <[email protected]>

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)
Quoted 9 lines
Quoted 5 lines
Quoted 1 line

This looks good to me now.

I noticed the patch uses "guest absolute address" instead of "physical address". This terminology was fixed in the original pmem patch and I guess the shared memory patch wasn't updated. Please don't use "guest".

So s/guest absolute/physical/ ? That can be fixed as an editorial update on top.

Yes, please.

Stefan

Reviewed-by on v5 1/1
#7 lore raw
In reply to [email protected]
Quoted 109 lines

Update the virtio-pmem spec to add support for describing the pmem region as a shared memory window. This is required to support virtio-pmem in Hyper-V, since Hyper-V only allows PCI devices to operate on memory ranges defined via BARs. When using the virtio PCI transport, shared memory regions are described via PCI BARs.

---
 conformance.tex | 14 ++++++++++++--
 virtio-pmem.tex | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 45 insertions(+), 9 deletions(-)
conformance.tex +12 -2
index 80547db..d9f2b45 100644
@@ -31,8 +31,9 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \ref{sec:Conformance / Driver Conformance / Sound Driver Conformance},
 \ref{sec:Conformance / Driver Conformance / Memory Driver Conformance},
 \ref{sec:Conformance / Driver Conformance / I2C Adapter Driver Conformance},
-\ref{sec:Conformance / Driver Conformance / SCMI Driver Conformance} or
-\ref{sec:Conformance / Driver Conformance / GPIO Driver Conformance}.
+\ref{sec:Conformance / Driver Conformance / SCMI Driver Conformance},
+\ref{sec:Conformance / Driver Conformance / GPIO Driver Conformance} or
+\ref{sec:Conformance / Driver Conformance / PMEM Driver Conformance}.
 
     \item Clause \ref{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance}.
   \end{itemize}
@@ -314,6 +315,14 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \item \ref{drivernormative:Device Types / GPIO Device / eventq Operation}
 \end{itemize}
 
+\conformance{\subsection}{PMEM Driver Conformance}\label{sec:Conformance / Driver Conformance / PMEM Driver Conformance}
+
+A PMEM driver MUST conform to the following normative statements:
+
+\begin{itemize}
+\item \ref{drivernormative:Device Types / PMEM Device / Device Initialization}
+\end{itemize}
+
 \conformance{\section}{Device Conformance}\label{sec:Conformance / Device Conformance}
 
 A device MUST conform to the following normative statements:
@@ -578,6 +587,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 A PMEM device MUST conform to the following normative statements:
 
 \begin{itemize}
+\item \ref{devicenormative:Device Types / PMEM Device / Device Initialization}
 \item \ref{devicenormative:Device Types / PMEM Device / Device Operation / Virtqueue flush}
 \item \ref{devicenormative:Device Types / PMEM Device / Device Operation / Virtqueue return}
 \end{itemize}
virtio-pmem.tex +26 -7
index 93ab3c1..384eaa3 100644
@@ -24,7 +24,10 @@ \subsection{Virtqueues}\label{sec:Device Types / PMEM Device / Virtqueues}
 
 \subsection{Feature bits}\label{sec:Device Types / PMEM Device / Feature bits}
 
-There are currently no feature bits defined for this device.
+\begin{description}
+\item[VIRTIO_PMEM_F_SHMEM_REGION (0)] The guest physical address range will be
+indicated as a shared memory region.
+\end{description}
 
 \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / Device configuration layout}
 
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.
+
+\drivernormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / · line 83

Quoted 10 lines
virtio-pmem.tex +7 -0
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the driver MUST query
+shared memory ID 0 for the physical address ranges, and MUST NOT use
+\field{start} or \field{stop}.
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the driver MUST read the
+physical address ranges from \field{start} and \field{stop}.
+
 \subsection{Driver Operations}\label{sec:Device Types / PMEM Driver / Driver Operation / Request Queues}
 
 Requests have the following format:

Looks good to me.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#8 lore raw
In reply to [email protected]

On Wed, 10 Nov 2021 10:55:55 -0800
[email protected] wrote:
[..]

Quoted 30 lines
virtio-pmem.tex +10 -6
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / · line 71

Quoted 10 lines
virtio-pmem.tex +10 -0
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.

Sorry for joining in this late.

I'm wondering if the quoted parts implies that the device SHOULD change its config space (fields start and size) when the driver sets FEATURES_OK, and the feature is VIRTIO_PMEM_F_SHMEM_REGION set at the device (it was offered, and got set).

My train of thought is: initially no feature is considered negotiated, and config space access is allowed before feature negotiation is completed. That means the at this stage the device must indicate via the config space, and thus \field{size} != 0. But as a part of the transition 'features not negotiated' -> 'features negotiated' the device needs to go '\field{size} != 0' -> '\field{size} == 0'

Is that what we want?

Also I understand that Hyper-V would make the device cease operation if the feature VIRTIO_PMEM_F_SHMEM_REGION was not accepted by the driver. I believe I've read that in some previous email.

Do we need a 'MAY fail to operate further if' statement? Anything that ain't guarded by feature bits ain't optional, and anything that is guarded by feature bits is optional unless explicitly stated otherwise.

Regards,
Halil

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#9 lore raw
Quoted 64 lines · Halil Pasic

On Wed, 10 Nov 2021 10:55:55 -0800
[email protected] wrote:
[..]

Quoted 30 lines
virtio-pmem.tex +10 -6
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / · line 71

Quoted 10 lines
virtio-pmem.tex +10 -0
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.

Sorry for joining in this late.

I'm wondering if the quoted parts implies that the device SHOULD change its config space (fields start and size) when the driver sets FEATURES_OK, and the feature is VIRTIO_PMEM_F_SHMEM_REGION set at the device (it was offered, and got set).

My train of thought is: initially no feature is considered negotiated, and config space access is allowed before feature negotiation is completed. That means the at this stage the device must indicate via the config space, and thus \field{size} != 0. But as a part of the transition 'features not negotiated' -> 'features negotiated' the device needs to go '\field{size} != 0' -> '\field{size} == 0'

Is that what we want?

Also I understand that Hyper-V would make the device cease operation if the feature VIRTIO_PMEM_F_SHMEM_REGION was not accepted by the driver. I believe I've read that in some previous email.

Quoted 6 lines

Do we need a 'MAY fail to operate further if' statement? Anything that ain't guarded by feature bits ain't optional, and anything that is guarded by feature bits is optional unless explicitly stated otherwise.

Regards,
Halil

No problem, thanks for joining in! :) I think your train of thought makes sense. You're right, for Hyper-V we can't support any driver that doesn't support VIRTIO_PMEM_F_SHMEM_REGION. So we have no choice but to set the config fields to zero from the start. There's no meaningful value we can place in those fields.

I'm trying to figure out how best to word this. Saying that devices should set config fields prior to FEATURE_OK is redundant. Should I add a section saying that config fields MAY be set to 0 if VIRTIO_PMEM_F_SHMEM_REGION is offerred (such as in the Hyper-V case)? And that if it's set to 0 and the feature isn't negotiated, the device must reject the driver? I'd also have to add in a section saying that the pmem region shouldn't be touched by the driver prior to DRIVER_OK being set, to save drivers from accessing the zeroed values placed in the config. Additionally, encouraging devices to set the config fields to 0 once the feature has been negotiated seems like the right thing to do, to catch bugs?

I'm not sure about the 'MAY fail to operate further statement'. Couldn't there be lots of reasons why a device rejects a driver? And wouldn't those be implementation details of each device? Is it common to document those?

Thoughts?

Thanks!
Taylor

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#10 lore raw
Quoted 33 lines · Taylor Stark
Quoted 32 lines · Halil Pasic

On Wed, 10 Nov 2021 10:55:55 -0800
[email protected] wrote:
[..]

Quoted 29 lines
virtio-pmem.tex +9 -6
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
Quoted 8 lines
Quoted 8 lines

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /

Quoted 8 lines
virtio-pmem.tex +8 -0
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate

[for the sake of the argument made below, note the 'has not been negotiated' here]

Quoted 11 lines
Quoted 11 lines

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / · line 70

Quoted 3 lines
virtio-pmem.tex +3 -0
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.

Sorry for joining in this late.

I'm wondering if the quoted parts implies that the device SHOULD change its config space (fields start and size) when the driver sets FEATURES_OK, and the feature is VIRTIO_PMEM_F_SHMEM_REGION set at the device (it was offered, and got set).

Quoted 6 lines
Quoted 6 lines

My train of thought is: initially no feature is considered negotiated, and config space access is allowed before feature negotiation is completed. That means the at this stage the device must indicate via the config space, and thus \field{size} != 0. But as a part of the transition 'features not negotiated' -> 'features negotiated' the device needs to go '\field{size} != 0' -> '\field{size} == 0'

It has been advised that it SHOULD put 0 there, but I don't consider that a must, just a good idea.

Quoted 10 lines
Quoted 10 lines

Is that what we want?

Also I understand that Hyper-V would make the device cease operation if the feature VIRTIO_PMEM_F_SHMEM_REGION was not accepted by the driver. I believe I've read that in some previous email.

Do we need a 'MAY fail to operate further if' statement? Anything that ain't guarded by feature bits ain't optional, and anything that is guarded by feature bits is optional unless explicitly stated otherwise.

The device can always fail the setting of FEATURES_OK, see 2.2.2:

"The device SHOULD accept any valid subset of features the driver accepts, otherwise it MUST fail to set the FEATURES_OK device status bit when the driver writes it."

I'd argue that "SHMEM_REGION not set" can be considered a non-valid subset by the device. (Also, there's a SHOULD in there.)

Quoted 8 lines
Quoted 3 lines

Regards,
Halil

No problem, thanks for joining in! :) I think your train of thought makes sense. You're right, for Hyper-V we can't support any driver that doesn't support VIRTIO_PMEM_F_SHMEM_REGION. So we have no choice but to set the config fields to zero from the start. There's no meaningful value we can place in those fields.

I think this is fine: we only require the fields to have meaningful contents if SHMEM_REGION has not been negotiated -- as long as the device has not accepted FEATURES_OK from the driver, nothing has been negotiated respectively not been negotiated.

Quoted 10 lines

I'm trying to figure out how best to word this. Saying that devices should set config fields prior to FEATURE_OK is redundant. Should I add a section saying that config fields MAY be set to 0 if VIRTIO_PMEM_F_SHMEM_REGION is offerred (such as in the Hyper-V case)? And that if it's set to 0 and the feature isn't negotiated, the device must reject the driver? I'd also have to add in a section saying that the pmem region shouldn't be touched by the driver prior to DRIVER_OK being set, to save drivers from accessing the zeroed values placed in the config. Additionally, encouraging devices to set the config fields to 0 once the feature has been negotiated seems like the right thing to do, to catch bugs?

I don't think we can prohibit the driver from reading the config space, that is explicitly allowed in the general case.

We probably can add a non-normative extra statement, though, that a device may present zeroes in the config space before feature negotiation has finished if it doesn't support !SHMEM_REGION. That would also give the driver a way to discover that it doesn't make sense to continue without negotiating SHMEM_REGION. No extra normative statement, but should make it a bit more clear to those trying to implement this.

Quoted 4 lines

I'm not sure about the 'MAY fail to operate further statement'. Couldn't there be lots of reasons why a device rejects a driver? And wouldn't those be implementation details of each device? Is it common to document those?

We probably don't need such a statement (see above).

As voting for this is open until tomorrow, we should quickly decide what to do. My preference would be to merge it as-is and do a clarification on top.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#11 lore raw
Quoted 65 lines · On Mon, 22 Nov 2021 12:58:01 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 33 lines · Taylor Stark
Quoted 32 lines · Halil Pasic

On Wed, 10 Nov 2021 10:55:55 -0800
[email protected] wrote:
[..]

Quoted 29 lines
virtio-pmem.tex +9 -6
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
Quoted 8 lines
Quoted 8 lines

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /

Quoted 8 lines
virtio-pmem.tex +8 -0
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate  

[for the sake of the argument made below, note the 'has not been negotiated' here]

Quoted 11 lines
Quoted 11 lines

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / · line 70

Quoted 3 lines
virtio-pmem.tex +3 -0
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.  

Sorry for joining in this late.

I'm wondering if the quoted parts implies that the device SHOULD change its config space (fields start and size) when the driver sets FEATURES_OK, and the feature is VIRTIO_PMEM_F_SHMEM_REGION set at the device (it was offered, and got set).

Quoted 6 lines
Quoted 6 lines

My train of thought is: initially no feature is considered negotiated, and config space access is allowed before feature negotiation is completed. That means the at this stage the device must indicate via the config space, and thus \field{size} != 0. But as a part of the transition 'features not negotiated' -> 'features negotiated' the device needs to go '\field{size} != 0' -> '\field{size} == 0'

Quoted 2 lines

It has been advised that it SHOULD put 0 there, but I don't consider that a must, just a good idea.

Right I agree with that. So we say that the config state update is a good idea as well?

BTW the definition of SHOULD:
SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.

Maybe I should have used 'should go' instead of 'needs to go' but that doesn't change the fact, that the spec currently regards this config space update as RECOMMENDED.

Conny, is that what we want?

Quoted 12 lines
Quoted 11 lines
Quoted 11 lines

Is that what we want?

Also I understand that Hyper-V would make the device cease operation if the feature VIRTIO_PMEM_F_SHMEM_REGION was not accepted by the driver. I believe I've read that in some previous email.

Do we need a 'MAY fail to operate further if' statement? Anything that ain't guarded by feature bits ain't optional, and anything that is guarded by feature bits is optional unless explicitly stated otherwise.

Quoted 8 lines

The device can always fail the setting of FEATURES_OK, see 2.2.2:

"The device SHOULD accept any valid subset of features the driver accepts, otherwise it MUST fail to set the FEATURES_OK device status bit when the driver writes it."

I'd argue that "SHMEM_REGION not set" can be considered a non-valid subset by the device. (Also, there's a SHOULD in there.)

Yes, but that is not very helpful for somebody reading the spec without having the context of this discussion.

To put it differently, why do we use the "MAY fail to operate further if" statement for:
* VIRTIO_F_ACCESS_PLATFORM not accepted
* VIRTIO_F_ORDER_PLATFORM not accepted
* VIRTIO_F_VERSION_1 not offered

If "SHMEM_REGION not accepted" is covered by 2.2.2 and thus superfluous, then so is "ACCESS_PLATFORM not accepted".

IMHO in the sentence quoted by you "valid subset" should be interpreted as a subset that is closed to the "depends on" relationship between feature sets. I.e. a subset is invalid, if it contains at least one feature such that some feature required by it is not in the subset.

If we say that any device implementation can proclaim any subset of features invalid, than the the first part of the sentence does not have any meaning to it.

You are definitively right about the SHOULD, it is not a MUST.

I believe the whole idea behind features is to facilitate extensibility and compatibility.

Usually a driver can safely ignore features it does not know or care about,and the device SHOULD still work according to the expectations of the driver.

But some feature bits are not like that. For example it ain't safe for the driver to ignore ACCESS_PLATFORM, for obvious reasons. And we express that with the "MAY fail to operate further if" statement.

Quoted 10 lines
Quoted 9 lines
Quoted 4 lines

Regards,
Halil

No problem, thanks for joining in! :) I think your train of thought makes sense. You're right, for Hyper-V we can't support any driver that doesn't support VIRTIO_PMEM_F_SHMEM_REGION. So we have no choice but to set the config fields to zero from the start. There's no meaningful value we can place in those fields.

Quoted 4 lines

I think this is fine: we only require the fields to have meaningful contents if SHMEM_REGION has not been negotiated -- as long as the device has not accepted FEATURES_OK from the driver, nothing has been negotiated respectively not been negotiated.

I have issues accepting the tristate nature of 'has been negotiated'. I don't think that is intuitive. If this really what we want we should be very clear about it.

In my opinion feature negotiation is usually an opt-in. Before the opt-in the device is to give us the "compatible" or "old" behavior.

Quoted 12 lines
Quoted 11 lines

I'm trying to figure out how best to word this. Saying that devices should set config fields prior to FEATURE_OK is redundant. Should I add a section saying that config fields MAY be set to 0 if VIRTIO_PMEM_F_SHMEM_REGION is offerred (such as in the Hyper-V case)? And that if it's set to 0 and the feature isn't negotiated, the device must reject the driver? I'd also have to add in a section saying that the pmem region shouldn't be touched by the driver prior to DRIVER_OK being set, to save drivers from accessing the zeroed values placed in the config. Additionally, encouraging devices to set the config fields to 0 once the feature has been negotiated seems like the right thing to do, to catch bugs?

Quoted 2 lines

I don't think we can prohibit the driver from reading the config space, that is explicitly allowed in the general case.

I agree. I believe the specification should tell us which features are essential by design, and which features are optional by design. We clearly have both kinds of features, right?

Quoted 4 lines

We probably can add a non-normative extra statement, though, that a device may present zeroes in the config space before feature negotiation has finished if it doesn't support !SHMEM_REGION.

I would not tie that to 'before feature negotiation' or 'does not support SHMEM'. We can say something like:

If the value of the field \filed{size} is 0, then the \field{start} does not contain the physical address of the persistent memory region, and the persistent memory region is advertised by means different than via the config space.

Quoted 11 lines

That would also give
the driver a way to discover that it doesn't make sense to continue without negotiating SHMEM_REGION. No extra normative statement, but should make it a bit more clear to those trying to implement this.

Quoted 4 lines

I'm not sure about the 'MAY fail to operate further statement'. Couldn't there be lots of reasons why a device rejects a driver? And wouldn't those be implementation details of each device? Is it common to document those?

We probably don't need such a statement (see above).

I disagree.

Quoted 5 lines

As voting for this is open until tomorrow, we should quickly decide what to do. My preference would be to merge it as-is and do a clarification on top.

I'm fine with merging as is.

Quoted 17 lines

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#12 lore raw
Quoted 80 lines · Halil Pasic
Quoted 65 lines · On Mon, 22 Nov 2021 12:58:01 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 33 lines · Taylor Stark
Quoted 32 lines · Halil Pasic

On Wed, 10 Nov 2021 10:55:55 -0800
[email protected] wrote:
[..]

Quoted 29 lines
virtio-pmem.tex +9 -6
@@ -36,22 +39,45 @@ \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /
 \end{lstlisting}
 
 \begin{description}
-\item[\field{start}] contains the physical address of the first byte of the persistent memory region.
+\item[\field{start}] contains the physical address of the first byte of the
+persistent memory region, if VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 
-\item[\field{size}] contains the length of this address range.
+\item[\field{size}] contains the length of this address range, if
+VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated.
 \end{description}
 
+\subsection{Device Initialization}\label{sec:Device Types / PMEM Device / Device Initialization}
+
+The device indicates the guest physical address to the driver in one of two ways:
 \begin{enumerate}
-\item Driver vpmem start is read from \field{start}.
-\item Driver vpmem end is read from \field{size}.
+\item As a guest absolute address, using virtio_pmem_config.
+\item As a shared memory region.
 \end{enumerate}
 
-\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization}
-
 The driver determines the start address and size of the persistent memory region in preparation for reading or writing data.
 
 The driver initializes req_vq in preparation for making flush requests.
 
Quoted 8 lines
Quoted 8 lines

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device /

Quoted 8 lines
virtio-pmem.tex +8 -0
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / PMEM Device / Device Initialization}
+
+If VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, the device MUST indicate the
+guest physical address as a shared memory region. The device MUST use shared
+memory region ID 0. The device SHOULD set \field{start} and \field{size} to zero.
+
+
+If VIRTIO_PMEM_F_SHMEM_REGION has not been negotiated, the device MUST indicate  

[for the sake of the argument made below, note the 'has not been negotiated' here]

Quoted 11 lines
Quoted 11 lines

virtio-pmem.tex \subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / · line 70

Quoted 3 lines
virtio-pmem.tex +3 -0
+the guest physical address as a guest absolute address. The device MUST set
+\field{start} to the absolute address and \field{size} to the size of the
+address range, in bytes.  

Sorry for joining in this late.

I'm wondering if the quoted parts implies that the device SHOULD change its config space (fields start and size) when the driver sets FEATURES_OK, and the feature is VIRTIO_PMEM_F_SHMEM_REGION set at the device (it was offered, and got set).

Quoted 6 lines
Quoted 6 lines

My train of thought is: initially no feature is considered negotiated, and config space access is allowed before feature negotiation is completed. That means the at this stage the device must indicate via the config space, and thus \field{size} != 0. But as a part of the transition 'features not negotiated' -> 'features negotiated' the device needs to go '\field{size} != 0' -> '\field{size} == 0'

Quoted 2 lines

It has been advised that it SHOULD put 0 there, but I don't consider that a must, just a good idea.

Right I agree with that. So we say that the config state update is a good idea as well?

BTW the definition of SHOULD:
SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.

Quoted 6 lines

Maybe I should have used 'should go' instead of 'needs to go' but that doesn't change the fact, that the spec currently regards this config space update as RECOMMENDED.

Conny, is that what we want?

I really don't understand your problem with this? Yes, we recommend that the device does this, but if there's anything that makes it difficult for the device, it can omit it.

Quoted 25 lines
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines

Is that what we want?

Quoted 7 lines
Quoted 7 lines

Also I understand that Hyper-V would make the device cease operation if the feature VIRTIO_PMEM_F_SHMEM_REGION was not accepted by the driver. I believe I've read that in some previous email.

Do we need a 'MAY fail to operate further if' statement? Anything that ain't guarded by feature bits ain't optional, and anything that is guarded by feature bits is optional unless explicitly stated otherwise.

Quoted 8 lines

The device can always fail the setting of FEATURES_OK, see 2.2.2:

"The device SHOULD accept any valid subset of features the driver accepts, otherwise it MUST fail to set the FEATURES_OK device status bit when the driver writes it."

I'd argue that "SHMEM_REGION not set" can be considered a non-valid subset by the device. (Also, there's a SHOULD in there.)

Yes, but that is not very helpful for somebody reading the spec without having the context of this discussion.

Quoted 8 lines

To put it differently, why do we use the "MAY fail to operate further if" statement for:
* VIRTIO_F_ACCESS_PLATFORM not accepted
* VIRTIO_F_ORDER_PLATFORM not accepted
* VIRTIO_F_VERSION_1 not offered

If "SHMEM_REGION not accepted" is covered by 2.2.2 and thus superfluous, then so is "ACCESS_PLATFORM not accepted".

I'd say it is indeed not really needed. The driver must be able to deal with the device rejecting FEATURES_OK for a certain feature set.

Quoted 15 lines

IMHO in the sentence quoted by you "valid subset" should be interpreted as a subset that is closed to the "depends on" relationship between feature sets. I.e. a subset is invalid, if it contains at least one feature such that some feature required by it is not in the subset.

If we say that any device implementation can proclaim any subset of features invalid, than the the first part of the sentence does not have any meaning to it.

You are definitively right about the SHOULD, it is not a MUST.

I believe the whole idea behind features is to facilitate extensibility and compatibility.

Quoted 7 lines

Usually a driver can safely ignore features it does not know or care about,and the device SHOULD still work according to the expectations of the driver.

But some feature bits are not like that. For example it ain't safe for the driver to ignore ACCESS_PLATFORM, for obvious reasons. And we express that with the "MAY fail to operate further if" statement.

Well if you want to highlight a bit, highlight it. But I don't think this is really needed here. Things like ACCESS_PLATFORM are not as obvious.

Quoted 17 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines

Regards,
Halil

No problem, thanks for joining in! :) I think your train of thought makes sense. You're right, for Hyper-V we can't support any driver that doesn't support VIRTIO_PMEM_F_SHMEM_REGION. So we have no choice but to set the config fields to zero from the start. There's no meaningful value we can place in those fields.

Quoted 4 lines

I think this is fine: we only require the fields to have meaningful contents if SHMEM_REGION has not been negotiated -- as long as the device has not accepted FEATURES_OK from the driver, nothing has been negotiated respectively not been negotiated.

Quoted 6 lines

I have issues accepting the tristate nature of 'has been negotiated'. I don't think that is intuitive. If this really what we want we should be very clear about it.

In my opinion feature negotiation is usually an opt-in. Before the opt-in the device is to give us the "compatible" or "old" behavior.

But a device is always free to require the "new" behaviour; otherwise, we could never ditch old interfaces.

Quoted 13 lines
Quoted 13 lines
Quoted 12 lines

I'm trying to figure out how best to word this. Saying that devices should set config fields prior to FEATURE_OK is redundant. Should I add a section saying that config fields MAY be set to 0 if VIRTIO_PMEM_F_SHMEM_REGION is offerred (such as in the Hyper-V case)? And that if it's set to 0 and the feature isn't negotiated, the device must reject the driver? I'd also have to add in a section saying that the pmem region shouldn't be touched by the driver prior to DRIVER_OK being set, to save drivers from accessing the zeroed values placed in the config. Additionally, encouraging devices to set the config fields to 0 once the feature has been negotiated seems like the right thing to do, to catch bugs?

Quoted 6 lines
Quoted 2 lines

I don't think we can prohibit the driver from reading the config space, that is explicitly allowed in the general case.

I agree. I believe the specification should tell us which features are essential by design, and which features are optional by design. We clearly have both kinds of features, right?

It also depends on the individual device which features are essential.

Quoted 6 lines
Quoted 5 lines

We probably can add a non-normative extra statement, though, that a device may present zeroes in the config space before feature negotiation has finished if it doesn't support !SHMEM_REGION.

Quoted 7 lines

I would not tie that to 'before feature negotiation' or 'does not support SHMEM'. We can say something like:

If the value of the field \filed{size} is 0, then the \field{start} does not contain the physical address of the persistent memory region, and the persistent memory region is advertised by means different than via the config space.

Isn't that self-evident? If the device does not provide the information in the config space, it uses a different mechanism.

Quoted 7 lines
Quoted 7 lines

That would also give
the driver a way to discover that it doesn't make sense to continue without negotiating SHMEM_REGION. No extra normative statement, but should make it a bit more clear to those trying to implement this.

Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

I'm not sure about the 'MAY fail to operate further statement'. Couldn't there be lots of reasons why a device rejects a driver? And wouldn't those be implementation details of each device? Is it common to document those?

We probably don't need such a statement (see above).

I disagree.

I still stand by my opinion.

Quoted 8 lines
Quoted 6 lines

As voting for this is open until tomorrow, we should quickly decide what to do. My preference would be to merge it as-is and do a clarification on top.

I'm fine with merging as is.

OK; if everyone agrees we need changes on top, we can add them later.

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/