Skip to contentSkip to conversation

Large shared memory regions

PATCH Superseded Dr. David Alan Gilbert (git) 2 participants 17 messages Mar 20, 2019 – Jun 27, 2019

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v4 cover Large shared memory regions

From: "Dr. David Alan Gilbert" <[email protected]>

This series formalises the idea of shared memory regions I'd
previously discussed and is intended for use with virtio-fs.

I've split it into three parts:

a) A general definition of the idea

b) A definition on virtio-pci using the layout previously
discussed; this still uses the list of capabilities.

c) A definition on virtio-mmio.

This v4 has one small tweak to the addressing comment in patch 1, based partially on Stefan's feedback to v3.

Dr. David Alan Gilbert (3):
shared memory: Define shared memory regions
shared memory: Define PCI capability
shared memory: Define mmio registers

 conformance.tex |  1 +
 content.tex     | 69 +++++++++++++++++++++++++++++++++++++++++++++++++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++
 3 files changed, 110 insertions(+)
 create mode 100644 shared-mem.tex
Signature · 18 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#1 lore raw

PATCH v4 1/3 shared memory: Define shared memory regions

From: "Dr. David Alan Gilbert" <[email protected]>

Define the requirements and idea behind shared memory regions.

---
 conformance.tex |  1 +
 content.tex     |  2 ++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)
 create mode 100644 shared-mem.tex
conformance.tex +1 -0
index ad7e82e..f6c0d50 100644
@@ -195,6 +195,7 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
content.tex +2 -0
index ede0ef6..8cd1a38 100644
@@ -371,6 +371,8 @@ making any more buffers available. When VIRTIO_F_NOTIFICATION_DATA
 has been negotiated, these notifications would then have
 identical \field{next_off} and \field{next_wrap} values.
 
+\input{shared-mem.tex}
+
 \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
 
 We start with an overview of device initialization, then expand on the
shared-mem.tex new file +40 -0
new file mode 100644
index 0000000..6dc506a
@@ -0,0 +1,40 @@
+\section{Shared Memory Regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+
+Shared memory regions are an additional facility
+available to devices that need a region of memory that's
+continuously shared between the host and the guest, rather
+than passed between them in the way virtqueue elements are.
+
+Example uses include shared caches and version pools for versioned
+data structures.
+
+The region is chosen by the host and presented to the guest, as
+such it is useful in situations where the memory is accessed on
+the host by other libraries that can't safely access guest RAM.
+
+A device may have multiple shared memory regions associated with
+it.  Each region has a \field{shmid} to identify it, the meaning
+of which is device-specific.
+
+Enumeration and location of shared memory regions is performed
+using a transport-specific data structure and mechanism.
+
+Memory consistency rules vary depending on the region and the
+device and they will be specified as required by each device.
+
+\subsection{Addressing within regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Addressing within regions }
+
+References into shared memory regions are represented as offsets from
+the beginning of the region instead of absolute memory addresses.
+Offsets are used both for references between structures stored
+within shared memory and for requests placed in virtqueues that
+refer to shared memory.
+The \field{shmid} may be explicit or may be inferred from the
+context of the reference.
+
+\devicenormative{\subsection}{Shared Memory Regions}{Basic Facilities of a Virtio
+Device / Shared Memory Regions}
+Shared memory regions MUST NOT expose shared memory regions which
+are used to control the operation of the device, nor to stream
+data.
+
Signature · 18 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#7 lore raw

PATCH v4 2/3 shared memory: Define PCI capability

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +37 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.
+
+While most capabilities can not be repeated, multiple
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields MUST be contained within the declared bar.
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 18 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#16 lore raw

PATCH v4 3/3 shared memory: Define mmio registers

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define an MMIO interface to discover and map shared memory regions.

---
 content.tex | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
content.tex +30 -0
index ddbf949..f2195be 100644
@@ -1726,6 +1726,33 @@ All register values are organized as Little Endian.
     selected by writing to \field{QueueSel}.
   }
   \hline 
+  \mmioreg{SHMSel}{Shared memory id}{0x0ac}{W}{%
+    Writing to this register selects the shared memory region \ref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+    following operations on \field{SHMLenLow}, \field{SHMLenHigh},
+    \field{SHMBaseLow} and \field{SHMBaseHigh} apply to.
+  }
+  \hline 
+  \mmiodreg{SHMLenLow}{SHMLenHigh}{Shared memory region 64 bit long length}{0x0b0}{0x0xb4}{R}{%
+    These registers return the length of the shared memory
+    region in bytes, as defined by the device for the region selected by
+    the \field{SHMSel} register.  The lower 32 bits of the length
+    are read from \field{SHMLenLow} and the higher 32 bits from
+    \field{SHMLenHigh}.  Reading from a non-existent
+    region (i.e. where the ID written to \field{SHMSel} is unused)
+    results in a length of -1.
+  }
+  \hline 
+  \mmiodreg{SHMBaseLow}{SHMBaseHigh}{Shared memory region 64 bit long physical address}{0x0b8}{0x0xbc}{R}{%
+    The driver reads these registers to discover the base address
+    of the region in physical address space.  This address is
+    chosen by the device (or other part of the VMM).
+    The lower 32 bits of the address are read from \field{SHMBaseLow}
+    with the higher 32 bits from \field{SHMBaseHigh}.  Reading
+    from a non-existent region (i.e. where the ID written to
+    \field{SHMSel} is unused) results in a base address of
+    0xffffffffffffffff.
+  }
+  \hline 
   \mmioreg{ConfigGeneration}{Configuration atomicity value}{0x0fc}{R}{
     Reading from this register returns a value describing a version of the device-specific configuration space (see \field{Config}).
     The driver can then access the configuration space and, when finished, read \field{ConfigGeneration} again.
@@ -1815,6 +1842,9 @@ If both values are valid, it MUST read \field{DeviceID}
 and if its value is zero (0x0) MUST abort initialization and
 MUST NOT access any other register.
 
+Drivers not expecting shared memory MUST NOT use the shared
+memory registers.
+
 Further initialization MUST follow the procedure described in
 \ref{sec:General Initialization And Device Operation / Device Initialization}~\nameref{sec:General Initialization And Device Operation / Device Initialization}.
 
Signature · 18 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
#2 lore raw

Re: [virtio-comment] [PATCH v4 1/3] shared memory: Define shared memory regions

In reply to Dr. David Alan Gilbert (git)
Quoted 44 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the requirements and idea behind shared memory regions.

---
 conformance.tex |  1 +
 content.tex     |  2 ++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)
 create mode 100644 shared-mem.tex
conformance.tex +1 -0
index ad7e82e..f6c0d50 100644
@@ -195,6 +195,7 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
content.tex +2 -0
index ede0ef6..8cd1a38 100644
@@ -371,6 +371,8 @@ making any more buffers available. When VIRTIO_F_NOTIFICATION_DATA
 has been negotiated, these notifications would then have
 identical \field{next_off} and \field{next_wrap} values.
 
+\input{shared-mem.tex}
+
 \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
 
 We start with an overview of device initialization, then expand on the
shared-mem.tex new file +0 -0
new file mode 100644
index 0000000..6dc506a

shared-mem.tex line 6

Quoted 6 lines
shared-mem.tex new file +6 -0
@@ -0,0 +1,40 @@
+\section{Shared Memory Regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+
+Shared memory regions are an additional facility
+available to devices that need a region of memory that's
+continuously shared between the host and the guest, rather
+than passed between them in the way virtqueue elements are.

Please rewrite host->device, guest->driver

shared-mem.tex line 13

Quoted 7 lines
shared-mem.tex new file +7 -0
+
+Example uses include shared caches and version pools for versioned
+data structures.
+
+The region is chosen by the host and presented to the guest, as
+such it is useful in situations where the memory is accessed on
+the host by other libraries that can't safely access guest RAM.

I'm not sure what does above mean in context of virtio. Is this an example?
So maybe:
For example, with device being part of a hypervisor and driver being part of a VM guest etc etc

shared-mem.tex line 20

Quoted 7 lines
shared-mem.tex new file +7 -0
+
+A device may have multiple shared memory regions associated with
+it.  Each region has a \field{shmid} to identify it, the meaning
+of which is device-specific.
+
+Enumeration and location of shared memory regions is performed
+using a transport-specific data structure and mechanism.

Let's just say in a transport-specific way.

Quoted 9 lines
shared-mem.tex new file +9 -0
+
+Memory consistency rules vary depending on the region and the
+device and they will be specified as required by each device.
+
+\subsection{Addressing within regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Addressing within regions }
+
+References into shared memory regions are represented as offsets from
+the beginning of the region instead of absolute memory addresses.
+Offsets are used both for references between structures stored

shared-mem.tex line 39

Quoted 10 lines
shared-mem.tex new file +10 -0
+within shared memory and for requests placed in virtqueues that
+refer to shared memory.
+The \field{shmid} may be explicit or may be inferred from the
+context of the reference.
+
+\devicenormative{\subsection}{Shared Memory Regions}{Basic Facilities of a Virtio
+Device / Shared Memory Regions}
+Shared memory regions MUST NOT expose shared memory regions which
+are used to control the operation of the device, nor to stream
+data.

regions must not expose regions. confused.
typo?

Also, what does stream data mean?

Quoted 20 lines
shared-mem.tex new file +1 -0
+
Signature · 18 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#8 lore raw

[virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Dr. David Alan Gilbert (git)
Quoted 32 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +6 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 10 lines
content.tex +10 -0
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 9 lines
content.tex +9 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 2 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

Quoted 14 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#9 lore raw

Re: [virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

Quoted 48 lines · Michael S. Tsirkin
Quoted 32 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +6 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 10 lines
content.tex +10 -0
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

Quoted 13 lines

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 9 lines
content.tex +9 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

And maybe we should set a bit in some other reserved byte so people can generally use 64 bit offsets/lengths?

Quoted 39 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 3 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

Quoted 14 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#3 lore raw

Re: [virtio-comment] [PATCH v4 1/3] shared memory: Define shared memory regions

In reply to Michael S. Tsirkin
Quoted 38 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 37 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the requirements and idea behind shared memory regions.

---
 conformance.tex |  1 +
 content.tex     |  2 ++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)
 create mode 100644 shared-mem.tex
conformance.tex +1 -0
index ad7e82e..f6c0d50 100644
@@ -195,6 +195,7 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
content.tex +2 -0
index ede0ef6..8cd1a38 100644
@@ -371,6 +371,8 @@ making any more buffers available. When VIRTIO_F_NOTIFICATION_DATA
 has been negotiated, these notifications would then have
 identical \field{next_off} and \field{next_wrap} values.
 
+\input{shared-mem.tex}
+
 \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
 
Quoted 15 lines

shared-mem.tex line 6

Quoted 13 lines
content.tex +0 -0
 We start with an overview of device initialization, then expand on the
shared-mem.tex new file +6 -0
new file mode 100644
index 0000000..6dc506a
@@ -0,0 +1,40 @@
+\section{Shared Memory Regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+
+Shared memory regions are an additional facility
+available to devices that need a region of memory that's
+continuously shared between the host and the guest, rather
+than passed between them in the way virtqueue elements are.

Please rewrite host->device, guest->driver

Done.

Quoted 6 lines
Quoted 6 lines
shared-mem.tex new file +6 -0
+
+Example uses include shared caches and version pools for versioned
+data structures.
+
+The region is chosen by the host and presented to the guest, as
+such it is useful in situations where the memory is accessed on
Quoted 8 lines

shared-mem.tex line 13

Quoted 1 line
shared-mem.tex new file +1 -0
+the host by other libraries that can't safely access guest RAM.

I'm not sure what does above mean in context of virtio. Is this an example?
So maybe:
For example, with device being part of a hypervisor and driver being part of a VM guest etc etc

It means the memory can be allocated by another library in the host and then passed into the guest; the alternative is where the guest gives a pointer into the guests own RAM.

Quoted 9 lines

shared-mem.tex line 20

Quoted 7 lines
shared-mem.tex new file +7 -0
+
+A device may have multiple shared memory regions associated with
+it.  Each region has a \field{shmid} to identify it, the meaning
+of which is device-specific.
+
+Enumeration and location of shared memory regions is performed
+using a transport-specific data structure and mechanism.

Let's just say in a transport-specific way.

Done.

Quoted 23 lines
Quoted 9 lines
shared-mem.tex new file +9 -0
+
+Memory consistency rules vary depending on the region and the
+device and they will be specified as required by each device.
+
+\subsection{Addressing within regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Addressing within regions }
+
+References into shared memory regions are represented as offsets from
+the beginning of the region instead of absolute memory addresses.
+Offsets are used both for references between structures stored

shared-mem.tex line 39

Quoted 10 lines
shared-mem.tex new file +10 -0
+within shared memory and for requests placed in virtqueues that
+refer to shared memory.
+The \field{shmid} may be explicit or may be inferred from the
+context of the reference.
+
+\devicenormative{\subsection}{Shared Memory Regions}{Basic Facilities of a Virtio
+Device / Shared Memory Regions}
+Shared memory regions MUST NOT expose shared memory regions which
+are used to control the operation of the device, nor to stream
+data.

regions must not expose regions. confused.
typo?

Quoted 1 line

Also, what does stream data mean?

I'm trying to find a way to say that people shouldn't side-step virtio queues by putting everything in a big blob of shared memory.

Better wording welcome.

Dave

Quoted 13 lines
Quoted 13 lines
shared-mem.tex new file +1 -0
+
Signature · 9 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#4 lore raw

Re: [virtio-comment] [PATCH v4 1/3] shared memory: Define shared memory regions

Quoted 72 lines · Dr. David Alan Gilbert
Quoted 38 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 37 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the requirements and idea behind shared memory regions.

---
 conformance.tex |  1 +
 content.tex     |  2 ++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)
 create mode 100644 shared-mem.tex
conformance.tex +1 -0
index ad7e82e..f6c0d50 100644
@@ -195,6 +195,7 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
content.tex +2 -0
index ede0ef6..8cd1a38 100644
@@ -371,6 +371,8 @@ making any more buffers available. When VIRTIO_F_NOTIFICATION_DATA
 has been negotiated, these notifications would then have
 identical \field{next_off} and \field{next_wrap} values.
 
+\input{shared-mem.tex}
+
 \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
 
Quoted 15 lines

shared-mem.tex line 6

Quoted 13 lines
content.tex +0 -0
 We start with an overview of device initialization, then expand on the
shared-mem.tex new file +6 -0
new file mode 100644
index 0000000..6dc506a
@@ -0,0 +1,40 @@
+\section{Shared Memory Regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+
+Shared memory regions are an additional facility
+available to devices that need a region of memory that's
+continuously shared between the host and the guest, rather
+than passed between them in the way virtqueue elements are.

Please rewrite host->device, guest->driver

Done.

Quoted 6 lines
Quoted 6 lines
shared-mem.tex new file +6 -0
+
+Example uses include shared caches and version pools for versioned
+data structures.
+
+The region is chosen by the host and presented to the guest, as
+such it is useful in situations where the memory is accessed on
Quoted 8 lines

shared-mem.tex line 13

Quoted 1 line
shared-mem.tex new file +1 -0
+the host by other libraries that can't safely access guest RAM.

I'm not sure what does above mean in context of virtio. Is this an example?
So maybe:
For example, with device being part of a hypervisor and driver being part of a VM guest etc etc

Quoted 3 lines

It means the memory can be allocated by another library in the host and then passed into the guest; the alternative is where the guest gives a pointer into the guests own RAM.

But we don't know driver is guest and device is host.

This is why I suggested that it's an example:
for example if driver is in guest and device is in host software then another host library can map it.

Quoted 35 lines
Quoted 9 lines

shared-mem.tex line 20

Quoted 7 lines
shared-mem.tex new file +7 -0
+
+A device may have multiple shared memory regions associated with
+it.  Each region has a \field{shmid} to identify it, the meaning
+of which is device-specific.
+
+Enumeration and location of shared memory regions is performed
+using a transport-specific data structure and mechanism.

Let's just say in a transport-specific way.

Done.

Quoted 23 lines
Quoted 9 lines
shared-mem.tex new file +9 -0
+
+Memory consistency rules vary depending on the region and the
+device and they will be specified as required by each device.
+
+\subsection{Addressing within regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Addressing within regions }
+
+References into shared memory regions are represented as offsets from
+the beginning of the region instead of absolute memory addresses.
+Offsets are used both for references between structures stored

shared-mem.tex line 39

Quoted 10 lines
shared-mem.tex new file +10 -0
+within shared memory and for requests placed in virtqueues that
+refer to shared memory.
+The \field{shmid} may be explicit or may be inferred from the
+context of the reference.
+
+\devicenormative{\subsection}{Shared Memory Regions}{Basic Facilities of a Virtio
+Device / Shared Memory Regions}
+Shared memory regions MUST NOT expose shared memory regions which
+are used to control the operation of the device, nor to stream
+data.

regions must not expose regions. confused.
typo?

Quoted 8 lines
Quoted 1 line

Also, what does stream data mean?

I'm trying to find a way to say that people shouldn't side-step virtio queues by putting everything in a big blob of shared memory.

Better wording welcome.

Dave

I think writes and read to these regions must not have side effects. Is this what you are trying to say?

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines
shared-mem.tex new file +1 -0
+
Signature · 9 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 2 lines

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#13 lore raw

[virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Michael S. Tsirkin
Quoted 39 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 38 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +12 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
Quoted 8 lines

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 4 lines
content.tex +4 -0
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

Do you mean changing the def of virtio_pci_cap to: ? struct virtio_pci_cap {

        __u8 cap_vndr;          /* Generic PCI field: PCI_CAP_ID_VNDR */
        __u8 cap_next;          /* Generic PCI field: next ptr. */
        __u8 cap_len;           /* Generic PCI field: capability length */
        __u8 cfg_type;          /* Identifies the structure. */
        __u8 bar;               /* Where to find it. */
        __u8 id;                /* Unique ID for this region */
        __u8 padding[2];        /* Pad to full dword. */
        __le32 offset;          /* Offset within bar. */
        __le32 length;          /* Length of the structure, in bytes. */

};

Quoted 5 lines
Quoted 5 lines
content.tex +5 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
Quoted 8 lines

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 4 lines
content.tex +4 -0
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

Would we then just rename virtio_pci_shm_cap to virtio_pci_cap64 after moving the id to virito_pci_cap?

Quoted 6 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 4 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

Done.

Quoted 10 lines

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

I've done that as:
The region defined by the combination of the \field {cap.offset}, \field {cap.offset_hi}, and \field {cap.length}, \field {cap.length_hi} fields MUST be contained within the declared bar.

is that right, but the base and hi are in separate structures.

Quoted 6 lines
Quoted 6 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
Quoted 8 lines
Quoted 8 lines

The VIRTIO_PCI_CAP_PCI_CFG capability

Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#10 lore raw

Re: [virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Michael S. Tsirkin
Quoted 63 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 48 lines · Michael S. Tsirkin
Quoted 32 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +6 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 10 lines
content.tex +10 -0
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

Quoted 13 lines

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 9 lines
content.tex +9 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

Quoted 2 lines

And maybe we should set a bit in some other reserved byte so people can generally use 64 bit offsets/lengths?

Is it actually any cleaner? I don't know - where would you put that bit?

Dave

Quoted 39 lines
Quoted 39 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 3 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

Quoted 14 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#14 lore raw

[virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Dr. David Alan Gilbert
Quoted 60 lines · Dr. David Alan Gilbert
Quoted 39 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 38 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +12 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
Quoted 8 lines

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 4 lines
content.tex +4 -0
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

Do you mean changing the def of virtio_pci_cap to: ? struct virtio_pci_cap {

        __u8 cap_vndr;          /* Generic PCI field: PCI_CAP_ID_VNDR */
        __u8 cap_next;          /* Generic PCI field: next ptr. */
        __u8 cap_len;           /* Generic PCI field: capability length */
        __u8 cfg_type;          /* Identifies the structure. */
        __u8 bar;               /* Where to find it. */
        __u8 id;                /* Unique ID for this region */
        __u8 padding[2];        /* Pad to full dword. */
        __le32 offset;          /* Offset within bar. */
        __le32 length;          /* Length of the structure, in bytes. */
Quoted 1 line

};

Something like this, yes.

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
content.tex +5 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
Quoted 8 lines

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 4 lines
content.tex +4 -0
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

Quoted 2 lines

Would we then just rename virtio_pci_shm_cap to virtio_pci_cap64 after moving the id to virito_pci_cap?

Right except we then need to document that only shm uses cap64, others use cap. And I guess for forward compatibility set a bit somewhere - within bar maybe?

Overall I'm no longer sure it's a good idea. you decide.

Quoted 20 lines
Quoted 6 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 4 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

Done.

Quoted 10 lines

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

Quoted 6 lines

I've done that as:
The region defined by the combination of the \field {cap.offset}, \field {cap.offset_hi}, and \field {cap.length}, \field {cap.length_hi} fields MUST be contained within the declared bar.

is that right, but the base and hi are in separate structures.

ok

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 1 line
2.20.1
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#11 lore raw

Re: [virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Dr. David Alan Gilbert
Quoted 64 lines · Dr. David Alan Gilbert
Quoted 63 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 61 lines · Michael S. Tsirkin
Quoted 32 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +6 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 10 lines
content.tex +10 -0
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 9 lines
content.tex +9 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

Quoted 6 lines
Quoted 2 lines

And maybe we should set a bit in some other reserved byte so people can generally use 64 bit offsets/lengths?

Is it actually any cleaner? I don't know - where would you put that bit?

Dave

We'd put it either in bar or cfg type.

No, I'm not sure it's strictly required.
It saves a bit of config space but increases
complexity for guests ....

Quoted 47 lines
Quoted 39 lines
Quoted 39 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 3 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

Quoted 14 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 2 lines

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#12 lore raw

[virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Dr. David Alan Gilbert
Quoted 64 lines · Dr. David Alan Gilbert
Quoted 63 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 61 lines · Michael S. Tsirkin
Quoted 32 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +6 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 10 lines
content.tex +10 -0
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 9 lines
content.tex +9 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

Quoted 6 lines
Quoted 2 lines

And maybe we should set a bit in some other reserved byte so people can generally use 64 bit offsets/lengths?

Is it actually any cleaner?

Dave

I am just saying this: let's assume some other config type will also want to use 64 bit offsets down the road. If we do it will be nice to be able, down the line, to write
if (bit set)

	64 bit
else
	32 bit

without special-casing this config type.

Quoted 1 line

I don't know - where would you put that bit?

options include cfg type and bar.
I think I prefer bar since their number is definitely limited by pci spec.

Quoted 47 lines
Quoted 39 lines
Quoted 39 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 3 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

Quoted 14 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
 
 The VIRTIO_PCI_CAP_PCI_CFG capability
Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 6 lines

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#15 lore raw

[virtio-comment] Re: [virtio-dev] [PATCH v4 2/3] shared memory: Define PCI capability

In reply to Dr. David Alan Gilbert
Quoted 76 lines · Dr. David Alan Gilbert
Quoted 39 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 38 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the PCI capability used for enumerating shared memory regions.

---
 content.tex | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
content.tex +12 -0
index 8cd1a38..ddbf949 100644
@@ -688,6 +688,8 @@ The fields are interpreted as follows:
 #define VIRTIO_PCI_CAP_DEVICE_CFG        4
 /* PCI configuration access */
 #define VIRTIO_PCI_CAP_PCI_CFG           5
+/* Shared memory region */
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
 \end{lstlisting}
 
         Any other value is reserved for future use.
@@ -1052,6 +1054,41 @@ any device type which has a device-specific configuration.
 
 The \field{offset} for the device-specific configuration MUST be 4-byte aligned.
 
+\subsubsection{Shared memory capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+Shared memory regions \ref{sec:Basic Facilities of a Virtio
+Device / Shared Memory Regions} are enumerated on the PCI transport
+as a sequence of VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities, one per region.
+
+The capability is immediately followed by an additional field like so:
+
+\begin{lstlisting}
+struct virtio_pci_shm_cap {
Quoted 8 lines

content.tex any device type which has a device-specific configuration. · line 1070

Quoted 4 lines
content.tex +4 -0
+        struct virtio_pci_cap cap;
+        u32 offset_hi;
+        u32 length_hi;
+        u8  id;

Pls pad, pci capabilities must be a multiple of 4 bytes. BTW virtio_pci_cap has 3 bytes of padding.
We can stick the id there maybe?

Do you mean changing the def of virtio_pci_cap to: ? struct virtio_pci_cap {

        __u8 cap_vndr;          /* Generic PCI field: PCI_CAP_ID_VNDR */
        __u8 cap_next;          /* Generic PCI field: next ptr. */
        __u8 cap_len;           /* Generic PCI field: capability length */
        __u8 cfg_type;          /* Identifies the structure. */
        __u8 bar;               /* Where to find it. */
        __u8 id;                /* Unique ID for this region */
        __u8 padding[2];        /* Pad to full dword. */
        __le32 offset;          /* Offset within bar. */
        __le32 length;          /* Length of the structure, in bytes. */

};

Quoted 5 lines
Quoted 5 lines
content.tex +5 -0
+};
+\end{lstlisting}
+
+Given that the \field{cap.length} and \field{cap.offset} fields
+are only 32 bit, the additional \field{offset_hi} and \field {length_hi}
Quoted 8 lines

content.tex any device type which has a device-specific configuration. · line 1079

Quoted 4 lines
content.tex +4 -0
+fields provide the most significant 32 bits of a total 64 bit offset and
+length within the bar specified by \field{cap.bar}.  \field{id}
+uniquely identifies the given shared memory region using a device
+specific identifier.

I didn't realize we will ever need so much space. I think I prefer moving offset_hi/length_hi to a general virtio_pci_cap64 that others can reuse.

Quoted 2 lines

Would we then just rename virtio_pci_shm_cap to virtio_pci_cap64 after moving the id to virito_pci_cap?

yes. and for simplicity we can for now specify that only shmem can be 64 bit.

Quoted 44 lines
Quoted 6 lines

content.tex any device type which has a device-specific configuration. · line 1081

Quoted 4 lines
content.tex +2 -0
+
+While most capabilities can not be repeated,

I'd drop this part.

Done.

Quoted 10 lines

content.tex any device type which has a device-specific configuration.

Quoted 8 lines

multiple

content.tex +7 -0
+VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capabilities are allowed as long
+as the \field{id} is unique.
+
+\devicenormative{\paragraph}{Device-specific configuration}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
+
+The region defined by the combination of the offsets and length
+fields 

let's do \field{cap.length} \field{cap.offset} etc.

I've done that as:
The region defined by the combination of the \field {cap.offset}, \field {cap.offset_hi}, and \field {cap.length}, \field {cap.length_hi} fields MUST be contained within the declared bar.

is that right, but the base and hi are in separate structures.

Quoted 6 lines
Quoted 6 lines

MUST be contained within the declared bar.

content.tex +3 -0
+
+The \field{id} MUST be unique for any one device instance.
+
 \subsubsection{PCI configuration access capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / PCI configuration access capability}
Quoted 8 lines
Quoted 8 lines

The VIRTIO_PCI_CAP_PCI_CFG capability

Signature · 6 lines
2.20.1


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Quoted 3 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#5 lore raw

Re: [virtio-comment] [PATCH v4 1/3] shared memory: Define shared memory regions

In reply to Dr. David Alan Gilbert
Quoted 72 lines · Dr. David Alan Gilbert
Quoted 38 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 37 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the requirements and idea behind shared memory regions.

---
 conformance.tex |  1 +
 content.tex     |  2 ++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)
 create mode 100644 shared-mem.tex
conformance.tex +1 -0
index ad7e82e..f6c0d50 100644
@@ -195,6 +195,7 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
content.tex +2 -0
index ede0ef6..8cd1a38 100644
@@ -371,6 +371,8 @@ making any more buffers available. When VIRTIO_F_NOTIFICATION_DATA
 has been negotiated, these notifications would then have
 identical \field{next_off} and \field{next_wrap} values.
 
+\input{shared-mem.tex}
+
 \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
 
Quoted 15 lines

shared-mem.tex line 6

Quoted 13 lines
content.tex +0 -0
 We start with an overview of device initialization, then expand on the
shared-mem.tex new file +6 -0
new file mode 100644
index 0000000..6dc506a
@@ -0,0 +1,40 @@
+\section{Shared Memory Regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+
+Shared memory regions are an additional facility
+available to devices that need a region of memory that's
+continuously shared between the host and the guest, rather
+than passed between them in the way virtqueue elements are.

Please rewrite host->device, guest->driver

Done.

Quoted 6 lines
Quoted 6 lines
shared-mem.tex new file +6 -0
+
+Example uses include shared caches and version pools for versioned
+data structures.
+
+The region is chosen by the host and presented to the guest, as
+such it is useful in situations where the memory is accessed on
Quoted 8 lines

shared-mem.tex line 13

Quoted 1 line
shared-mem.tex new file +1 -0
+the host by other libraries that can't safely access guest RAM.

I'm not sure what does above mean in context of virtio. Is this an example?
So maybe:
For example, with device being part of a hypervisor and driver being part of a VM guest etc etc

Quoted 3 lines

It means the memory can be allocated by another library in the host and then passed into the guest; the alternative is where the guest gives a pointer into the guests own RAM.

OK so let's say this like this: memory is allocated by the device as opposed to by driver.
As such this allows configurations where access to guest memory is limited. For example and so .

Quoted 35 lines
Quoted 9 lines

shared-mem.tex line 20

Quoted 7 lines
shared-mem.tex new file +7 -0
+
+A device may have multiple shared memory regions associated with
+it.  Each region has a \field{shmid} to identify it, the meaning
+of which is device-specific.
+
+Enumeration and location of shared memory regions is performed
+using a transport-specific data structure and mechanism.

Let's just say in a transport-specific way.

Done.

Quoted 23 lines
Quoted 9 lines
shared-mem.tex new file +9 -0
+
+Memory consistency rules vary depending on the region and the
+device and they will be specified as required by each device.
+
+\subsection{Addressing within regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Addressing within regions }
+
+References into shared memory regions are represented as offsets from
+the beginning of the region instead of absolute memory addresses.
+Offsets are used both for references between structures stored

shared-mem.tex line 39

Quoted 10 lines
shared-mem.tex new file +10 -0
+within shared memory and for requests placed in virtqueues that
+refer to shared memory.
+The \field{shmid} may be explicit or may be inferred from the
+context of the reference.
+
+\devicenormative{\subsection}{Shared Memory Regions}{Basic Facilities of a Virtio
+Device / Shared Memory Regions}
+Shared memory regions MUST NOT expose shared memory regions which
+are used to control the operation of the device, nor to stream
+data.

regions must not expose regions. confused.
typo?

Quoted 8 lines
Quoted 1 line

Also, what does stream data mean?

I'm trying to find a way to say that people shouldn't side-step virtio queues by putting everything in a big blob of shared memory.

Better wording welcome.

Dave

The way to improve wording would be by specifying what are the disadvantages of storing data in shared memory, and when it should be avoided.

Quoted 31 lines
Quoted 13 lines
Quoted 13 lines
shared-mem.tex new file +1 -0
+
Signature · 9 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

#6 lore raw

[virtio-comment] Re: [virtio-dev] Re: [virtio-comment] [PATCH v4 1/3] shared memory: Define shared memory regions

Quoted 73 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 72 lines · Dr. David Alan Gilbert
Quoted 38 lines · * Michael S. Tsirkin ([email protected]) wrote:
Quoted 37 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Define the requirements and idea behind shared memory regions.

---
 conformance.tex |  1 +
 content.tex     |  2 ++
 shared-mem.tex  | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 43 insertions(+)
 create mode 100644 shared-mem.tex
conformance.tex +1 -0
index ad7e82e..f6c0d50 100644
@@ -195,6 +195,7 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
content.tex +2 -0
index ede0ef6..8cd1a38 100644
@@ -371,6 +371,8 @@ making any more buffers available. When VIRTIO_F_NOTIFICATION_DATA
 has been negotiated, these notifications would then have
 identical \field{next_off} and \field{next_wrap} values.
 
+\input{shared-mem.tex}
+
 \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
 
Quoted 15 lines

shared-mem.tex line 6

Quoted 13 lines
content.tex +0 -0
 We start with an overview of device initialization, then expand on the
shared-mem.tex new file +6 -0
new file mode 100644
index 0000000..6dc506a
@@ -0,0 +1,40 @@
+\section{Shared Memory Regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
+
+Shared memory regions are an additional facility
+available to devices that need a region of memory that's
+continuously shared between the host and the guest, rather
+than passed between them in the way virtqueue elements are.

Please rewrite host->device, guest->driver

Done.

Quoted 14 lines

shared-mem.tex line 13

Quoted 7 lines
shared-mem.tex new file +7 -0
+
+Example uses include shared caches and version pools for versioned
+data structures.
+
+The region is chosen by the host and presented to the guest, as
+such it is useful in situations where the memory is accessed on
+the host by other libraries that can't safely access guest RAM.

I'm not sure what does above mean in context of virtio. Is this an example?
So maybe:
For example, with device being part of a hypervisor and driver being part of a VM guest etc etc

Quoted 8 lines
Quoted 3 lines

It means the memory can be allocated by another library in the host and then passed into the guest; the alternative is where the guest gives a pointer into the guests own RAM.

OK so let's say this like this: memory is allocated by the device as opposed to by driver.
As such this allows configurations where access to guest memory is limited. For example and so .

How about:

The memory region is allocated by the device and presented to the driver. Where the device is implemented in software on a host, this arrangement allows the memory region to be allocated by a library on the host, which the device may not have full control over.

Dave

Quoted 90 lines
Quoted 36 lines
Quoted 10 lines

shared-mem.tex line 20

Quoted 8 lines
shared-mem.tex new file +7 -0
+
+A device may have multiple shared memory regions associated with
+it.  Each region has a \field{shmid} to identify it, the meaning
+of which is device-specific.
+
+Enumeration and location of shared memory regions is performed
+using a transport-specific data structure and mechanism.

Let's just say in a transport-specific way.

Done.

Quoted 23 lines
Quoted 9 lines
shared-mem.tex new file +9 -0
+
+Memory consistency rules vary depending on the region and the
+device and they will be specified as required by each device.
+
+\subsection{Addressing within regions}\label{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Addressing within regions }
+
+References into shared memory regions are represented as offsets from
+the beginning of the region instead of absolute memory addresses.
+Offsets are used both for references between structures stored

shared-mem.tex line 39

Quoted 10 lines
shared-mem.tex new file +10 -0
+within shared memory and for requests placed in virtqueues that
+refer to shared memory.
+The \field{shmid} may be explicit or may be inferred from the
+context of the reference.
+
+\devicenormative{\subsection}{Shared Memory Regions}{Basic Facilities of a Virtio
+Device / Shared Memory Regions}
+Shared memory regions MUST NOT expose shared memory regions which
+are used to control the operation of the device, nor to stream
+data.

regions must not expose regions. confused.
typo?

Quoted 8 lines
Quoted 1 line

Also, what does stream data mean?

I'm trying to find a way to say that people shouldn't side-step virtio queues by putting everything in a big blob of shared memory.

Better wording welcome.

Dave

The way to improve wording would be by specifying what are the disadvantages of storing data in shared memory, and when it should be avoided.

Quoted 31 lines
Quoted 13 lines
Quoted 13 lines
shared-mem.tex new file +1 -0
+
Signature · 9 lines
2.20.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Quoted 8 lines
Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Quoted 8 lines

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Quoted 4 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.

Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/