virtio-pmem: PMEM device spec
PATCH v3 1/1 virtio-pmem: PMEM device spec
Posting virtio specification for virtio pmem device. Virtio pmem is a paravirtualized device which allows the guest to bypass page cache. Virtio pmem kernel driver is merged in Upstream Kernel 5.3. Also, Qemu device is merged in Qemu 4.1.
Signed-off-by Pankaj Gupta <[email protected]>
---
v2->v3:
- Suggested text changes - Stefan
- Removed Virtio flush command details from conformance section {Cornelia, Stefan}
- More generic security text - Cornelia
conformance.tex | 16 ++++++- content.tex | 1 + virtio-pmem.tex | 122 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 137 insertions(+), 2 deletions(-) create mode 100644 virtio-pmem.tex
conformance.tex +14 -2
@@ -31,7 +31,8 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \ref{sec:Conformance / Driver Conformance / Sound Driver Conformance}, \ref{sec:Conformance / Driver Conformance / Memory Driver Conformance}, \ref{sec:Conformance / Driver Conformance / I2C Adapter Driver Conformance} or -\ref{sec:Conformance / Driver Conformance / SCMI Driver Conformance}. +\ref{sec:Conformance / Driver Conformance / SCMI Driver Conformance}, +\ref{sec:Conformance / Driver Conformance / PMEM Driver Conformance}. \item Clause \ref{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance}. \end{itemize}
@@ -55,7 +56,8 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \ref{sec:Conformance / Device Conformance / Sound Device Conformance}, \ref{sec:Conformance / Device Conformance / Memory Device Conformance}, \ref{sec:Conformance / Device Conformance / I2C Adapter Device Conformance} or -\ref{sec:Conformance / Device Conformance / SCMI Device Conformance}. +\ref{sec:Conformance / Device Conformance / SCMI Device Conformance}, +\ref{sec:Conformance / Device Conformance / PMEM Driver Conformance}. \item Clause \ref{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance}. \end{itemize}
@@ -301,6 +303,16 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \item \ref{drivernormative:Device Types / SCMI Device / Device Operation / Setting Up eventq Buffers} \end{itemize} +\conformance{\subsection}{PMEM Driver Conformance}\label{sec:Conformance / Driver Conformance / PMEM Driver Conformance} + +A PMEM driver MUST conform to the following normative statements: + +\begin{itemize} +\item \ref{devicenormative:Device Types / PMEM Device / Device Initialization} +\item \ref{devicenormative:Device Types / PMEM Device / Device Operation / Virtqueue flush} +\item \ref{devicenormative:Device Types / PMEM Device / Device Operation / Virtqueue return} +\end{itemize} + \conformance{\section}{Device Conformance}\label{sec:Conformance / Device Conformance} A device MUST conform to the following normative statements:
content.tex +1 -0
@@ -6583,6 +6583,7 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device \input{virtio-mem.tex} \input{virtio-i2c.tex} \input{virtio-scmi.tex} +\input{virtio-pmem.tex} \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
virtio-pmem.tex new file +122 -0
122 lines not on this page · raw message
Signature · 1 line
2.25.1
Quoted 8 lines · Pankaj Gupta
virtio-pmem.tex new file +2
-0@@ -0,0 +1,122 @@ +\section{PMEM Device}\label{sec:Device Types / PMEM Device} +
virtio-pmem.tex line 12
Quoted 10 lines
virtio-pmem.tex new file +10
-0+The virtio pmem device is a persistent memory (NVDIMM) device +that provides a virtio based asynchronous flush mechanism. This avoids +the need of a separate page cache in the guest and keeps the page cache +only in the host. Under memory pressure, the host makes use of +efficient memory reclaim decisions for page cache pages of all the +guests. This helps to reduce the memory footprint and fits more guests +in the host system. + +The virtio pmem device provides access to byte-addressable persistent +memory. The persist memory is directly accessible range of system memory.
s/is directly accessible range/is a directly accessible range/
Quoted 43 lines
virtio-pmem.tex new file +43
-0+Data written to this memory is made persistent by separately sending a +flush command. Writes that have been flushed are preserved across device +reset and power failure. + +\subsection{Device ID}\label{sec:Device Types / PMEM Device / Device ID} + 27 + +\subsection{Virtqueues}\label{sec:Device Types / PMEM Device / Virtqueues} +\begin{description} +\item[0] req_vq +\end{description} + +\subsection{Feature bits}\label{sec:Device Types / PMEM Device / Feature bits} + +There are currently no feature bits defined for this device. + +\subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / Device configuration layout} + +\begin{lstlisting} +struct virtio_pmem_config { + le64 start; + le64 size; +}; +\end{lstlisting} + +\begin{description} +\item[\field{start}] contains the physical address of the first byte of the persistent memory region. + +\item[\field{size}] contains the length of this address range. +\end{description} + +\begin{enumerate} +\item Driver vpmem start is read from \field{start}. +\item Driver vpmem end is read from \field{size}. +\end{enumerate} + +\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization} + +The driver determines the start address and size of the persistent memory region in preparation for reading or writing data. + +The driver initializes req_vq in preparation for making flush requests. + +\subsection{Driver Operations}\label{sec:Device Types / PMEM Driver / Driver Operation / Request Queues}
virtio-pmem.tex line 65
Quoted 10 lines
virtio-pmem.tex new file +10
-0+ +Requests have the following format: + +\begin{lstlisting} +struct virtio_pmem_req { + le32 type; +}; +\end{lstlisting} + +\field{type} is the request command type.
Where are the type constants defined? I was expecting a flush request type constant:
/* Request types */ #define VIRTIO_PMEM_REQ_T_FLUSH 0
virtio-pmem.tex line 70
Quoted 5 lines
virtio-pmem.tex new file +5
-0+ +\subsection{Device Operations}\label{sec:Device Types / PMEM Driver / Device Operation} +\devicenormative{\subsubsection}{Device Operation: Virtqueue flush}{Device Types / PMEM Device / Device Operation / Virtqueue flush} + +The device MUST ensure that all writes made before a flush request will persist across device reset and power failure before completing the flush request.
"made" is vague, does it mean "submitted" or "completed". I suggest tweaking the language:
"writes completed before a flush request persist ..."
Quoted 12 lines
Quoted 12 lines
virtio-pmem.tex new file +6
-0@@ -0,0 +1,122 @@ +\section{PMEM Device}\label{sec:Device Types / PMEM Device} + +The virtio pmem device is a persistent memory (NVDIMM) device +that provides a virtio based asynchronous flush mechanism. This avoids +the need of a separate page cache in the guest and keeps the page cache +only in the host. Under memory pressure, the host makes use of
Quoted 8 lines
virtio-pmem.tex line 12
Quoted 6 lines
virtio-pmem.tex new file +6
-0+efficient memory reclaim decisions for page cache pages of all the +guests. This helps to reduce the memory footprint and fits more guests +in the host system. + +The virtio pmem device provides access to byte-addressable persistent +memory. The persist memory is directly accessible range of system memory.s/is directly accessible range/is a directly accessible range/
o.k
Quoted 58 lines
Quoted 44 lines
virtio-pmem.tex new file +43
-0+Data written to this memory is made persistent by separately sending a +flush command. Writes that have been flushed are preserved across device +reset and power failure. + +\subsection{Device ID}\label{sec:Device Types / PMEM Device / Device ID} + 27 + +\subsection{Virtqueues}\label{sec:Device Types / PMEM Device / Virtqueues} +\begin{description} +\item[0] req_vq +\end{description} + +\subsection{Feature bits}\label{sec:Device Types / PMEM Device / Feature bits} + +There are currently no feature bits defined for this device. + +\subsection{Device configuration layout}\label{sec:Device Types / PMEM Device / Device configuration layout} + +\begin{lstlisting} +struct virtio_pmem_config { + le64 start; + le64 size; +}; +\end{lstlisting} + +\begin{description} +\item[\field{start}] contains the physical address of the first byte of the persistent memory region. + +\item[\field{size}] contains the length of this address range. +\end{description} + +\begin{enumerate} +\item Driver vpmem start is read from \field{start}. +\item Driver vpmem end is read from \field{size}. +\end{enumerate} + +\subsection{Driver Initialization}\label{sec:Device Types / PMEM Driver / Driver Initialization} + +The driver determines the start address and size of the persistent memory region in preparation for reading or writing data. + +The driver initializes req_vq in preparation for making flush requests. + +\subsection{Driver Operations}\label{sec:Device Types / PMEM Driver / Driver Operation / Request Queues}virtio-pmem.tex line 65
Quoted 10 lines
virtio-pmem.tex new file +10
-0+ +Requests have the following format: + +\begin{lstlisting} +struct virtio_pmem_req { + le32 type; +}; +\end{lstlisting} + +\field{type} is the request command type.Where are the type constants defined? I was expecting a flush request type constant:
Quoted 2 lines
/* Request types */ #define VIRTIO_PMEM_REQ_T_FLUSH 0
Removed this in current version. Thinking this might be driver specific detail. Will add again.
Quoted 11 lines
virtio-pmem.tex line 70
Quoted 6 lines
virtio-pmem.tex new file +5
-0+ +\subsection{Device Operations}\label{sec:Device Types / PMEM Driver / Device Operation} +\devicenormative{\subsubsection}{Device Operation: Virtqueue flush}{Device Types / PMEM Device / Device Operation / Virtqueue flush} + +The device MUST ensure that all writes made before a flush request will persist across device reset and power failure before completing the flush request."made" is vague, does it mean "submitted" or "completed". I suggest tweaking the language:
"writes completed before a flush request persist ..."
yes, "completed"
Thanks,
Pankaj
Quoted 13 lines · Pankaj Gupta
Quoted 13 lines
virtio-pmem.tex line 65
Quoted 9 lines
virtio-pmem.tex new file +9
-0+Requests have the following format: + +\begin{lstlisting} +struct virtio_pmem_req { + le32 type; +}; +\end{lstlisting} + +\field{type} is the request command type.Where are the type constants defined? I was expecting a flush request type constant:
Quoted 5 lines
Quoted 2 lines
/* Request types */ #define VIRTIO_PMEM_REQ_T_FLUSH 0Removed this in current version. Thinking this might be driver specific detail. Will add again.
Constant values that the driver places into the "type" field need to be defined in the specification. Otherwise there's a risk that drivers place different values into the field and we cannot add new types of requests in the future.
Stefan
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
virtio-pmem.tex line 65
Quoted 9 lines
virtio-pmem.tex new file +9
-0+Requests have the following format: + +\begin{lstlisting} +struct virtio_pmem_req { + le32 type; +}; +\end{lstlisting} + +\field{type} is the request command type.Where are the type constants defined? I was expecting a flush request type constant:
/* Request types */ #define VIRTIO_PMEM_REQ_T_FLUSH 0
Quoted 7 lines
Quoted 2 lines
Removed this in current version. Thinking this might be driver specific detail. Will add again.
Constant values that the driver places into the "type" field need to be defined in the specification. Otherwise there's a risk that drivers place different values into the field and we cannot add new types of requests in the future.
Agree.
Waiting for any other comment before I send next version with the minor changes suggested
by Stefan in v3.
Thanks,
Pankaj