Skip to contentSkip to conversation

virtio-net: support device stats

PATCH Superseded Xuan Zhuo 2 participants 5 messages Sep 28, 2021 – Dec 9, 2021

v4 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Xuan Zhuo

author
#0 lore raw

PATCH v4 1/1 virtio-net: support device stats

This patch allows the driver to obtain some statistics from the device.

In the back-end implementation, we can count a lot of such information, which can be used for debugging and judging the running status of the back-end. We hope to directly display it to the user through ethtool.

---

v4:
1. remove dev_stats_num, {rx|tx}_stats_num
2. Use two commands to get the stats of queue pair and dev respectively

v3 changes:
1. add dev_version
2. use queue_pair_index replace rx_num, tx_num
3. Explain the processing when the device and driver support different numbers of stats

 content.tex | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)
content.tex +77 -1
index 7cec1c3..ad8d781 100644
@@ -2972,6 +2972,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_CTRL_STATS(55)] Device can provide device-level statistics
+    to the driver through the control channel.
+
 \item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
  (fragmenting the packet) the USO splits large UDP packet
  to several segments when each of these smaller packets has UDP header.
@@ -3017,6 +3020,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_CTRL_STATS] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
@@ -3895,6 +3899,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
         u8 command;
         u8 command-specific-data[];
         u8 ack;
+        u8 command-specific-data-reply[];
 };
 
 /* ack values */
@@ -3903,7 +3908,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \end{lstlisting}
 
 The \field{class}, \field{command} and command-specific-data are set by the
-driver, and the device sets the \field{ack} byte. There is little it can
+driver, and the device sets the \field{ack} byte and optionally
+\field{command-specific-data-reply}. There is little it can
 do except issue a diagnostic if \field{ack} is not
 VIRTIO_NET_OK.
 
@@ -4384,6 +4390,76 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 See \ref{sec:Basic
 Facilities of a Virtio Device / Virtqueues / Message Framing}.
 
+\paragraph{Device stats}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device stats}
+
+If the VIRTIO_NET_F_CTRL_STATS feature is negotiated, the driver can
+get device stats from the device by \field{command-specific-data-reply}.
+
+To get the stats, the following definitions are used:
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_STATS                  6
+#define VIRTIO_NET_CTRL_STATS_GET_DEV          0
+#define VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR   1
+\end{lstlisting}
+
+The following layout structure are used:
+
+\field{command-specific-data}
+\begin{lstlisting}
+struct virtio_net_ctrl_stats_queue_pair {
+    le64 queue_pair_index;
+}
+\end{lstlisting}
+
+\field{command-specific-data-reply}
+\begin{lstlisting}
+struct virtio_net_ctrl_reply_stats_dev {
+    le64 dev_version;		// The number of device version.
+    le64 dev_reset;			// The number of device reset.
+}
+
+struct virtio_net_ctrl_reply_stats_queue_pair {
+    le64 queue_pair_index;
+    le64 rx_inter;			// The number of interrupts sent by the rx queue.
+    le64 rx_drop;			// The number of packets dropped by the rx queue. Contains all kinds of packet loss.
+    le64 rx_drop_overruns;	// The number of packets dropped by the rx queue when no more avail desc.
+    le64 rx_csum_bad;		// The number of packets with abnormal csum in the rx queue.
+    le64 rx_gso_packets;	// The number of gso packets received by rx.
+    le64 rx_gso_bytes;		// The number of gso bytes received by rx.
+    le64 rx_oversize_pkts;	// The number of oversized packets received by the rx queue.
+    le64 tx_inter;			// The number of interrupts sent by the tx queue.
+    le64 tx_drop;			// The number of packets dropped by the tx queue. Contains all kinds of packet loss.
+    le64 tx_csum;			// The number of packets that completes csum by the device.
+    le64 tx_gso_packets;	// The number of gso packets transmitted.
+    le64 tx_gso_bytes;		// The number of gso bytes transmitted.
+}
+\end{lstlisting}
+
+All device stats is divided into two categories, one is the stats of the device,
+and the other is the stats of the queue pair. The queue pair stats contains the
+stats of rx and tx.
+
+The command VIRTIO_NET_CTRL_STATS_GET_DEV is used to obtain the stats of the device.
+The command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR is used to obtain the stats of the queue pair.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_DEV,
+\field{command-specific-data} MUST be empty. The structure
+\field{virtio_net_ctrl_reply_stats_dev} MUST be used as
+\field{command-specific-data-reply}.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR,
+\field{command-specific-data} MUST be \field{virtio_net_ctrl_stats_queue_pair}. At the same
+time, the structure \field{virtio_net_ctrl_reply_stats_queue_pair} MUST be used as
+\field{command-specific-data-reply}.
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index} specify the queue pair
+index of the queue that the driver wants to get stats.
+
+The device MUST set \field{command-specific-data-reply} based on command.
+
+When the command is VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR, the device MUST set
+\field{virtio_net_ctrl_reply_stats_queue_pair.queue_pair_index} equal to
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index}
+
 \section{Block Device}\label{sec:Device Types / Block Device}
 
 The virtio block device is a simple virtual block device (ie.

--
2.31.0

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#1 lore raw

On Tue, 28 Sep 2021 17:13:40 +0800, Xuan Zhuo <[email protected]> wrote:

Hi, everybody, is there anything else I need to do with this patch? When will this patch be merged into, so that I can carry out follow-up work.

Thanks.

Quoted 144 lines

This patch allows the driver to obtain some statistics from the device.

In the back-end implementation, we can count a lot of such information, which can be used for debugging and judging the running status of the back-end. We hope to directly display it to the user through ethtool.

---

v4:
1. remove dev_stats_num, {rx|tx}_stats_num
2. Use two commands to get the stats of queue pair and dev respectively

v3 changes:
1. add dev_version
2. use queue_pair_index replace rx_num, tx_num
3. Explain the processing when the device and driver support different numbers of stats

 content.tex | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)
content.tex +77 -2
index 7cec1c3..ad8d781 100644
@@ -2972,6 +2972,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_CTRL_STATS(55)] Device can provide device-level statistics
+    to the driver through the control channel.
+
 \item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
  (fragmenting the packet) the USO splits large UDP packet
  to several segments when each of these smaller packets has UDP header.
@@ -3017,6 +3020,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_CTRL_STATS] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
@@ -3895,6 +3899,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
         u8 command;
         u8 command-specific-data[];
         u8 ack;
+        u8 command-specific-data-reply[];
 };
 
 /* ack values */
@@ -3903,7 +3908,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \end{lstlisting}
 
 The \field{class}, \field{command} and command-specific-data are set by the
-driver, and the device sets the \field{ack} byte. There is little it can
+driver, and the device sets the \field{ack} byte and optionally
+\field{command-specific-data-reply}. There is little it can
 do except issue a diagnostic if \field{ack} is not
 VIRTIO_NET_OK.
 
@@ -4384,6 +4390,76 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 See \ref{sec:Basic
 Facilities of a Virtio Device / Virtqueues / Message Framing}.
 
+\paragraph{Device stats}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device stats}
+
+If the VIRTIO_NET_F_CTRL_STATS feature is negotiated, the driver can
+get device stats from the device by \field{command-specific-data-reply}.
+
+To get the stats, the following definitions are used:
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_STATS                  6
+#define VIRTIO_NET_CTRL_STATS_GET_DEV          0
+#define VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR   1
+\end{lstlisting}
+
+The following layout structure are used:
+
+\field{command-specific-data}
+\begin{lstlisting}
+struct virtio_net_ctrl_stats_queue_pair {
+    le64 queue_pair_index;
+}
+\end{lstlisting}
+
+\field{command-specific-data-reply}
+\begin{lstlisting}
+struct virtio_net_ctrl_reply_stats_dev {
+    le64 dev_version;		// The number of device version.
+    le64 dev_reset;			// The number of device reset.
+}
+
+struct virtio_net_ctrl_reply_stats_queue_pair {
+    le64 queue_pair_index;
+    le64 rx_inter;			// The number of interrupts sent by the rx queue.
+    le64 rx_drop;			// The number of packets dropped by the rx queue. Contains all kinds of packet loss.
+    le64 rx_drop_overruns;	// The number of packets dropped by the rx queue when no more avail desc.
+    le64 rx_csum_bad;		// The number of packets with abnormal csum in the rx queue.
+    le64 rx_gso_packets;	// The number of gso packets received by rx.
+    le64 rx_gso_bytes;		// The number of gso bytes received by rx.
+    le64 rx_oversize_pkts;	// The number of oversized packets received by the rx queue.
+    le64 tx_inter;			// The number of interrupts sent by the tx queue.
+    le64 tx_drop;			// The number of packets dropped by the tx queue. Contains all kinds of packet loss.
+    le64 tx_csum;			// The number of packets that completes csum by the device.
+    le64 tx_gso_packets;	// The number of gso packets transmitted.
+    le64 tx_gso_bytes;		// The number of gso bytes transmitted.
+}
+\end{lstlisting}
+
+All device stats is divided into two categories, one is the stats of the device,
+and the other is the stats of the queue pair. The queue pair stats contains the
+stats of rx and tx.
+
+The command VIRTIO_NET_CTRL_STATS_GET_DEV is used to obtain the stats of the device.
+The command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR is used to obtain the stats of the queue pair.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_DEV,
+\field{command-specific-data} MUST be empty. The structure
+\field{virtio_net_ctrl_reply_stats_dev} MUST be used as
+\field{command-specific-data-reply}.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR,
+\field{command-specific-data} MUST be \field{virtio_net_ctrl_stats_queue_pair}. At the same
+time, the structure \field{virtio_net_ctrl_reply_stats_queue_pair} MUST be used as
+\field{command-specific-data-reply}.
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index} specify the queue pair
+index of the queue that the driver wants to get stats.
+
+The device MUST set \field{command-specific-data-reply} based on command.
+
+When the command is VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR, the device MUST set
+\field{virtio_net_ctrl_reply_stats_queue_pair.queue_pair_index} equal to
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index}
+
 \section{Block Device}\label{sec:Device Types / Block Device}
 
 The virtio block device is a simple virtual block device (ie.
--

2.31.0

Quoted 4 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#2 lore raw
Quoted 75 lines · Xuan Zhuo

On Tue, 28 Sep 2021 17:13:40 +0800, Xuan Zhuo <[email protected]> wrote:

Hi, everybody, is there anything else I need to do with this patch? When will this patch be merged into, so that I can carry out follow-up work.

Thanks.

Quoted 67 lines

This patch allows the driver to obtain some statistics from the device.

In the back-end implementation, we can count a lot of such information, which can be used for debugging and judging the running status of the back-end. We hope to directly display it to the user through ethtool.

---

v4:
1. remove dev_stats_num, {rx|tx}_stats_num
2. Use two commands to get the stats of queue pair and dev respectively

v3 changes:
1. add dev_version
2. use queue_pair_index replace rx_num, tx_num
3. Explain the processing when the device and driver support different numbers of stats

 content.tex | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)
content.tex +7 -1
index 7cec1c3..ad8d781 100644
@@ -2972,6 +2972,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_CTRL_STATS(55)] Device can provide device-level statistics
+    to the driver through the control channel.
+
 \item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
  (fragmenting the packet) the USO splits large UDP packet
  to several segments when each of these smaller packets has UDP header.
@@ -3017,6 +3020,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_CTRL_STATS] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
@@ -3895,6 +3899,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
         u8 command;
         u8 command-specific-data[];
         u8 ack;
+        u8 command-specific-data-reply[];
 };
 
 /* ack values */
@@ -3903,7 +3908,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \end{lstlisting}
 
 The \field{class}, \field{command} and command-specific-data are set by the
-driver, and the device sets the \field{ack} byte. There is little it can
+driver, and the device sets the \field{ack} byte and optionally
+\field{command-specific-data-reply}. There is little it can
 do except issue a diagnostic if \field{ack} is not
 VIRTIO_NET_OK.
 
@@ -4384,6 +4390,76 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 See \ref{sec:Basic
 Facilities of a Virtio Device / Virtqueues / Message Framing}.
 
Quoted 10 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4402

Quoted 10 lines
content.tex +10 -0
+\paragraph{Device stats}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device stats}
+
+If the VIRTIO_NET_F_CTRL_STATS feature is negotiated, the driver can
+get device stats from the device by \field{command-specific-data-reply}.
+
+To get the stats, the following definitions are used:
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_STATS                  6
+#define VIRTIO_NET_CTRL_STATS_GET_DEV          0
+#define VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR   1

We need cvq stats as well.

Quoted 7 lines
Quoted 7 lines
content.tex +7 -0
+\end{lstlisting}
+
+The following layout structure are used:
+
+\field{command-specific-data}
+\begin{lstlisting}
+struct virtio_net_ctrl_stats_queue_pair {
Quoted 8 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4417

Quoted 8 lines
content.tex +8 -0
+    le64 queue_pair_index;
+}
+\end{lstlisting}
+
+\field{command-specific-data-reply}
+\begin{lstlisting}
+struct virtio_net_ctrl_reply_stats_dev {
+    le64 dev_version;                // The number of device version.

What's the use case for this field?

Quoted 11 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4428

Quoted 11 lines
content.tex +11 -0
+    le64 dev_reset;                  // The number of device reset.
+}
+
+struct virtio_net_ctrl_reply_stats_queue_pair {
+    le64 queue_pair_index;
+    le64 rx_inter;                   // The number of interrupts sent by the rx queue.
+    le64 rx_drop;                    // The number of packets dropped by the rx queue. Contains all kinds of packet loss.
+    le64 rx_drop_overruns;   // The number of packets dropped by the rx queue when no more avail desc.
+    le64 rx_csum_bad;                // The number of packets with abnormal csum in the rx queue.
+    le64 rx_gso_packets;     // The number of gso packets received by rx.
+    le64 rx_gso_bytes;               // The number of gso bytes received by rx.

It's kind of strange that we count gso packets without the the packets/bytes regardless of gso or not.

Quoted 3 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4431

Quoted 3 lines
content.tex +3 -0
+    le64 rx_oversize_pkts;   // The number of oversized packets received by the rx queue.
+    le64 tx_inter;                   // The number of interrupts sent by the tx queue.
+    le64 tx_drop;                    // The number of packets dropped by the tx queue. Contains all kinds of packet loss.

Is there any reason that can cause the packet to be dropped in tx by the device?

Quoted 7 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4438

Quoted 7 lines
content.tex +7 -0
+    le64 tx_csum;                    // The number of packets that completes csum by the device.
+    le64 tx_gso_packets;     // The number of gso packets transmitted.
+    le64 tx_gso_bytes;               // The number of gso bytes transmitted.
+}
+\end{lstlisting}
+
+All device stats is divided into two categories, one is the stats of the device,

s/is/are/

Btw, I wonder how easily we can extend the stats based on this.

Thanks

Quoted 31 lines
Quoted 31 lines
content.tex +24 -1
+and the other is the stats of the queue pair. The queue pair stats contains the
+stats of rx and tx.
+
+The command VIRTIO_NET_CTRL_STATS_GET_DEV is used to obtain the stats of the device.
+The command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR is used to obtain the stats of the queue pair.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_DEV,
+\field{command-specific-data} MUST be empty. The structure
+\field{virtio_net_ctrl_reply_stats_dev} MUST be used as
+\field{command-specific-data-reply}.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR,
+\field{command-specific-data} MUST be \field{virtio_net_ctrl_stats_queue_pair}. At the same
+time, the structure \field{virtio_net_ctrl_reply_stats_queue_pair} MUST be used as
+\field{command-specific-data-reply}.
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index} specify the queue pair
+index of the queue that the driver wants to get stats.
+
+The device MUST set \field{command-specific-data-reply} based on command.
+
+When the command is VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR, the device MUST set
+\field{virtio_net_ctrl_reply_stats_queue_pair.queue_pair_index} equal to
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index}
+
 \section{Block Device}\label{sec:Device Types / Block Device}
 
 The virtio block device is a simple virtual block device (ie.
--

2.31.0

Quoted 5 lines
Quoted 4 lines

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

author
#3 lore raw
Quoted 76 lines · Jason Wang
Quoted 75 lines · Xuan Zhuo

On Tue, 28 Sep 2021 17:13:40 +0800, Xuan Zhuo <[email protected]> wrote:

Hi, everybody, is there anything else I need to do with this patch? When will this patch be merged into, so that I can carry out follow-up work.

Thanks.

Quoted 67 lines

This patch allows the driver to obtain some statistics from the device.

In the back-end implementation, we can count a lot of such information, which can be used for debugging and judging the running status of the back-end. We hope to directly display it to the user through ethtool.

---

v4:
1. remove dev_stats_num, {rx|tx}_stats_num
2. Use two commands to get the stats of queue pair and dev respectively

v3 changes:
1. add dev_version
2. use queue_pair_index replace rx_num, tx_num
3. Explain the processing when the device and driver support different numbers of stats

 content.tex | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)
content.tex +7 -1
index 7cec1c3..ad8d781 100644
@@ -2972,6 +2972,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_CTRL_STATS(55)] Device can provide device-level statistics
+    to the driver through the control channel.
+
 \item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
  (fragmenting the packet) the USO splits large UDP packet
  to several segments when each of these smaller packets has UDP header.
@@ -3017,6 +3020,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_CTRL_STATS] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
@@ -3895,6 +3899,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
         u8 command;
         u8 command-specific-data[];
         u8 ack;
+        u8 command-specific-data-reply[];
 };
 
 /* ack values */
@@ -3903,7 +3908,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \end{lstlisting}
 
 The \field{class}, \field{command} and command-specific-data are set by the
-driver, and the device sets the \field{ack} byte. There is little it can
+driver, and the device sets the \field{ack} byte and optionally
+\field{command-specific-data-reply}. There is little it can
 do except issue a diagnostic if \field{ack} is not
 VIRTIO_NET_OK.
 
@@ -4384,6 +4390,76 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 See \ref{sec:Basic
 Facilities of a Virtio Device / Virtqueues / Message Framing}.
 
Quoted 12 lines
Quoted 10 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4402

Quoted 10 lines
content.tex +10 -0
+\paragraph{Device stats}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device stats}
+
+If the VIRTIO_NET_F_CTRL_STATS feature is negotiated, the driver can
+get device stats from the device by \field{command-specific-data-reply}.
+
+To get the stats, the following definitions are used:
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_STATS                  6
+#define VIRTIO_NET_CTRL_STATS_GET_DEV          0
+#define VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR   1

We need cvq stats as well.

Yes, I will add cvq related in the next version.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
content.tex +9 -0
+\end{lstlisting}
+
+The following layout structure are used:
+
+\field{command-specific-data}
+\begin{lstlisting}
+struct virtio_net_ctrl_stats_queue_pair {
+    le64 queue_pair_index;
+}
Quoted 8 lines
Quoted 6 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4417

Quoted 6 lines
content.tex +6 -0
+\end{lstlisting}
+
+\field{command-specific-data-reply}
+\begin{lstlisting}
+struct virtio_net_ctrl_reply_stats_dev {
+    le64 dev_version;                // The number of device version.

What's the use case for this field?

Used to distinguish different versions of devices. For example, in a cloud environment, the device is provided by a cloud service provider. However, there may be multiple different versions. There will be some differences in functions and features between these different versions. Based on this Field users can know which version of the backend device they are currently using.

Of course, for back-end devices using different series, this value may be meaningless.

Quoted 15 lines
Quoted 12 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4428

Quoted 12 lines
content.tex +11 -0
+    le64 dev_reset;                  // The number of device reset.
+}
+
+struct virtio_net_ctrl_reply_stats_queue_pair {
+    le64 queue_pair_index;
+    le64 rx_inter;                   // The number of interrupts sent by the rx queue.
+    le64 rx_drop;                    // The number of packets dropped by the rx queue. Contains all kinds of packet loss.
+    le64 rx_drop_overruns;   // The number of packets dropped by the rx queue when no more avail desc.
+    le64 rx_csum_bad;                // The number of packets with abnormal csum in the rx queue.
+    le64 rx_gso_packets;     // The number of gso packets received by rx.
+    le64 rx_gso_bytes;               // The number of gso bytes received by rx.

It's kind of strange that we count gso packets without the the packets/bytes regardless of gso or not.

Since the driver already has statistics packets/bytes, and for the driver, this information can be easily counted. Do you think we need to add packets/bytes?

Quoted 6 lines
Quoted 4 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4431

Quoted 4 lines
content.tex +3 -0
+    le64 rx_oversize_pkts;   // The number of oversized packets received by the rx queue.
+    le64 tx_inter;                   // The number of interrupts sent by the tx queue.
+    le64 tx_drop;                    // The number of packets dropped by the tx queue. Contains all kinds of packet loss.

Is there any reason that can cause the packet to be dropped in tx by the device?

For example, some scenarios where the flow rate is limited. And scenarios where the desc message sent by the driver is wrong.

Quoted 11 lines
Quoted 8 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4438

Quoted 8 lines
content.tex +7 -0
+    le64 tx_csum;                    // The number of packets that completes csum by the device.
+    le64 tx_gso_packets;     // The number of gso packets transmitted.
+    le64 tx_gso_bytes;               // The number of gso bytes transmitted.
+}
+\end{lstlisting}
+
+All device stats is divided into two categories, one is the stats of the device,

s/is/are/

Quoted 1 line

Btw, I wonder how easily we can extend the stats based on this.

"Probably not, looking at other NIC drivers the "keys" are pretty stable. We can start from a large set anyhow."

As we discussed earlier, expanding stats is a relatively rare thing. We will start with a large stats collection.

Thanks.

Quoted 40 lines

Thanks

Quoted 36 lines
Quoted 35 lines
content.tex +24 -1
+and the other is the stats of the queue pair. The queue pair stats contains the
+stats of rx and tx.
+
+The command VIRTIO_NET_CTRL_STATS_GET_DEV is used to obtain the stats of the device.
+The command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR is used to obtain the stats of the queue pair.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_DEV,
+\field{command-specific-data} MUST be empty. The structure
+\field{virtio_net_ctrl_reply_stats_dev} MUST be used as
+\field{command-specific-data-reply}.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR,
+\field{command-specific-data} MUST be \field{virtio_net_ctrl_stats_queue_pair}. At the same
+time, the structure \field{virtio_net_ctrl_reply_stats_queue_pair} MUST be used as
+\field{command-specific-data-reply}.
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index} specify the queue pair
+index of the queue that the driver wants to get stats.
+
+The device MUST set \field{command-specific-data-reply} based on command.
+
+When the command is VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR, the device MUST set
+\field{virtio_net_ctrl_reply_stats_queue_pair.queue_pair_index} equal to
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index}
+
 \section{Block Device}\label{sec:Device Types / Block Device}
 
 The virtio block device is a simple virtual block device (ie.
--

2.31.0

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

#4 lore raw
Quoted 112 lines · Xuan Zhuo
Quoted 76 lines · Jason Wang
Quoted 75 lines · Xuan Zhuo

On Tue, 28 Sep 2021 17:13:40 +0800, Xuan Zhuo <[email protected]> wrote:

Hi, everybody, is there anything else I need to do with this patch? When will this patch be merged into, so that I can carry out follow-up work.

Thanks.

Quoted 67 lines

This patch allows the driver to obtain some statistics from the device.

In the back-end implementation, we can count a lot of such information, which can be used for debugging and judging the running status of the back-end. We hope to directly display it to the user through ethtool.

---

v4:
1. remove dev_stats_num, {rx|tx}_stats_num
2. Use two commands to get the stats of queue pair and dev respectively

v3 changes:
1. add dev_version
2. use queue_pair_index replace rx_num, tx_num
3. Explain the processing when the device and driver support different numbers of stats

 content.tex | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)
content.tex +7 -1
index 7cec1c3..ad8d781 100644
@@ -2972,6 +2972,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_CTRL_STATS(55)] Device can provide device-level statistics
+    to the driver through the control channel.
+
 \item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
  (fragmenting the packet) the USO splits large UDP packet
  to several segments when each of these smaller packets has UDP header.
@@ -3017,6 +3020,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_CTRL_STATS] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
@@ -3895,6 +3899,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
         u8 command;
         u8 command-specific-data[];
         u8 ack;
+        u8 command-specific-data-reply[];
 };
 
 /* ack values */
@@ -3903,7 +3908,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \end{lstlisting}
 
 The \field{class}, \field{command} and command-specific-data are set by the
-driver, and the device sets the \field{ack} byte. There is little it can
+driver, and the device sets the \field{ack} byte and optionally
+\field{command-specific-data-reply}. There is little it can
 do except issue a diagnostic if \field{ack} is not
 VIRTIO_NET_OK.
 
@@ -4384,6 +4390,76 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 See \ref{sec:Basic
 Facilities of a Virtio Device / Virtqueues / Message Framing}.
 
Quoted 12 lines
Quoted 10 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4402

Quoted 10 lines
content.tex +10 -0
+\paragraph{Device stats}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device stats}
+
+If the VIRTIO_NET_F_CTRL_STATS feature is negotiated, the driver can
+get device stats from the device by \field{command-specific-data-reply}.
+
+To get the stats, the following definitions are used:
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_STATS                  6
+#define VIRTIO_NET_CTRL_STATS_GET_DEV          0
+#define VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR   1

We need cvq stats as well.

Yes, I will add cvq related in the next version.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
content.tex +9 -0
+\end{lstlisting}
+
+The following layout structure are used:
+
+\field{command-specific-data}
+\begin{lstlisting}
+struct virtio_net_ctrl_stats_queue_pair {
+    le64 queue_pair_index;
+}
Quoted 8 lines
Quoted 6 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4417

Quoted 6 lines
content.tex +6 -0
+\end{lstlisting}
+
+\field{command-specific-data-reply}
+\begin{lstlisting}
+struct virtio_net_ctrl_reply_stats_dev {
+    le64 dev_version;                // The number of device version.

What's the use case for this field?

Quoted 8 lines

Used to distinguish different versions of devices. For example, in a cloud environment, the device is provided by a cloud service provider. However, there may be multiple different versions. There will be some differences in functions and features between these different versions. Based on this Field users can know which version of the backend device they are currently using.

Of course, for back-end devices using different series, this value may be meaningless.

I'd expect to use features instead of the version here. As we've discussed, it looks to me it can introduce troubles for migration compatibility.

Quoted 18 lines
Quoted 17 lines
Quoted 14 lines
Quoted 4 lines
content.tex +1 -0
+    le64 dev_reset;                  // The number of device reset.

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4428

Quoted 10 lines
content.tex +10 -0
+}
+
+struct virtio_net_ctrl_reply_stats_queue_pair {
+    le64 queue_pair_index;
+    le64 rx_inter;                   // The number of interrupts sent by the rx queue.
+    le64 rx_drop;                    // The number of packets dropped by the rx queue. Contains all kinds of packet loss.
+    le64 rx_drop_overruns;   // The number of packets dropped by the rx queue when no more avail desc.
+    le64 rx_csum_bad;                // The number of packets with abnormal csum in the rx queue.
+    le64 rx_gso_packets;     // The number of gso packets received by rx.
+    le64 rx_gso_bytes;               // The number of gso bytes received by rx.

It's kind of strange that we count gso packets without the the packets/bytes regardless of gso or not.

Quoted 2 lines

Since the driver already has statistics packets/bytes, and for the driver, this information can be easily counted. Do you think we need to add packets/bytes?

I think so, the counters at device level are even more important which can be used for debugging.

Quoted 10 lines
Quoted 7 lines
Quoted 5 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4431

Quoted 5 lines
content.tex +3 -0
+    le64 rx_oversize_pkts;   // The number of oversized packets received by the rx queue.
+    le64 tx_inter;                   // The number of interrupts sent by the tx queue.
+    le64 tx_drop;                    // The number of packets dropped by the tx queue. Contains all kinds of packet loss.

Is there any reason that can cause the packet to be dropped in tx by the device?

For example, some scenarios where the flow rate is limited. And scenarios where the desc message sent by the driver is wrong.

Yes.

Quoted 12 lines
Quoted 12 lines
Quoted 9 lines

content.tex \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device · line 4438

Quoted 9 lines
content.tex +7 -0
+    le64 tx_csum;                    // The number of packets that completes csum by the device.
+    le64 tx_gso_packets;     // The number of gso packets transmitted.
+    le64 tx_gso_bytes;               // The number of gso bytes transmitted.
+}
+\end{lstlisting}
+
+All device stats is divided into two categories, one is the stats of the device,

s/is/are/

Quoted 8 lines
Quoted 1 line

Btw, I wonder how easily we can extend the stats based on this.

"Probably not, looking at other NIC drivers the "keys" are pretty stable. We can start from a large set anyhow."

As we discussed earlier, expanding stats is a relatively rare thing. We will start with a large stats collection.

Right, I forget that.

Thanks

Quoted 44 lines

Thanks.

Quoted 40 lines

Thanks

Quoted 36 lines
Quoted 35 lines
content.tex +24 -1
+and the other is the stats of the queue pair. The queue pair stats contains the
+stats of rx and tx.
+
+The command VIRTIO_NET_CTRL_STATS_GET_DEV is used to obtain the stats of the device.
+The command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR is used to obtain the stats of the queue pair.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_DEV,
+\field{command-specific-data} MUST be empty. The structure
+\field{virtio_net_ctrl_reply_stats_dev} MUST be used as
+\field{command-specific-data-reply}.
+
+When the driver uses the command VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR,
+\field{command-specific-data} MUST be \field{virtio_net_ctrl_stats_queue_pair}. At the same
+time, the structure \field{virtio_net_ctrl_reply_stats_queue_pair} MUST be used as
+\field{command-specific-data-reply}.
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index} specify the queue pair
+index of the queue that the driver wants to get stats.
+
+The device MUST set \field{command-specific-data-reply} based on command.
+
+When the command is VIRTIO_NET_CTRL_STATS_GET_QUEUE_PAIR, the device MUST set
+\field{virtio_net_ctrl_reply_stats_queue_pair.queue_pair_index} equal to
+\field{virtio_net_ctrl_stats_queue_pair.queue_pair_index}
+
 \section{Block Device}\label{sec:Device Types / Block Device}
 
 The virtio block device is a simple virtual block device (ie.
--

2.31.0

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]